Make sure to use lib instead of lib64 for LLDB_LIB_DIR

[fjricci]

Summary

$(lldb -P)/../../ is assumed to be the lldb library directory
by the test suite. However, it is possible that the python
libs would be installed in build/lib64 instead of build/lib.
Since liblldb.so is always installed in lib,
make sure this is always used as LLDB_LIB_DIR.

In cases where the python libs were already in build/lib, this
patch will not affect LLDB_LIB_DIR.

[labath]

Actually, the destination for liblldb.so depends on the LLVM_LIBDIR_SUFFIX cmake variable. If you're on a system which likes to shove things into lib64, maybe you could follow suit and define the variable when you build llvm/lldb. (otherwise, this patch will break the logic for anyone who does use the variable.) Would that work for your use case?

[fjricci]

When I use LLVM_LIBDIR_SUFFIX=64, it looks like the python _lldb.so symlink still points to build/lib/liblldb.so, instead of using the lib64 directory. So if I do this, none of the tests will run on the check-lldb target, since importing lldb in python fails.

Perhaps this symlink is the bug that should be fixed? However, it appears that finish-swig-Python-LLDB.sh already has the correct logic for this, so it's not immediately clear to me why the symlink is incorrect (if this seems like a bug we want fixed, I'll investigate):

ln -s "../../../liblldb${SOEXT}" _lldb.so

[labath]

In D19067#401416, @fjricci wrote:

When I use LLVM_LIBDIR_SUFFIX=64, it looks like the python _lldb.so symlink still points to build/lib/liblldb.so, instead of using the lib64 directory. So if I do this, none of the tests will run on the check-lldb target, since importing lldb in python fails.

Perhaps this symlink is the bug that should be fixed? However, it appears that finish-swig-Python-LLDB.sh already has the correct logic for this, so it's not immediately clear to me why the symlink is incorrect (if this seems like a bug we want fixed, I'll investigate):

ln -s "../../../liblldb${SOEXT}" _lldb.so

Yes, I believe this would be a better fix for your problem.

Note that finish-swig-Python-LLDB.sh is not used anymore in the cmake build. It's replacement is finishSwigPythonLLDB.py so you'll need to look for the problem there.

[fjricci]

So I looked at finishSwigPythonLLDB.py, and it does look like that's where the bug is. This script sets the end of the symlink path with:

strSrc = os.path.join("lib", "liblldb" + strLibFileExtn)

Note here that "lib" is hardcoded, and doesn't use the LLVM_LIBDIR_SUFFIX, which is why the symlink is broken. The only way I can think of to communicate this suffix from cmake into python would be by having cmake set an environment variable (presumably with the same name). Does this seem reasonable, or do you think there's a better solution?

The solution from finish-swig-Python-LLDB.sh would also probably work (enforcing that liblldb.so and python2.7/site-packages/lldb be in the same directory). That solution actually would be more consistent with the way the unit test suite works anyway. It does seem less flexible though.

[zturner]

Don't use an environment variable, add a command line flag to the script
instead

[labath]

Yes, please use a command-line argument to pass that information into python.

[fjricci]

Pass the lldb lib directory to python swig scripts

lldb/trunk/CMakeLists.txt

 add_subdirectory(test)
 add_subdirectory(tools)
 add_subdirectory(unittests)
 add_subdirectory(lit)
 
 if (NOT LLDB_DISABLE_PYTHON)
     # Add a Post-Build Event to copy over Python files and create the symlink to liblldb.so for the Python API(hardlink on Windows)
     add_custom_target( finish_swig ALL
-        COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/scripts/finishSwigWrapperClasses.py "--srcRoot=${LLDB_SOURCE_DIR}" "--targetDir=${CMAKE_CURRENT_BINARY_DIR}/scripts" "--cfgBldDir=${CMAKE_CURRENT_BINARY_DIR}/scripts" "--prefix=${CMAKE_BINARY_DIR}" "--cmakeBuildConfiguration=${CMAKE_CFG_INTDIR}" -m
+        COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/scripts/finishSwigWrapperClasses.py "--srcRoot=${LLDB_SOURCE_DIR}" "--targetDir=${CMAKE_CURRENT_BINARY_DIR}/scripts" "--cfgBldDir=${CMAKE_CURRENT_BINARY_DIR}/scripts" "--prefix=${CMAKE_BINARY_DIR}" "--cmakeBuildConfiguration=${CMAKE_CFG_INTDIR}" "--lldbLibDir=lib${LLVM_LIBDIR_SUFFIX}" -m
         DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/scripts/finishSwigWrapperClasses.py
         DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/scripts/lldb.py
         COMMENT "Python script sym-linking LLDB Python API")
     # We depend on liblldb being built before we can do this step.
     add_dependencies(finish_swig liblldb lldb-argdumper)
 
     # If we build the readline module, we depend on that happening
     # first.

lldb/trunk/scripts/Python/finishSwigPythonLLDB.py

 
 
 #++---------------------------------------------------------------------------
 # Details:  Make the symbolic that the script bridge for Python will need in
 #           the Python framework directory.
 # Args:     vDictArgs               - (R) Program input parameters.
 #           vstrFrameworkPythonDir  - (R) Python framework directory.
 #           vstrLiblldbName         - (R) File name for _lldb library.
+#           vstrLiblldbDir          - (R) liblldb directory.
 # Returns:  Bool - True = function success, False = failure.
 #           Str - Error description on task failure.
 # Throws:   None.
 #--
-def make_symlink_liblldb(vDictArgs, vstrFrameworkPythonDir, vstrLiblldbFileName):
+def make_symlink_liblldb(vDictArgs, vstrFrameworkPythonDir, vstrLiblldbFileName, vstrLldbLibDir):
     dbg = utilsDebug.CDebugFnVerbose("Python script make_symlink_liblldb()")
     bOk = True
     strErrMsg = ""
     strTarget = vstrLiblldbFileName
     strSrc = ""
 
     eOSType = utilsOsType.determine_os_type()
     if eOSType == utilsOsType.EnumOsType.Windows:
         # When importing an extension module using a debug version of python, you
         # write, for example, "import foo", but the interpreter searches for
         # "foo_d.pyd"
         if is_debug_interpreter():
             strTarget += "_d"
         strTarget += ".pyd"
     else:
         strTarget += ".so"
 
     bMakeFileCalled = "-m" in vDictArgs
     if not bMakeFileCalled:
-        strSrc = os.path.join("lib", "LLDB")
+        strSrc = os.path.join(vstrLldbLibDir, "LLDB")
     else:
         strLibFileExtn = ""
         if eOSType == utilsOsType.EnumOsType.Windows:
             strSrc = os.path.join("bin", "liblldb.dll")
         else:
             if eOSType == utilsOsType.EnumOsType.Darwin:
                 strLibFileExtn = ".dylib"
             else:
                 strLibFileExtn = ".so"
-            strSrc = os.path.join("lib", "liblldb" + strLibFileExtn)
+            strSrc = os.path.join(vstrLldbLibDir, "liblldb" + strLibFileExtn)
 
     bOk, strErrMsg = make_symlink(vDictArgs, vstrFrameworkPythonDir, strSrc, strTarget)
 
     return (bOk, strErrMsg)
 
 #++---------------------------------------------------------------------------
 # Details:  Make the symbolic link to the darwin-debug.
 # Args:     vDictArgs               - (R) Program input parameters.
def make_symlink_lldb_argdumper(vDictArgs, vstrFrameworkPythonDir, vstrArgdumperFileName):
 
     return (bOk, strErrMsg)
 
 #++---------------------------------------------------------------------------
 # Details:  Make the symlink that the script bridge for Python will need in
 #           the Python framework directory.
 # Args:     vDictArgs               - (R) Program input parameters.
 #           vstrFrameworkPythonDir  - (R) Python framework directory.
+#           vstrLldbLibDir          - (R) liblldb directory.
 # Returns:  Bool - True = function success, False = failure.
 #           strErrMsg - Error description on task failure.
 # Throws:   None.
 #--
-def create_symlinks(vDictArgs, vstrFrameworkPythonDir):
+def create_symlinks(vDictArgs, vstrFrameworkPythonDir, vstrLldbLibDir):
     dbg = utilsDebug.CDebugFnVerbose("Python script create_symlinks()")
     bOk = True
     strErrMsg = ""
     eOSType = utilsOsType.determine_os_type()
 
     # Make symlink for _lldb
     strLibLldbFileName = "_lldb"
     if bOk:
         bOk, strErrMsg = make_symlink_liblldb(vDictArgs,
                                               vstrFrameworkPythonDir,
-                                              strLibLldbFileName)
+                                              strLibLldbFileName,
+                                              vstrLldbLibDir)
 
     # Make symlink for darwin-debug on Darwin
     strDarwinDebugFileName = "darwin-debug"
     if bOk and eOSType == utilsOsType.EnumOsType.Darwin:
         bOk, strErrMsg = make_symlink_darwin_debug(vDictArgs,
                                                    vstrFrameworkPythonDir,
                                                    strDarwinDebugFileName)
 
if eOSType == utilsOsType.EnumOsType.Unknown:
         strErrMsg = strErrMsgOsTypeUnknown
     elif eOSType == utilsOsType.EnumOsType.Windows:
         bOk, strWkDir, strErrMsg = get_framework_python_dir_windows(vDictArgs)
     else:
         bOk, strWkDir, strErrMsg = get_framework_python_dir_other_platforms(vDictArgs)
 
     return (bOk, strWkDir, strErrMsg)
 
+#++---------------------------------------------------------------------------
+# Details:  Retrieve the liblldb directory path, if it exists and is valid.
+# Args:     vDictArgs               - (R) Program input parameters.
+# Returns:  Bool - True = function success, False = failure.
+#           Str - liblldb directory path.
+#           strErrMsg - Error description on task failure.
+# Throws:   None.
+#--
+def get_liblldb_dir(vDictArgs):
+    dbg = utilsDebug.CDebugFnVerbose("Python script get_liblldb_dir()")
+    bOk = True
+    strErrMsg = ""
+
+    strLldbLibDir = ""
+    bHaveLldbLibDir = "--lldbLibDir" in vDictArgs
+    if bHaveLldbLibDir:
+        strLldbLibDir = vDictArgs["--lldbLibDir"]
+    if (bHaveLldbLibDir == False) or (strLldbLibDir.__len__() == 0):
+        strLldbLibDir = "lib"
+
+    return (bOk, strLldbLibDir, strErrMsg)
+
 #-----------------------------------------------------------------------------
 #-----------------------------------------------------------------------------
 #-----------------------------------------------------------------------------
 
 """ Details: Program main entry point fn. Called by another Python script.
 
     --------------------------------------------------------------------------
     Details: This script is to be called by another Python script. It is not
Args:   vDictArgs   - (R) Map of parameter names to values.
             --cfgBlddir     Where the buildSwigPythonLLDB.py program will
             (optional)      put the lldb.py file it generated from running
                             SWIG.
             --prefix        Is the root directory used to determine where
             (optional)      third-party modules for scripting languages should
                             be installed. Where non-Darwin systems want to put
                             the .py and .so files so that Python can find them
                             automatically. Python install directory.
+            --lldbLibDir    The name of the directory containing liblldb.so.
+            (optional)      "lib" by default.
     Results:    0       Success
                 -100+   Error from this script to the caller script.
                 -100    Error program failure with optional message.
 
     --------------------------------------------------------------------------
 
 """
 def main(vDictArgs):
def main(vDictArgs):
 
     if bOk:
         bOk, strCfgBldDir, strMsg = get_config_build_dir(vDictArgs, strFrameworkPythonDir)
     if bOk and bDbg:
         print((strMsgPyFileLocatedHere % strFrameworkPythonDir))
         print((strMsgConfigBuildDir % strCfgBldDir))
 
     if bOk:
+        bOk, strLldbLibDir, strMsg = get_liblldb_dir(vDictArgs)
+
+    if bOk:
         bOk, strMsg = find_or_create_python_dir(vDictArgs, strFrameworkPythonDir)
 
     if bOk:
-        bOk, strMsg = create_symlinks(vDictArgs, strFrameworkPythonDir)
+        bOk, strMsg = create_symlinks(vDictArgs, strFrameworkPythonDir, strLldbLibDir)
 
     if bOk:
         bOk, strMsg = copy_six(vDictArgs, strFrameworkPythonDir)
 
     if bOk:
         bOk, strMsg = copy_lldbpy_file_to_lldb_pkg_dir(vDictArgs,
                                                        strFrameworkPythonDir,
                                                        strCfgBldDir)

lldb/trunk/scripts/finishSwigWrapperClasses.py

Args:   -h              (optional) Print help information on this program.\n\
     --cfgBldDir=            (optional) Where the build-swig-Python-LLDB.py program \n\
                     will put the lldb.py file it generated from running\n\
                     SWIG.\n\
     --prefix=           (optional) Is the root directory used to determine where\n\
                     third-party modules for scripting languages should\n\
                     be installed. Where non-Darwin systems want to put\n\
                     the .py and .so files so that Python can find them\n\
                     automatically. Python install directory.\n\
+    --lldbLibDir    (optional) The name of the directory containing liblldb.so.\n\
+                    \"lib\" by default.\n\
     --cmakeBuildConfiguration=  (optional) Is the build configuration(Debug, Release, RelWithDebugInfo)\n\
                     used to determine where the bin and lib directories are \n\
                     created for a Windows build.\n\
     --argsFile=         The args are read from a file instead of the\n\
                     command line. Other command line args are ignored.\n\
 \n\
 Usage:\n\
     finishSwigWrapperClasses.py --srcRoot=ADirPath --targetDir=ADirPath\n\
-    --cfgBldDir=ADirPath --prefix=ADirPath -m -d\n\
+    --cfgBldDir=ADirPath --prefix=ADirPath --lldbLibDir=ADirPath -m -d\n\
 \n\
 " #TAG_PROGRAM_HELP_INFO
 
 #++---------------------------------------------------------------------------
 # Details:  Exit the program on success. Called on program successfully done
 #           its work. Returns a status result to the caller.
 # Args:     vnResult    - (R) 0 or greater indicating success.
 #           vMsg        - (R) Success message if any to show success to user.

 #--
 def validate_arguments(vArgv):
     dbg = utilsDebug.CDebugFnVerbose("validate_arguments()")
     strMsg = ""
     dictArgs = {}
     nResult = 0
     strListArgs = "hdm" # Format "hiox:" = -h -i -o -x <arg>
     listLongArgs = ["srcRoot=", "targetDir=", "cfgBldDir=", "prefix=", "cmakeBuildConfiguration=",
-                    "argsFile"]
+                    "lldbLibDir=", "argsFile"]
     dictArgReq = {  "-h": "o",          # o = optional, m = mandatory
                     "-d": "o",
                     "-m": "o",
                     "--srcRoot": "m",
                     "--targetDir": "m",
                     "--cfgBldDir": "o",
                     "--prefix": "o",
                     "--cmakeBuildConfiguration": "o",
+                    "--lldbLibDir": "o",
                     "--argsFile": "o" }
 
     # Check for mandatory parameters
     nResult, dictArgs, strMsg = utilsArgsParse.parse(vArgv, strListArgs,
                                                      listLongArgs,
                                                      dictArgReq,
                                                      strHelpInfo)
     if nResult < 0:
Args:   -h (optional)   Print help information on this program.
             --prefix=       Is the root directory used to determine where
             (optional)      third-party modules for scripting languages should
                             be installed. Where non-Darwin systems want to put
                             the .py and .so files so that Python can find them
                             automatically. Python install directory.
             --cmakeBuildConfiguration=  (optional) Is the build configuration(Debug, Release, RelWithDebugInfo)\n\
                             used to determine where the bin and lib directories are \n\
                             created for a Windows build.\n\
+            --lldbLibDir=   The name of the directory containing liblldb.so.
+            (optional)      "lib" by default.
             --argsFile=     The args are read from a file instead of the
                             command line. Other command line args are ignored.
     Usage:
             finishSwigWrapperClasses.py --srcRoot=ADirPath --targetDir=ADirPath
-            --cfgBldDir=ADirPath --prefix=ADirPath -m -d
+            --cfgBldDir=ADirPath --prefix=ADirPath --lldbLibDir=ADirPath -m -d
 
     Results:    0 Success
                 -1 Error - invalid parameters passed.
                 -2 Error - incorrect number of mandatory parameters passed.
 
                 -4 Error - unable to determine OS type.
                 -5 Error - program not run with name of "__main__".
                 -8 Error - unable to locate the scripts folder.