Make it possible for lit.site.cfg to contain relative paths, and use it for llvm and clang

[thakis]

Summary

Currently, all generated lit.site.cfg files contain absolute paths.

This makes it impossible to build on one machine, and then transfer the build output
to another machine for test execution. Being able to do this is useful for several use cases:

1. When running tests on an ARM machine, it would be possible to build on a fast x86 machine and then copy build artifacts over after building.

2. It allows running several test suites (clang, llvm, lld) on 3 different machines, reducing test time from sum(each test suite time) to max(each test suite time).

This patch makes it possible to pass a list of variables that should be relative in the
generated lit.site.cfg.py file to configure_lit_site_cfg(). The lit.site.cfg.py.in file needs
to call path() on these variables, so that the paths are converted to absolute form
at lit start time.

The testers would have to have an LLVM checkout at the same revision, and the build
dir would have to be at the same relative path as on the builder.

This does not yet cover how to figure out which files to copy from the builder machine
to the tester machines. (One idea is to look at the --graphviz=test.dot output and
copy all inputs of the check-llvm target.)

[hans]

Looks reasonable to me. Just some nits.

llvm/cmake/modules/AddLLVM.cmake

1483

Maybe comment what this does?

1485

typo: Passign -> Passing

1495

Nit: maybe len_vars to match the 'PATHS' name?

[thakis]

Thanks!

llvm/cmake/modules/AddLLVM.cmake

1483

Turns out the saving and undoing isn't needed since cmake copies all variables to new scopes, and this changes just the local copy. So this is gone now.

[ldionne]

Just trying to wrap my head around this. Basically, this makes lit.cfg files relocatable, however doesn't everything else need to be relocatable too for anything to work? For example, if you set the rpath on some executable, that needs to be relative too, right?

So, assuming this patch, the basic idea is:

1. Build on machine A
2. Copy the whole build directory and LLVM tree to machine B
3. Run lit -sv <whatever> on machine B -- everything should work if all paths in lit.cfg and all paths embedded in build artifacts are relative to the build directory.

It also requires that machine B is able to execute all the tests, so for example for the Clang test suite it needs to be able to run Clang, run FileCheck, etc. Drawing a parallel with the libc++ test suite, this would require machine B to be able to build executables for machine B (from the .pass.cpp sources) and run them on machine B.

Is that the idea?

[thakis]

I somehow edited my comment instead of replying to it. Editing again to restore what I deleted:

Yes, that's all correct.

2. Copy the whole build directory and LLVM tree to machine B

(Except for implementation tweaks. For example, the test machine could possibly pull the llvm tree at the same revision that the builder builds at while the builder's still building, instead of copying (parts of) the LLVM tree from the builder after a build.)

[CarlosAlbertoEnciso]

In D77184#1957632, @thakis wrote:

Thanks!

Hi,

After the series of changes you have done, I am experiencing a build error on Windows. These are some of the errors:

• Attribute ignored -- Loadable modules not supported on this platform.

Traceback (most recent call last):

File "<string>", line 1, in <module>
File "<string>", line 1, in <genexpr>
File "C:\ProgramData\Python27\lib\ntpath.py", line 529, in relpath
  % (path_prefix, start_prefix))

ValueError: path is on drive X:, start on drive E:
CMake Error at X:/llvm-root/llvm-project/llvm/cmake/modules/AddLLVM.cmake:1508 (message):

PATHS lengths got confused

Call Stack (most recent call first):

X:/llvm-root/llvm-project/clang/test/CMakeLists.txt:33 (configure_lit_site_cfg)

CMake Error at X:/llvm-root/llvm-project/llvm/cmake/modules/AddLLVM.cmake:1517 (list):

list GET given empty list

Call Stack (most recent call first):

X:/llvm-root/llvm-project/clang/test/CMakeLists.txt:33 (configure_lit_site_cfg)

CMake Error at X:/llvm-root/llvm-project/llvm/cmake/modules/AddLLVM.cmake:1517 (list):

list GET given empty list

Call Stack (most recent call first):

X:/llvm-root/llvm-project/clang/test/CMakeLists.txt:33 (configure_lit_site_cfg)

CMake Error at X:/llvm-root/llvm-project/llvm/cmake/modules/AddLLVM.cmake:1517 (list):

list GET given empty list

Call Stack (most recent call first):

X:/llvm-root/llvm-project/clang/test/CMakeLists.txt:33 (configure_lit_site_cfg)

CMake Error at X:/llvm-root/llvm-project/llvm/cmake/modules/AddLLVM.cmake:1517 (list):

list GET given empty list

Call Stack (most recent call first):

X:/llvm-root/llvm-project/clang/test/CMakeLists.txt:33 (configure_lit_site_cfg)

My CMake command line is:

E:\llvm-root\builds\win\debug\x64>cmake -G"Visual Studio 15 2017 Win64" -Thost=x64 -DCMAKE_BUILD_TYPE=Debug -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-gnu
-DLLVM_ENABLE_PROJECTS="clang" x:\llvm-root\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF

Drive E: contains the build files
Drive X: contains the llvm sources

The only way to have a successful build is to revert : ab11b9eefa16661017c2c7b3b34c46b069f43fb7

Thanks

[andrewng]

The following patch fixes my issues on Windows, but I haven't tested that it doesn't break anything else:

diff --git a/llvm/cmake/modules/AddLLVM.cmake b/llvm/cmake/modules/AddLLVM.cmake
index 91bec7d8081..f630af211fd 100644
--- a/llvm/cmake/modules/AddLLVM.cmake
+++ b/llvm/cmake/modules/AddLLVM.cmake
@@ -1495,7 +1495,7 @@ function(configure_lit_site_cfg site_in site_out)
     string(REPLACE ";" "\\;" ARG_PATH_VALUES_ESCAPED "${ARG_PATH_VALUES}")
     get_filename_component(OUTPUT_DIR ${site_out} DIRECTORY)
     execute_process(COMMAND "${PYTHON_EXECUTABLE}" "-c"
-      "import os, sys; sys.stdout.write(';'.join(os.path.relpath(p, sys.argv[1]).replace(os.sep, '/') if p else '' for p in sys.argv[2].split(';')))"
+      "import os, sys; drive = os.path.splitdrive(sys.argv[1])[0]; sys.stdout.write(';'.join('' if not p else p if os.path.splitdrive(p)[0] != drive else os.path.relpath(p, sys.argv[1]).replace(os.sep, '/') for p in sys.argv[2].split(';')))"
       ${OUTPUT_DIR}
       ${ARG_PATH_VALUES_ESCAPED}
       OUTPUT_VARIABLE ARG_PATH_VALUES_RELATIVE)

[grimar]

In D77184#1959351, @andrewng wrote:

The following patch fixes my issues on Windows, but I haven't tested that it doesn't break anything else:

diff --git a/llvm/cmake/modules/AddLLVM.cmake b/llvm/cmake/modules/AddLLVM.cmake
index 91bec7d8081..f630af211fd 100644
--- a/llvm/cmake/modules/AddLLVM.cmake
+++ b/llvm/cmake/modules/AddLLVM.cmake
@@ -1495,7 +1495,7 @@ function(configure_lit_site_cfg site_in site_out)
     string(REPLACE ";" "\\;" ARG_PATH_VALUES_ESCAPED "${ARG_PATH_VALUES}")
     get_filename_component(OUTPUT_DIR ${site_out} DIRECTORY)
     execute_process(COMMAND "${PYTHON_EXECUTABLE}" "-c"
-      "import os, sys; sys.stdout.write(';'.join(os.path.relpath(p, sys.argv[1]).replace(os.sep, '/') if p else '' for p in sys.argv[2].split(';')))"
+      "import os, sys; drive = os.path.splitdrive(sys.argv[1])[0]; sys.stdout.write(';'.join('' if not p else p if os.path.splitdrive(p)[0] != drive else os.path.relpath(p, sys.argv[1]).replace(os.sep, '/') for p in sys.argv[2].split(';')))"
       ${OUTPUT_DIR}
       ${ARG_PATH_VALUES_ESCAPED}
       OUTPUT_VARIABLE ARG_PATH_VALUES_RELATIVE)

This helped me, thanks for posting this! (I've just updated and faced the build issue on windows. Seems it doesn't like that I build it on D: drive)

[thakis]

grimar, andrewng: You both have checkout and build on different drives too, yes?

[thakis]

I pushed Andrew's fix (thanks!) (with minor formatting tweaks) in dbb0d8ecb3a024bd6817ebd8ad8c5c199a51d933 . Let me know if you still see issues.

[rsmith]

This has broken my ability to run the check-clang target on Linux. There are symlinks in the path from which I run my builds, and this patch is computing incorrect relative paths in that situation.

[rsmith]

In D77184#1961208, @rsmith wrote:

This has broken my ability to run the check-clang target on Linux. There are symlinks in the path from which I run my builds, and this patch is computing incorrect relative paths in that situation.

This patch appears to fix the problem:

diff --git a/llvm/cmake/modules/AddLLVM.cmake b/llvm/cmake/modules/AddLLVM.cmake
index e0ceb364c3e..10010ce8caa 100644
--- a/llvm/cmake/modules/AddLLVM.cmake
+++ b/llvm/cmake/modules/AddLLVM.cmake
@@ -1505,7 +1505,7 @@ drive = os.path.splitdrive(sys.argv[1])[0]\n
 def relpath(p):\n
     if not p: return ''\n
     if os.path.splitdrive(p)[0] != drive: return p\n
-    return os.path.relpath(p, sys.argv[1]).replace(os.sep, '/')\n
+    return os.path.relpath(os.path.realpath(p), os.path.realpath(sys.argv[1])).replace(os.sep, '/')\n
 sys.stdout.write(';'.join(relpath(p) for p in sys.argv[2].split(';')))"
       ${OUTPUT_DIR}
       ${ARG_PATH_VALUES_ESCAPED}

[rsmith]

In D77184#1961214, @rsmith wrote:

In D77184#1961208, @rsmith wrote:

This has broken my ability to run the check-clang target on Linux. There are symlinks in the path from which I run my builds, and this patch is computing incorrect relative paths in that situation.

This patch appears to fix the problem:

Turns out that's only a partial fix (it fixes ninja clang-check but not the general problem). Running lit from inside the source directory is still broken. Eg, this used to work:

clang/test$ $BUILDDIR/bin/llvm-lit SemaCXX/constant-expression.cpp

... and doesn't work any more because the relative paths in lit.site.cfg.py don't seem to resolve to the right places. Please can you fix or revert?

[CarlosAlbertoEnciso]

In D77184#1960473, @thakis wrote:

I pushed Andrew's fix (thanks!) (with minor formatting tweaks) in dbb0d8ecb3a024bd6817ebd8ad8c5c199a51d933 . Let me know if you still see issues.

It fixes the Windows issues when building from different drive. Thanks

[thakis]

In D77184#1961220, @rsmith wrote:

In D77184#1961214, @rsmith wrote:

In D77184#1961208, @rsmith wrote:

This has broken my ability to run the check-clang target on Linux. There are symlinks in the path from which I run my builds, and this patch is computing incorrect relative paths in that situation.

This patch appears to fix the problem:

Turns out that's only a partial fix (it fixes ninja clang-check but not the general problem). Running lit from inside the source directory is still broken. Eg, this used to work:

clang/test$ $BUILDDIR/bin/llvm-lit SemaCXX/constant-expression.cpp

... and doesn't work any more because the relative paths in lit.site.cfg.py don't seem to resolve to the right places. Please can you fix or revert?

Can you describe your symlink setup in enough detail that I can recreate it locally please?

[thakis]

Can you describe your symlink setup in enough detail that I can recreate it locally please?

I've landed a workaround that should make things go for you again in 7db64e202f9. I'd still be curious to hear details about your setup, so that I can see if I can make it work better. (Right now, this now bails out when it sees a symlink and keeps an absolute path as before, which is a bit lame.)

[rsmith]

In D77184#1961495, @thakis wrote:

In D77184#1961220, @rsmith wrote:

In D77184#1961214, @rsmith wrote:

In D77184#1961208, @rsmith wrote:

This has broken my ability to run the check-clang target on Linux. There are symlinks in the path from which I run my builds, and this patch is computing incorrect relative paths in that situation.

This patch appears to fix the problem:

Turns out that's only a partial fix (it fixes ninja clang-check but not the general problem). Running lit from inside the source directory is still broken. Eg, this used to work:

clang/test$ $BUILDDIR/bin/llvm-lit SemaCXX/constant-expression.cpp

... and doesn't work any more because the relative paths in lit.site.cfg.py don't seem to resolve to the right places. Please can you fix or revert?

Can you describe your symlink setup in enough detail that I can recreate it locally please?

Two directories $s and $d within $HOME.

Source git checkout in $s
Build dir is $d/build
Symlink $d/src -> $s
Symlink $s/clang/build -> $d/build
Symlink $c -> $s/clang

Configured from $d/build with cmake ../src
Actual build is run from $c/build
Manual lit commands are run from $c/test

I can provide you with the scripts I use to create my checkouts, symlinks, and to run cmake if you like.

[grimar]

In D77184#1960437, @thakis wrote:

grimar, andrewng: You both have checkout and build on different drives too, yes?

Nope. I think my configuration is just normal, except I do not use "C:".
My checkout is "D:\Work3\LLVM\llvm-project" and the build folder is inside: "D:\Work3\LLVM\llvm-project\build".

(Just in case, to clarify: I have no issues with building the todays LLVM).

[andrewng]

In D77184#1960437, @thakis wrote:

grimar, andrewng: You both have checkout and build on different drives too, yes?

Sorry for not replying earlier. Yes, for most of my setups I have the source checkout on one drive and build output on another drive if available.

Thanks for committing the Windows fix.

Cheers,
Andrew

[vzakhari]

It does not work on Windows (msbuild) for me, because ${pathlist_escaped} contains paths like %(build_mode)s/bin (caused by set_llvm_build_mode). This seems to break something so that python process does not produce any output at all: I checked both OUTPUT_VARIABLE and ERROR_VARIABLE and they were empty on any invocation of execute_process. @thakis, do you have any ideas what might be wrong?

FWIW, the code does not seem to work as expected for multi-configuration environments, since the %(build_mode) substitution is not done yet and the paths are not real filesystem paths.

[vzakhari]

In D77184#2056306, @vzakhari wrote:

It does not work on Windows (msbuild) for me, because ${pathlist_escaped} contains paths like %(build_mode)s/bin (caused by set_llvm_build_mode). This seems to break something so that python process does not produce any output at all: I checked both OUTPUT_VARIABLE and ERROR_VARIABLE and they were empty on any invocation of execute_process. @thakis, do you have any ideas what might be wrong?

FWIW, the code does not seem to work as expected for multi-configuration environments, since the %(build_mode) substitution is not done yet and the paths are not real filesystem paths.

Please ignore this. It turned out the issue was in a python wrapper stuck in the middle.

clang/test/CMakeLists.txt

llvm_canonicalize_cmake_booleans(
   LLVM_ENABLE_PLUGINS
   LLVM_ENABLE_THREADS)
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
+  PATHS
+  "LLVM_SOURCE_DIR"
+  "LLVM_BINARY_DIR"
+  "LLVM_TOOLS_DIR"
+  "LLVM_LIBS_DIR"
+  "SHLIBDIR"
+  "LLVM_LIT_TOOLS_DIR"
+  "CLANG_BINARY_DIR"
+  "CLANG_SOURCE_DIR"
+  "CLANG_TOOLS_DIR"
   )
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/Unit/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.cfg.py
+  PATHS
+  "LLVM_SOURCE_DIR"
+  "LLVM_BINARY_DIR"
+  "LLVM_TOOLS_DIR"
+  "LLVM_LIBS_DIR"
+  "CLANG_BINARY_DIR"
+  "SHLIBDIR"
   )
 
 option(CLANG_TEST_USE_VG "Run Clang tests under Valgrind" OFF)
 if(CLANG_TEST_USE_VG)
   set(CLANG_TEST_EXTRA_ARGS ${CLANG_TEST_EXTRA_ARGS} "--vg")
 endif ()
 
 list(APPEND CLANG_TEST_DEPS

clang/test/Unit/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
-config.llvm_src_root = "@LLVM_SOURCE_DIR@"
-config.llvm_obj_root = "@LLVM_BINARY_DIR@"
-config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
-config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
+config.llvm_src_root = path("@LLVM_SOURCE_DIR@")
+config.llvm_obj_root = path("@LLVM_BINARY_DIR@")
+config.llvm_tools_dir = path("@LLVM_TOOLS_DIR@")
+config.llvm_libs_dir = path("@LLVM_LIBS_DIR@")
 config.llvm_build_mode = "@LLVM_BUILD_MODE@"
-config.clang_obj_root = "@CLANG_BINARY_DIR@"
+config.clang_obj_root = path("@CLANG_BINARY_DIR@")
 config.enable_shared = @ENABLE_SHARED@
-config.shlibdir = "@SHLIBDIR@"
+config.shlibdir = path("@SHLIBDIR@")
 config.target_triple = "@TARGET_TRIPLE@"
 
 # Support substitution of the tools_dir, libs_dirs, and build_mode with user
 # parameters. This is used when we can't determine the tool dir at
 # configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
     config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@CLANG_SOURCE_DIR@/test/Unit/lit.cfg.py")
+lit_config.load_config(
+    config, os.path.join(path("@CLANG_SOURCE_DIR@"), "test/Unit/lit.cfg.py"))

clang/test/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
-config.llvm_src_root = "@LLVM_SOURCE_DIR@"
-config.llvm_obj_root = "@LLVM_BINARY_DIR@"
-config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
-config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
-config.llvm_shlib_dir = "@SHLIBDIR@"
+config.llvm_src_root = path("@LLVM_SOURCE_DIR@")
+config.llvm_obj_root = path("@LLVM_BINARY_DIR@")
+config.llvm_tools_dir = path("@LLVM_TOOLS_DIR@")
+config.llvm_libs_dir = path("@LLVM_LIBS_DIR@")
+config.llvm_shlib_dir = path("@SHLIBDIR@")
 config.llvm_plugin_ext = "@LLVM_PLUGIN_EXT@"
-config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
-config.clang_obj_root = "@CLANG_BINARY_DIR@"
-config.clang_src_dir = "@CLANG_SOURCE_DIR@"
-config.clang_tools_dir = "@CLANG_TOOLS_DIR@"
+config.lit_tools_dir = path("@LLVM_LIT_TOOLS_DIR@")
+config.clang_obj_root = path("@CLANG_BINARY_DIR@")
+config.clang_src_dir = path("@CLANG_SOURCE_DIR@")
+config.clang_tools_dir = path("@CLANG_TOOLS_DIR@")
 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.host_cxx = "@CMAKE_CXX_COMPILER@"
 config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
 config.have_zlib = @HAVE_LIBZ@
 config.clang_arcmt = @CLANG_ENABLE_ARCMT@
 config.clang_default_cxx_stdlib = "@CLANG_DEFAULT_CXX_STDLIB@"
 config.clang_staticanalyzer = @CLANG_ENABLE_STATIC_ANALYZER@

 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 if not "@CLANG_DEFAULT_LINKER@":
     config.available_features.add('platform-linker')
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@CLANG_SOURCE_DIR@/test/lit.cfg.py")
+lit_config.load_config(
+    config, os.path.join(config.clang_src_dir, "test/lit.cfg.py"))

llvm/cmake/modules/AddLLVM.cmake

macro(set_llvm_build_mode)
   endif ()
 endmacro()
 
 # This function provides an automatic way to 'configure'-like generate a file
 # based on a set of common and custom variables, specifically targeting the
 # variables needed for the 'lit.site.cfg' files. This function bundles the
 # common variables that any Lit instance is likely to need, and custom
 # variables can be passed in.
+# The keyword PATHS is followed by a list of cmake variable names that are
+# mentioned as `path("@varname@")` in the lit.cfg.py.in file. Variables in that
+# list are treated as paths that are relative to the directory the generated
+# lit.cfg.py file is in, and the `path()` function converts the relative
+# path back to absolute form. This makes it possible to move a build directory
+# containing lit.cfg.py files from one machine to another.
 function(configure_lit_site_cfg site_in site_out)
-  cmake_parse_arguments(ARG "" "" "MAIN_CONFIG;OUTPUT_MAPPING" ${ARGN})
+  cmake_parse_arguments(ARG "" "" "MAIN_CONFIG;OUTPUT_MAPPING;PATHS" ${ARGN})
 
   if ("${ARG_MAIN_CONFIG}" STREQUAL "")
     get_filename_component(INPUT_DIR ${site_in} DIRECTORY)
     set(ARG_MAIN_CONFIG "${INPUT_DIR}/lit.cfg")
   endif()
   if ("${ARG_OUTPUT_MAPPING}" STREQUAL "")
     set(ARG_OUTPUT_MAPPING "${site_out}")
   endif()
function(configure_lit_site_cfg site_in site_out)
 
   set(HOST_OS ${CMAKE_SYSTEM_NAME})
   set(HOST_ARCH ${CMAKE_SYSTEM_PROCESSOR})
 
   set(HOST_CC "${CMAKE_C_COMPILER} ${CMAKE_C_COMPILER_ARG1}")
   set(HOST_CXX "${CMAKE_CXX_COMPILER} ${CMAKE_CXX_COMPILER_ARG1}")
   set(HOST_LDFLAGS "${CMAKE_EXE_LINKER_FLAGS}")
 
-  set(LIT_SITE_CFG_IN_HEADER  "## Autogenerated from ${site_in}\n## Do not edit!")
+  set(LIT_SITE_CFG_IN_HEADER  "# Autogenerated from ${site_in}\n# Do not edit!")
+
+  string(CONCAT LIT_SITE_CFG_IN_HEADER "${LIT_SITE_CFG_IN_HEADER}\n\n"
+    "# Allow generated lit.site.cfg.py to be relocatable.\n"
+    "def path(p): return os.path.join(os.path.dirname(__file__), p) if p else ''\n"
+    )
 
   # Override config_target_triple (and the env)
   if(LLVM_TARGET_TRIPLE_ENV)
     # This is expanded into the heading.
-    string(CONCAT LIT_SITE_CFG_IN_HEADER "${LIT_SITE_CFG_IN_HEADER}\n\n"
+    string(CONCAT LIT_SITE_CFG_IN_HEADER "${LIT_SITE_CFG_IN_HEADER}"
       "import os\n"
       "target_env = \"${LLVM_TARGET_TRIPLE_ENV}\"\n"
       "config.target_triple = config.environment[target_env] = os.environ.get(target_env, \"${TARGET_TRIPLE}\")\n"
       )
 
     # This is expanded to; config.target_triple = ""+config.target_triple+""
     set(TARGET_TRIPLE "\"+config.target_triple+\"")
   endif()
 
+  # Walk ARG_PATHS and collect the current value of the variables in there.
+  foreach(path ${ARG_PATHS})
+    list(APPEND ARG_PATH_VALUES "${${path}}")
+  endforeach()
+
+  get_filename_component(OUTPUT_DIR ${site_out} DIRECTORY)
+
+  set(arg_path_limit 0)
+  if (ARG_PATH_VALUES)
+    # Passign ARG_PATH_VALUES as-is to execute_process() makes cmake strip
+    # empty list entries. So escape the ;s in the list and do the splitting
+    # outselves. cmake has no relpath function, so use Python for that.
+    string(REPLACE ";" "\\;" ARG_PATH_VALUES_ESCAPED "${ARG_PATH_VALUES}")
+    execute_process(COMMAND "${PYTHON_EXECUTABLE}" "-c"
+      "import os, sys; sys.stdout.write(';'.join(os.path.relpath(p, sys.argv[1]) if p else '' for p in sys.argv[2].split(';')))"
+      ${OUTPUT_DIR}
+      ${ARG_PATH_VALUES_ESCAPED}
+      OUTPUT_VARIABLE ARG_PATH_VALUES_RELATIVE)
+
+    list(LENGTH ARG_PATHS len_vars)
+    list(LENGTH ARG_PATH_VALUES len_values)
+    list(LENGTH ARG_PATH_VALUES_RELATIVE len_rels)
+    if ((NOT ${len_vars} EQUAL ${len_values}) OR
+        (NOT ${len_vars} EQUAL ${len_rels}))
+      message(SEND_ERROR "PATHS lengths got confused")
+    endif()
+
+    math(EXPR arg_path_limit "${len_rels} - 1")
+
+    # Transform variables mentioned in ARG_PATHS to relative paths for
+    # the configure_file() call.
+    foreach(i RANGE ${arg_path_limit})
+      list(GET ARG_PATHS ${i} val1)
+      list(GET ARG_PATH_VALUES_RELATIVE ${i} val2)
+      set(${val1} ${val2})
+    endforeach()
+  endif()
+
   configure_file(${site_in} ${site_out} @ONLY)
+
+  # Restore ARG_PATHS variables to their original values.
+  foreach(i RANGE ${arg_path_limit})
+    list(GET ARG_PATHS ${i} val1)
+    list(GET ARG_PATH_VALUES ${i} val2)
+    set(${val1} ${val2})
+  endforeach()
+
   if (EXISTS "${ARG_MAIN_CONFIG}")
     set(PYTHON_STATEMENT "map_config('${ARG_MAIN_CONFIG}', '${site_out}')")
     get_property(LLVM_LIT_CONFIG_MAP GLOBAL PROPERTY LLVM_LIT_CONFIG_MAP)
     set(LLVM_LIT_CONFIG_MAP "${LLVM_LIT_CONFIG_MAP}\n${PYTHON_STATEMENT}")
     set_property(GLOBAL PROPERTY LLVM_LIT_CONFIG_MAP ${LLVM_LIT_CONFIG_MAP})
   endif()
 endfunction()
 

llvm/test/CMakeLists.txt

llvm_canonicalize_cmake_booleans(
   LLVM_BYE_LINK_INTO_TOOLS
   )
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
+  PATHS
+  "LLVM_SOURCE_DIR"
+  "LLVM_BINARY_DIR"
+  "LLVM_TOOLS_DIR"
+  "LLVM_LIBRARY_DIR"
+  "SHLIBDIR"
   )
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/Unit/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/Unit/lit.cfg.py
+  PATHS
+  "LLVM_SOURCE_DIR"
+  "LLVM_BINARY_DIR"
+  "LLVM_TOOLS_DIR"
+  "SHLIBDIR"
   )
 
 # Set the depends list as a variable so that it can grow conditionally.
 # NOTE: Sync the substitutions in test/lit.cfg when adding to this list.
 set(LLVM_TEST_DEPENDS
           BugpointPasses
           FileCheck
           LLVMHello

llvm/test/Unit/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
-config.llvm_src_root = "@LLVM_SOURCE_DIR@"
-config.llvm_obj_root = "@LLVM_BINARY_DIR@"
-config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
+config.llvm_src_root = path("@LLVM_SOURCE_DIR@")
+config.llvm_obj_root = path("@LLVM_BINARY_DIR@")
+config.llvm_tools_dir = path("@LLVM_TOOLS_DIR@")
 config.llvm_build_mode = "@LLVM_BUILD_MODE@"
 config.enable_shared = @ENABLE_SHARED@
-config.shlibdir = "@SHLIBDIR@"
+config.shlibdir = path("@SHLIBDIR@")
 
 # Support substitution of the tools_dir and build_mode with user parameters.
 # This is used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_build_mode = config.llvm_build_mode % lit_config.params
 except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LLVM_SOURCE_DIR@/test/Unit/lit.cfg.py")
+lit_config.load_config(
+    config, os.path.join(config.llvm_src_root, "test/Unit/lit.cfg.py"))

llvm/test/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 import sys
 
 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.target_triple = "@TARGET_TRIPLE@"
-config.llvm_src_root = "@LLVM_SOURCE_DIR@"
-config.llvm_obj_root = "@LLVM_BINARY_DIR@"
-config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
-config.llvm_lib_dir = "@LLVM_LIBRARY_DIR@"
-config.llvm_shlib_dir = "@SHLIBDIR@"
+config.llvm_src_root = path("@LLVM_SOURCE_DIR@")
+config.llvm_obj_root = path("@LLVM_BINARY_DIR@")
+config.llvm_tools_dir = path("@LLVM_TOOLS_DIR@")
+config.llvm_lib_dir = path("@LLVM_LIBRARY_DIR@")
+config.llvm_shlib_dir = path("@SHLIBDIR@")
 config.llvm_shlib_ext = "@SHLIBEXT@"
 config.llvm_exe_ext = "@EXEEXT@"
-config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
+config.lit_tools_dir = path("@LLVM_LIT_TOOLS_DIR@")
 config.python_executable = "@PYTHON_EXECUTABLE@"
 config.gold_executable = "@GOLD_EXECUTABLE@"
 config.ld64_executable = "@LD64_EXECUTABLE@"
 config.ocamlfind_executable = "@OCAMLFIND@"
 config.have_ocamlopt = @HAVE_OCAMLOPT@
 config.have_ocaml_ounit = @HAVE_OCAML_OUNIT@
 config.ocaml_flags = "@OCAMLFLAGS@"
 config.include_go_tests = @LLVM_INCLUDE_GO_TESTS@
except KeyError:
     e = sys.exc_info()[1]
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
-lit_config.load_config(config, "@LLVM_SOURCE_DIR@/test/lit.cfg.py")
+lit_config.load_config(
+    config, os.path.join(config.llvm_src_root, "test/lit.cfg.py"))