[runtimes] Add a mechanism to use cache files for a runtimes build

[smeenai]

Summary

Projects like libc++ include their own cache files, and it's convenient
to just be able to reuse those cache files as part of a runtimes build
instead of having to duplicate the settings inside a special runtimes
cache file (which will inevitably get out of sync).

I'm not completely happy about overloading the existing argument passing
behavior based on the argument name, but I also couldn't think of a good
alternative. Suggestions are welcome.

[phosek]

The fragments like https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/Fuchsia-stage2.cmake#L132-L184 and in particular https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/Fuchsia-stage2.cmake#L186-L289 have gotten quite unwieldy so I was already thinking about breaking those up into separate cache files and this feature is going to be needed so thank you!

The open question is if we should try and encourage developers towards using separate caches to the point where we can eventually completely deprecate and remove the support for passthrough or we want to be maintaining both forever?

Regarding the argument handling, I thought about alternatives but haven't come up with anything better. I have a question regarding the name though, we generally avoid the CMAKE_ prefixed variables to potential (future) collisions (for example for CMake starts using CMAKE_CACHES in the future). Perhaps we should call this something like LLVM_CACHE_FILES or just CACHE_FILES for simplicty?

[smeenai]

In D158791#4623904, @phosek wrote:

The fragments like https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/Fuchsia-stage2.cmake#L132-L184 and in particular https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/Fuchsia-stage2.cmake#L186-L289 have gotten quite unwieldy so I was already thinking about breaking those up into separate cache files and this feature is going to be needed so thank you!

The open question is if we should try and encourage developers towards using separate caches to the point where we can eventually completely deprecate and remove the support for passthrough or we want to be maintaining both forever?

Hmm. I think separate cache files are going to be generally cleaner, but I can imagine cases where you only need to specify a couple of arguments and it's easier to inline them instead of creating a separate file. I think we can start with adding the support and converting our own runtimes build setups over and use that to get a sense of if there's still good use cases for separate passthroughs.

Regarding the argument handling, I thought about alternatives but haven't come up with anything better. I have a question regarding the name though, we generally avoid the CMAKE_ prefixed variables to potential (future) collisions (for example for CMake starts using CMAKE_CACHES in the future). Perhaps we should call this something like LLVM_CACHE_FILES or just CACHE_FILES for simplicty?

Good point, I'll change it.

[smeenai]

Change to CACHE_FILES

[smeenai]

Ping.

[phosek]

LGTM

llvm/runtimes/CMakeLists.txt

function(builtin_register_target compiler_rt_path name)
   cmake_parse_arguments(ARG "" "" "DEPENDS;CMAKE_ARGS;EXTRA_ARGS" ${ARGN})
 
   set(${name}_extra_args ${ARG_CMAKE_ARGS})
   get_cmake_property(variable_names VARIABLES)
   foreach(variable_name ${variable_names})
     string(FIND "${variable_name}" "BUILTINS_${name}" out)
     if("${out}" EQUAL 0)
       string(REPLACE "BUILTINS_${name}_" "" new_name ${variable_name})
+      if(new_name STREQUAL CACHE_FILES)
+        foreach(cache IN LISTS ${variable_name})
+          list(APPEND ${name}_extra_args -C ${cache})
+        endforeach()
+      else()
         string(REPLACE ";" "|" new_value "${${variable_name}}")
         list(APPEND ${name}_extra_args "-D${new_name}=${new_value}")
       endif()
+    endif()
   endforeach()
 
   llvm_ExternalProject_Add(builtins-${name}
                            ${compiler_rt_path}/lib/builtins
                            DEPENDS ${ARG_DEPENDS}
                            CMAKE_ARGS -DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_DIR}
                                       -DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_TOOLS_BINARY_DIR}
                                       -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON
function(runtime_register_target name)
   list(APPEND ${name}_extra_args -DLLVM_USE_LINKER=${LLVM_USE_LINKER})
 
   get_cmake_property(variable_names VARIABLES)
   foreach(extra_name IN ITEMS ${ARG_BASE_NAME} ${name})
     foreach(variable_name ${variable_names})
       string(FIND "${variable_name}" "RUNTIMES_${extra_name}_" out)
       if("${out}" EQUAL 0)
         string(REPLACE "RUNTIMES_${extra_name}_" "" new_name ${variable_name})
+        if(new_name STREQUAL CACHE_FILES)
+          foreach(cache IN LISTS ${variable_name})
+            list(APPEND ${name}_extra_args -C ${cache})
+          endforeach()
+        else()
           string(REPLACE ";" "|" new_value "${${variable_name}}")
           list(APPEND ${name}_extra_args "-D${new_name}=${new_value}")
         endif()
+      endif()
     endforeach()
   endforeach()
 
   set_enable_per_target_runtime_dir()
 
   llvm_ExternalProject_Add(runtimes-${name}
                            ${CMAKE_CURRENT_SOURCE_DIR}/../../runtimes
                            DEPENDS ${ARG_DEPENDS}