Harmonize cmake_policy() across standalone builds of all projects

[mgorny]

Summary

Move cmake_policy() settings from llvm/CMakeLists.txt into a shared
cmake/modules/CMakePolicy.cmake. Include it from all relevant
projects that support standalone builds, in order to ensure that
the policies are consistently set whether they are built in-tree
or stand-alone.

[mgorny]

Replace with one patch for all the projects.

[nickdesaulniers]

clang/CMakeLists.txt

6

Seems error prone. Does cmake have an include system for including fragments from elsewhere? If so, that seems like a more concise and less error prone approach.

https://cmake.org/cmake/help/latest/command/include.html

[mgorny]

clang/CMakeLists.txt

6

I suppose that makes sense. I think that at this point all these components require the top-level cmake directory from monorepo anyway.

[mgorny]

Use a shared policy file via include() instead.

[mgorny]

Thanks. Since this is not urgent, I'll let it wait for others to chime in a while more.

[mstorsjo]

This broke builds where clang/lld/lldb are symlinked into llvm/tools instead of specified via LLVM_ENABLE_PROJECTS - since ${CMAKE_CURRENT_SOURCE_DIR}/../cmake doesn't find anything in that context.

(The reason for wanting to stick to such a legacy setup, is that ccache caches can't be shared between workdirs for source files that are found outside of the base llvm-project/llvm directory - and I'm heavily reliant on ccache for keeping build times manageable in my dev workflow.)

[mstorsjo]

In D136572#3888177, @mstorsjo wrote:

This broke builds where clang/lld/lldb are symlinked into llvm/tools instead of specified via LLVM_ENABLE_PROJECTS - since ${CMAKE_CURRENT_SOURCE_DIR}/../cmake doesn't find anything in that context.

This aspect, for finding the shared cmake directory, is handled elsewhere by the main LLVM build setting the LLVM_COMMON_CMAKE_UTILS variable, and if not found, it's assumed to be in ${CMAKE_CURRENT_SOURCE_DIR}/... See e.g. https://github.com/llvm/llvm-project/blob/llvmorg-15.0.0/lld/CMakeLists.txt#L198-L206.

Since this is invoked very early in the per-project cmakefiles, I guess we can't assume to have all those variables set up yet, but e.g. something like this fixes the issue for me:

diff --git a/lld/CMakeLists.txt b/lld/CMakeLists.txt
index 964bc3fd8b17..530f52b05bac 100644
--- a/lld/CMakeLists.txt
+++ b/lld/CMakeLists.txt
@@ -1,6 +1,11 @@
 cmake_minimum_required(VERSION 3.13.4)
-include(${CMAKE_CURRENT_SOURCE_DIR}/../cmake/Modules/CMakePolicy.cmake
-  NO_POLICY_SCOPE)
+if(DEFINED LLVM_COMMON_CMAKE_UTILS)
+  include(${LLVM_COMMON_CMAKE_UTILS}/Modules/CMakePolicy.cmake
+    NO_POLICY_SCOPE)
+else()
+  include(${CMAKE_CURRENT_SOURCE_DIR}/../cmake/Modules/CMakePolicy.cmake
+    NO_POLICY_SCOPE)
+endif()
 
 # If we are not building as a part of LLVM, build LLD as an
 # standalone project, using LLVM as an external library:

(And the same applied for all the other subprojects.) It's admittedly not very pretty though...

[mgorny]

I've reverted it right now, and I'll update the patch later today.

[mgorny]

Move LLVM_COMMON_CMAKE_UTILS earlier and use it for CMakePolicy path.

[mgorny]

@mstorsjo, could you check this version?

[MaskRay]

It'll be good if we have a wiki describing how downstream users invoke cmake, so that large cmake refactoring can be verified beforehand.
(like usage verification https://github.com/opencollab/llvm-toolchain-integration-test-suite, but for cmake invocations)

[mstorsjo]

In D136572#3888630, @mgorny wrote:

@mstorsjo, could you check this version?

LGTM, this seems to work for my usecase (with lld,lldb,clang enabled this way). Thanks for the quick revert and fix!

[mstorsjo]

Also for the record, I'd love to get rid of this symlink based setup, if I could make cmake produce project files where caching works in the same way. The main requirement for that would be to have the whole cmake build started from the toplevel llvm-project directory (so that all source from all subprojects are subdirectories to this), instead of the llvm-project/llvm directory.

[mgorny]

In D136572#3889317, @mstorsjo wrote:

Also for the record, I'd love to get rid of this symlink based setup, if I could make cmake produce project files where caching works in the same way. The main requirement for that would be to have the whole cmake build started from the toplevel llvm-project directory (so that all source from all subprojects are subdirectories to this), instead of the llvm-project/llvm directory.

To be honest, I also find it weird to start monorepo builds from within llvm subdirectory. Perhaps adding a top-level CMakeLists.txt that would work and wouldn't break people's workflows wouldn't be that hard, though.

[mstorsjo]

In D136572#3890835, @mgorny wrote:

In D136572#3889317, @mstorsjo wrote:

Also for the record, I'd love to get rid of this symlink based setup, if I could make cmake produce project files where caching works in the same way. The main requirement for that would be to have the whole cmake build started from the toplevel llvm-project directory (so that all source from all subprojects are subdirectories to this), instead of the llvm-project/llvm directory.

To be honest, I also find it weird to start monorepo builds from within llvm subdirectory. Perhaps adding a top-level CMakeLists.txt that would work and wouldn't break people's workflows wouldn't be that hard, though.

This has been discussed briefly before - the main issue is that LLVM does some cmake trickery to recreate the installed directory structure (with executables in $(pwd)/bin etc) directly in the build folder, and if llvm-project/llvm is included from a toplevel llvm-project/CMakeLists.txt, this all ends up in $(pwd)/llvm/bin as things stand right now. See e.g. https://discourse.llvm.org/t/rfc-llvm-build-system-future-direction/53430/18. Hopefully it wouldn't be too hard to resolve - I had a brief look at it (just creating a stub CMakeLists.txt at the root and including the llvm subdirectory) a couple years ago, but didn't spend further time on looking into how things are set up and what it would take to fix it.

clang/CMakeLists.txt

 cmake_minimum_required(VERSION 3.13.4)
 
 # If we are not building as a part of LLVM, build Clang as an
 # standalone project, using LLVM as an external library:
 if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  # Please keep policies in sync with llvm/CMakeLists.txt.
+  if(POLICY CMP0114)
+    cmake_policy(SET CMP0114 OLD)
+  endif()
+  if(POLICY CMP0116)
+    cmake_policy(SET CMP0116 OLD)
+  endif()
   project(Clang)
   set(CLANG_BUILT_STANDALONE TRUE)
 endif()
 
 # Must go below project(..)
 include(GNUInstallDirs)
 
 if(CLANG_BUILT_STANDALONE)

flang/CMakeLists.txt

 cmake_minimum_required(VERSION 3.13.4)
 
+# Check for a standalone build and configure as appropriate from
+# there.
+if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  # Please keep policies in sync with llvm/CMakeLists.txt.
+  if(POLICY CMP0114)
+    cmake_policy(SET CMP0114 OLD)
+  endif()
+  if(POLICY CMP0116)
+    cmake_policy(SET CMP0116 OLD)
+  endif()
+  message("Building Flang as a standalone project.")
+  project(Flang)
+  set(FLANG_STANDALONE_BUILD ON)
+else()
+  set(FLANG_STANDALONE_BUILD OFF)
+endif()
+
 set(CMAKE_BUILD_WITH_INSTALL_NAME_DIR ON)
 
 # Flang requires C++17.
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
 set(FLANG_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 
 if (CMAKE_SOURCE_DIR STREQUAL CMAKE_BINARY_DIR AND NOT MSVC_IDE)
   message(FATAL_ERROR "In-source builds are not allowed. \
           Please create a directory and run cmake from there,\
           passing the path to this source directory as the last argument.\
           This process created the file `CMakeCache.txt' and the directory\
           `CMakeFiles'. Please delete them.")
 endif()
 
 option(FLANG_ENABLE_WERROR "Fail and stop building flang if a warning is triggered." OFF)
 
-# Check for a standalone build and configure as appropriate from
-# there.
-if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
-  message("Building Flang as a standalone project.")
-  project(Flang)
-  set(FLANG_STANDALONE_BUILD ON)
-else()
-  set(FLANG_STANDALONE_BUILD OFF)
-endif()
-
 # Must go below project(..)
 include(GNUInstallDirs)
 
 if (FLANG_STANDALONE_BUILD)
   set(FLANG_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
   if (NOT MSVC_IDE)
     set(LLVM_ENABLE_ASSERTIONS ${ENABLE_ASSERTIONS}
       CACHE BOOL "Enable assertions")

lld/CMakeLists.txt

 cmake_minimum_required(VERSION 3.13.4)
 
 # If we are not building as a part of LLVM, build LLD as an
 # standalone project, using LLVM as an external library:
 if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  # Please keep policies in sync with llvm/CMakeLists.txt.
+  if(POLICY CMP0114)
+    cmake_policy(SET CMP0114 OLD)
+  endif()
+  if(POLICY CMP0116)
+    cmake_policy(SET CMP0116 OLD)
+  endif()
   project(lld)
   set(LLD_BUILT_STANDALONE TRUE)
 endif()
 
 # Must go below project(..)
 include(GNUInstallDirs)
 
 if(LLD_BUILT_STANDALONE)

lldb/CMakeLists.txt

 cmake_minimum_required(VERSION 3.13.4)
 
-# Add path for custom modules.
-set(CMAKE_MODULE_PATH
-  ${CMAKE_MODULE_PATH}
-  "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
-  "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
-  )
-
 # If we are not building as part of LLVM, build LLDB as a standalone project,
 # using LLVM as an external library.
 if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  # Please keep policies in sync with llvm/CMakeLists.txt.
+  if(POLICY CMP0114)
+    cmake_policy(SET CMP0114 OLD)
+  endif()
+  if(POLICY CMP0116)
+    cmake_policy(SET CMP0116 OLD)
+  endif()
   project(lldb)
   set(LLDB_BUILT_STANDALONE TRUE)
 endif()
 
+# Add path for custom modules.
+set(CMAKE_MODULE_PATH
+  ${CMAKE_MODULE_PATH}
+  "${CMAKE_CURRENT_SOURCE_DIR}/cmake"
+  "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules"
+  )
+
 # Must go below project(..)
 include(GNUInstallDirs)
 
 if(LLDB_BUILT_STANDALONE)
   include(LLDBStandalone)
 
   set(CMAKE_CXX_STANDARD 17 CACHE STRING "C++ standard to conform to")
   set(CMAKE_CXX_STANDARD_REQUIRED YES)

lldb/cmake/modules/LLDBStandalone.cmake

-# CMP0116: Ninja generators transform `DEPFILE`s from `add_custom_command()`
-# New in CMake 3.20. https://cmake.org/cmake/help/latest/policy/CMP0116.html
-if(POLICY CMP0116)
-  cmake_policy(SET CMP0116 OLD)
-endif()
-
 if(NOT DEFINED LLVM_COMMON_CMAKE_UTILS)
   set(LLVM_COMMON_CMAKE_UTILS ${CMAKE_CURRENT_SOURCE_DIR}/../cmake)
 endif()
 
 list(APPEND CMAKE_MODULE_PATH "${LLVM_COMMON_CMAKE_UTILS}/Modules")
 
 option(LLVM_INSTALL_TOOLCHAIN_ONLY "Only include toolchain files in the 'install' target." OFF)
 

mlir/CMakeLists.txt

 # MLIR project.
 
+cmake_minimum_required(VERSION 3.13.4)
+
 # Check if MLIR is built as a standalone project.
 if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
+  # Please keep policies in sync with llvm/CMakeLists.txt.
+  if(POLICY CMP0114)
+    cmake_policy(SET CMP0114 OLD)
+  endif()
+  if(POLICY CMP0116)
+    cmake_policy(SET CMP0116 OLD)
+  endif()
   project(mlir)
   set(MLIR_STANDALONE_BUILD TRUE)
 endif()
 
 # Must go below project(..)
 include(GNUInstallDirs)
 
 if(MLIR_STANDALONE_BUILD)
-  cmake_minimum_required(VERSION 3.13.4)
-
   find_package(LLVM CONFIG REQUIRED)
   set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${LLVM_CMAKE_DIR})
   include(HandleLLVMOptions)
   include(AddLLVM)
   include(TableGen)
 
   include_directories(${LLVM_INCLUDE_DIRS})