Add experimental support for unofficial monorepo-like directory layout

[mehdi_amini]

Summary

This allows to have clang and llvm and the other subprojects
side-by-side instead of nested. This can be used with the monorepo or
multiple repos.

It will help having a single set of sources checked out but allows to
have a build directory with llvm and another one with llvm+clang.
Basically it abstracts LLVM_EXTERNAL_xxxx_SOURCE_DIR making it more
convenient by adopting a convention.

[mehdi_amini]

Document the new cmake flag

[mehdi_amini]

Update the comment in CMake

[mehdi_amini]

Add a comment on the magic for clang-tools-extra

[beanz]

We should abstract this into a module that can be re-used and not have it implemented directly in the LLVM top-level CMakeLists. As an example other projects could use this code for out-of-tree builds to identify source directories for projects.

[mehdi_amini]

In D26365#588857, @beanz wrote:

We should abstract this into a module that can be re-used and not have it implemented directly in the LLVM top-level CMakeLists. As an example other projects could use this code for out-of-tree builds to identify source directories for projects.

I can try!

Can you elaborate a little about the "other projects" case? You mean out-of-tree projects like Swift or in-tree projects like compiler-rt (or both)?

[beanz]

As an example libcxx has goop to find libcxxabi and libunwind's source directories. This code could be in a module so that those projects could use a common module instead of hand-rolling the code.

[mehdi_amini]

In D26365#588952, @beanz wrote:

As an example libcxx has goop to find libcxxabi and libunwind's source directories. This code could be in a module so that those projects could use a common module instead of hand-rolling the code.

OK, makes sense, I will work on this.

llvm/CMakeLists.txt

foreach(lang ${languages})
              "${dyld_envar} ${cmd}")
       endforeach()
       set(CMAKE_${lang}_CREATE_STATIC_LIBRARY
         ${CMAKE_${lang}_CREATE_STATIC_LIBRARY_NEW})
     endforeach()
   endif()
 endif()
 
+# Side-by-side subprojects layout: automatically set the
+# LLVM_EXTERNAL_${project}_SOURCE_DIR using LLVM_ALL_PROJECTS
+# This allows an easy way of setting up a build directory for llvm and another
+# one for llvm+clang+... using the same sources.
+set(LLVM_ALL_PROJECTS "clang;libcxx;libcxxabi;lldb;compiler-rt;lld;polly")
+set(LLVM_ENABLE_PROJECTS "" CACHE STRING
+  "Semicolon-separated list of projects to build (${LLVM_ALL_PROJECTS}), or \"all\".")
+if( LLVM_ENABLE_PROJECTS STREQUAL "all" )
+  set( LLVM_ENABLE_PROJECTS ${LLVM_ALL_PROJECTS})
+endif()
+foreach(proj ${LLVM_ENABLE_PROJECTS})
+  set(PROJ_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../${proj}")
+  if(NOT EXISTS "${PROJ_DIR}" OR NOT IS_DIRECTORY "${PROJ_DIR}")
+    message(FATAL_ERROR "LLVM_ENABLE_PROJECTS requests ${proj} but directory not found: ${PROJ_DIR}")
+  endif()
+  string(TOUPPER "${proj}" upper_proj)
+  set(LLVM_EXTERNAL_${upper_proj}_SOURCE_DIR   "${CMAKE_CURRENT_SOURCE_DIR}/../${proj}")
+  if (proj STREQUAL "clang")
+    set(LLVM_EXTERNAL_CLANG_TOOLS_EXTRA_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../clang-tools-extra")
+  endif()
+endforeach()
+
 # The following only works with the Ninja generator in CMake >= 3.0.
 set(LLVM_PARALLEL_COMPILE_JOBS "" CACHE STRING
   "Define the maximum number of concurrent compilation jobs.")
 if(LLVM_PARALLEL_COMPILE_JOBS)
   if(NOT CMAKE_MAKE_PROGRAM MATCHES "ninja")
     message(WARNING "Job pooling is only available with Ninja generators.")
   else()
     set_property(GLOBAL APPEND PROPERTY JOB_POOLS compile_job_pool=${LLVM_PARALLEL_COMPILE_JOBS})

llvm/docs/CMake.rst

 **LLVM_EXTERNAL_{CLANG,LLD,POLLY}_SOURCE_DIR**:PATH
   These variables specify the path to the source directory for the external
   LLVM projects Clang, lld, and Polly, respectively, relative to the top-level
   source directory.  If the in-tree subdirectory for an external project
   exists (e.g., llvm/tools/clang for Clang), then the corresponding variable
   will not be used.  If the variable for an external project does not point
   to a valid path, then that project will not be built.
 
+**LLVM_ENABLE_PROJECTS**:STRING
+  Semicolon-separated list of projects to build, or *all* for building all
+  (clang, libcxx, libcxxabi, lldb, compiler-rt, lld, polly) projects.
+  This flag assumes that projects are checked out side-by-side and not nested,
+  i.e. clang needs to be in parallel of llvm instead of nested in `llvm/tools`.
+  This feature allows to have one build for only LLVM and another for clang+llvm
+  using the same source checkout.
+
 **LLVM_EXTERNAL_PROJECTS**:STRING
   Semicolon-separated list of additional external projects to build as part of
   llvm. For each project LLVM_EXTERNAL_<NAME>_SOURCE_DIR have to be specified
   with the path for the source code of the project. Example:
   ``-DLLVM_EXTERNAL_PROJECTS="Foo;Bar"
   -DLLVM_EXTERNAL_FOO_SOURCE_DIR=/src/foo
   -DLLVM_EXTERNAL_BAR_SOURCE_DIR=/src/bar``.