[cmake] With utils disabled, don't build tblgen in cross mode

[loladiro]

Summary

In cross mode, we build a separate NATIVE tblgen that runs on the
host and is used during the build. Separately, we have a flag that
disables building all executables in utils/. Of course generally,
this doesn't turn off tblgen, since we need that during the build.
In cross mode, however, that tblegen is useless since we never
actually use it. Furthermore, it can be actively problematic if the
cross toolchain doesn't like building executables for whatever reason.
And even if building executables works fine, we can at least save
compile time by omitting it from the target build. There's two changes
needed to make this happen:

• Stop creating a dependency from the native tool to the target tool. No such dependency is required for a correct build, so I'm not entirely sure why it was there in the first place.
• If utils were disabled on the CMake command line and we're in cross mode, respect that by excluding it from the install target (using EXCLUDE_FROM_ALL).

[smeenai]

Thanks for working on this!

llvm/cmake/modules/TableGen.cmake

125

Thank you for fixing this.

142

I'm not a fan of this, especially since add_tablegen is a macro, so setting this variable will leak out of it.

Unfortunately, it doesn't seem like there's any other way to pass this information to add_llvm_executable. Ideally, you'd add a new argument to that method and use it here. Failing that, you should save the old value of EXCLUDE_FROM_ALL and restore it after your add_llvm_executable call, similar to what's being done for LLVM_LINK_COMPONENTS.

[loladiro]

Use set_target_properties to set EXCLUDE_FROM_ALL on the target rather than using
the global setting.

[smeenai]

LGTM

[beanz]

This patch is actually really problematic. We have the native tablegen depend on the target tablegen so that the native one rebuilds whenever the sources for the target one rebuild. Without that dependency the native tablegen may not rebuild at all, or (if it does) it will likely rebuild at the wrong time.

This patch makes iterative builds on a cross-target incorrect.

[beanz]

This patch is effectively un-done as of rL367895. I'm happy to discuss alternative approaches to the problem you're trying to solve, but we need correct builds, which means we need the dependency between the native and target tool.

[loladiro]

Is there a way to just add the whatever the underlying dependencies are that are missing for the native tool? My concern is that I don't want to build target executables (because the toolchain for that is broken).

[beanz]

You'd need to make the NATIVE target depend on all the source files and headers that the the target depends on, which is theoretically possible, although I'm not sure of a clean way to do it in CMake.

If your target toolchain doesn't support executables you can always feed in a pre-built tablegen, which will skip all of this, and skip building one, but then it is on you to ensure that your tablegen is getting rebuilt properly.

[loladiro]

Hmm, we do have a list of all the .cpp files in the tablegen tool in ${ARGN}, and I'd be fine depending on the target libraries. Do you know of a way to emulate whatever handling add_executable is doing for .cpp files to get at the included .h files?

[beanz]

I don't know of any way to emulate that because it isn't something CMake exports or even knows about. It is something the underlying build system detects from the compiler's depfiles. One thought that might work would be to generate an object library target containing the tool source files, and instead depend on the object library and all the target static archives. I think that would get the ordering right and would not require the link step to run.

[loladiro]

Ok, I think that'd work for my use case. I'll try it out and post a revision to that extent.

llvm/cmake/modules/TableGen.cmake

macro(add_tablegen target project)
   set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
 
   # CMake-3.9 doesn't let compilation units depend on their dependent libraries.
   if(NOT (CMAKE_GENERATOR STREQUAL "Ninja" AND NOT CMAKE_VERSION VERSION_LESS 3.9) AND NOT XCODE)
     # FIXME: It leaks to user, callee of add_tablegen.
     set(LLVM_ENABLE_OBJLIB ON)
   endif()
 
-  add_llvm_executable(${target} DISABLE_LLVM_LINK_LLVM_DYLIB ${ARGN})
-  set(LLVM_LINK_COMPONENTS ${${target}_OLD_LLVM_LINK_COMPONENTS})
-
   set(${project}_TABLEGEN "${target}" CACHE
       STRING "Native TableGen executable. Saves building one when cross-compiling.")
 
   # Effective tblgen executable to be used:
   set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN} PARENT_SCOPE)
   set(${project}_TABLEGEN_TARGET ${${project}_TABLEGEN} PARENT_SCOPE)
 
   if(LLVM_USE_HOST_TOOLS)
     if( ${${project}_TABLEGEN} STREQUAL "${target}" )
-      build_native_tool(${target} ${project}_TABLEGEN_EXE DEPENDS ${target})
+      build_native_tool(${target} ${project}_TABLEGEN_EXE)
       set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN_EXE} PARENT_SCOPE)
 
       add_custom_target(${project}-tablegen-host DEPENDS ${${project}_TABLEGEN_EXE})
       set(${project}_TABLEGEN_TARGET ${project}-tablegen-host PARENT_SCOPE)
 
       # Create an artificial dependency between tablegen projects, because they
       # compile the same dependencies, thus using the same build folders.
       # FIXME: A proper fix requires sequentially chaining tablegens.
       if (NOT ${project} STREQUAL LLVM AND TARGET ${project}-tablegen-host AND
           TARGET LLVM-tablegen-host)
         add_dependencies(${project}-tablegen-host LLVM-tablegen-host)
       endif()
+
+      # If we're using the host tablegen, and utils were not requested, we have no
+      # need to build this tablegen.
+      if ( NOT LLVM_BUILD_UTILS )
+        set(EXCLUDE_FROM_ALL ON)
+      endif()
     endif()
   endif()
 
+  add_llvm_executable(${target} DISABLE_LLVM_LINK_LLVM_DYLIB ${ARGN})
+  set(LLVM_LINK_COMPONENTS ${${target}_OLD_LLVM_LINK_COMPONENTS})
+
   if (${project} STREQUAL LLVM AND NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
     set(export_to_llvmexports)
     if(${target} IN_LIST LLVM_DISTRIBUTION_COMPONENTS OR
         NOT LLVM_DISTRIBUTION_COMPONENTS)
       set(export_to_llvmexports EXPORT LLVMExports)
     endif()
 
     install(TARGETS ${target}
             ${export_to_llvmexports}
             RUNTIME DESTINATION ${LLVM_TOOLS_INSTALL_DIR})
   endif()
   set_property(GLOBAL APPEND PROPERTY LLVM_EXPORTS ${target})
 endmacro()