Add new LLVM_OPTIMIZED_TABLEGEN build setting which configures, builds and uses a release tablegen build when LLVM is configured with assertions enabled.

[beanz]

Summary

This change leverages the cross-compiling functionality in the build system to build a release tablegen executable for use during the build.

[beanz]

I just did a test building libLLVMARMCodeGen.a. The process I followed was:

cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg> rm -rf ./*
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg> cmake -G "Sublime Text 2 - Ninja" -DCMAKE_BUILD_TYPE=Debug -DLLVM_OPTIMIZED_TABLEGEN=On -DLLVM_TARGETS_TO_BUILD="ARM" ../llvm
...
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg> ninja native/bin/llvm-tblgen
...
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg> ninja llvm-tblgen
...
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg> time ninja lib/libLLVMARMCodeGen.a
[56/56] Linking CXX static library lib/libLLVMARMCodeGen.a

31.50 real       101.88 user         7.93 sys

cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg-2> rm -rf ./*
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg-2> cmake -G "Sublime Text 2 - Ninja" -DCMAKE_BUILD_TYPE=Debug -DLLVM_OPTIMIZED_TABLEGEN=Off -DLLVM_TARGETS_TO_BUILD="ARM" ../llvm
...
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg-2> ninja llvm-tblgen
...
cbieneman@MacBook-Pro ~/d/o/llvm-build-dbg-2> time ninja lib/libLLVMARMCodeGen.a
[56/56] Linking CXX static library lib/libLLVMARMCodeGen.a

40.85 real       134.16 user         8.14 sys

Using a release tablegen shaved 9 seconds of the libLLVMARMCodeGen.a build time.

In my test you will notice the use of different build directories, and that I pre-built tablegen so that its build time wasn’t factored into the build. My changes do cause tablegen to be built twice, which can be a bit of a slowdown, but the savings on incremental builds, and when building all backends is substantial.

-Chris

[mehdi_amini]

Why not defaulting to ON?
I expect that debugging tablegen itself is not very common and we may want the default to optimize for the common case?

[beanz]

When adding new build options my general feeling is to default to "no change" and if people prefer the behavior we can change the default after people have gotten used to it.

Also since this hasn't been tested with all the various generators that people are using I think it is best to keep it off initially. If people want to test Visual Studio & Xcode I'd feel a lot more comfortable about making this the default.

[rnk]

This is the first time I looked at our cross-compling support, and I'd really rather just add -O2 -DNDEBUG to lib/Support, lib/TableGen, and utils/TableGen and call it good. Shelling back out to 'cmake --build' in another config is... pretty hacky.

[beanz]

Owen proposed a similar approach internally, but I don’t think that is the right approach.

For one, setting -DNDEBUG on parts (but not all) of the code isn’t safe. We have headers in libSupport that change ifdef on NDEBUG, and that could cause subtle bugs.

Also I don’t like the idea of bleeding the non-debug code into what you might be debugging. It would kinda suck to be debugging something and suddenly lose your debug granularity because you stepped into optimized code.

One of the big appeals to me about using the cross-compiling support for this is it means that your output from a debug build remains the same (barring debug tablegen behaving wildly different from a release one).

-Chris

[beanz]

Updating to reflect Justin's feedback. The non-cross compiling specific code is now gated on LLVM_USE_HOST_TOOLS instead of CMAKE_CROSSCOMPILING. This has the pleasent side effect of having LLVM_OPTIMIZED_TABLEGEN not generate optimized llvm-config along with the optimized tablegen.

[rnk]

lgtm

I still think this is super hacky, but if we already have CrossCompile, I don't see any reason to prohibit using it this way.

CMakeLists.txt

option (LLVM_BUILD_EXTERNAL_COMPILER_RT
   "Build compiler-rt as an external project." OFF)
 
 option(LLVM_BUILD_LLVM_DYLIB "Build libllvm dynamic library" OFF)
 option(LLVM_DISABLE_LLVM_DYLIB_ATEXIT "Disable llvm-shlib's atexit destructors." ON)
 if(LLVM_DISABLE_LLVM_DYLIB_ATEXIT)
   set(DISABLE_LLVM_DYLIB_ATEXIT 1)
 endif()
 
+option(LLVM_OPTIMIZED_TABLEGEN "Force TableGen to be built with optimization" OFF)
+if(CMAKE_CROSSCOMPILING OR (LLVM_OPTIMIZED_TABLEGEN AND LLVM_ENABLE_ASSERTIONS))
+  set(LLVM_USE_HOST_TOOLS ON)
+endif()
+
 # All options referred to from HandleLLVMOptions have to be specified
 # BEFORE this include, otherwise options will not be correctly set on
 # first cmake run
 include(config-ix)
 
 # By default, we target the host, but this can be overridden at CMake
 # invocation time.
 set(LLVM_DEFAULT_TARGET_TRIPLE "${LLVM_HOST_TRIPLE}" CACHE STRING
set(CMAKE_EXE_LINKER_FLAGS
       "${CMAKE_EXE_LINKER_FLAGS} -Wl,-allow-shlib-undefined")
 endif()
 
 set(CMAKE_INCLUDE_CURRENT_DIR ON)
 
 include_directories( ${LLVM_INCLUDE_DIR} ${LLVM_MAIN_INCLUDE_DIR})
 
 # when crosscompiling import the executable targets from a file
-if(CMAKE_CROSSCOMPILING)
+if(LLVM_USE_HOST_TOOLS)
   include(CrossCompile)
-endif(CMAKE_CROSSCOMPILING)
+endif(LLVM_USE_HOST_TOOLS)
 
 if( ${CMAKE_SYSTEM_NAME} MATCHES FreeBSD )
   # On FreeBSD, /usr/local/* is not used by default. In order to build LLVM
   # with libxml2, iconv.h, etc., we must add /usr/local paths.
   include_directories("/usr/local/include")
   link_directories("/usr/local/lib")
 endif( ${CMAKE_SYSTEM_NAME} MATCHES FreeBSD )
 

cmake/modules/TableGen.cmake

if(${LLVM_TABLEGEN} STREQUAL tblgen)
           STRING "Native TableGen executable. Saves building one when cross-compiling."
           FORCE)
     endif()
   endif()
 
   # Effective tblgen executable to be used:
   set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN} PARENT_SCOPE)
 
-  if(CMAKE_CROSSCOMPILING)
+  if(LLVM_USE_HOST_TOOLS)
     if( ${${project}_TABLEGEN} STREQUAL "${target}" )
       set(${project}_TABLEGEN_EXE "${LLVM_NATIVE_BUILD}/bin/${target}")
       set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN_EXE} PARENT_SCOPE)
 
       add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
         COMMAND ${CMAKE_COMMAND} --build . --target ${target} --config $<CONFIGURATION>
         DEPENDS ${LLVM_NATIVE_BUILD}/CMakeCache.txt
         WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}