With CMake 3.8+ force Visual Studio generator builds to use the x64 host toolchain if the user is on a 64-bit system

[gbedwell]

Summary

With older CMake versions, or if the user is on a 32-bit system we'll still output a warning suggesting using the 64-bit host toolchain.

Hopefully this will alleviate some of the teething problems new developers may run into using the Visual Studio generators (assuming the workflow is to go and grab the latest version of CMake. The "Getting Started with the LLVM System using Microsoft Visual Studio" documentation suggests a CMake 3.8.0 requirement anyway. See https://cmake.org/cmake/help/latest/variable/CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE.html

On my 64-bit Windows system, I get the following behaviour:

CMake version	Generator	warning (CMakeLists.txt:31) produced	CMAKE_CPP_COMPILER
3.4.3	Visual Studio 14 2015	yes	C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe
3.4.3	Visual Studio 14 2015 Win64	yes	C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
3.8.2	Visual Studio 14 2015	no	C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64_x86/cl.exe
3.8.2	Visual Studio 14 2015 Win64	no	C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe
3.8.2	Visual Studio 15 2017	no	C:/Program Files (x86)/Microsoft Visual Studio/Preview/Professional/VC/Tools/MSVC/14.16.26926/bin/Hostx64/x86/cl.exe
3.8.2	Visual Studio 15 2017 Win64	no	C:/Program Files (x86)/Microsoft Visual Studio/Preview/Professional/VC/Tools/MSVC/14.16.26926/bin/Hostx64/x64/cl.exe

[gbedwell]

I find the CMake documentation very unclear here. On the related page for https://cmake.org/cmake/help/latest/variable/CMAKE_GENERATOR_TOOLSET.html it says:

The value of this variable should never be modified by project code. A toolchain file specified by the CMAKE_TOOLCHAIN_FILE variable may initialize CMAKE_GENERATOR_TOOLSET. Once a given build tree has been initialized with a particular value for this variable, changing the value has undefined behavior.

However, as this is within the existing check that CMAKE_GENERATOR_TOOLSET is empty, and it happens prior to the project command where the tree actually appears to be initialized, (and that in my testing this does precisely what I'd want it to) I'm pretty convinced that this change is safe and correct.

[aaron.ballman]

This area of CMake is pretty mysterious to me, but the behavior seems to be... different than what I'd expect. I'm using CMake 3.10.2.

With Patch and without -Thost=x64
CMAKE
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe -- works

Process Explorer
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\bin\Hostx86\x64\cl.exe

Without Patch and with -Thost=x64
CMAKE
-- Found assembler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Professional/VC/Tools/MSVC/14.15.26726/bin/Hostx64/x64/cl.exe -- works

Process Explorer
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.15.26726\bin\Hostx64\x64\cl.exe

So with the patch and when not using -Thost=x64, CMake finds the compiler at Hostx64/x64/cl.exe but MSBuild executes Hostx86\x64\cl.exe and without the patch when using -Thost=x64, CMake finds the compiler at Hostx64/x64/cl.exe and MSBuild executes the compiler at Hostx64\x64\cl.exe.

[gbedwell]

Oh drat. Well, I guess maybe this does manage to fall into the area of undefined behaviour then. Weirdly, that didn't happen on my local system, although I did see something similar when setting the PreferredToolArchitecture environment variable for the environment that CMake spawns in, but not in the global environment when running the VS IDE (CMAKE_C[XX]_COMPILER would not match the actual compiler being invoked). Thanks for taking a look anyway!

[aaron.ballman]

In D53073#1263453, @gbedwell wrote:

Oh drat. Well, I guess maybe this does manage to fall into the area of undefined behaviour then. Weirdly, that didn't happen on my local system, although I did see something similar when setting the PreferredToolArchitecture environment variable for the environment that CMake spawns in, but not in the global environment when running the VS IDE (CMAKE_C[XX]_COMPILER would not match the actual compiler being invoked). Thanks for taking a look anyway!

Thank you for trying to improve the situation!

CMakeLists.txt

 endif()
 
 if (NOT PACKAGE_VERSION)
   set(PACKAGE_VERSION
     "${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}.${LLVM_VERSION_PATCH}${LLVM_VERSION_SUFFIX}")
 endif()
 
 if ((CMAKE_GENERATOR MATCHES "Visual Studio") AND (CMAKE_GENERATOR_TOOLSET STREQUAL ""))
+
+  if ((CMAKE_VERSION VERSION_LESS 3.8) OR (NOT "$ENV{PROCESSOR_ARCHITECTURE}" STREQUAL "AMD64"))
     message(WARNING "Visual Studio generators use the x86 host compiler by "
-                  "default, even for 64-bit targets. This can result in linker "
-                  "instability and out of memory errors. To use the 64-bit "
-                  "host compiler, pass -Thost=x64 on the CMake command line.")
+                    "default, even for 64-bit targets. This can result in "
+                    "linker instability and out of memory errors. To use the "
+                    "64-bit host compiler, pass -Thost=x64 on the CMake "
+                    "command line or update to a newer version of CMake "
+                    "(3.8+) where this will be set automatically on 64-bit "
+                    "systems.")
+  else()
+    # CMake 3.8 introduces the following variable to allow us to select the
+    # 64-bit host Visual Studio toolchain.
+    set(CMAKE_VS_PLATFORM_TOOLSET_HOST_ARCHITECTURE "x64")
+  endif()
+
 endif()
 
 project(LLVM
   VERSION ${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}.${LLVM_VERSION_PATCH}
   LANGUAGES C CXX ASM)
 
 if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
   message(STATUS "No build type selected, default to Debug")