[cmake] Add support for case-sensitive Windows SDKs

[smeenai]

Summary

When the Windows SDK is hosted on a case-sensitive filesystem (e.g. when
compiling on Linux and not using ciopfs), we can automatically generate
a VFS overlay for headers and symlinks for libraries.

[zturner]

cmake/platforms/WinMsvc.cmake

78–93

Assuming that the vfs overlay works as you claim it does, is there any advantage to using the WINSDK_CASE_SENSITIVE=NO path? Why would someone ever want to use ciopfs instead of the vfs overlay? Would it be good enough to just say "If CMAKE_SYSTEM_NAME is not windows, use the overlay"?

[compnerd]

cmake/platforms/WinMsvc.cmake

78–93

Yes, the VFS overlay works as claimed. The thing is that it would only alleviate it for the compilation portion, not the linking portion. In fact, I used the same trick to deal with the cross-compilation on Linux for Windows in swift. The one thing which is slightly annoying is that the VFS overlay needs to walk over the file system to be generated, as opposed to the swift case, where I manually hand constructed the mapping for the handful of files used in the swift build.

The only advantage that I see for the ciopfs is that it also accounts for the linking portion. It is good enough to just enable the VFS overlay for all of the compilation though based on:

CMAKE_SYSTEM_NAME MATCHES Windows AND CMAKE_HOST_SYSTEM_NAME NOT STREQUAL CMAKE_SYSTEM_NAME

[smeenai]

cmake/platforms/WinMsvc.cmake

78–93

Like compnerd said, the VFS overlay only works for the compilation portion, but I'm also generating symlinks for the linking portion.

I intentionally made the decision to have the VFS overlay generated via a filesystem walk instead of manually constructured, and to have all the libraries symlinked instead of just the ones currently used by LLVM. It's significantly more future proof, and it's only a one time cost per clean build. I'd be happy to reconsider if there's a significant perf win from using a smaller VFS overlay, though I didn't see any in some simple testing.

I don't think there's any inherent advantage to one approach over the other, except that the VFS overlay and library symlinking is fully taken care of by the toolchain file. It's easy enough to remove support for the ciopfs path, since the other path should just work.

As far as determining case-sensitivity goes:

• Would anyone even be using this toolchain file on Windows, i.e. is that something we need to consider?
• NTFS can technically be case-sensitive, though I don't think anyone uses it. (I personally think all Windows developers should be forced to use that until they can get their damn casing right, but that's a separate issue.)
• macOS is case-insensitive by default, though it can also be configured case-sensitive (again, don't know how many people do that).

NOT CMAKE_HOST_SYSTEM_NAME STREQUAL "Darwin" and NOT CMAKE_HOST_SYSTEM_NAME STREQUAL "Windows"`

seems like a reasonable condition to me. CMake doesn't have a direct way of determining filesystem case sensitivity, as far as I can tell (though you could always also do something like create a temp file in the build directory and try to access it with a different case.

[smeenai]

Drop ciopfs support

[zturner]

Looks good. I dislike that we have to emit a file from CMake, but I can't really think of a better way.

cmake/platforms/WinMsvc.cmake

 # Cross toolchain configuration for using clang-cl on non-Windows hosts to
 # target MSVC.
 #
 # Usage:
 # cmake -G Ninja
 #    -DCMAKE_TOOLCHAIN_FILE=/path/to/this/file
 #    -DLLVM_NATIVE_TOOLCHAIN=/path/to/llvm/installation
 #    -DMSVC_BASE=/path/to/MSVC/system/libraries/and/includes
 #    -DWINSDK_BASE=/path/to/windows-sdk
 #    -DWINSDK_VER=windows sdk version folder name
+#    -DWINSDK_CASE_SENSITIVE=[YES|NO]
 #
 # LLVM_NATIVE_TOOLCHAIN:
 #   *Absolute path* to a folder containing the toolchain which will be used to
 #   build.  At a minimum, this folder should have a bin directory with a
 #   copy of clang-cl, clang, clang++, and lld-link, as well as a lib directory
 #   containing clang's system resource directory.
 #
 # MSVC_BASE:

 #           ucrt.lib
 #           etc...
 #       um
 #         x64
 #         x86
 #           kernel32.lib
 #           etc
 #
-# IMPORTANT: In order for this to work, you will need a valid copy of the Windows
-# SDK and C++ STL headers and libraries on your host.  Additionally, since the
-# Windows libraries and headers are not case-correct, you will need to have these
-# mounted in a case-insensitive mount.  This requires one command to set up.
+# WINSDK_CASE_SENSITIVE:
+#   The Windows SDK headers and libraries are not case-correct. You can mount
+#   the SDK in a case-insensitive mount to work around this. Run the following
+#   commands to set the mount up, and then copy the SDK headers and libraries to
+#   the winsdk.icase folder and use *that* folder as the path when configuring
+#   CMake.
 #
-# ~/src: mkdir winsdk
-# ~/src: mkdir winsdk.icase
-# ~/src: ciopfs winsdk/ winsdk.icase
+#   ~/src: mkdir winsdk
+#   ~/src: mkdir winsdk.icase
+#   ~/src: ciopfs winsdk/ winsdk.icase
 #
-# Now copy or otherwise install your headers and libraries to the winsdk.icase folder
-# and use *that* folder as the path when configuring CMake.
-#
-# TODO: We could also provide a CMake option -DUSE_ICASE_VFS_OVERLAY=ON/OFF that would
-# make this optional.  For now, we require ciopfs.
+#   Alternatively, you can set WINSDK_CASE_SENSITIVE to YES, in which case this
+#   toolchain file will assume the SDK is hosted on a case-sensitive filesystem,
+#   and will create a case-insensitive clang VFS overlay for the SDK headers and
+#   case-correcting symlinks for the libraries. WINSDK_CASE_SENSITIVE defaults
+#   to OFF; i.e., the default assumption is a case-insensitive mount.
 
 
 # When configuring CMake with a toolchain file against a top-level CMakeLists.txt,
 # it will actually run CMake many times, once for each small test program used to
 # determine what features a compiler supports.  Unfortunately, none of these
 # invocations share a CMakeCache.txt with the top-level invocation, meaning they
 # won't see the value of any arguments the user passed via -D.  Since these are
 # necessary to properly configure MSVC in both the top-level configuration as well as
 # all feature-test invocations, we set environment variables with the values so that
 # these environments get inherited by child invocations.
 function(init_user_prop prop)
   if(${prop})
     set(ENV{_${prop}} "${${prop}}")
   else()
     set(${prop} "$ENV{_${prop}}" PARENT_SCOPE)
   endif()
 endfunction()
 
+function(generate_winsdk_vfs_overlay winsdk_include_dir output_path)
+  set(include_dirs)
+  file(GLOB_RECURSE entries LIST_DIRECTORIES true "${winsdk_include_dir}/*")
+  foreach(entry ${entries})
+    if(IS_DIRECTORY "${entry}")
+      list(APPEND include_dirs "${entry}")
+    endif()
+  endforeach()
+
+  file(WRITE "${output_path}"  "version: 0\n")
+  file(APPEND "${output_path}" "case-sensitive: false\n")
+  file(APPEND "${output_path}" "roots:\n")
+
+  foreach(dir ${include_dirs})
+    file(GLOB headers RELATIVE "${dir}" "${dir}/*.h")
+    if(NOT headers)
+      continue()
+    endif()
+
+    file(APPEND "${output_path}" "  - name: \"${dir}\"\n")
+    file(APPEND "${output_path}" "    type: directory\n")
+    file(APPEND "${output_path}" "    contents:\n")
+
+    foreach(header ${headers})
+      file(APPEND "${output_path}" "      - name: \"${header}\"\n")
+      file(APPEND "${output_path}" "        type: file\n")
+      file(APPEND "${output_path}" "        external-contents: \"${dir}/${header}\"\n")
+    endforeach()
+  endforeach()
+endfunction()
+
+function(generate_winsdk_lib_symlinks winsdk_um_lib_dir output_dir)
+  execute_process(COMMAND "${CMAKE_COMMAND}" -E make_directory "${output_dir}")
+  file(GLOB libraries RELATIVE "${winsdk_um_lib_dir}" "${winsdk_um_lib_dir}/*")
+  foreach(library ${libraries})
+    string(TOLOWER "${library}" symlink_name)
+    execute_process(COMMAND "${CMAKE_COMMAND}"
+                            -E create_symlink
+                            "${winsdk_um_lib_dir}/${library}"
+                            "${output_dir}/${symlink_name}")
+  endforeach()
+endfunction()
+
 # FIXME: We should support target architectures other than x64
 set(CMAKE_SYSTEM_NAME Windows)
 set(CMAKE_SYSTEM_VERSION 10.0)
 set(CMAKE_SYSTEM_PROCESSOR AMD64)
 
 init_user_prop(LLVM_NATIVE_TOOLCHAIN)
 init_user_prop(MSVC_BASE)
 init_user_prop(WINSDK_BASE)
 init_user_prop(WINSDK_VER)
+init_user_prop(WINSDK_CASE_SENSITIVE)
 
 set(MSVC_INCLUDE "${MSVC_BASE}/include")
 set(MSVC_LIB "${MSVC_BASE}/lib")
 set(WINSDK_INCLUDE "${WINSDK_BASE}/Include/${WINSDK_VER}")
 set(WINSDK_LIB "${WINSDK_BASE}/Lib/${WINSDK_VER}")
 
 # Do some sanity checking to make sure we can find a native toolchain and
 # that the Windows SDK / MSVC STL directories look kosher.

 set(COMPILE_FLAGS
     -D_CRT_SECURE_NO_WARNINGS
     -imsvc "${MSVC_INCLUDE}"
     -imsvc "${WINSDK_INCLUDE}/ucrt"
     -imsvc "${WINSDK_INCLUDE}/shared"
     -imsvc "${WINSDK_INCLUDE}/um"
     -imsvc "${WINSDK_INCLUDE}/winrt")
 
+if (WINSDK_CASE_SENSITIVE)
+  # Ensure all sub-configures use the top-level VFS overlay instead of generating their own.
+  init_user_prop(winsdk_vfs_overlay_path)
+  if(NOT winsdk_vfs_overlay_path)
+    set(winsdk_vfs_overlay_path "${CMAKE_BINARY_DIR}/winsdk_vfs_overlay.yaml")
+    generate_winsdk_vfs_overlay("${WINSDK_BASE}/Include/${WINSDK_VER}" "${winsdk_vfs_overlay_path}")
+    init_user_prop(winsdk_vfs_overlay_path)
+  endif()
+  list(APPEND COMPILE_FLAGS
+       -Xclang -ivfsoverlay -Xclang "${winsdk_vfs_overlay_path}")
+endif()
+
 string(REPLACE ";" " " COMPILE_FLAGS "${COMPILE_FLAGS}")
 
 # We need to preserve any flags that were passed in by the user. However, we
 # can't append to CMAKE_C_FLAGS and friends directly, because toolchain files
 # will be re-invoked on each reconfigure and therefore need to be idempotent.
 # The assignments to the _INITIAL cache variables don't use FORCE, so they'll
 # only be populated on the initial configure, and their values won't change
 # afterward.
 set(_CMAKE_C_FLAGS_INITIAL "${CMAKE_C_FLAGS}" CACHE STRING "")
 set(CMAKE_C_FLAGS "${_CMAKE_C_FLAGS_INITIAL} ${COMPILE_FLAGS}" CACHE STRING "" FORCE)
 
 set(_CMAKE_CXX_FLAGS_INITIAL "${CMAKE_CXX_FLAGS}" CACHE STRING "")
 set(CMAKE_CXX_FLAGS "${_CMAKE_CXX_FLAGS_INITIAL} ${COMPILE_FLAGS}" CACHE STRING "" FORCE)
 
 set(LINK_FLAGS
     # Prevent CMake from attempting to invoke mt.exe. It only recognizes the slashed form and not the dashed form.
     /manifest:no
 
     # FIXME: We should support target architectures other than x64.
     -libpath:"${MSVC_LIB}/x64"
     -libpath:"${WINSDK_LIB}/ucrt/x64"
     -libpath:"${WINSDK_LIB}/um/x64")
 
+if(WINSDK_CASE_SENSITIVE)
+  # Ensure all sub-configures use the top-level symlinks dir instead of generating their own.
+  init_user_prop(winsdk_lib_symlinks_dir)
+  if(NOT winsdk_lib_symlinks_dir)
+    set(winsdk_lib_symlinks_dir "${CMAKE_BINARY_DIR}/winsdk_lib_symlinks")
+    generate_winsdk_lib_symlinks("${WINSDK_BASE}/Lib/${WINSDK_VER}/um/x64" "${winsdk_lib_symlinks_dir}")
+    init_user_prop(winsdk_lib_symlinks_dir)
+  endif()
+  list(APPEND LINK_FLAGS
+       -libpath:"${winsdk_lib_symlinks_dir}")
+endif()
+
 string(REPLACE ";" " " LINK_FLAGS "${LINK_FLAGS}")
 
 # See explanation for compiler flags above for the _INITIAL variables.
 set(_CMAKE_EXE_LINKER_FLAGS_INITIAL "${CMAKE_EXE_LINKER_FLAGS}" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "${_CMAKE_EXE_LINKER_FLAGS_INITIAL} ${LINK_FLAGS}" CACHE STRING "" FORCE)
 
 set(_CMAKE_MODULE_LINKER_FLAGS_INITIAL "${CMAKE_MODULE_LINKER_FLAGS}" CACHE STRING "")
 set(CMAKE_MODULE_LINKER_FLAGS "${_CMAKE_MODULE_LINKER_FLAGS_INITIAL} ${LINK_FLAGS}" CACHE STRING "" FORCE)