[windows-itanium] [WIP] Windows Itanium Build Recipe

[bd1976llvm]

Summary

Script and patches to automate the building of a Windows Itanium environment.

[bd1976llvm]

Removed the code patch and --patch functionality. These are not needed given that this review *is* a patch and having a patch in a patch is confusing.

[mstorsjo]

libcxx/src/CMakeLists.txt

68

I guess this could be made shorter as

list(APPEND LIBCXX_SOURCES
    support/win32/locale_win32.cpp
    support/win32/support.cpp)
if (LIBCXX_ENABLE_THREADS)
    list(APPEND LIBCXX_SOURCES
        support/win32/thread_win32.cpp)
endif()

199

Can this be made an elseif (MSVC) or similar? (Do windows-itanium environments have MSVC set in cmake?) Because for MinGW targets, the syntax below should still be used.

EDIT: From reading further here, I see that it should be covered by "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC" at least.

[bd1976llvm]

CMAKE improvements.

[bd1976llvm]

Hi mstorsjo, thanks for looking at this. Please add anyone else to this review that might be interested.

Some questions:

• One thing I don't understand is why I had to edit locale_win32.h to avoid collisions with MS sdk definitions but mingw doesn't have to?

• I initially wanted to use link.exe but it doesn't like the COMDATs emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?

• A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

libcxx/src/CMakeLists.txt

199

Thanks - of course mingw is also "WIN32". This should definitely be for "MSVC"like toolchains.

Do windows-itanium environments have MSVC set in cmake?

They shouldn't; however, until the clang driver code is improved, I am hijacking (forcing the triple with the -triple cc1 layer option) the msvc driver - this choice reduces the amount of options I have to set via cmake flags.

[mstorsjo]

In D88124#2294077, @bd1976llvm wrote:

Hi mstorsjo, thanks for looking at this. Please add anyone else to this review that might be interested.

Adding @smeenai who I think also is familiar with the windows-itanium target.

Some questios:

• One thing I don't understand is why I had to edit locale_win32.h to avoid collisions with MS sdk definitions but mingw doesn't have to?

I don't know why you need to do that either - I can also build libcxx with clang-cl for the normal MSVC target without having to edit anything. Or maybe it's specific to the windows-itanium case, because for the normal libcxx/msvc configuration, it still uses the new/delete operators from MSVC (i.e. not using libcxxabi).

• I initially wanted to use link.exe but it doesn't like the COMDAT's emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?

Are those the leader-less comdats for xdata/pdata, with a trailing function name, like .text$myfunc, .xdata$myfunc, .pdata$myfunc? That's not a problem with mingw, as that's exactly the mingw specific behaviour here. (GNU ld only handles this form, not the proper COFF associative comdats - llvm generates this form to be interoperable with GNU ld for mingw targets, and lld can handle it fine as well.)

I don't really see why you'd be getting them though; this is controlled by https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCStreamer.cpp#L783-L794, based on a flag set at https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCAsmInfoCOFF.cpp#L58-L62, so unless the itanium target uses MCAsmInfo derived from MCAsmInfoGNUCOFF (as opposed to directly from MCAsmInfoCOFF or from MCAsmInfoMicrosoft), I don't see how you'd end up with these. (Or are there any lowlevel bits in libcxxabi that ends up forcing this kinds of comdats?)

• A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

Not sure - maybe?

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

Yes, and that was one of the driving forces for completing the auto-importing feature. As long as you're only linking statically it should work just fine.

In https://reviews.llvm.org/D43184 I experimented with adding an option to clang, to generate constructors that would run at runtime, fixing this up, but it wasn't very pretty. As the autoimport feature was useful for other cases, and is how GNU tools handle it, I went with that approach.

So in principle, as far as I understand it, the itanium C++ ABI is incompatible with windows with dynamic linking, unless you have an autoimport feature in the linker that can fix it up.

[bd1976llvm]

In D88124#2294372, @mstorsjo wrote:

I don't know why you need to do that either - I can also build libcxx with clang-cl for the normal MSVC target without having to edit anything. Or maybe it's specific to the windows-itanium case, because for the normal libcxx/msvc configuration, it still uses the new/delete operators from MSVC (i.e. not using libcxxabi).

I think this answers the question. It is specific to the windows-itanium case. The clang-cl build wants libcxx backed by the MS VS CRT. However, with windows-itanium, we only want the "C" parts of the MS VS CRT - with the C++ parts supplied by libcxxabi. Unfortunately, the MS VS CRT headers are not factored cleanly into C and C++ paths and it is easy to pull in unwanted headers which is what I found was occurring here. Mingw supplies its own headers and so it wouldn't be affected by this, IIUC.

• I initially wanted to use link.exe but it doesn't like the COMDAT's emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?

Are those the leader-less comdats for xdata/pdata, with a trailing function name, like .text$myfunc, .xdata$myfunc, .pdata$myfunc? That's not a problem with mingw, as that's exactly the mingw specific behaviour here. (GNU ld only handles this form, not the proper COFF associative comdats - llvm generates this form to be interoperable with GNU ld for mingw targets, and lld can handle it fine as well.)

I don't really see why you'd be getting them though; this is controlled by https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCStreamer.cpp#L783-L794, based on a flag set at https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCAsmInfoCOFF.cpp#L58-L62, so unless the itanium target uses MCAsmInfo derived from MCAsmInfoGNUCOFF (as opposed to directly from MCAsmInfoCOFF or from MCAsmInfoMicrosoft), I don't see how you'd end up with these. (Or are there any lowlevel bits in libcxxabi that ends up forcing this kinds of comdats?)

• A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

Not sure - maybe?

Thanks for the pointer to leaderless comdats. I need to investigate this more. The problems I am seeing seem to be from comdats that are associated with the use of the "__inline" keyword.

There is another comdat related problem which is that if I compile straight to a coff file it works. However, if I compile to asm then when I try to assemble that asm I get errors like this:

src/CMakeFiles/cxx_shared.dir\locale.s:5787:2: error: section '.xdata$' is already linkonce
      .linkonce       discard

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

Yes, and that was one of the driving forces for completing the auto-importing feature. As long as you're only linking statically it should work just fine.

In https://reviews.llvm.org/D43184 I experimented with adding an option to clang, to generate constructors that would run at runtime, fixing this up, but it wasn't very pretty. As the autoimport feature was useful for other cases, and is how GNU tools handle it, I went with that approach.

So in principle, as far as I understand it, the itanium C++ ABI is incompatible with windows with dynamic linking, unless you have an autoimport feature in the linker that can fix it up.

Thanks! I had worked some of this out on my own - however these pointers really helped to understand the problem.

[bd1976llvm]

Use lld's mingw autolinking feature. Add a testcase that will fail without this.

[bd1976llvm]

Added LIT based tests.

Incorporated improvements discussed in https://reviews.llvm.org/D89518.

[bd1976llvm]

Added type_info tests.

[bd1976llvm]

@mstorsjo I'm looking at this again after the winter break. Do you have any further comments?

[mstorsjo]

libcxx/include/CMakeLists.txt

242

IMO this whole if/else could be dropped, as cl and clang-cl can take options starting both with dashes and slashes. But I guess that's orthogonal to the change at hand here (and it's essential for the change below), so I guess it looks ok.

libcxx/include/support/win32/locale_win32.h

15

This change breaks mingw builds, as the exact contents of corecrt.h and xlocinfo.h differs a bit there (I end up with errors like this):

include/c++/v1/__locale:134:20: error: use of undeclared identifier '_M_COLLATE'
        collate  = LC_COLLATE_MASK,
                   ^
include/c++/v1/support/win32/locale_win32.h:18:25: note: expanded from macro 'LC_COLLATE_MASK'
#define LC_COLLATE_MASK _M_COLLATE
                        ^

libcxx/src/CMakeLists.txt

63

This change looks like it could be split out and landed separately from the rest?

199

On this one, I'm a bit unsure here in which cases cmake calls the compiler driver for linking (requiring -Wl) and when it just calls (lld-)link directly - but if this is what's needed and works for you, then sure. As this is within an if (MSVC) so it shouldn't affect my builds in any case.

libcxxabi/CMakeLists.txt

262

This bit has changed and been improved upstream, so I'd suggest you rebase it and see if this change really is needed any longer.

libcxxabi/src/cxa_guard_impl.h

44

This can't really proceed in this form, you need to fix it properly somehow.

[bd1976llvm]

Thanks for your comments. I tried to do a rebase right up to the head of main - but it proved very difficult to resolve all the problems this caused. Instead I have rebased to your https://github.com/llvm/llvm-project/commit/8a73aa8c4c3ee82e7d650c6a957937713a4a9ab4

There are a few pieces that I can upstream but I think the rest of the changes are due to the working around the lack of a proper driver for Windows Itanium and modifications for trying to build using only the MS sdk excluding any VS runtime stuff.

I am actually not looking to commit this build script to LLVM. I am happy to upstream the changes which make sense and close this review - this build script will still be available on phabricator as an example.

I would like to commit the documentation change in https://reviews.llvm.org/D89518 though.

Thanks!

libcxx/include/support/win32/locale_win32.h

15

Yes. If you look at the block below I had to fish these defines out of xlocinfo.h header. I'm not sure if there is another way to solve this problem. Obviously, this is not something that can be upstream.

libcxx/src/CMakeLists.txt

63

Yes. Thanks.

199

Do you think this part might be acceptable upstream on its own?

libcxxabi/CMakeLists.txt

262

Thanks :)

libcxxabi/src/cxa_guard_impl.h

44

Turned out I was incorrect here (my comment had rotted I think) unistd.h just isn't available in the MS sdk. I think that this part can be landed in a separate change?

[bd1976llvm]

rebased build script and some minor improvements.

[mstorsjo]

libcxx/src/CMakeLists.txt

199

Hmm, unsure - if you try to upstream it, it needs a thorough explanation of how the linker is called in your case compared to other cases (calling it via the compiler driver vs calling *link directly) at least.

libcxxabi/src/cxa_guard_impl.h

44

Right - this one works in mingw because unistd.h exists there, and we don't build libcxxabi for MSVC targets otherwise, normally, so something like __has_include() for it might work.

I'm not entirely sure what unistd.h is used for here though - the file builds fine without it for mingw targets at least, so putting it wihtin #ifndef _WIN32 or something like that might work as well - or check the contents of the source file to see if it's just conditionally needed with some other condition.

[bd1976llvm]

libcxx/src/CMakeLists.txt

199

Actually, this is another part that is really part of how I have fudged things to get everything to work without a proper driver. I don't think it is suitable for upstreaming.

libcxxabi/CMakeLists.txt

437

Do you know why this part isn't required for mingw? I definitely to make these match for my builds.

libcxxabi/src/cxa_guard_impl.h

44

Thanks - I'll look into it after the weekend.

[bd1976llvm]

Added the patch from https://reviews.llvm.org/D93203 and the scei vendor variant of windows itanium to the relevant tests.

[mstorsjo]

libcxxabi/CMakeLists.txt

437

Mingw headers don't have those concepts at all. If libcxx sets such defines, we do need to match it here (and within the tests) - so I think this is upstreamable - just try to match the libcxx logic.

[bd1976llvm]

libcxxabi/CMakeLists.txt

437

Thanks - I gave this a go but I got stuck trying to get the libcxx testing running for a clang-cl build of libcxx. I still intend to upstream - just need the clang-cl build sorted first!

[mstorsjo]

libcxxabi/CMakeLists.txt

437

Ah, I see.

FWIW, I'm planning on writing up some docs on how to test libcxx on windows - I have one version of how to do that at https://github.com/mstorsjo/llvm-project/blob/e5fb5c31ae09b2bafab876c0b539d84668713715/libcxx/test-instructions.txt.

Currently lots of libcxx tests fail due to other minor incompatibilities - I'm planning on trying to upstream workarounds for that soon at the same time too. If you run into issues, try picking commits from https://github.com/mstorsjo/llvm-project/commits/999b8b75867bb7f92eecc027ade9c1a7574dde65 if you run into issues. It's a bit mixed in with other (less upstreamable I think) fixes for cross-testing and other uncommon build configurations.

[bd1976llvm]

libcxxabi/CMakeLists.txt

437

Thanks very much. Still looking at this but slow progress due to other priorities.

[bd1976llvm]

Set DLL storage class for guard variables correctly to avoid a link error when code using a guard variable is inlined into a module that imports the code.

libcxx/include/CMakeLists.txt

add_custom_command(OUTPUT ${dst}
       COMMAND ${CMAKE_COMMAND} -E copy_if_different ${src} ${dst}
       COMMENT "Copying CXX __config")
   list(APPEND out_files ${dst})
   add_custom_target(generate-cxx-headers DEPENDS ${out_files})
 
   add_library(${CXX_HEADER_TARGET} INTERFACE)
   add_dependencies(${CXX_HEADER_TARGET} generate-cxx-headers ${LIBCXX_CXX_ABI_HEADER_TARGET})
   # TODO: Use target_include_directories once we figure out why that breaks the runtimes build
-  if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC" OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC")
+
+  if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC" OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC" AND NOT ${CMAKE_CXX_COMPILER_FRONTEND_VARIANT} STREQUAL "GNU")
     target_compile_options(${CXX_HEADER_TARGET} INTERFACE /I "${output_dir}")
   else()
     target_compile_options(${CXX_HEADER_TARGET} INTERFACE -I "${output_dir}")
   endif()
 
   # Make sure the generated __config_site header is included when we build the library.
-  if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC" OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC")
+  if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC" OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC" AND NOT ${CMAKE_CXX_COMPILER_FRONTEND_VARIANT} STREQUAL "GNU")
     target_compile_options(${CXX_HEADER_TARGET} INTERFACE /FI "${LIBCXX_BINARY_DIR}/__config_site")
   else()
     target_compile_options(${CXX_HEADER_TARGET} INTERFACE -include "${LIBCXX_BINARY_DIR}/__config_site")
   endif()
 else()
   add_library(${CXX_HEADER_TARGET} INTERFACE)
 endif()
 

libcxx/include/support/win32/locale_win32.h

 // -*- C++ -*-
 //===--------------------- support/win32/locale_win32.h -------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef _LIBCPP_SUPPORT_WIN32_LOCALE_WIN32_H
 #define _LIBCPP_SUPPORT_WIN32_LOCALE_WIN32_H
 
 #include <__config>
 #include <stdio.h>
-#include <xlocinfo.h> // _locale_t
+#include <corecrt.h> // _locale_t
 #include <__nullptr>
 
+<<<!!!!! 
+
+The xlocinfo.h include was replaced with corecrt.h above, as xlocinfo.h drags in
+unwanted definitions e.g. of new and delete from the MS sdk. Due to that replacement
+there are missing #defines that were provided by xlocinfo.h. To allow this file to
+compile this comment should be replaced with the necessary set of defines copied from
+xlocinfo.h. On my sdk from VS2017 xlocinfo.h is located here:
+
+C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\include\xlocinfo.h
+
+It provides the following set of defines:
+
+_X_ALL
+_X_COLLATE
+_X_CTYPE
+ _X_MONETARY
+_X_NUMERIC
+_X_TIME
+_X_MAX
+X_MESSAGES
+_NCAT
+_CATMASK(n)
+_M_COLLATE
+_M_CTYPE
+_M_MONETARY
+_M_NUMERIC
+_M_TIME
+_M_MESSAGES
+_M_ALL
+
+ !!!!!!>>>>
+
 #define LC_COLLATE_MASK _M_COLLATE
 #define LC_CTYPE_MASK _M_CTYPE
 #define LC_MONETARY_MASK _M_MONETARY
 #define LC_NUMERIC_MASK _M_NUMERIC
 #define LC_TIME_MASK _M_TIME
 #define LC_MESSAGES_MASK _M_MESSAGES
 #define LC_ALL_MASK (  LC_COLLATE_MASK \
                      | LC_CTYPE_MASK \

libcxx/src/CMakeLists.txt

set(LIBCXX_SOURCES
   typeinfo.cpp
   utility.cpp
   valarray.cpp
   variant.cpp
   vector.cpp
   )
 
 if(WIN32)
-  list(APPEND LIBCXX_SOURCES
     support/win32/locale_win32.cpp
-    support/win32/support.cpp
-    support/win32/thread_win32.cpp
-    )
+    support/win32/support.cpp)
+if (LIBCXX_ENABLE_THREADS)
+    list(APPEND LIBCXX_SOURCES
+        support/win32/thread_win32.cpp)
+endif()
+
 elseif("${CMAKE_SYSTEM_NAME}" STREQUAL "SunOS")
   list(APPEND LIBCXX_SOURCES
     support/solaris/mbsnrtowcs.inc
     support/solaris/wcsnrtombs.inc
     support/solaris/xlocale.cpp
     )
 endif()
 
 if (LIBCXX_ENABLE_FILESYSTEM)
else()
       target_link_libraries(cxx_shared PUBLIC unwind)
     endif()
   endif()
 
   # Link against libc++abi
   if (LIBCXX_STATICALLY_LINK_ABI_IN_SHARED_LIBRARY)
     if (APPLE)
       target_link_libraries(cxx_shared PRIVATE "-Wl,-force_load" "${LIBCXX_CXX_STATIC_ABI_LIBRARY}")
+    elseif (MSVC OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC")
+      target_link_libraries(cxx_shared PRIVATE "-Wl,/WHOLEARCHIVE:${LIBCXX_CXX_STATIC_ABI_LIBRARY}${CMAKE_STATIC_LIBRARY_SUFFIX}")
     else()
       target_link_libraries(cxx_shared PRIVATE "-Wl,--whole-archive,-Bstatic" "${LIBCXX_CXX_STATIC_ABI_LIBRARY}" "-Wl,-Bdynamic,--no-whole-archive")
     endif()
   else()
     target_link_libraries(cxx_shared PUBLIC "${LIBCXX_CXX_SHARED_ABI_LIBRARY}")
   endif()
 
   # Maybe re-export symbols from libc++abi
if (LIBCXX_ENABLE_STATIC)
   # Attempt to merge the libc++.a archive and the ABI library archive into one.
   if (LIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY)
     set(MERGE_ARCHIVES_SEARCH_PATHS "")
     if (LIBCXX_CXX_ABI_LIBRARY_PATH)
       set(MERGE_ARCHIVES_SEARCH_PATHS "-L${LIBCXX_CXX_ABI_LIBRARY_PATH}")
     endif()
     if (TARGET "${LIBCXX_CXX_STATIC_ABI_LIBRARY}" OR HAVE_LIBCXXABI)
       set(MERGE_ARCHIVES_ABI_TARGET "$<TARGET_LINKER_FILE:${LIBCXX_CXX_STATIC_ABI_LIBRARY}>")
+    elseif (MSVC OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC")
+      set(MERGE_ARCHIVES_ABI_TARGET
+        "${LIBCXX_CXX_STATIC_ABI_LIBRARY}${CMAKE_STATIC_LIBRARY_SUFFIX}")
     else()
       set(MERGE_ARCHIVES_ABI_TARGET
         "${CMAKE_STATIC_LIBRARY_PREFIX}${LIBCXX_CXX_STATIC_ABI_LIBRARY}${CMAKE_STATIC_LIBRARY_SUFFIX}")
     endif()
     if (APPLE)
       set(MERGE_ARCHIVES_LIBTOOL "--use-libtool" "--libtool" "${CMAKE_LIBTOOL}")
     endif()
     add_custom_command(TARGET cxx_static POST_BUILD

libcxx/utils/merge_archives.py

 
 def print_and_exit(msg):
     sys.stderr.write(msg + '\n')
     exit_with_cleanups(1)
 
 def find_and_diagnose_missing(lib, search_paths):
     if os.path.exists(lib):
         return os.path.abspath(lib)
-    if not lib.startswith('lib') or not lib.endswith('.a'):
-        print_and_exit(("input file '%s' not not name a static library. "
-                       "It should start with 'lib' and end with '.a") % lib)
     for sp in search_paths:
         assert type(sp) is list and len(sp) == 1
         path = os.path.join(sp[0], lib)
         if os.path.exists(path):
             return os.path.abspath(path)
     print_and_exit("input '%s' does not exist" % lib)
 
 

libcxxabi/CMakeLists.txt

 # the libc++abi that is being built. There are two problems with testing a
 # static libc++abi. In the case of a standalone build, the tests will link the
 # system's libc++, which might not have been built against our libc++abi. In the
 # case of an in tree build, libc++ will prefer a dynamic libc++abi from the
 # system over a static libc++abi from the output directory.
 option(LIBCXXABI_ENABLE_SHARED "Build libc++abi as a shared library." ON)
 option(LIBCXXABI_ENABLE_STATIC "Build libc++abi as a static library." ON)
 
+option(LIBCXXABI_ENABLE_STATIC_FOR_LIBCXX_SHARED "Build libc++abi as a static library to be linked into a libcxx shared library." ON)
+
 cmake_dependent_option(LIBCXXABI_INSTALL_STATIC_LIBRARY
   "Install the static libc++abi library." ON
   "LIBCXXABI_ENABLE_STATIC;LIBCXXABI_INSTALL_LIBRARY" OFF)
 cmake_dependent_option(LIBCXXABI_INSTALL_SHARED_LIBRARY
   "Install the shared libc++abi library." ON
   "LIBCXXABI_ENABLE_SHARED;LIBCXXABI_INSTALL_LIBRARY" OFF)
 
 cmake_dependent_option(LIBCXXABI_STATICALLY_LINK_UNWINDER_IN_STATIC_LIBRARY
if (LIBCXXABI_HAS_NOSTDINCXX_FLAG)
   string(REPLACE "-stdlib=libstdc++" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
 endif()
 
 # Let the library headers know they are currently being used to build the
 # library.
 add_definitions(-D_LIBCXXABI_BUILDING_LIBRARY)
 
 # Disable DLL annotations on Windows for static builds.
-if (WIN32 AND LIBCXXABI_ENABLE_STATIC AND NOT LIBCXXABI_ENABLE_SHARED)
+if (WIN32 AND LIBCXXABI_ENABLE_STATIC AND NOT LIBCXXABI_ENABLE_SHARED AND NOT LIBCXXABI_ENABLE_STATIC_FOR_LIBCXX_SHARED)
   add_definitions(-D_LIBCXXABI_DISABLE_VISIBILITY_ANNOTATIONS)
 endif()
 
+# We are building libcxxabi as a static library and linking it into libcxx;
+# therefore, we need the export annotations to be in place so that the
+# required symobls will be exported from the final libcxx dll.
+if (LIBCXXABI_ENABLE_STATIC_FOR_LIBCXX_SHARED)
+  add_definitions(-D_LIBCPP_BUILDING_LIBRARY)
+endif()
+
 add_compile_flags_if_supported(-Werror=return-type)
 
 # Get warning flags
 add_compile_flags_if_supported(-W)
 add_compile_flags_if_supported(-Wall)
 add_compile_flags_if_supported(-Wchar-subscripts)
 add_compile_flags_if_supported(-Wconversion)
 add_compile_flags_if_supported(-Wmismatched-tags)

 
 # Prevent libc++abi from having library dependencies on libc++
 add_definitions(-D_LIBCPP_DISABLE_EXTERN_TEMPLATE)
 
 # Bring back `std::unexpected`, which is removed in C++17, to support
 # pre-C++17.
 add_definitions(-D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS)
 
-if (MSVC)
+if (MSVC OR "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC")
   add_definitions(-D_CRT_SECURE_NO_WARNINGS)
+  if (LIBCXXABI_ENABLE_STATIC_FOR_LIBCXX_SHARED)
+    # There is a symbol dependency between libc++ and libc++abi. To break this dependency
+    # we build libc++abi as a library and then statically link that library into the
+    # libc++ dll. In order to make this work I have copied this block of compiler flags from
+    # libc++'s' CMakeLists.txt. Unless these flags are normalized link errors are emitted
+    # when trying to link the libc++ dll, for example:
+    #   lld-link: error: /failifmismatch: mismatch detected for '_CRT_STDIO_ISO_WIDE_SPECIFIERS':
+    #   >>> src/CMakeFiles/cxx_shared.dir/support/win32/support.cpp.obj has value 1
+    #   >>> c++abi.lib(cxa_handlers.cpp.obj) has value 0
+    add_definitions(-D_ALLOW_MSC_VER_MISMATCH)
+    add_definitions(-D_ALLOW_ITERATOR_DEBUG_LEVEL_MISMATCH)
+    add_definitions(-D_CRTBLD)
+    add_definitions(-D_CRT_STDIO_ISO_WIDE_SPECIFIERS)
+  endif()
 endif()
 
 # Define LIBCXXABI_USE_LLVM_UNWINDER for conditional compilation.
 if (LIBCXXABI_USE_LLVM_UNWINDER)
   add_definitions(-DLIBCXXABI_USE_LLVM_UNWINDER)
 endif()
 
 if (LIBCXXABI_SILENT_TERMINATE)
   add_definitions(-DLIBCXXABI_SILENT_TERMINATE)

libcxxabi/src/cxa_guard_impl.h

  *  Overall Design:
  *    The implementation was designed to allow each implementation to be tested
  *    independent of the C++ runtime or platform support.
  *
  */
 
 #include "__cxxabi_config.h"
 #include "include/atomic_support.h"
-#include <unistd.h>
+// Avoid dragging in vcruntime headers
+// TODO: Control this properly with the preprocessor.
+//#include <unistd.h>
 #if defined(__has_include)
 # if __has_include(<sys/syscall.h>)
 #   include <sys/syscall.h>
 # endif
 #endif
 
 #include <stdlib.h>
 #include <__threading_support>

libunwind/CMakeLists.txt

 option(LIBUNWIND_ENABLE_STATIC "Build libunwind as a static library." ON)
 option(LIBUNWIND_ENABLE_CROSS_UNWINDING "Enable cross-platform unwinding support." OFF)
 option(LIBUNWIND_ENABLE_ARM_WMMX "Enable unwinding support for ARM WMMX registers." OFF)
 option(LIBUNWIND_ENABLE_THREADS "Build libunwind with threading support." ON)
 option(LIBUNWIND_WEAK_PTHREAD_LIB "Use weak references to refer to pthread functions." OFF)
 option(LIBUNWIND_USE_COMPILER_RT "Use compiler-rt instead of libgcc" OFF)
 option(LIBUNWIND_INCLUDE_DOCS "Build the libunwind documentation." ${LLVM_INCLUDE_DOCS})
 option(LIBUNWIND_IS_BAREMETAL "Build libunwind for baremetal targets." OFF)
+option(LIBUNWIND_NO_LIBRARIES "Don't add any libraries to the link line." OFF)
 
 set(LIBUNWIND_LIBDIR_SUFFIX "${LLVM_LIBDIR_SUFFIX}" CACHE STRING
     "Define suffix of library directory name (32/64)")
 option(LIBUNWIND_INSTALL_LIBRARY "Install the libunwind library." ON)
 cmake_dependent_option(LIBUNWIND_INSTALL_STATIC_LIBRARY
   "Install the static libunwind library." ON
   "LIBUNWIND_ENABLE_STATIC;LIBUNWIND_INSTALL_LIBRARY" OFF)
 cmake_dependent_option(LIBUNWIND_INSTALL_SHARED_LIBRARY

libunwind/src/CMakeLists.txt

 endif()
 
 set(LIBUNWIND_SOURCES
     ${LIBUNWIND_CXX_SOURCES}
     ${LIBUNWIND_C_SOURCES}
     ${LIBUNWIND_ASM_SOURCES})
 
 # Generate library list.
+if (NOT LIBUNWIND_NO_LIBRARIES)
   add_library_flags_if(LIBUNWIND_HAS_C_LIB c)
   if (LIBUNWIND_USE_COMPILER_RT)
     add_library_flags("${LIBUNWIND_BUILTINS_LIBRARY}")
   else()
     add_library_flags_if(LIBUNWIND_HAS_GCC_S_LIB gcc_s)
     add_library_flags_if(LIBUNWIND_HAS_GCC_LIB gcc)
   endif()
   add_library_flags_if(LIBUNWIND_HAS_DL_LIB dl)
   if (LIBUNWIND_ENABLE_THREADS)
     add_library_flags_if(LIBUNWIND_HAS_PTHREAD_LIB pthread)
     add_compile_flags_if(LIBUNWIND_WEAK_PTHREAD_LIB -DLIBUNWIND_USE_WEAK_PTHREAD=1)
   endif()
+endif()
 
 # Setup flags.
 add_link_flags_if_supported(-nodefaultlibs)
 
 # MINGW_LIBRARIES is defined in config-ix.cmake
 add_library_flags_if(MINGW "${MINGW_LIBRARIES}")
 
 if (LIBUNWIND_ENABLE_SHARED AND

wi.py

+#!/usr/bin/env python3
+
+# Build a windows-itanium environment:
+#   We require libunwind, libcxx-abi, and libcxx and the MS ucrt.
+#
+# This script is windows specific; however, a similar process
+# could be used to cross-compile from linux.
+#
+# The process *should be* to first build a window-itanium clang
+# and then do a stand-alone compilation of each of the
+# libraries using that compiler. However, this doesn't
+# work (currently) due to the lack of a suitable driver for
+# windows-itanium (in the clang driver code). Instead we build
+# an msvc targeting clang and then we invoke that clang but force
+# the triple to windows-itanium using the -cc1 layer -triple option.
+#
+# This script automates a build of the required components using cmake
+# and ninja. Build products are left in the source tree for simplicity. The
+# result is a "wi" directory containing the toolchain binaries, include
+# files and libraries. Due to the lack of a suitable windows-itanium
+# driver this script creates a .bat file in the toolchain bin directory
+# named "wi.bat" which assists in driving the windows-itanium compiler.
+#
+# usage:
+#   1. Ensure pre-requisites are available as described in "pre-requisites:".
+#   2. cd into a git workspace and apply the patch from D88124.
+#   3. Manually fix up "locale_win32.h" (see notes in the patch).
+#   4. Run "wi.py" to build.
+#
+#   Note: The "--clean-all" option can be supplied to force a rebuild.
+#
+# pre-requisites:
+#   This script assumes it's run from within an llvm git checkout (tested 
+#   using: 4cb016cd2d8467c572b2e5c5d34f376ee79e4ac1) with the patch from
+#   https://reviews.llvm.org/D88124 applied.
+#
+#   Ninja and cmake must be on your path. Tested with:
+#   - Ninga version v1.10.1, binaries from https://github.com/ninja-build/ninja/releases
+#   - cmake version 3.18.2, binary zip from https://cmake.org/download/
+#
+#   Note: We need a version of cmake that is high enough to have:
+#         https://gitlab.kitware.com/cmake/cmake/-/issues/16439
+#
+#   The build needs to be able to find the installed windows sdk. I tested
+#   the script with an sdk installed via VS2017. The following include files
+#   and library locations were added when running vcvarsall.bat on my system:
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\include
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\include
+#       C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.17763.0\\ucrt
+#       C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.17763.0\\shared
+#       C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.17763.0\\um
+#       C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.17763.0\\winrt
+#       C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.17763.0\\cppwinrt
+#
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64
+#       C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.17763.0\\ucrt\\x64
+#       C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.17763.0\\um\\x64
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64
+#       C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x86\\store\\references
+#       C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.17763.0
+#       C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.17763.0
+#       C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319
+#
+# Notes on inter-dependencies:
+#   libunwind can be built as a dll and is not dependent on other projects.
+#
+#   In libcxx/include/exception the "class _LIBCPP_TYPE_VIS exception_ptr" is declared. This has these three
+#   functions
+#     exception_ptr(const exception_ptr&) _NOEXCEPT;
+#     exception_ptr& operator=(const exception_ptr&) _NOEXCEPT;
+#     ~exception_ptr() _NOEXCEPT;
+#
+#   This header is included when building stdlib_exception.cpp in libcxxabi. The definitions of the above three
+#   functions are in libcxx not libcxxabi. Therefore, there is a circular dependency and you get a link error when
+#   trying to link libcxxabi into a DLL. Maybe this works with other itanium toolchains as they would generate
+#   ELF output and the missing symbols would become references to be resolved later by the dynamic linker? We
+#   tackle this by statically linking libcxxabi into libcxx when building the libcxx DLL.
+#
+
+import os
+import shutil
+import stat
+import subprocess
+import sys
+import winreg
+
+curr_dir = os.getcwd()
+
+def do_exit(exit_code):
+    print('Build {}'.format('failed' if exit_code != 0 else 'succeeded'))
+    exit(exit_code)
+
+def remove_file(file):
+    if os.path.exists(file):
+        os.remove(file)
+
+def remove_readonly(func, path, _):
+    "Clear the readonly bit and reattempt the removal"
+    os.chmod(path, stat.S_IWRITE)
+    func(path)
+
+def remove_dir(dir):
+    if os.path.exists(dir):
+        shutil.rmtree(dir, onerror = remove_readonly)
+
+def run_cmd(cmds, bat_file, input=None):
+    with open('{}'.format(bat_file), 'w') as tf:
+        for cmd in cmds:
+            tf.write('{}\n@if %ERRORLEVEL% NEQ 0 exit %ERRORLEVEL%\n\n'.format(cmd));
+        tf.write('exit %ERRORLEVEL%\n')
+        tf.flush()
+
+        proc = subprocess.Popen(['cmd', '/c', tf.name], stdin = subprocess.PIPE)
+        proc.communicate(input.encode() if input else None)
+        if proc.returncode != 0:
+            do_exit(proc.returncode)
+
+vcvars_cmd_line = None
+def get_vcvars_cmd():
+    global vcvars_cmd_line
+    if not vcvars_cmd_line:
+        try:
+            with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r'SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VS7') as key:
+                path = winreg.QueryValueEx(key, r'15.0')
+                vcvars_cmd_line = '@call "' + path[0] + r'VC\Auxiliary\Build\vcvarsall.bat" amd64'
+        except:
+            vcvars_cmd_line = r'@echo "Warning: could not find vcvarsall.bat"'
+    return vcvars_cmd_line
+
+build_dirs_name = 'build-wi'
+
+toolchain_build_path = os.path.join(curr_dir, build_dirs_name)
+
+install_path = os.path.join(curr_dir, 'wi')
+
+toolchain_bin_path = os.path.join(install_path, 'bin')
+
+def build_clang_and_lld():
+    '''
+    Build a windows-msvc (default triple) toolchain
+    with support for the windows-itanium triple. See comments
+    on the lack of a suitable windows-itanium driver elsewhere.
+    '''
+    if not os.path.exists(toolchain_build_path):
+        os.mkdir(toolchain_build_path)
+
+    cd_build_dir = '@cd {}'.format(toolchain_build_path)
+
+    cmake_cmd  = ' cmake'
+    cmake_cmd += ' -GNinja'
+    cmake_cmd += ' -DCMAKE_CXX_COMPILER=cl'
+    cmake_cmd += ' -DCMAKE_C_COMPILER=cl'
+    cmake_cmd += ' -DCMAKE_MAKE_PROGRAM=ninja'
+    cmake_cmd += ' -DLLVM_TARGETS_TO_BUILD:STRING=X86'
+    cmake_cmd += ' -DLLVM_OPTIMIZED_TABLEGEN:BOOL=ON'
+    cmake_cmd += ' -DLLVM_ENABLE_ASSERTIONS=ON'
+    cmake_cmd += ' -DLLVM_INCLUDE_EXAMPLES=OFF'
+    cmake_cmd += ' -DLLVM_ENABLE_TIMESTAMPS=OFF'
+    cmake_cmd += ' -DLLVM_VERSION_SUFFIX='
+    cmake_cmd += ' -DLLVM_BUILD_RUNTIME=OFF'
+    cmake_cmd += ' -DLLVM_APPEND_VC_REV=OFF'
+    cmake_cmd += ' -DLLVM_ENABLE_PROJECTS="clang;lld"'
+    cmake_cmd += ' -DCMAKE_BUILD_TYPE=Release'
+    cmake_cmd += ' -DCLANG_ENABLE_STATIC_ANALYZER=OFF'
+    cmake_cmd += ' -DCLANG_ENABLE_ARCMT=OFF'
+    cmake_cmd += ' ../llvm'
+    cmake_cmd += ' -DCMAKE_INSTALL_PREFIX=\"{}\"'.format(install_path)
+
+    run_cmd([get_vcvars_cmd(),
+             cd_build_dir,
+             cmake_cmd,
+             'ninja -v',
+             'ninja install'],
+             'toolchain.bat')
+
+ar_bin = os.path.join(toolchain_bin_path, 'llvm-ar')
+ranlib_bin = os.path.join(toolchain_bin_path, 'llvm-ranlib')
+
+unwind_build_dir = os.path.join('libunwind', build_dirs_name)
+
+llvm_dir = os.path.join(curr_dir, 'llvm')
+
+install_libs_path = os.path.join(install_path, 'lib')
+install_bin_path = os.path.join(install_path, 'bin')
+
+# EXPLAIN: Use -Xclang -triple rather than -target (see other comments about the lack of a suitable windows-itanium driver.)
+common_clang_options  = ' -Xclang -triple -Xclang x86_64-unknown-windows-itanium'
+
+# EXPLAIN: When -windows-itanium was being actively developed lld wasn't ready to replace link.exe but now we need it to as
+#          link.exe doesn't like the COMDATs in the generated COFF files.
+common_clang_options += ' --ld-path='.format(os.path.join(toolchain_bin_path, 'lld-link'))
+
+# EXPLAIN: My understanding is that we shouldn't need to set any _LIBCPP_* flags when
+#          building libcxxabi. Unfortunately, I found that if you don't set this then you get
+#          compiler errors for "missing refstring" because refstring is declared in libcxx/include/stdexcept
+#          but the declaration is guarded by: #ifndef _LIBCPP_ABI_VCRUNTIME and _LIBCPP_ABI_VCRUNTIME is on. This
+#          seemed the best way to force it off. It also seems right as it makes sense to set this for windows-itanium.
+common_clang_options  += ' -D_LIBCPP_ABI_FORCE_ITANIUM'
+
+# EXPLAIN: We rather crudely specify linker options even when doing compilation.
+# suppress warnings about this.
+common_clang_options  += ' -Qunused-arguments'
+
+# EXPLAIN: Use -fsjlj-exceptions for now!
+# TODO: Get DWARF zero cost exceptions working properly!
+common_clang_options  += ' -fsjlj-exceptions'
+
+clang_builtins_path = os.path.join(toolchain_build_path, 'lib', 'clang', '12.0.0', 'include')
+
+def build_unwind():
+    if not os.path.exists(unwind_build_dir):
+      os.mkdir(unwind_build_dir)
+
+    if not os.path.exists(install_path):
+      os.mkdir(install_path)
+
+    cd_build_dir = '@cd {}'.format(unwind_build_dir)
+
+    run_cmake =  'cmake'
+    run_cmake += ' --debug-trycompile'
+    run_cmake += ' -G \"Ninja\"'
+    run_cmake += ' -DCMAKE_MAKE_PROGRAM=ninja'
+    run_cmake += ' -DCMAKE_BUILD_TYPE=Release'
+    run_cmake += ' -DCMAKE_INSTALL_PREFIX=\"{}\"'.format(install_path)
+    run_cmake += ' -DCMAKE_C_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CXX_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CROSSCOMPILING=ON'
+    run_cmake += ' -DCMAKE_SYSTEM_NAME=Windows'
+    run_cmake += ' -DCMAKE_C_COMPILER_WORKS=ON'
+    run_cmake += ' -DCMAKE_CXX_COMPILER_WORKS=ON'
+    run_cmake += ' -DLLVM_PATH=\"{}\"'.format(llvm_dir)
+    run_cmake += ' -DLLVM_COMPILER_CHECKED=ON'
+    run_cmake += ' -DCMAKE_AR=\"{}\"'.format(ar_bin)
+    run_cmake += ' -DCMAKE_RANLIB=\"{}\"'.format(ranlib_bin)
+    run_cmake += ' -DCXX_SUPPORTS_CXX11=ON'
+    run_cmake += ' -DCXX_SUPPORTS_CXX_STD=ON'
+    run_cmake += ' -DLIBUNWIND_USE_COMPILER_RT=OFF'
+    run_cmake += ' -DLIBUNWIND_ENABLE_THREADS=OFF'
+    run_cmake += ' -DLIBUNWIND_ENABLE_SHARED=ON'
+    run_cmake += ' -DLIBUNWIND_ENABLE_STATIC=OFF'
+    run_cmake += ' -DLIBUNWIND_ENABLE_CROSS_UNWINDING=OFF'
+    run_cmake += ' -DLIBUNWIND_INSTALL_LIBRARY=ON'
+
+    # EXPLAIN: I added this flag to the cmake files to prevent cmake adding
+    #          the following libraries: -lc.lib -lgcc_s.lib -lgcc.lib -ldl.lib. I presume
+    #          that building for windows is just not a thing that libunwind really supports?
+    #          Can we upstream something similar?
+    run_cmake += ' -DLIBUNWIND_NO_LIBRARIES=ON'
+
+    # EXPLAN: Adding these two libraries fixes the following error during linking:
+    #           lld-link: warning: undefined symbol: fprintf
+    #           >>> referenced by src/CMakeFiles/unwind_shared.dir/Unwind-sjlj.c.obj:(_Unwind_SjLj_RaiseException)
+    #           >>> referenced 33 more times
+    #         However it introduces this:
+    #           lld-link: warning: duplicate symbol: _snprintf
+    #           >>> defined at src/CMakeFiles/unwind_shared.dir/UnwindLevel1.c.obj
+    #           >>> defined at legacy_stdio_definitions.lib(legacy_stdio_definitions.obj)
+    libunwind_c_flags   = ' -lmsvcrt -llegacy_stdio_definitions'
+
+    # EXPLAN: This works-around the multiply defined error messages when linking, e.g:
+    #           lld-link: warning: duplicate symbol: sprintf
+    #           >>> defined at src/CMakeFiles/unwind_shared.dir/UnwindLevel1.c.obj
+    #           >>> defined at src/CMakeFiles/unwind_shared.dir/UnwindLevel1-gcc-ext.c.obj
+    #         I have set it on the basis that the mingw build sets this flag to work around COMDAT
+    #         problems and we see the same comdat errors from lld when linking.
+    # TODO: work out if this is the correct thing to be doing here!
+    libunwind_c_flags  += ' -Xlinker /force:multiple'
+
+    run_cmake += ' -DCMAKE_CXX_FLAGS=\"-Wno-dll-attribute-on-redeclaration {} {}\"'.format(common_clang_options, libunwind_c_flags)
+    run_cmake += ' -DCMAKE_C_FLAGS=\"-Wno-dll-attribute-on-redeclaration {} {}\"'.format(common_clang_options, libunwind_c_flags)
+
+    # EXPLAIN: Don't run the linker on compiler check
+    run_cmake += ' -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY'
+    run_cmake += ' ..'
+
+    run_cmd([get_vcvars_cmd(),
+             'set PATH={};%PATH%'.format(toolchain_bin_path),
+             cd_build_dir,
+             run_cmake,
+             'ninja -v',
+             'ninja install'],
+             'libunwind.bat')
+
+    # TODO: Why doesn't "ninja install" put the dll
+    #       into the correct install location? Currently, it
+    #       doesn't so we manually do this here:
+    if not os.path.exists(install_bin_path):
+        os.mkdir(install_bin_path)
+    built_dll = os.path.join(unwind_build_dir, 'lib', 'unwind.dll')
+    installed_dll = os.path.join(install_bin_path, 'unwind.dll')
+    shutil.copyfile(built_dll, installed_dll)
+
+cxx_build_dir = os.path.join('libcxx', build_dirs_name)
+
+def build_cxx():
+    if not os.path.exists(cxx_build_dir):
+        os.mkdir(cxx_build_dir)
+
+    cd_build_dir = '@cd {}'.format(cxx_build_dir)
+
+    run_cmake =  'cmake'
+    # EXPLAIN: --debug-trycompile leaves around the files and dirs that cmake
+    #          uses to decide which features are supported. This process can easily go
+    #          awry due to setting incorrect flags here and leaving this stuff around
+    #          allows for investigation.
+    run_cmake += ' --debug-trycompile'
+    run_cmake += ' -G \"Ninja\"'
+    run_cmake += ' -DCMAKE_MAKE_PROGRAM=ninja'
+    run_cmake += ' -DCMAKE_BUILD_TYPE=Release'
+    run_cmake += ' -DCMAKE_INSTALL_PREFIX=\"{}\"'.format(install_path)
+    run_cmake += ' -DCMAKE_C_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CXX_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CROSSCOMPILING=ON'
+    run_cmake += ' -DCMAKE_SYSTEM_NAME=Windows'
+    run_cmake += ' -DCMAKE_C_COMPILER_WORKS=ON'
+    run_cmake += ' -DCMAKE_CXX_COMPILER_WORKS=ON'
+    run_cmake += ' -DLLVM_PATH=\"{}\"'.format(llvm_dir)
+    run_cmake += ' -DLLVM_COMPILER_CHECKED=ON'
+    run_cmake += ' -DCMAKE_AR=\"{}\"'.format(ar_bin)
+    run_cmake += ' -DCMAKE_RANLIB=\"{}\"'.format(ranlib_bin)
+    run_cmake += ' -DLIBCXX_INSTALL_HEADERS=ON'
+    run_cmake += ' -DLIBCXX_ENABLE_SHARED=ON'
+    run_cmake += ' -DLIBCXX_ENABLE_STATIC=OFF'
+    run_cmake += ' -DLIBCXX_ENABLE_EXCEPTIONS=ON'
+
+    # EXPLAIN: for windows-itanium we want to use the MS supplied runtime.
+    run_cmake += ' -DLIBCXX_USE_COMPILER_RT=OFF'
+
+    # EXPLAIN: turn threads off for now - hopefully simplifies the build.
+    # TODO: enable threads
+    run_cmake += ' -DLIBCXX_ENABLE_THREADS=OFF'
+
+    # EXPLAIN: _LIBCPP_HAS_THREAD_API_WIN32 can't be set if -DLIBCXXABI_ENABLE_THREADS" is "OFF"
+    # run_cmake += ' -DLIBCXX_HAS_WIN32_THREAD_API=ON'
+
+    run_cmake += ' -DLIBCXX_ENABLE_MONOTONIC_CLOCK=ON'
+    run_cmake += ' -DLIBCXX_SUPPORTS_STD_EQ_CXX11_FLAG=ON'
+    run_cmake += ' -DLIBCXX_HAVE_CXX_ATOMICS_WITHOUT_LIB=ON'
+    run_cmake += ' -DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=OFF'
+    run_cmake += ' -DLIBCXX_ENABLE_FILESYSTEM=OFF'
+    run_cmake += ' -DLIBCXX_ENABLE_STATIC_ABI_LIBRARY=ON'
+    run_cmake += ' -DLIBCXX_CXX_ABI=libcxxabi'
+    run_cmake += ' -DLIBCXX_CXX_ABI_INCLUDE_PATHS=../../libcxxabi/include'
+    run_cmake += ' -DLIBCXX_CXX_ABI_LIBRARY_PATH=../../libcxxabi/{}/lib'.format(build_dirs_name)
+    run_cmake += ' -DLIBCXX_LIBDIR_SUFFIX=""'
+    run_cmake += ' -DLIBCXX_INCLUDE_TESTS=OFF'
+    run_cmake += ' -DLIBCXX_ENABLE_ABI_LINKER_SCRIPT=OFF'
+
+    # EXPLAIN: Try to remove any dependency on the VC runtime - we
+    #          need libc++abi to supply the C++ runtime.
+    run_cmake += ' -DLIBCXX_NO_VCRUNTIME=ON'
+
+    # EXPLAIN: As we are statically linking against libcxxabi we need
+    #          to link against the unwind import library to resolve
+    #          unwind references from the libcxxabi objects.
+    libcxx_c_flags  = ' {}'.format(os.path.join(install_path, 'lib', 'unwind.lib'))
+
+    # EXPLAIN: force this to be undefined to prevent the inclusion of sys/time that MS doesn't provide.
+    libcxx_c_flags += ' -UCLOCK_REALTIME'
+
+    # EXPLAN: I have set it on the basis that the mingw build sets this flag to work around COMDAT
+    #         problems and we see the same comdat errors from lld when linking.
+    # TODO: work out if this is the correct thing to be doing here!
+    libcxx_c_flags  += ' -Xlinker /force:multiple'
+
+    run_cmake += ' -DCMAKE_CXX_FLAGS=\"{} {}\"'.format(common_clang_options, libcxx_c_flags)
+    run_cmake += ' -DCMAKE_C_FLAGS=\"{} {}\"'.format(common_clang_options, libcxx_c_flags)
+
+    # EXPLAIN: Don't run the linker on CMAKE compiler check.
+    run_cmake += ' -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY'
+
+    run_cmake += ' ..'
+
+    run_cmd(['set PATH={};%PATH%'.format(toolchain_bin_path),
+             cd_build_dir,
+             run_cmake,
+             'ninja -v',
+             'ninja install'],
+             'libcxx.bat')
+
+cxxabi_build_dir = os.path.join('libcxxabi', build_dirs_name)
+
+def build_cxxabi():
+    if not os.path.exists(cxxabi_build_dir):
+      os.mkdir(cxxabi_build_dir)
+
+    cd_build_dir = '@cd {}'.format(cxxabi_build_dir)
+
+    run_cmake =  'cmake'
+    run_cmake += ' --debug-trycompile'
+    run_cmake += ' -G \"Ninja\"'
+    run_cmake += ' -DCMAKE_MAKE_PROGRAM=ninja'
+    run_cmake += ' -DCMAKE_BUILD_TYPE=Release'
+    run_cmake += ' -DCMAKE_INSTALL_PREFIX=\"{}\"'.format(install_path)
+    run_cmake += ' -DCMAKE_C_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CXX_COMPILER=clang'
+    run_cmake += ' -DCMAKE_CROSSCOMPILING=ON'
+    run_cmake += ' -DCMAKE_SYSTEM_NAME=Windows'
+    run_cmake += ' -DCMAKE_C_COMPILER_WORKS=ON'
+    run_cmake += ' -DCMAKE_CXX_COMPILER_WORKS=ON'
+    run_cmake += ' -DLLVM_PATH=\"{}\"'.format(llvm_dir)
+    run_cmake += ' -DLLVM_COMPILER_CHECKED=ON'
+    run_cmake += ' -DCMAKE_AR=\"{}\"'.format(ar_bin)
+    run_cmake += ' -DCMAKE_RANLIB=\"{}\"'.format(ranlib_bin)
+    run_cmake += ' -DLIBCXXABI_ENABLE_EXCEPTIONS=ON'
+
+    # EXPLAIN: we want to use the MS CRT.
+    run_cmake += ' -DLIBUNWIND_USE_COMPILER_RT=OFF'
+
+    # EXPLAIN: turn threads off - hopefully simplifies the build.
+    # TODO: enable threads
+    run_cmake += ' -DLIBCXXABI_ENABLE_THREADS=OFF'
+
+    run_cmake += ' -DLIBCXXABI_ENABLE_SHARED=OFF'
+    run_cmake += ' -DLIBCXXABI_ENABLE_STATIC=ON'
+
+    # EXPLAIN: Add visibility annotations as if building a dll.
+    #          This cmake option does not exist upstream. See
+    #          comments in code changes for details.
+    run_cmake += ' -LIBCXXABI_ENABLE_STATIC_FOR_LIBCXX_SHARED=ON'
+
+    run_cmake += ' -DLIBCXXABI_LIBCXX_INCLUDES=../../libcxx/include'
+    run_cmake += ' -DLIBCXXABI_LIBDIR_SUFFIX=""'
+    run_cmake += ' -DLIBCXXABI_ENABLE_NEW_DELETE_DEFINITIONS=ON'
+    run_cmake += ' -DCXX_SUPPORTS_CXX_STD=ON'
+
+    # EXPLAIN: Unless we force this off then cxxabi will contain
+    #          references to threading symbols that will not be defined
+    #          by libc++ e.g:
+    #            lld-link: error: undefined symbol: std::__1::__libcpp_thread_yield()
+    #            >>> referenced by c++abi.lib(cxa_default_handlers.cpp.obj):(std::__1::__libcpp_timed_backoff_policy::operator()(std::__1::chrono::duration<long long, std::__1::ratio<1ll, 1000000000ll> >) const)
+    #            >>> referenced by c++abi.lib(cxa_demangle.cpp.obj)
+    #            >>> referenced by c++abi.lib(cxa_exception_storage.cpp.obj)
+    libcxxabi_c_flags =  ' -D_LIBCPP_HAS_NO_THREADS'
+
+    # EXPLAIN: Use -Xclang -triple rather than -target to avoid: clang: error: invalid linker name in argument '-fuse-ld=lld-link'
+    run_cmake += ' -DCMAKE_CXX_FLAGS=\"{} {}\"'.format(libcxxabi_c_flags, common_clang_options)
+    run_cmake += ' -DCMAKE_C_FLAGS=\"{} {}\"'.format(libcxxabi_c_flags, common_clang_options)
+
+    run_cmake += ' ..'
+
+    run_cmd(['set PATH={};%PATH%'.format(toolchain_bin_path),
+             cd_build_dir,
+             run_cmake,
+             'ninja -v'],
+             'libcxxabi.bat')
+
+def write_driver_script():
+    script  = get_vcvars_cmd()
+    script += '\nset PATH={};%PATH%\n'.format(os.path.join(install_path, 'bin'))
+    script += '{} ^\n'.format(os.path.join(toolchain_bin_path, 'clang.exe'))
+    script += ' -fuse-ld=lld-link ^\n'
+    script += ' -Xclang -triple -Xclang x86_64-unknown-windows-itanium ^\n'
+    script += ' -fsjlj-exceptions ^\n'
+    script += ' -isystem {} ^\n'.format(os.path.join(install_path, 'include'))
+    script += ' -isystem {} ^\n'.format(os.path.join(install_path, 'include', 'c++', 'v1'))
+    script += ' -lkernel32 ^\n'
+    script += ' -luser32 ^\n'
+    script += ' -lgdi32 ^\n'
+    script += ' -lwinspool ^\n'
+    script += ' -lshell32 ^\n'
+    script += ' -lole32 ^\n'
+    script += ' -loleaut32 ^\n'
+    script += ' -luuid ^\n'
+    script += ' -lcomdlg32 ^\n'
+    script += ' -ladvapi32 ^\n'
+    script += ' -loldnames ^\n'
+    script += ' {} ^\n'.format(os.path.join(install_path, 'lib', 'c++.lib'))
+    script += ' {} ^\n'.format(os.path.join(install_path, 'lib', 'unwind.lib'))
+    script += ' -D_LIBCPP_HAS_NO_THREADS ^\n'
+    script += ' -Wl,/nodefaultlib ^\n'
+    script += ' -lmsvcrt ^\n'
+    script += ' -lucrt ^\n'
+    script += ' -lvcruntime ^\n'
+    script += ' %*'
+    
+    with open('{}'.format(os.path.join(install_path, 'bin', 'clang-wi.bat')), 'w') as file:
+        file.write(script)
+
+def test_hello_world():
+    test_path = os.path.join(install_path, 'test')
+    if not os.path.exists(test_path):
+        os.mkdir(test_path)
+    with open('{}'.format(os.path.join(test_path, 'hello.cpp')), 'w') as file:
+        file.write('#include <iostream>\nint main() {std::cout << "Hello!";}')
+
+    run_cmd(['@set PATH={};%PATH%'.format(toolchain_bin_path),
+             '@cd {}'.format(test_path),
+             'call {} hello.cpp -o hello.exe'.format(os.path.join(install_path, 'bin', 'clang-wi.bat')),
+             'hello.exe >NUL',
+             '@hello.exe | findstr "Hello!" || (echo "hello.exe UNSUCCESSFUL: did not print: Hello!" && exit 1)',
+             '@echo "hello.exe SUCCESSFUL!'],
+             'test.bat')
+
+if '--clean-libs' in sys.argv or '--clean-all' in sys.argv:
+    remove_file('toolchain.bat')
+    remove_file('libunwind.bat')
+    remove_file('libcxxabi.bat')
+    remove_file('libcxx.bat')
+    remove_file('test.bat')
+    remove_file(os.path.join(install_libs_path, 'unwind.lib'))
+    remove_file(os.path.join(install_bin_path, 'unwind.dll'))
+    remove_file(os.path.join(install_libs_path, 'c++.lib'))
+    remove_file(os.path.join(install_bin_path, 'c++.dll'))
+    remove_dir(unwind_build_dir)
+    remove_dir(cxxabi_build_dir)
+    remove_dir(cxx_build_dir)
+
+if '--clean-all' in sys.argv:
+    remove_dir(toolchain_build_path)
+    remove_dir(install_path)
+
+if '--debug' in sys.argv:
+    # EXPLAIN: produce more debugging output from the linker in the log files.
+    common_clang_options  += ' -Xlinker /verbose'
+
+    # EXPLAIN: Log which include files were pulled in.
+    common_clang_options  += ' -H'
+
+    # EXPLAIN: Save the preprocessed output files next to the object files
+    #          Note: I have seen this option cause COMDAT errors to be reported.
+    #                It seems that source -> assembly -> obj may report comdat
+    #                errors (when assembling) that are not reported doing source-> obj.
+    # TODO: Investigate these comdat errors.
+    common_clang_options  += ' -save-temps=obj'
+
+build_clang_and_lld()
+build_unwind()
+build_cxxabi()
+build_cxx()
+write_driver_script()
+test_hello_world()