This is an archive of the discontinued LLVM Phabricator instance.

[windows-itanium] [WIP] Windows Itanium Build Recipe
Needs ReviewPublic

Authored by bd1976llvm on Sep 22 2020, 5:21 PM.

Details

Reviewers
compnerd
smeenai
Summary

Script and patches to automate the building of a Windows Itanium environment.

Diff Detail

Event Timeline

bd1976llvm created this revision.Sep 22 2020, 5:21 PM
bd1976llvm requested review of this revision.Sep 22 2020, 5:21 PM
bd1976llvm edited the summary of this revision. (Show Details)

Removed the code patch and --patch functionality. These are not needed given that this review *is* a patch and having a patch in a patch is confusing.

bd1976llvm edited the summary of this revision. (Show Details)Sep 23 2020, 7:54 AM
mstorsjo added inline comments.Sep 24 2020, 12:59 AM
libcxx/src/CMakeLists.txt
68

I guess this could be made shorter as

list(APPEND LIBCXX_SOURCES
    support/win32/locale_win32.cpp
    support/win32/support.cpp)
if (LIBCXX_ENABLE_THREADS)
    list(APPEND LIBCXX_SOURCES
        support/win32/thread_win32.cpp)
endif()
199

Can this be made an elseif (MSVC) or similar? (Do windows-itanium environments have MSVC set in cmake?) Because for MinGW targets, the syntax below should still be used.

EDIT: From reading further here, I see that it should be covered by "${CMAKE_CXX_SIMULATE_ID}" STREQUAL "MSVC" at least.

CMAKE improvements.

bd1976llvm marked 2 inline comments as done.EditedSep 24 2020, 5:44 PM

Hi mstorsjo, thanks for looking at this. Please add anyone else to this review that might be interested.

Some questions:

  • One thing I don't understand is why I had to edit locale_win32.h to avoid collisions with MS sdk definitions but mingw doesn't have to?
  • I initially wanted to use link.exe but it doesn't like the COMDATs emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?
  • A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

libcxx/src/CMakeLists.txt
199

Thanks - of course mingw is also "WIN32". This should definitely be for "MSVC"like toolchains.

Do windows-itanium environments have MSVC set in cmake?

They shouldn't; however, until the clang driver code is improved, I am hijacking (forcing the triple with the -triple cc1 layer option) the msvc driver - this choice reduces the amount of options I have to set via cmake flags.

mstorsjo added a subscriber: smeenai.

Hi mstorsjo, thanks for looking at this. Please add anyone else to this review that might be interested.

Adding @smeenai who I think also is familiar with the windows-itanium target.

Some questios:

  • One thing I don't understand is why I had to edit locale_win32.h to avoid collisions with MS sdk definitions but mingw doesn't have to?

I don't know why you need to do that either - I can also build libcxx with clang-cl for the normal MSVC target without having to edit anything. Or maybe it's specific to the windows-itanium case, because for the normal libcxx/msvc configuration, it still uses the new/delete operators from MSVC (i.e. not using libcxxabi).

  • I initially wanted to use link.exe but it doesn't like the COMDAT's emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?

Are those the leader-less comdats for xdata/pdata, with a trailing function name, like .text$myfunc, .xdata$myfunc, .pdata$myfunc? That's not a problem with mingw, as that's exactly the mingw specific behaviour here. (GNU ld only handles this form, not the proper COFF associative comdats - llvm generates this form to be interoperable with GNU ld for mingw targets, and lld can handle it fine as well.)

I don't really see why you'd be getting them though; this is controlled by https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCStreamer.cpp#L783-L794, based on a flag set at https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCAsmInfoCOFF.cpp#L58-L62, so unless the itanium target uses MCAsmInfo derived from MCAsmInfoGNUCOFF (as opposed to directly from MCAsmInfoCOFF or from MCAsmInfoMicrosoft), I don't see how you'd end up with these. (Or are there any lowlevel bits in libcxxabi that ends up forcing this kinds of comdats?)

  • A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

Not sure - maybe?

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

Yes, and that was one of the driving forces for completing the auto-importing feature. As long as you're only linking statically it should work just fine.

In https://reviews.llvm.org/D43184 I experimented with adding an option to clang, to generate constructors that would run at runtime, fixing this up, but it wasn't very pretty. As the autoimport feature was useful for other cases, and is how GNU tools handle it, I went with that approach.

So in principle, as far as I understand it, the itanium C++ ABI is incompatible with windows with dynamic linking, unless you have an autoimport feature in the linker that can fix it up.

bd1976llvm marked an inline comment as done.Sep 28 2020, 5:09 PM

I don't know why you need to do that either - I can also build libcxx with clang-cl for the normal MSVC target without having to edit anything. Or maybe it's specific to the windows-itanium case, because for the normal libcxx/msvc configuration, it still uses the new/delete operators from MSVC (i.e. not using libcxxabi).

I think this answers the question. It is specific to the windows-itanium case. The clang-cl build wants libcxx backed by the MS VS CRT. However, with windows-itanium, we only want the "C" parts of the MS VS CRT - with the C++ parts supplied by libcxxabi. Unfortunately, the MS VS CRT headers are not factored cleanly into C and C++ paths and it is easy to pull in unwanted headers which is what I found was occurring here. Mingw supplies its own headers and so it wouldn't be affected by this, IIUC.

  • I initially wanted to use link.exe but it doesn't like the COMDAT's emitted by LLVM (lld-link is happy). Does mingw hit these COMDAT problems?

Are those the leader-less comdats for xdata/pdata, with a trailing function name, like .text$myfunc, .xdata$myfunc, .pdata$myfunc? That's not a problem with mingw, as that's exactly the mingw specific behaviour here. (GNU ld only handles this form, not the proper COFF associative comdats - llvm generates this form to be interoperable with GNU ld for mingw targets, and lld can handle it fine as well.)

I don't really see why you'd be getting them though; this is controlled by https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCStreamer.cpp#L783-L794, based on a flag set at https://github.com/llvm/llvm-project/blob/master/llvm/lib/MC/MCAsmInfoCOFF.cpp#L58-L62, so unless the itanium target uses MCAsmInfo derived from MCAsmInfoGNUCOFF (as opposed to directly from MCAsmInfoCOFF or from MCAsmInfoMicrosoft), I don't see how you'd end up with these. (Or are there any lowlevel bits in libcxxabi that ends up forcing this kinds of comdats?)

  • A problem with this build is that I see lots of multiply defined symbol errors. I think these are related to the COMDAT problems?

Not sure - maybe?

Thanks for the pointer to leaderless comdats. I need to investigate this more. The problems I am seeing seem to be from comdats that are associated with the use of the "__inline" keyword.

There is another comdat related problem which is that if I compile straight to a coff file it works. However, if I compile to asm then when I try to assemble that asm I get errors like this:

src/CMakeFiles/cxx_shared.dir\locale.s:5787:2: error: section '.xdata$' is already linkonce
      .linkonce       discard

I also discovered a problem today that typeinfo's reference the "vtable for cxxabiv1::class_type_info". However, that symbol is in an import library and MC doesn't emit a relocation against an "__impl_" prefixed symbol name for the reference so the link fails:

lld-link: error: undefined symbol: vtable for cxxabiv1::class_type_info

referenced by C:\Users\BDUNBO~1\AppData\Local\Temp\plugin-8ae1e2.o:(typeinfo for Foo)

I can resolve this by using the auto-importing feature you added to lld-link. Have you hit this problem with mingw?

Yes, and that was one of the driving forces for completing the auto-importing feature. As long as you're only linking statically it should work just fine.

In https://reviews.llvm.org/D43184 I experimented with adding an option to clang, to generate constructors that would run at runtime, fixing this up, but it wasn't very pretty. As the autoimport feature was useful for other cases, and is how GNU tools handle it, I went with that approach.

So in principle, as far as I understand it, the itanium C++ ABI is incompatible with windows with dynamic linking, unless you have an autoimport feature in the linker that can fix it up.

Thanks! I had worked some of this out on my own - however these pointers really helped to understand the problem.

Use lld's mingw autolinking feature. Add a testcase that will fail without this.

Added LIT based tests.

Incorporated improvements discussed in https://reviews.llvm.org/D89518.

Added type_info tests.

@mstorsjo I'm looking at this again after the winter break. Do you have any further comments?

mstorsjo added inline comments.Jan 7 2021, 4:11 AM
libcxx/include/CMakeLists.txt
242

IMO this whole if/else could be dropped, as cl and clang-cl can take options starting both with dashes and slashes. But I guess that's orthogonal to the change at hand here (and it's essential for the change below), so I guess it looks ok.

libcxx/include/support/win32/locale_win32.h
15

This change breaks mingw builds, as the exact contents of corecrt.h and xlocinfo.h differs a bit there (I end up with errors like this):

include/c++/v1/__locale:134:20: error: use of undeclared identifier '_M_COLLATE'
        collate  = LC_COLLATE_MASK,
                   ^
include/c++/v1/support/win32/locale_win32.h:18:25: note: expanded from macro 'LC_COLLATE_MASK'
#define LC_COLLATE_MASK _M_COLLATE
                        ^
libcxx/src/CMakeLists.txt
63

This change looks like it could be split out and landed separately from the rest?

199

On this one, I'm a bit unsure here in which cases cmake calls the compiler driver for linking (requiring -Wl) and when it just calls (lld-)link directly - but if this is what's needed and works for you, then sure. As this is within an if (MSVC) so it shouldn't affect my builds in any case.

libcxxabi/CMakeLists.txt
262

This bit has changed and been improved upstream, so I'd suggest you rebase it and see if this change really is needed any longer.

libcxxabi/src/cxa_guard_impl.h
44

This can't really proceed in this form, you need to fix it properly somehow.

bd1976llvm marked 5 inline comments as done.Jan 22 2021, 3:58 AM

Thanks for your comments. I tried to do a rebase right up to the head of main - but it proved very difficult to resolve all the problems this caused. Instead I have rebased to your https://github.com/llvm/llvm-project/commit/8a73aa8c4c3ee82e7d650c6a957937713a4a9ab4

There are a few pieces that I can upstream but I think the rest of the changes are due to the working around the lack of a proper driver for Windows Itanium and modifications for trying to build using only the MS sdk excluding any VS runtime stuff.

I am actually not looking to commit this build script to LLVM. I am happy to upstream the changes which make sense and close this review - this build script will still be available on phabricator as an example.

I would like to commit the documentation change in https://reviews.llvm.org/D89518 though.

Thanks!

libcxx/include/support/win32/locale_win32.h
15

Yes. If you look at the block below I had to fish these defines out of xlocinfo.h header. I'm not sure if there is another way to solve this problem. Obviously, this is not something that can be upstream.

libcxx/src/CMakeLists.txt
63

Yes. Thanks.

199

Do you think this part might be acceptable upstream on its own?

libcxxabi/CMakeLists.txt
262

Thanks :)

libcxxabi/src/cxa_guard_impl.h
44

Turned out I was incorrect here (my comment had rotted I think) unistd.h just isn't available in the MS sdk. I think that this part can be landed in a separate change?

bd1976llvm marked 4 inline comments as done.

rebased build script and some minor improvements.

mstorsjo added inline comments.Jan 22 2021, 4:06 AM
libcxx/src/CMakeLists.txt
199

Hmm, unsure - if you try to upstream it, it needs a thorough explanation of how the linker is called in your case compared to other cases (calling it via the compiler driver vs calling *link directly) at least.

libcxxabi/src/cxa_guard_impl.h
44

Right - this one works in mingw because unistd.h exists there, and we don't build libcxxabi for MSVC targets otherwise, normally, so something like __has_include() for it might work.

I'm not entirely sure what unistd.h is used for here though - the file builds fine without it for mingw targets at least, so putting it wihtin #ifndef _WIN32 or something like that might work as well - or check the contents of the source file to see if it's just conditionally needed with some other condition.

bd1976llvm marked 3 inline comments as done.Jan 22 2021, 3:40 PM
bd1976llvm added inline comments.
libcxx/src/CMakeLists.txt
199

Actually, this is another part that is really part of how I have fudged things to get everything to work without a proper driver. I don't think it is suitable for upstreaming.

libcxxabi/CMakeLists.txt
437

Do you know why this part isn't required for mingw? I definitely to make these match for my builds.

libcxxabi/src/cxa_guard_impl.h
44

Thanks - I'll look into it after the weekend.

bd1976llvm marked 2 inline comments as done.

Added the patch from https://reviews.llvm.org/D93203 and the scei vendor variant of windows itanium to the relevant tests.

mstorsjo added inline comments.Jan 22 2021, 10:27 PM
libcxxabi/CMakeLists.txt
437

Mingw headers don't have those concepts at all. If libcxx sets such defines, we do need to match it here (and within the tests) - so I think this is upstreamable - just try to match the libcxx logic.

bd1976llvm marked an inline comment as done.Feb 1 2021, 5:53 PM
bd1976llvm added inline comments.
libcxxabi/CMakeLists.txt
437

Thanks - I gave this a go but I got stuck trying to get the libcxx testing running for a clang-cl build of libcxx. I still intend to upstream - just need the clang-cl build sorted first!

mstorsjo added inline comments.Feb 1 2021, 11:38 PM
libcxxabi/CMakeLists.txt
437

Ah, I see.

FWIW, I'm planning on writing up some docs on how to test libcxx on windows - I have one version of how to do that at https://github.com/mstorsjo/llvm-project/blob/e5fb5c31ae09b2bafab876c0b539d84668713715/libcxx/test-instructions.txt.

Currently lots of libcxx tests fail due to other minor incompatibilities - I'm planning on trying to upstream workarounds for that soon at the same time too. If you run into issues, try picking commits from https://github.com/mstorsjo/llvm-project/commits/999b8b75867bb7f92eecc027ade9c1a7574dde65 if you run into issues. It's a bit mixed in with other (less upstreamable I think) fixes for cross-testing and other uncommon build configurations.

bd1976llvm marked an inline comment as done.Feb 17 2021, 5:59 AM
bd1976llvm added inline comments.
libcxxabi/CMakeLists.txt
437

Thanks very much. Still looking at this but slow progress due to other priorities.

smeenai resigned from this revision.Apr 28 2022, 2:05 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 28 2022, 2:05 PM

Set DLL storage class for guard variables correctly to avoid a link error when code using a guard variable is inlined into a module that imports the code.