This is an archive of the discontinued LLVM Phabricator instance.

[MinGW][clang-shlib] Build by default on MinGW
AbandonedPublic

Authored by mati865 on Sep 11 2020, 12:59 PM.

Details

Summary

It builds without errors and makes possible to use CLANG_LINK_CLANG_DYLIB=1.

Diff Detail

Event Timeline

mati865 created this revision.Sep 11 2020, 12:59 PM
mati865 requested review of this revision.Sep 11 2020, 12:59 PM
mati865 edited the summary of this revision. (Show Details)
mstorsjo accepted this revision.Sep 11 2020, 10:54 PM

This should be ok, I think.

This revision is now accepted and ready to land.Sep 11 2020, 10:54 PM
This revision was automatically updated to reflect the committed changes.
ASDenysPetrov reopened this revision.Oct 9 2020, 7:01 AM
ASDenysPetrov added a subscriber: ASDenysPetrov.

Hi, @mati865
Currently I got an error while building with GCC10 on Win10.

[2325/2656] Linking CXX shared library bin\libclang-cpp.dll
FAILED: bin/libclang-cpp.dll lib/libclang-cpp.dll.a
cmd.exe /C "cd . && D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing  -O2   -shared -o bin\libclang-cpp.dll -Wl,--out-implib,lib\libclang-cpp.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @tools\clang\tools\clang-shlib\CMakeFiles\clang-cpp.rsp  && cd ."
D:/WORK/Utilities/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: error: export ordinal too large: 66444
collect2.exe: error: ld returned 1 exit status
[2328/2656] Building CXX object tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndex.cpp.obj
ninja: build stopped: subcommand failed.

I've tryed this on two PCs. Could you, please, revise you solution. I'm not familiar with this part but this affects me (and I think other Win-people).

This revision is now accepted and ready to land.Oct 9 2020, 7:01 AM
ASDenysPetrov requested changes to this revision.Oct 9 2020, 7:01 AM
This revision now requires changes to proceed.Oct 9 2020, 7:01 AM

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

ASDenysPetrov added a comment.EditedOct 9 2020, 12:26 PM

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

Sorry for not mentioning that. This is my cmd:

cmake -GNinja ../llvm -DLLVM_LIT_ARGS=-sv -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../install -DLLVM_ENABLE_ASSERTIONS=ON

I've tried GCC8 and GCC10 versions.

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

60995 with cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_IGNORE_PATH=/usr/lib -DLLVM_ENABLE_PROJECTS="clang;lld" ../llvm-project/llvm so it's very close.
Another build where I don't have exact command line but it uses Clang+LLD and LLVM_LINK_LLVM_DYLIB:BOOL=ON has only 35125 exports.

I'll try to dig a bit soonish (maybe enable clang-shlib only if LLVM is linked dynamically?) but you can revert it in meantime.

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

60995 with cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_IGNORE_PATH=/usr/lib -DLLVM_ENABLE_PROJECTS="clang;lld" ../llvm-project/llvm so it's very close.
Another build where I don't have exact command line but it uses Clang+LLD and LLVM_LINK_LLVM_DYLIB:BOOL=ON has only 35125 exports.

I'll try to dig a bit soonish (maybe enable clang-shlib only if LLVM is linked dynamically?) but you can revert it in meantime.

I looked at my recent cross-builds (built with llvm-mingw, i.e. libc++) with -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_TARGETS_TO_BUILD="ARM;AArch64;X86", and I'm seeing ~64000 exports.

Reproduced by adding -DLLVM_ENABLE_ASSERTIONS=ON.

With -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON libLLVM.dll also fails to link.

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

Older versions of lld didn't error out if exceeding the limit, this is fixed in D86701, unfortunately after the 11 branch.

Older versions of lld didn't error out if exceeding the limit, this is fixed in D86701, unfortunately after the 11 branch.

Thanks, I'll backport it for MSYS2 LLVM 11 package.

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

After adding -DLLVM_BUILD_LLVM_DYLIB=ON libclang-cpp.dll has 37660 exports, libLLVM.dll gets truncated.
Considering my previous findings linking libclang-cpp.dll to libLLVM.dll seems to reduce amount of exports to safer 30k-40k range.

Possible solutions:

  • revert this diff so I carry downstream MSYS2 patch (easy)
  • gate building of libclang-cpp.dll for MinGW on LLVM_BUILD_LLVM_DYLIB=ON (easy)
  • export less symbols from libclang-cpp.dll (probably hard or very hard)

Out of these, the second (only enable on mingw if libllvm is available) sounds sensible to me. Or maybe even make it into a cmake option?

mati865 added a comment.EditedOct 10 2020, 12:08 PM

Or maybe even make it into a cmake option?

I've considered it with default to true for unix and false for others but I'm not sure if complicating Clang build options like that is worth it.

Or maybe even make it into a cmake option?

I've considered it with default to true for unix and false for others but I'm not sure if complicating Clang build options like that is worth it.

Yeah, it does sound a bit like overkill, so something like what you suggested above probably is sanest then.