[MinGW][clang-shlib] Build by default on MinGW

[mati865]

Summary

It builds without errors and makes possible to use CLANG_LINK_CLANG_DYLIB=1.

[mstorsjo]

This should be ok, I think.

[ASDenysPetrov]

Hi, @mati865
Currently I got an error while building with GCC10 on Win10.

[2325/2656] Linking CXX shared library bin\libclang-cpp.dll
FAILED: bin/libclang-cpp.dll lib/libclang-cpp.dll.a
cmd.exe /C "cd . && D:\WORK\Utilities\MSYS2\mingw64\bin\c++.exe -Wa,-mbig-obj -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -fno-common -Woverloaded-virtual -fno-strict-aliasing  -O2   -shared -o bin\libclang-cpp.dll -Wl,--out-implib,lib\libclang-cpp.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @tools\clang\tools\clang-shlib\CMakeFiles\clang-cpp.rsp  && cd ."
D:/WORK/Utilities/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: error: export ordinal too large: 66444
collect2.exe: error: ld returned 1 exit status
[2328/2656] Building CXX object tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndex.cpp.obj
ninja: build stopped: subcommand failed.

I've tryed this on two PCs. Could you, please, revise you solution. I'm not familiar with this part but this affects me (and I think other Win-people).

[mati865]

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

[mstorsjo]

In D87547#2322325, @mati865 wrote:

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

[ASDenysPetrov]

In D87547#2322325, @mati865 wrote:

@ASDenysPetrov what CMake options do you use?
I'm unable to reproduce this error with Clang+LLD and GCC+Binutils from MSYS2.

Sorry for not mentioning that. This is my cmd:

cmake -GNinja ../llvm -DLLVM_LIT_ARGS=-sv -DLLVM_ENABLE_PROJECTS=clang -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../install -DLLVM_ENABLE_ASSERTIONS=ON

I've tried GCC8 and GCC10 versions.

[mati865]

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

60995 with cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_IGNORE_PATH=/usr/lib -DLLVM_ENABLE_PROJECTS="clang;lld" ../llvm-project/llvm so it's very close.
Another build where I don't have exact command line but it uses Clang+LLD and LLVM_LINK_LLVM_DYLIB:BOOL=ON has only 35125 exports.

I'll try to dig a bit soonish (maybe enable clang-shlib only if LLVM is linked dynamically?) but you can revert it in meantime.

[mstorsjo]

In D87547#2322643, @mati865 wrote:

How many exports does your DLL end up with - is it close to the 64k limit, or far from it? Is there maybe some other library that is linked in, that increases the number of exported symbols (=all), or does this DLL only export explicitly intended symbols with dllexport?

60995 with cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_IGNORE_PATH=/usr/lib -DLLVM_ENABLE_PROJECTS="clang;lld" ../llvm-project/llvm so it's very close.
Another build where I don't have exact command line but it uses Clang+LLD and LLVM_LINK_LLVM_DYLIB:BOOL=ON has only 35125 exports.

I'll try to dig a bit soonish (maybe enable clang-shlib only if LLVM is linked dynamically?) but you can revert it in meantime.

I looked at my recent cross-builds (built with llvm-mingw, i.e. libc++) with -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_TARGETS_TO_BUILD="ARM;AArch64;X86", and I'm seeing ~64000 exports.

[mati865]

Reproduced by adding -DLLVM_ENABLE_ASSERTIONS=ON.

With -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON libLLVM.dll also fails to link.

[mati865]

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

[mstorsjo]

In D87547#2322900, @mati865 wrote:

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

Older versions of lld didn't error out if exceeding the limit, this is fixed in D86701, unfortunately after the 11 branch.

[mati865]

Older versions of lld didn't error out if exceeding the limit, this is fixed in D86701, unfortunately after the 11 branch.

Thanks, I'll backport it for MSYS2 LLVM 11 package.

With -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_RANLIB_COMPILER=llvm-ranlib -DCMAKE_AR_COMPILER=llvm-ar -DCMAKE_LINKER=ld.lld -DLLVM_ENABLE_LLD=ON libclang-cpp.dll has 65535 exports, seems it got truncated?

After adding -DLLVM_BUILD_LLVM_DYLIB=ON libclang-cpp.dll has 37660 exports, libLLVM.dll gets truncated.
Considering my previous findings linking libclang-cpp.dll to libLLVM.dll seems to reduce amount of exports to safer 30k-40k range.

Possible solutions:

• revert this diff so I carry downstream MSYS2 patch (easy)
• gate building of libclang-cpp.dll for MinGW on LLVM_BUILD_LLVM_DYLIB=ON (easy)
• export less symbols from libclang-cpp.dll (probably hard or very hard)

[mstorsjo]

Out of these, the second (only enable on mingw if libllvm is available) sounds sensible to me. Or maybe even make it into a cmake option?

[mati865]

Or maybe even make it into a cmake option?

I've considered it with default to true for unix and false for others but I'm not sure if complicating Clang build options like that is worth it.

[mstorsjo]

In D87547#2323605, @mati865 wrote:

Or maybe even make it into a cmake option?

I've considered it with default to true for unix and false for others but I'm not sure if complicating Clang build options like that is worth it.

Yeah, it does sound a bit like overkill, so something like what you suggested above probably is sanest then.

[mati865]

Opened https://reviews.llvm.org/D89225

clang/tools/CMakeLists.txt

 add_clang_subdirectory(clang-offload-bundler)
 add_clang_subdirectory(clang-offload-wrapper)
 add_clang_subdirectory(clang-scan-deps)
 
 add_clang_subdirectory(c-index-test)
 
 add_clang_subdirectory(clang-rename)
 add_clang_subdirectory(clang-refactor)
-if(UNIX)
+if(UNIX OR MINGW)
   add_clang_subdirectory(clang-shlib)
 endif()
 
 if(CLANG_ENABLE_ARCMT)
   add_clang_subdirectory(arcmt-test)
   add_clang_subdirectory(c-arcmt-test)
 endif()