[LLD] Enable --no-undefined-version by default.

[danalbert]

Summary

Allowing incorrect version scripts is not a helpful default. Flip that
to help users find their bugs at build time rather than at run time.

[MaskRay]

Thanks for catching this. The new default makes more sense and aligns with the other enabled-by-default diagnostic (warn("attempt to reassign symbol linking glibc libc.so is known to trigger some warnings).
I think the blast radius is very small since projects using exact patterns in a version script are rare. Default version symbols don't need patterns while non-default version symbols are rarely used.
When a pattern does not specify a symbol, it's very clear a user error.

Please update lld/docs/ReleaseNotes.rst and flip lld/docs/ld.lld.1 to use --undefined-version instead.

[danalbert]

Update docs.

[danalbert]

Please update lld/docs/ReleaseNotes.rst and flip lld/docs/ld.lld.1 to use --undefined-version instead.

Thanks for the heads up! Done, I think? Never had to edit a man page before so no idea if that part is right :)

[MaskRay]

LGTM.

lld/docs/ld.lld.1

355

Mentioning just Fl -undefined-version is fine. For boolean options we only mention the one which is not the default.

[danalbert]

Remove man doc for default behavior.

[ronlieb]

hi, our amdgpu buildbot which uses lld has been failing since this commit landed
https://lab.llvm.org/buildbot/#/builders/193/builds/20055

[jhuber6]

In D135402#3855331, @ronlieb wrote:

hi, our amdgpu buildbot which uses lld has been failing since this commit landed
https://lab.llvm.org/buildbot/#/builders/193/builds/20055

I addressed those issues by adding --undefined-version to everywhere that used the version scripts.

I've recently been getting a few spurious build failures after this patch for. Getting rid of the following exported symbols seems to fix it, but I'm not sure if this is the correct solution or these are spuriously undefined.

diff --git a/llvm/tools/lto/lto.exports b/llvm/tools/lto/lto.exports
index 3abae5f0fcba..4164c3919a97 100644
--- a/llvm/tools/lto/lto.exports
+++ b/llvm/tools/lto/lto.exports
@@ -45,12 +45,6 @@ lto_codegen_compile_optimized
 lto_codegen_set_should_internalize
 lto_codegen_set_should_embed_uselists
 lto_set_debug_options
-LLVMCreateDisasm
-LLVMCreateDisasmCPU
-LLVMDisasmDispose
-LLVMDisasmInstruction
-LLVMSetDisasmOptions
-LLVMCreateDisasmCPUFeatures
 thinlto_create_codegen
 thinlto_codegen_dispose
 thinlto_codegen_add_module

This seems to happen every time I rebuild,

[133/897] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git 
: && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=lld -Wl,--color-diagnostics   -Wl,--gc-sections  -Wl,--version-script,"/home/jhuber/Documents/llvm/llvm-project/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64AsmParser.so.16git  lib/libLLVMAMDGPUAsmParser.so.16git  lib/libLLVMARMAsmParser.so.16git  lib/libLLVMAVRAsmParser.so.16git  lib/libLLVMBPFAsmParser.so.16git  lib/libLLVMMipsAsmParser.so.16git  lib/libLLVMMSP430AsmParser.so.16git  lib/libLLVMPowerPCAsmParser.so.16git  lib/libLLVMRISCVAsmParser.so.16git  lib/libLLVMSparcAsmParser.so.16git  lib/libLLVMSystemZAsmParser.so.16git  lib/libLLVMVEAsmParser.so.16git  lib/libLLVMWebAssemblyAsmParser.so.16git  lib/libLLVMX86AsmParser.so.16git  lib/libLLVMAArch64CodeGen.so.16git  lib/libLLVMAMDGPUCodeGen.so.16git  lib/libLLVMARMCodeGen.so.16git  lib/libLLVMAVRCodeGen.so.16git  lib/libLLVMBPFCodeGen.so.16git  lib/libLLVMHexagonCodeGen.so.16git  lib/libLLVMLanaiCodeGen.so.16git  lib/libLLVMMipsCodeGen.so.16git  lib/libLLVMMSP430CodeGen.so.16git  lib/libLLVMNVPTXCodeGen.so.16git  lib/libLLVMPowerPCCodeGen.so.16git  lib/libLLVMRISCVCodeGen.so.16git  lib/libLLVMSparcCodeGen.so.16git  lib/libLLVMSystemZCodeGen.so.16git  lib/libLLVMVECodeGen.so.16git  lib/libLLVMWebAssemblyCodeGen.so.16git  lib/libLLVMX86CodeGen.so.16git  lib/libLLVMXCoreCodeGen.so.16git  lib/libLLVMAVRDesc.so.16git  lib/libLLVMBPFDesc.so.16git  lib/libLLVMMipsDesc.so.16git  lib/libLLVMMSP430Desc.so.16git  lib/libLLVMNVPTXDesc.so.16git  lib/libLLVMPowerPCDesc.so.16git  lib/libLLVMSparcDesc.so.16git  lib/libLLVMVEDesc.so.16git  lib/libLLVMX86Desc.so.16git  lib/libLLVMXCoreDesc.so.16git  lib/libLLVMAArch64Disassembler.so.16git  lib/libLLVMAMDGPUDisassembler.so.16git  lib/libLLVMARMDisassembler.so.16git  lib/libLLVMAVRDisassembler.so.16git  lib/libLLVMBPFDisassembler.so.16git  lib/libLLVMHexagonDisassembler.so.16git  lib/libLLVMLanaiDisassembler.so.16git  lib/libLLVMMipsDisassembler.so.16git  lib/libLLVMMSP430Disassembler.so.16git  lib/libLLVMPowerPCDisassembler.so.16git  lib/libLLVMRISCVDisassembler.so.16git  lib/libLLVMSparcDisassembler.so.16git  lib/libLLVMSystemZDisassembler.so.16git  lib/libLLVMVEDisassembler.so.16git  lib/libLLVMWebAssemblyDisassembler.so.16git  lib/libLLVMX86Disassembler.so.16git  lib/libLLVMXCoreDisassembler.so.16git  lib/libLLVMAVRInfo.so.16git  lib/libLLVMBPFInfo.so.16git  lib/libLLVMMipsInfo.so.16git  lib/libLLVMMSP430Info.so.16git  lib/libLLVMNVPTXInfo.so.16git  lib/libLLVMPowerPCInfo.so.16git  lib/libLLVMSparcInfo.so.16git  lib/libLLVMVEInfo.so.16git  lib/libLLVMX86Info.so.16git  lib/libLLVMXCoreInfo.so.16git  lib/libLLVMLTO.so.16git  lib/libLLVMHexagonAsmParser.so.16git  lib/libLLVMLanaiAsmParser.so.16git  lib/libLLVMAArch64Desc.so.16git  lib/libLLVMAArch64Info.so.16git  lib/libLLVMAMDGPUDesc.so.16git  lib/libLLVMAMDGPUInfo.so.16git  lib/libLLVMARMDesc.so.16git  lib/libLLVMARMInfo.so.16git  lib/libLLVMHexagonDesc.so.16git  lib/libLLVMHexagonInfo.so.16git  lib/libLLVMLanaiDesc.so.16git  lib/libLLVMLanaiInfo.so.16git  lib/libLLVMRISCVDesc.so.16git  lib/libLLVMRISCVInfo.so.16git  lib/libLLVMSystemZDesc.so.16git  lib/libLLVMSystemZInfo.so.16git  lib/libLLVMWebAssemblyDesc.so.16git  lib/libLLVMWebAssemblyInfo.so.16git  lib/libLLVMMCDisassembler.so.16git  lib/libLLVMCodeGen.so.16git  lib/libLLVMTarget.so.16git  lib/libLLVMMC.so.16git  lib/libLLVMBitReader.so.16git  lib/libLLVMCore.so.16git  lib/libLLVMSupport.so.16git  -Wl,-rpath-link,/home/jhuber/Documents/llvm/llvm-project/build/lib && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined

[vzakhari]

--undefined-version is not supported by GNU ld, so now my build is failing :)

[vzakhari]

FYI, I reverted these three commits:
https://reviews.llvm.org/rG096f93e73dc3
https://reviews.llvm.org/rG3f62314c235bd2475c8e2b5b874b2932a444e823
https://reviews.llvm.org/rG7ec8b0d162e354c703f5390784287054601f9c69

[danalbert]

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

[vzakhari]

In D135402#3856895, @danalbert wrote:

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

The option addition was an attempt to resolve lld build failure reported by @ronlieb above: https://reviews.llvm.org/D135402#3855331
So I had to revert both commits so that the builds pass with both ld and lld

[jhuber6]

In D135402#3856895, @danalbert wrote:

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

The problem was that this patch put LLVM in a state that it could not be built using lld anymore. GNU ld not supporting the flag simply made it difficult to work around the regression. The problems observed were with the libomp and lto version scripts. This patch should be re-landed once those issues are addressed more directly.

[danalbert]

Okay. Unfortunately it's unrealistic that I'll ever have the time to chase every build bot, so unless someone else can volunteer to do that I don't think this default will ever improve.

[vzakhari]

I am not volunteering, but here is my thoughts.

@jhuber6, @ronlieb, do you know why the lld build started failing after this change? Is there --fatal-warnings option specified for the linker invocations?
If we remove --fatal-warnings this should allow @danalbert to reland this patch, and then the warnings can be phased out incrementally. Does it make sense?
Or, alternatively, lld default may be changed to always warn-on-undefined-symbols-in-version-script but never treat this as an error, unless --fatal-warnings and --undefined-version are both explicitly specified (to match legacy behavior). This may result in some convoluted user experience.

I have a general question though: is lld supposed to be a verbatim replacement for ld and vice versa? Is it okay to allow different user-visible behavior for the two tools?

[MaskRay]

AFAICT openmp uses an incorrect version script where some exact patterns don't match symbols. This is error-prone but before this default switch it was unnoticed.

If you use --no-undefined-version with GNU ld, you likely get similar errors.

"drop-in replacement" does not mean ld.lld copies every behavior of GNU ld. We consider many factors and what default makes the most sense.
I've made comments why the new default makes more sense and I don't want to repeat.

[jhuber6]

In D135402#3857039, @MaskRay wrote:

AFAICT openmp uses an incorrect version script where some exact patterns don't match symbols. This is error-prone but before this default switch it was unnoticed.

Yes, I originally attempted to rectify the offending version scripts. The only difficult one to rectify is the one for libomp. This is because some symbols are only defined in a debug build, or some other configuration, it contains lines like this.

#if USE_DEBUGGER
        __kmp_debugging;
        __kmp_omp_debug_struct_info;
#endif /* USE_DEBUGGER */

What would you suggest for solving this? Should we just include dummy symbols in the case that these symbols aren't included?

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

diff --git a/llvm/tools/lto/lto.exports b/llvm/tools/lto/lto.exports
index 3abae5f0fcba..4164c3919a97 100644
--- a/llvm/tools/lto/lto.exports
+++ b/llvm/tools/lto/lto.exports
@@ -45,12 +45,6 @@ lto_codegen_compile_optimized
 lto_codegen_set_should_internalize
 lto_codegen_set_should_embed_uselists
 lto_set_debug_options
-LLVMCreateDisasm
-LLVMCreateDisasmCPU
-LLVMDisasmDispose
-LLVMDisasmInstruction
-LLVMSetDisasmOptions
-LLVMCreateDisasmCPUFeatures
 thinlto_create_codegen
 thinlto_codegen_dispose
 thinlto_codegen_add_module

If you use --no-undefined-version with GNU ld, you likely get similar errors.

"drop-in replacement" does not mean ld.lld copies every behavior of GNU ld. We consider many factors and what default makes the most sense.
I've made comments why the new default makes more sense and I don't want to repeat.

I'm fine with the philosophy and this patch, but I don't want the OpenMP builds to be totally broken until we fix it.

[danalbert]

What would you suggest for solving [differing exported symbols between debug and release builds]?

Two version scripts. In Android we have similar issues, especially in places like libc that differ from ABI to ABI, and we solve this by preprocessing the version script before giving it to the linker.

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

tbh that sounds like a bug. If you're expecting symbols to be public and LTO is making that not the case (DCE?), the expected ABI is not being fulfilled. Or maybe those weren't meant to be public anyway so it doesn't actually matter and they should just be removed.

[jhuber6]

In D135402#3857064, @danalbert wrote:

What would you suggest for solving [differing exported symbols between debug and release builds]?

Two version scripts. In Android we have similar issues, especially in places like libc that differ from ABI to ABI, and we solve this by preprocessing the version script before giving it to the linker.

I was considering if preprocessing the version script was a possibility given the #if usage, basically feeding it through the compiler's preprocessor, but I wasn't sure if there was an easy way to achieve that via CMake.

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

tbh that sounds like a bug. If you're expecting symbols to be public and LTO is making that not the case (DCE?), the expected ABI is not being fulfilled. Or maybe those weren't meant to be public anyway so it doesn't actually matter and they should just be removed.

It seems like they were put there deliberately from the commit history, but it may be out of date following some other changes. It was a bizarre failure that went away if I did a completely clean build but showed up again whenever I did a pull & build so I was unsure what the issue was.

[MaskRay]

I relanded a OpenMP workaround to expect to check the fallout. But I made cherry-pick of this patch as a dependent patch of the OpenMP workaround so I ended up reland this patch as well...
Well, let's see whether the OpenMP workaround works...

[ronlieb]

amdgpu buildbot sees issues.

[ronlieb]

https://lab.llvm.org/buildbot/#/builders/193/builds/21412

[MaskRay]

In D135402#3913463, @ronlieb wrote:

https://lab.llvm.org/buildbot/#/builders/193/builds/21412

I cannot reproduce.

cmake -GNinja -Sllvm -B/tmp/out/s2-custom -DCMAKE_BUILD_TYPE=Release ${LLVM} -DCMAKE_CXX_COMPILER=/tmp/Rel/bin/clang++ -DCMAKE_C_COMPILER=/tmp/Rel/bin/clang -DCLANG_DEFAULT_LINKER=lld '-DLLVM_TARGETS_TO_BUILD=X86;AMDGPU' -DLLVM_ENABLE_RUNTIMES=openmp '-DLLVM_ENABLE_PROJECTS=clang;llvm;lld'
ninja -C /tmp/out/s2-custom check-openmp

I checked moving openmp from LLVM_ENABLE_RUNTIMES to LLVM_ENABLE_PROJECTS as well. Beside, I cannot find --version-script in the failed log. How does the build use the problematic linker script?

[ronlieb]

This cmake recipe reproduces issue for me:

/usr/bin/cmake \

-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;lld;llvm" \
-DLLVM_LIT_ARGS="-vv --show-unsupported --show-xfail -j 32" \
-DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
-DLLVM_ENABLE_ASSERTIONS=ON                        \
-DLLVM_ENABLE_RUNTIMES="openmp" \
-DLIBOMPTARGET_ENABLE_DEBUG=ON \
-DCLANG_DEFAULT_LINKER=lld                         \
../llvm -GNinja

[MaskRay]

In D135402#3913753, @ronlieb wrote:

This cmake recipe reproduces issue for me:

/usr/bin/cmake \

-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;lld;llvm" \
-DLLVM_LIT_ARGS="-vv --show-unsupported --show-xfail -j 32" \
-DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
-DLLVM_ENABLE_ASSERTIONS=ON                        \
-DLLVM_ENABLE_RUNTIMES="openmp" \
-DLIBOMPTARGET_ENABLE_DEBUG=ON \
-DCLANG_DEFAULT_LINKER=lld                         \
../llvm -GNinja

ninja check-openmp gives me

CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins/amdgpu/CMakeLists.txt:92 (add_dependencies):
  The dependency target "omptarget.devicertl.amdgpu" of target
  "omptarget.rtl.amdgpu" does not exist.


CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins/cuda/CMakeLists.txt:85 (add_dependencies):
  The dependency target "omptarget.devicertl.nvptx" of target
  "omptarget.rtl.cuda" does not exist.


CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins-nextgen/cuda/CMakeLists.txt:88 (add_dependencies):
  The dependency target "omptarget.devicertl.nvptx" of target
  "omptarget.rtl.cuda.nextgen" does not exist.


-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_NM
    CMAKE_OBJCOPY
    CMAKE_READELF

[nathanchance]

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

[ronlieb]

my build systems have ROCm installed, which might make a difference.
since i can reproduce the issue locally

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

i would like to request the change be reverted so we can restore the bot to green, and work the issue locally , and push up a patch sometime tomorrow to attempt to resolve for our bot.

are you seeing any other bots failing from this change ?

[MaskRay]

In D135402#3913787, @ronlieb wrote:

my build systems have ROCm installed, which might make a difference.
since i can reproduce the issue locally

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

i would like to request the change be reverted so we can restore the bot to green, and work the issue locally , and push up a patch sometime tomorrow to attempt to resolve for our bot.

are you seeing any other bots failing from this change ?

I am ok with a temporary revert.

The first revert was done nearly a month ago but no action was taken to properly fix the issue 88da0de14f135bee034b286cc736dacd6274e77d. I wish that OpenMP folks can fix this properly:)

[MaskRay]

In D135402#3913785, @nathanchance wrote:

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

[jhuber6]

In D135402#3913815, @MaskRay wrote:

In D135402#3913785, @nathanchance wrote:

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

IIRC this was a somewhat weird scenario. I only noticed these errors after doing a re-build. If I deleted everything and did a fresh build it would work fine but would start failing again once I started trying to recompile without starting from scratch. Makes me thing it's a CMake problem.

FWIW I fixed all the issues libomptarget had. libomp had the issues I outlined above and I didn't get a solid recommendation on what to do about it.

[nathanchance]

In D135402#3913815, @MaskRay wrote:

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

This set of options and commands reproduces the issue for me. This is the first stage building the second (instrumented) stage of a PGO build.

$ cmake \
-B build/stage1 \
-G Ninja \
-DCMAKE_AR=$(command -v llvm-ar) \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=$(command -v clang) \
-DCMAKE_CXX_COMPILER=$(command -v clang++) \
-DCMAKE_RANLIB=$(command -v llvm-ranlib) \
-DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
-DCOMPILER_RT_BUILD_CRT=OFF \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_BUILD_SANITIZERS=OFF \
-DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_USE_LINKER=lld \
-S llvm >/dev/null

$ ninja -C build/stage1

$ cmake \
-B build/stage2 \
-G Ninja \
-DCLANG_TABLEGEN=$PWD/build/stage1/bin/clang-tblgen \
-DCMAKE_AR=$PWD/build/stage1/bin/llvm-ar \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=$PWD/build/stage1/bin/clang \
-DCMAKE_CXX_COMPILER=$PWD/build/stage1/bin/clang++ \
-DCMAKE_RANLIB=$PWD/build/stage1/bin/llvm-ranlib \
-DLLVM_BINUTILS_INCDIR=/usr/include \
-DLLVM_BUILD_INSTRUMENTED=IR \
-DLLVM_BUILD_RUNTIME=OFF \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_ENABLE_PLUGINS=ON \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_TABLEGEN=$PWD/build/stage1/bin/llvm-tblgen \
-DLLVM_USE_LINKER=$PWD/build/stage1/bin/ld.lld \
-DLLVM_VP_COUNTERS_PER_SITE=6 \
-S llvm >/dev/null

$ ninja -C build/stage2
...
[2802/2803] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /home/nathan/cbl/src/llvm-project/build/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/home/nathan/cbl/src/llvm-project/build/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles"   -Wl,--gc-sections  -Wl,--version-script,"/home/nathan/cbl/src/llvm-project/build/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
clang-16: warning: '-fuse-ld=' taking a path is deprecated; use '--ld-path=' instead [-Wfuse-ld-path]
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[2803/2803] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /home/nathan/cbl/src/llvm-project/build/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/home/nathan/cbl/src/llvm-project/build/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles"   -Wl,--gc-sections  -Wl,--version-script,"/home/nathan/cbl/src/llvm-project/build/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
clang-16: warning: '-fuse-ld=' taking a path is deprecated; use '--ld-path=' instead [-Wfuse-ld-path]
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

[ronlieb]

Please revert this change, so our bots can go green.
we are struggling to figure out how to fix openmp/runtime

we will keep working it , but a broken bot leaves us vulnerable to other changes.
also, we would like the authors assistance to help fix the issue so the patch can reland.

[JonChesterfield]

This looks conclusive to me. It's a sensible change to LLD which has found things that look broken in LLVM and in OpenMP, both of which should be fixed, but we can't leave multistage builds and openmp dead in the meantime. I'm going to revert. https://lab.llvm.org/buildbot/#/builders/193 green immediately following the revert, thankfully.

For openmp defining different symbols in release and debug, that's nasty - suggest we define functions that do nothing for the release build. I don't recognise lto.exports but the fix is likely to be removing those lines from the version script, or having multiple version scripts, or patching the source to always define those symbols.

edit: I see this was reverted and then relanded two days ago, which explains why the bots went red then. The most recent commit is eedbe44b8755, reverted in b1fdeeec1a5a570be

Looking at the buildbot logs, the openmp failures were along the lines of:

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

which seems to be something that is broken in a way that doesn't matter...

So I'm inclined to say we should pass -Wl,--undefined-version through CMake, optimistically hoping that ld will ignore flags it doesn't understand and/or there is some mechanism to pass different flags to different linkers in CMake, at which point the openmp side of things will stop falling over. Maybe the same workaround is legitimate for lto export

[jhuber6]

In D135402#3917463, @JonChesterfield wrote:

So I'm inclined to say we should pass -Wl,--undefined-version through CMake, optimistically hoping that ld will ignore flags it doesn't understand and/or there is some mechanism to pass different flags to different linkers in CMake, at which point the openmp side of things will stop falling over. Maybe the same workaround is legitimate for lto export

That was the original approach, but this didn't work out as the GNU linker doesn't ignore those and I don't think there's a way to configure CMake based off of the linker. Users are free to configure clang to use whatever linker by default, and we don't necessarily know if the compiler is clang. I'm not very familiar with the libomp code, but we could potentially just export a wildcard and accept a few new symbols leaking out in different configuraitons.

[ronlieb]

this patch when applied on top of a previous successful build, passes.
however, if i start with a fresh build directory and cmake/build it fails.
normally our amdgpu buildbot would not notice.

buildbot https://lab.llvm.org/staging/#/builders/183/builds/2704

unfortunately our buildbot died 3-4 days ago, and had to be reimaged yesterday and is still in staging.
when i restarted the buildbot yesterday it picked up this change and started failing.
I am not sure why patch from JPeyton did not resovle this. looking into... @jhuber6 ?

[ronlieb]

In D135402#3981172, @ronlieb wrote:

this patch when applied on top of a previous successful build, passes.
however, if i start with a fresh build directory and cmake/build it fails.
normally our amdgpu buildbot would not notice.

buildbot https://lab.llvm.org/staging/#/builders/183/builds/2704

unfortunately our buildbot died 3-4 days ago, and had to be reimaged yesterday and is still in staging.
when i restarted the buildbot yesterday it picked up this change and started failing.
I am not sure why patch from JPeyton did not resovle this. looking into... @jhuber6 ?

Joseph Huber pushed in a fix that gets our bot to go green. thx
[OpenMP][libomp] Fix version scripts after undefined version script changes

[jhuber6]

I'm still getting this when doing incremental builds. It's very annoying to need to completely remove my build directly every time I do a git pull.

FAILED: lib/libLTO.so.16git 
: && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=lld -Wl,--color-diagnostics   -Wl,--gc-sections  -Wl,--version-script,"/home/jhuber/Documents/llvm/llvm-project/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64AsmParser.so.16git  lib/libLLVMAMDGPUAsmParser.so.16git  lib/libLLVMARMAsmParser.so.16git  lib/libLLVMAVRAsmParser.so.16git  lib/libLLVMBPFAsmParser.so.16git  lib/libLLVMMipsAsmParser.so.16git  lib/libLLVMMSP430AsmParser.so.16git  lib/libLLVMPowerPCAsmParser.so.16git  lib/libLLVMRISCVAsmParser.so.16git  lib/libLLVMSparcAsmParser.so.16git  lib/libLLVMSystemZAsmParser.so.16git  lib/libLLVMVEAsmParser.so.16git  lib/libLLVMWebAssemblyAsmParser.so.16git  lib/libLLVMX86AsmParser.so.16git  lib/libLLVMAArch64CodeGen.so.16git  lib/libLLVMAMDGPUCodeGen.so.16git  lib/libLLVMARMCodeGen.so.16git  lib/libLLVMAVRCodeGen.so.16git  lib/libLLVMBPFCodeGen.so.16git  lib/libLLVMHexagonCodeGen.so.16git  lib/libLLVMLanaiCodeGen.so.16git  lib/libLLVMMipsCodeGen.so.16git  lib/libLLVMMSP430CodeGen.so.16git  lib/libLLVMNVPTXCodeGen.so.16git  lib/libLLVMPowerPCCodeGen.so.16git  lib/libLLVMRISCVCodeGen.so.16git  lib/libLLVMSparcCodeGen.so.16git  lib/libLLVMSystemZCodeGen.so.16git  lib/libLLVMVECodeGen.so.16git  lib/libLLVMWebAssemblyCodeGen.so.16git  lib/libLLVMX86CodeGen.so.16git  lib/libLLVMXCoreCodeGen.so.16git  lib/libLLVMAVRDesc.so.16git  lib/libLLVMBPFDesc.so.16git  lib/libLLVMMipsDesc.so.16git  lib/libLLVMMSP430Desc.so.16git  lib/libLLVMNVPTXDesc.so.16git  lib/libLLVMPowerPCDesc.so.16git  lib/libLLVMSparcDesc.so.16git  lib/libLLVMVEDesc.so.16git  lib/libLLVMX86Desc.so.16git  lib/libLLVMXCoreDesc.so.16git  lib/libLLVMAArch64Disassembler.so.16git  lib/libLLVMAMDGPUDisassembler.so.16git  lib/libLLVMARMDisassembler.so.16git  lib/libLLVMAVRDisassembler.so.16git  lib/libLLVMBPFDisassembler.so.16git  lib/libLLVMHexagonDisassembler.so.16git  lib/libLLVMLanaiDisassembler.so.16git  lib/libLLVMMipsDisassembler.so.16git  lib/libLLVMMSP430Disassembler.so.16git  lib/libLLVMPowerPCDisassembler.so.16git  lib/libLLVMRISCVDisassembler.so.16git  lib/libLLVMSparcDisassembler.so.16git  lib/libLLVMSystemZDisassembler.so.16git  lib/libLLVMVEDisassembler.so.16git  lib/libLLVMWebAssemblyDisassembler.so.16git  lib/libLLVMX86Disassembler.so.16git  lib/libLLVMXCoreDisassembler.so.16git  lib/libLLVMAVRInfo.so.16git  lib/libLLVMBPFInfo.so.16git  lib/libLLVMMipsInfo.so.16git  lib/libLLVMMSP430Info.so.16git  lib/libLLVMNVPTXInfo.so.16git  lib/libLLVMPowerPCInfo.so.16git  lib/libLLVMSparcInfo.so.16git  lib/libLLVMVEInfo.so.16git  lib/libLLVMX86Info.so.16git  lib/libLLVMXCoreInfo.so.16git  lib/libLLVMLTO.so.16git  lib/libLLVMHexagonAsmParser.so.16git  lib/libLLVMLanaiAsmParser.so.16git  lib/libLLVMAArch64Desc.so.16git  lib/libLLVMAArch64Info.so.16git  lib/libLLVMAMDGPUDesc.so.16git  lib/libLLVMAMDGPUInfo.so.16git  lib/libLLVMARMDesc.so.16git  lib/libLLVMARMInfo.so.16git  lib/libLLVMHexagonDesc.so.16git  lib/libLLVMHexagonInfo.so.16git  lib/libLLVMLanaiDesc.so.16git  lib/libLLVMLanaiInfo.so.16git  lib/libLLVMRISCVDesc.so.16git  lib/libLLVMRISCVInfo.so.16git  lib/libLLVMSystemZDesc.so.16git  lib/libLLVMSystemZInfo.so.16git  lib/libLLVMWebAssemblyDesc.so.16git  lib/libLLVMWebAssemblyInfo.so.16git  lib/libLLVMMCDisassembler.so.16git  lib/libLLVMCodeGen.so.16git  lib/libLLVMTarget.so.16git  lib/libLLVMMC.so.16git  lib/libLLVMBitReader.so.16git  lib/libLLVMCore.so.16git  lib/libLLVMSupport.so.16git  -Wl,-rpath-link,/home/jhuber/Documents/llvm/llvm-project/build/lib && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined

[glandium]

ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined

This is happening reliably when linking libLTO in a build with LLVM_LINK_LLVM_DYLIB=ON and LLVM_ENABLE_LLD=ON because in that case these symbols are in libLLVM instead of libLTO.

[kongyi]

libomp build with OPENMP_ENABLE_OMPTARGET=FALSE;OPENMP_ENABLE_OMP_TOOLS=FALSE continues to be broken.

ld.lld: error: version script assignment of 'VERSION' to symbol 'ompt_start_tool' failed: symbol not defined
ld.lld: error: version script assignment of 'VERSION' to symbol 'ompt_libomp_connect' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

[zatrazz]

Another project that is also breaks with this change is glibc. Although I could be fixed on glibc side, the undefined symbols are used to simplify the version symbol for multiple ABI with different base ABIs.

[xen0n]

Hi, as I've recently tried to bootstrap Gentoo on LoongArch with LLVM toolchain, I've run into multiple build failures that needed to be worked around; given how many high-profile projects are impacted, is it prudent to temporarily downgrade this to a warning? mold chose to emit a warning for such cases, for example.

[jhuber6]

In D135402#4054415, @xen0n wrote:

Hi, as I've recently tried to bootstrap Gentoo on LoongArch with LLVM toolchain, I've run into multiple build failures that needed to be worked around; given how many high-profile projects are impacted, is it prudent to temporarily downgrade this to a warning? mold chose to emit a warning for such cases, for example.

I'd be in favor of a warning. Now whenever I build LLVM with lld as my linker I need to completely remove any previous installs or I'll get the libLTO.so errors above.

[MaskRay]

GNU ld --no-undefined-version uses errors, so we'd better not to loosen the diagnostic.
We could have a tri-state that we default to a warning, but that is some code burden on the lld side and we need very strong arguments to support that.
For distributions that allow using lld as the default linker, there is actually a very straightforward workaround for the projects with issues: add -Wl,--undefined-version to a Clang configuration file.
(I added support for --undefined-version to the just-released GNU ld 2.40. 16.0.0 release will be after it.)
I went through the list on https://github.com/gentoo/gentoo/pull/29097#issuecomment-1383304538 and sent patches to fix some projects or reported bugs.

[treapster]

It seems to have broken BOLT build, with similar error

FAILED: lib/libRemarks.so.16git
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)

Cmake:

cmake -G Ninja -S ~/llvm-project/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt" -DLLVM_LIT_ARGS="-v --xunit-xml-output test-results.xml" -DLLVM_ENABLE_ZSTD=OFF -DBUILD_SHARED_LIBS=ON -DCMAKE_C_COMPILER="$HOME/bin/clang" -DCMAKE_CXX_COMPILER="$HOME/bin/clang++" -DLLVM_USE_LINKER="$HOME/bin/ld.lld"

[treapster]

If you try to pass -Wl,--undefined-version to CMake, you get erros in CMakeTestCCompiler because it uses system linker which errors out on unknown flag.

[raj.khem]

I am late to party I know. I am trying to enable lld as default system linker for one of yocto based distros and this option turned on by default now breaks quite a few packages on linux with this error

riscv64-yoe-linux-ld.lld: error: version script assignment of <SYMBOL>' to symbol '<ALIAS>' failed: symbol not defined

I guess I have to add -Wl,--undefined-version to global LDFLAGS for now until these packages are fixed one by one.

lld/ELF/Driver.cpp

static void readConfigs(opt::InputArgList &args) {
   config->thinLTOModulesToCompile =
       args::getStrings(args, OPT_thinlto_single_module_eq);
   config->timeTraceEnabled = args.hasArg(OPT_time_trace_eq);
   config->timeTraceGranularity =
       args::getInteger(args, OPT_time_trace_granularity, 500);
   config->trace = args.hasArg(OPT_trace);
   config->undefined = args::getStrings(args, OPT_undefined);
   config->undefinedVersion =
-      args.hasFlag(OPT_undefined_version, OPT_no_undefined_version, true);
+      args.hasFlag(OPT_undefined_version, OPT_no_undefined_version, false);
   config->unique = args.hasArg(OPT_unique);
   config->useAndroidRelrTags = args.hasFlag(
       OPT_use_android_relr_tags, OPT_no_use_android_relr_tags, false);
   config->warnBackrefs =
       args.hasFlag(OPT_warn_backrefs, OPT_no_warn_backrefs, false);
   config->warnCommon = args.hasFlag(OPT_warn_common, OPT_no_warn_common, false);
   config->warnSymbolOrdering =
       args.hasFlag(OPT_warn_symbol_ordering, OPT_no_warn_symbol_ordering, true);

lld/ELF/Options.td

defm undefined_glob: EEq<"undefined-glob", "Force undefined symbol during linking">,
   MetaVarName<"<pattern>">;
 
 def unique: F<"unique">, HelpText<"Creates a separate output section for every orphan input section">;
 
 defm unresolved_symbols:
   Eq<"unresolved-symbols", "Determine how to handle unresolved symbols">;
 
 defm undefined_version: B<"undefined-version",
-  "Allow unused version in version script (default)",
+  "Allow unused version in version script (disabled by default)",
   "Report version scripts that refer undefined symbols">;
 
 defm rsp_quoting: EEq<"rsp-quoting", "Quoting style for response files">,
   MetaVarName<"[posix,windows]">;
 
 def v: Flag<["-"], "v">, HelpText<"Display the version number">;
 
 def verbose: F<"verbose">, HelpText<"Verbose mode">;

lld/docs/ReleaseNotes.rst

 ELF Improvements
 ----------------
 
 * ``ELFCOMPRESS_ZSTD`` compressed input sections are now supported.
   (`D129406 <https://reviews.llvm.org/D129406>`_)
 * ``--compress-debug-sections=zstd`` is now available to compress debug
   sections with zstd (``ELFCOMPRESS_ZSTD``).
   (`D133548 <https://reviews.llvm.org/D133548>`_)
+* ``--no-undefined-version`` is now the default; symbols named in version
+  scripts that have no matching symbol in the output will be reported. Use
+  ``--undefined-version`` to revert to the old behavior.
 
 Breaking changes
 ----------------
 
 COFF Improvements
 -----------------
 
 * ...

lld/docs/ld.lld.1

 .It Fl -no-nmagic
 Page align sections.
 .It Fl -no-omagic
 Do not set the text data sections to be writable, page align sections.
 .It Fl -no-relax
 Disable target-specific relaxations. For x86-64 this disables R_X86_64_GOTPCRELX and R_X86_64_REX_GOTPCRELX GOT optimization.
 .It Fl -no-rosegment
 Do not put read-only non-executable sections in their own segment.
-.It Fl -no-undefined-version
-Report version scripts that refer undefined symbols.
+.It Fl -undefined-version
+Do not report version scripts that refer to undefined symbols.
 .It Fl -no-undefined
 Report unresolved symbols even if the linker is creating a shared library.
 .It Fl -no-warn-symbol-ordering
 Do not warn about problems with the symbol ordering file or call graph profile.
 .It Fl -no-whole-archive
 Restores the default behavior of loading archive members.
 .It Fl -no-pie , Fl -no-pic-executable
 Do not create a position independent executable.

lld/test/ELF/verdef-defaultver.s

 # REQUIRES: x86
 
 # RUN: rm -rf %t && split-file %s %t && cd %t
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/verdef-defaultver.s -o %t1
 # RUN: echo "V1 { global: a; b; local: *; };" > %t.script
 # RUN: echo "V2 { global: b; c; } V1;" >> %t.script
-# RUN: ld.lld --hash-style=sysv -shared -soname shared %t1 --version-script %t.script -o %t.so
+# RUN: ld.lld --hash-style=sysv -shared -soname shared %t1 --version-script %t.script --undefined-version -o %t.so
 # RUN: llvm-readobj -V --dyn-syms %t.so | FileCheck --check-prefix=DSO %s
 
 # DSO:      DynamicSymbols [
 # DSO-NEXT:    Symbol {
 # DSO-NEXT:      Name:
 # DSO-NEXT:      Value: 0x0
 # DSO-NEXT:      Size: 0
 # DSO-NEXT:      Binding: Local

 # EXE-NEXT:          Index: 3
 # EXE-NEXT:          Name: V2
 # EXE-NEXT:        }
 # EXE-NEXT:      ]
 # EXE-NEXT:    }
 # EXE-NEXT:  ]
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64 b.s -o b.o
-# RUN: ld.lld -shared --version-script=%t.script --fatal-warnings %t.so b.o -o b.so
+# RUN: ld.lld -shared --version-script=%t.script --fatal-warnings --undefined-version %t.so b.o -o b.so
 # RUN: llvm-readelf --dyn-syms b.so | FileCheck %s --check-prefix=PREEMPT
-# RUN: ld.lld -shared --version-script=%t.script --fatal-warnings b.o %t.so -o b.so
+# RUN: ld.lld -shared --version-script=%t.script --fatal-warnings --undefined-version b.o %t.so -o b.so
 # RUN: llvm-readelf --dyn-syms b.so | FileCheck %s --check-prefix=PREEMPT
 
 # PREEMPT-DAG: a@@V1
 # PREEMPT-DAG: c@@V2
 
 #--- a.s
 .globl _start
 _start:
   .long a - .
   .long b - .
   .long c - .
 
 #--- b.s
 .globl a, c
 a:
 c:

lld/test/ELF/verdef-dependency.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
 # RUN: echo "LIBSAMPLE_1.0 { global: a; local: *; };" > %t.script
 # RUN: echo "LIBSAMPLE_2.0 { global: b; local: *; } LIBSAMPLE_1.0;" >> %t.script
 # RUN: echo "LIBSAMPLE_3.0 { global: c; } LIBSAMPLE_2.0;" >> %t.script
-# RUN: ld.lld --version-script %t.script -shared -soname shared %t.o -o %t.so
+# RUN: ld.lld --version-script %t.script --undefined-version -shared -soname shared %t.o -o %t.so
 # RUN: llvm-readobj -V --dyn-syms %t.so | FileCheck --check-prefix=DSO %s
 
 # DSO:      VersionDefinitions [
 # DSO-NEXT:   Definition {
 # DSO-NEXT:     Version: 1
 # DSO-NEXT:     Flags [ (0x1)
 # DSO-NEXT:       Base (0x1)
 # DSO-NEXT:     ]

lld/test/ELF/verneed.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %S/Inputs/verneed1.s -o %t1.o
 # RUN: echo "v1 {}; v2 {}; v3 { global: f1; local: *; };" > %t.script
-# RUN: ld.lld -shared %t1.o --version-script %t.script -o %t1.so -soname verneed1.so.0
+# RUN: ld.lld -shared %t1.o --version-script %t.script --undefined-version -o %t1.so -soname verneed1.so.0
 # RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %S/Inputs/verneed2.s -o %t2.o
-# RUN: ld.lld -shared %t2.o --version-script %t.script -o %t2.so -soname verneed2.so.0
+# RUN: ld.lld -shared %t2.o --version-script %t.script --undefined-version -o %t2.so -soname verneed2.so.0
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
 # RUN: ld.lld --hash-style=sysv %t.o %t1.so %t2.so -o %t
 # RUN: llvm-readobj -S -d --section-data --dyn-syms -V %t | FileCheck %s
 
 # CHECK:        Section {
 # CHECK:          Index: 1
 # CHECK-NEXT:     Name: .dynsym

lld/test/ELF/version-script-extern-undefined.s

 # REQUIRES: x86
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
 # RUN: echo "FOO { global: extern \"C++\" { \"abb(int)\"; }; };" > %t.script
-# RUN: ld.lld --version-script %t.script -shared %t.o -o %t.so
+# RUN: ld.lld --version-script %t.script --undefined-version -shared %t.o -o %t.so
 # RUN: llvm-readobj -V %t.so | FileCheck %s
 
 # CHECK:      VersionSymbols [
 # CHECK-NEXT:   Symbol {
 # CHECK-NEXT:     Version: 0
 # CHECK-NEXT:     Name:
 # CHECK-NEXT:   }
 # CHECK-NEXT:   Symbol {
 # CHECK-NEXT:     Version: 1
 # CHECK-NEXT:     Name: _Z3abbi
 # CHECK-NEXT:   }
 # CHECK-NEXT: ]
 
 .globl _Z3abbi

lld/test/ELF/version-script-local-preemptible.s

 # REQUIRES: x86
 
 ## Version scripts are handled before relocation processing, so
 ## { local: *; } does not make a canonical PLT VER_NDX_LOCAL.
 
 # RUN: echo '.global foo; .type foo, @function; foo:' | \
 # RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t.so.o
 # RUN: ld.lld %t.so.o -o %t.so -shared
 
 # RUN: echo "{ global: main; local: *; };" > %t.script
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
-# RUN: ld.lld %t.o %t.so -o %t -version-script %t.script
+# RUN: ld.lld %t.o %t.so -o %t -version-script %t.script --undefined-version
 # RUN: llvm-readelf -r --symbols %t | FileCheck %s
 
 # CHECK:      Relocation section '.rela.plt' at offset {{.*}} contains 1 entries:
 # CHECK:        R_X86_64_JUMP_SLOT [[ADDR:[0-9a-f]+]] foo + 0
 
 # CHECK:      Symbol table '.dynsym' contains 2 entries:
 # CHECK-NEXT:   Num:    Value          Size Type    Bind   Vis      Ndx Name
 # CHECK-NEXT:     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
 # CHECK-NEXT:     1: [[ADDR]]             0 FUNC    GLOBAL DEFAULT  UND foo
 
 .globl _start
 _start:
   movl $foo - ., %eax

lld/test/ELF/version-script-noundef.s

 # REQUIRES: x86
 
 # RUN: echo "VERSION_1.0 { global: bar; };" > %t.script
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
-# RUN: ld.lld --version-script %t.script -shared %t.o -o /dev/null --fatal-warnings
+# RUN: not ld.lld --version-script %t.script -shared %t.o -o /dev/null \
+# RUN:    --fatal-warnings 2>&1 | FileCheck -check-prefix=ERR1 %s
 # RUN: ld.lld --version-script %t.script -shared --undefined-version %t.o -o %t.so
 # RUN: not ld.lld --version-script %t.script -shared --no-undefined-version \
 # RUN:   %t.o -o %t.so 2>&1 | FileCheck -check-prefix=ERR1 %s
 # ERR1: version script assignment of 'VERSION_1.0' to symbol 'bar' failed: symbol not defined
 
 # RUN: echo "VERSION_1.0 { global: und; };" > %t2.script
 # RUN: not ld.lld --version-script %t2.script -shared --no-undefined-version \
 # RUN:   %t.o -o %t.so 2>&1 | FileCheck -check-prefix=ERR2 %s

lld/test/ELF/version-script-reassign.s

 # RUN: llvm-readelf --dyn-syms %t.so | FileCheck --check-prefix=GLOBAL-SYM %s
 
 ## Note, ld.bfd silently accepts this case.
 # RUN: ld.lld -shared %t.o --version-script %t1.ver --version-script %t2.ver \
 # RUN:   -o %t.so 2>&1 | FileCheck --check-prefix=V1-WARN %s
 # RUN: llvm-readelf --dyn-syms %t.so | FileCheck --check-prefix=V1-SYM %s
 
 # RUN: ld.lld -shared %t.o --version-script %t1.ver --version-script %t2w.ver \
-# RUN:   -o %t.so --fatal-warnings
+# RUN:   -o %t.so --fatal-warnings --undefined-version
 # RUN: llvm-readelf --dyn-syms %t.so | FileCheck --check-prefix=V1-SYM %s
 
 # LOCAL: warning: attempt to reassign symbol 'foo' of VER_NDX_LOCAL to version 'V1'
 # LOCAL-SYM-NOT: foo
 
 # GLOBAL: warning: attempt to reassign symbol 'foo' of VER_NDX_GLOBAL to version 'V1'
 # GLOBAL-SYM: foo{{$}}
 
 # V1-WARN: warning: attempt to reassign symbol 'foo' of version 'V1' to version 'V2'
 # V1-SYM: foo@@V1
 
 .globl foo
 foo: