Page MenuHomePhabricator

[LLD] Enable --no-undefined-version by default.
ClosedPublic

Authored by danalbert on Oct 6 2022, 2:15 PM.

Details

Summary

Allowing incorrect version scripts is not a helpful default. Flip that
to help users find their bugs at build time rather than at run time.

Diff Detail

Event Timeline

danalbert created this revision.Oct 6 2022, 2:15 PM
Herald added a project: Restricted Project. · View Herald Transcript
danalbert published this revision for review.Oct 6 2022, 2:16 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 6 2022, 2:16 PM
pirama added a subscriber: pirama.Oct 6 2022, 2:18 PM

Thanks for catching this. The new default makes more sense and aligns with the other enabled-by-default diagnostic (warn("attempt to reassign symbol linking glibc libc.so is known to trigger some warnings).
I think the blast radius is very small since projects using exact patterns in a version script are rare. Default version symbols don't need patterns while non-default version symbols are rarely used.
When a pattern does not specify a symbol, it's very clear a user error.

Please update lld/docs/ReleaseNotes.rst and flip lld/docs/ld.lld.1 to use --undefined-version instead.

danalbert updated this revision to Diff 465884.Oct 6 2022, 2:42 PM

Update docs.

Please update lld/docs/ReleaseNotes.rst and flip lld/docs/ld.lld.1 to use --undefined-version instead.

Thanks for the heads up! Done, I think? Never had to edit a man page before so no idea if that part is right :)

MaskRay accepted this revision.Oct 6 2022, 2:46 PM

LGTM.

lld/docs/ld.lld.1
355

Mentioning just Fl -undefined-version is fine. For boolean options we only mention the one which is not the default.

This revision is now accepted and ready to land.Oct 6 2022, 2:46 PM
danalbert updated this revision to Diff 465902.Oct 6 2022, 3:22 PM

Remove man doc for default behavior.

This revision was automatically updated to reflect the committed changes.

hi, our amdgpu buildbot which uses lld has been failing since this commit landed
https://lab.llvm.org/buildbot/#/builders/193/builds/20055

jhuber6 added a subscriber: jhuber6.EditedOct 13 2022, 6:59 AM

hi, our amdgpu buildbot which uses lld has been failing since this commit landed
https://lab.llvm.org/buildbot/#/builders/193/builds/20055

I addressed those issues by adding --undefined-version to everywhere that used the version scripts.

I've recently been getting a few spurious build failures after this patch for. Getting rid of the following exported symbols seems to fix it, but I'm not sure if this is the correct solution or these are spuriously undefined.

diff --git a/llvm/tools/lto/lto.exports b/llvm/tools/lto/lto.exports
index 3abae5f0fcba..4164c3919a97 100644
--- a/llvm/tools/lto/lto.exports
+++ b/llvm/tools/lto/lto.exports
@@ -45,12 +45,6 @@ lto_codegen_compile_optimized
 lto_codegen_set_should_internalize
 lto_codegen_set_should_embed_uselists
 lto_set_debug_options
-LLVMCreateDisasm
-LLVMCreateDisasmCPU
-LLVMDisasmDispose
-LLVMDisasmInstruction
-LLVMSetDisasmOptions
-LLVMCreateDisasmCPUFeatures
 thinlto_create_codegen
 thinlto_codegen_dispose
 thinlto_codegen_add_module

This seems to happen every time I rebuild,

[133/897] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git 
: && /usr/bin/c++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wno-misleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=lld -Wl,--color-diagnostics   -Wl,--gc-sections  -Wl,--version-script,"/home/jhuber/Documents/llvm/llvm-project/build/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64AsmParser.so.16git  lib/libLLVMAMDGPUAsmParser.so.16git  lib/libLLVMARMAsmParser.so.16git  lib/libLLVMAVRAsmParser.so.16git  lib/libLLVMBPFAsmParser.so.16git  lib/libLLVMMipsAsmParser.so.16git  lib/libLLVMMSP430AsmParser.so.16git  lib/libLLVMPowerPCAsmParser.so.16git  lib/libLLVMRISCVAsmParser.so.16git  lib/libLLVMSparcAsmParser.so.16git  lib/libLLVMSystemZAsmParser.so.16git  lib/libLLVMVEAsmParser.so.16git  lib/libLLVMWebAssemblyAsmParser.so.16git  lib/libLLVMX86AsmParser.so.16git  lib/libLLVMAArch64CodeGen.so.16git  lib/libLLVMAMDGPUCodeGen.so.16git  lib/libLLVMARMCodeGen.so.16git  lib/libLLVMAVRCodeGen.so.16git  lib/libLLVMBPFCodeGen.so.16git  lib/libLLVMHexagonCodeGen.so.16git  lib/libLLVMLanaiCodeGen.so.16git  lib/libLLVMMipsCodeGen.so.16git  lib/libLLVMMSP430CodeGen.so.16git  lib/libLLVMNVPTXCodeGen.so.16git  lib/libLLVMPowerPCCodeGen.so.16git  lib/libLLVMRISCVCodeGen.so.16git  lib/libLLVMSparcCodeGen.so.16git  lib/libLLVMSystemZCodeGen.so.16git  lib/libLLVMVECodeGen.so.16git  lib/libLLVMWebAssemblyCodeGen.so.16git  lib/libLLVMX86CodeGen.so.16git  lib/libLLVMXCoreCodeGen.so.16git  lib/libLLVMAVRDesc.so.16git  lib/libLLVMBPFDesc.so.16git  lib/libLLVMMipsDesc.so.16git  lib/libLLVMMSP430Desc.so.16git  lib/libLLVMNVPTXDesc.so.16git  lib/libLLVMPowerPCDesc.so.16git  lib/libLLVMSparcDesc.so.16git  lib/libLLVMVEDesc.so.16git  lib/libLLVMX86Desc.so.16git  lib/libLLVMXCoreDesc.so.16git  lib/libLLVMAArch64Disassembler.so.16git  lib/libLLVMAMDGPUDisassembler.so.16git  lib/libLLVMARMDisassembler.so.16git  lib/libLLVMAVRDisassembler.so.16git  lib/libLLVMBPFDisassembler.so.16git  lib/libLLVMHexagonDisassembler.so.16git  lib/libLLVMLanaiDisassembler.so.16git  lib/libLLVMMipsDisassembler.so.16git  lib/libLLVMMSP430Disassembler.so.16git  lib/libLLVMPowerPCDisassembler.so.16git  lib/libLLVMRISCVDisassembler.so.16git  lib/libLLVMSparcDisassembler.so.16git  lib/libLLVMSystemZDisassembler.so.16git  lib/libLLVMVEDisassembler.so.16git  lib/libLLVMWebAssemblyDisassembler.so.16git  lib/libLLVMX86Disassembler.so.16git  lib/libLLVMXCoreDisassembler.so.16git  lib/libLLVMAVRInfo.so.16git  lib/libLLVMBPFInfo.so.16git  lib/libLLVMMipsInfo.so.16git  lib/libLLVMMSP430Info.so.16git  lib/libLLVMNVPTXInfo.so.16git  lib/libLLVMPowerPCInfo.so.16git  lib/libLLVMSparcInfo.so.16git  lib/libLLVMVEInfo.so.16git  lib/libLLVMX86Info.so.16git  lib/libLLVMXCoreInfo.so.16git  lib/libLLVMLTO.so.16git  lib/libLLVMHexagonAsmParser.so.16git  lib/libLLVMLanaiAsmParser.so.16git  lib/libLLVMAArch64Desc.so.16git  lib/libLLVMAArch64Info.so.16git  lib/libLLVMAMDGPUDesc.so.16git  lib/libLLVMAMDGPUInfo.so.16git  lib/libLLVMARMDesc.so.16git  lib/libLLVMARMInfo.so.16git  lib/libLLVMHexagonDesc.so.16git  lib/libLLVMHexagonInfo.so.16git  lib/libLLVMLanaiDesc.so.16git  lib/libLLVMLanaiInfo.so.16git  lib/libLLVMRISCVDesc.so.16git  lib/libLLVMRISCVInfo.so.16git  lib/libLLVMSystemZDesc.so.16git  lib/libLLVMSystemZInfo.so.16git  lib/libLLVMWebAssemblyDesc.so.16git  lib/libLLVMWebAssemblyInfo.so.16git  lib/libLLVMMCDisassembler.so.16git  lib/libLLVMCodeGen.so.16git  lib/libLLVMTarget.so.16git  lib/libLLVMMC.so.16git  lib/libLLVMBitReader.so.16git  lib/libLLVMCore.so.16git  lib/libLLVMSupport.so.16git  -Wl,-rpath-link,/home/jhuber/Documents/llvm/llvm-project/build/lib && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined

--undefined-version is not supported by GNU ld, so now my build is failing :)

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

The option addition was an attempt to resolve lld build failure reported by @ronlieb above: https://reviews.llvm.org/D135402#3855331
So I had to revert both commits so that the builds pass with both ld and lld

jhuber6 added a comment.EditedOct 13 2022, 2:18 PM

Why was an LLD patch reverted as a result of GNU ld not supporting a flag?

The problem was that this patch put LLVM in a state that it could not be built using lld anymore. GNU ld not supporting the flag simply made it difficult to work around the regression. The problems observed were with the libomp and lto version scripts. This patch should be re-landed once those issues are addressed more directly.

Okay. Unfortunately it's unrealistic that I'll ever have the time to chase every build bot, so unless someone else can volunteer to do that I don't think this default will ever improve.

I am not volunteering, but here is my thoughts.

@jhuber6, @ronlieb, do you know why the lld build started failing after this change? Is there --fatal-warnings option specified for the linker invocations?
If we remove --fatal-warnings this should allow @danalbert to reland this patch, and then the warnings can be phased out incrementally. Does it make sense?
Or, alternatively, lld default may be changed to always warn-on-undefined-symbols-in-version-script but never treat this as an error, unless --fatal-warnings and --undefined-version are both explicitly specified (to match legacy behavior). This may result in some convoluted user experience.

I have a general question though: is lld supposed to be a verbatim replacement for ld and vice versa? Is it okay to allow different user-visible behavior for the two tools?

AFAICT openmp uses an incorrect version script where some exact patterns don't match symbols. This is error-prone but before this default switch it was unnoticed.

If you use --no-undefined-version with GNU ld, you likely get similar errors.

"drop-in replacement" does not mean ld.lld copies every behavior of GNU ld. We consider many factors and what default makes the most sense.
I've made comments why the new default makes more sense and I don't want to repeat.

AFAICT openmp uses an incorrect version script where some exact patterns don't match symbols. This is error-prone but before this default switch it was unnoticed.

Yes, I originally attempted to rectify the offending version scripts. The only difficult one to rectify is the one for libomp. This is because some symbols are only defined in a debug build, or some other configuration, it contains lines like this.

#if USE_DEBUGGER
        __kmp_debugging;
        __kmp_omp_debug_struct_info;
#endif /* USE_DEBUGGER */

What would you suggest for solving this? Should we just include dummy symbols in the case that these symbols aren't included?

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

diff --git a/llvm/tools/lto/lto.exports b/llvm/tools/lto/lto.exports
index 3abae5f0fcba..4164c3919a97 100644
--- a/llvm/tools/lto/lto.exports
+++ b/llvm/tools/lto/lto.exports
@@ -45,12 +45,6 @@ lto_codegen_compile_optimized
 lto_codegen_set_should_internalize
 lto_codegen_set_should_embed_uselists
 lto_set_debug_options
-LLVMCreateDisasm
-LLVMCreateDisasmCPU
-LLVMDisasmDispose
-LLVMDisasmInstruction
-LLVMSetDisasmOptions
-LLVMCreateDisasmCPUFeatures
 thinlto_create_codegen
 thinlto_codegen_dispose
 thinlto_codegen_add_module

If you use --no-undefined-version with GNU ld, you likely get similar errors.

"drop-in replacement" does not mean ld.lld copies every behavior of GNU ld. We consider many factors and what default makes the most sense.
I've made comments why the new default makes more sense and I don't want to repeat.

I'm fine with the philosophy and this patch, but I don't want the OpenMP builds to be totally broken until we fix it.

What would you suggest for solving [differing exported symbols between debug and release builds]?

Two version scripts. In Android we have similar issues, especially in places like libc that differ from ABI to ABI, and we solve this by preprocessing the version script before giving it to the linker.

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

tbh that sounds like a bug. If you're expecting symbols to be public and LTO is making that not the case (DCE?), the expected ABI is not being fulfilled. Or maybe those weren't meant to be public anyway so it doesn't actually matter and they should just be removed.

What would you suggest for solving [differing exported symbols between debug and release builds]?

Two version scripts. In Android we have similar issues, especially in places like libc that differ from ABI to ABI, and we solve this by preprocessing the version script before giving it to the linker.

I was considering if preprocessing the version script was a possibility given the #if usage, basically feeding it through the compiler's preprocessor, but I wasn't sure if there was an easy way to achieve that via CMake.

Also, I mentioned previously in this thread the errors I was getting with the lto version script. Do you happen to know why these aren't defined?

tbh that sounds like a bug. If you're expecting symbols to be public and LTO is making that not the case (DCE?), the expected ABI is not being fulfilled. Or maybe those weren't meant to be public anyway so it doesn't actually matter and they should just be removed.

It seems like they were put there deliberately from the commit history, but it may be out of date following some other changes. It was a bizarre failure that went away if I did a completely clean build but showed up again whenever I did a pull & build so I was unsure what the issue was.

I relanded a OpenMP workaround to expect to check the fallout. But I made cherry-pick of this patch as a dependent patch of the OpenMP workaround so I ended up reland this patch as well...
Well, let's see whether the OpenMP workaround works...

amdgpu buildbot sees issues.

I cannot reproduce.

cmake -GNinja -Sllvm -B/tmp/out/s2-custom -DCMAKE_BUILD_TYPE=Release ${LLVM} -DCMAKE_CXX_COMPILER=/tmp/Rel/bin/clang++ -DCMAKE_C_COMPILER=/tmp/Rel/bin/clang -DCLANG_DEFAULT_LINKER=lld '-DLLVM_TARGETS_TO_BUILD=X86;AMDGPU' -DLLVM_ENABLE_RUNTIMES=openmp '-DLLVM_ENABLE_PROJECTS=clang;llvm;lld'
ninja -C /tmp/out/s2-custom check-openmp

I checked moving openmp from LLVM_ENABLE_RUNTIMES to LLVM_ENABLE_PROJECTS as well. Beside, I cannot find --version-script in the failed log. How does the build use the problematic linker script?

This cmake recipe reproduces issue for me:

/usr/bin/cmake \

-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;lld;llvm" \
-DLLVM_LIT_ARGS="-vv --show-unsupported --show-xfail -j 32" \
-DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
-DLLVM_ENABLE_ASSERTIONS=ON                        \
-DLLVM_ENABLE_RUNTIMES="openmp" \
-DLIBOMPTARGET_ENABLE_DEBUG=ON \
-DCLANG_DEFAULT_LINKER=lld                         \
../llvm -GNinja

This cmake recipe reproduces issue for me:

/usr/bin/cmake \

-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;lld;llvm" \
-DLLVM_LIT_ARGS="-vv --show-unsupported --show-xfail -j 32" \
-DLLVM_TARGETS_TO_BUILD="X86;AMDGPU" \
-DLLVM_ENABLE_ASSERTIONS=ON                        \
-DLLVM_ENABLE_RUNTIMES="openmp" \
-DLIBOMPTARGET_ENABLE_DEBUG=ON \
-DCLANG_DEFAULT_LINKER=lld                         \
../llvm -GNinja

ninja check-openmp gives me

CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins/amdgpu/CMakeLists.txt:92 (add_dependencies):
  The dependency target "omptarget.devicertl.amdgpu" of target
  "omptarget.rtl.amdgpu" does not exist.


CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins/cuda/CMakeLists.txt:85 (add_dependencies):
  The dependency target "omptarget.devicertl.nvptx" of target
  "omptarget.rtl.cuda" does not exist.


CMake Error at /home/maskray/llvm/openmp/libomptarget/plugins-nextgen/cuda/CMakeLists.txt:88 (add_dependencies):
  The dependency target "omptarget.devicertl.nvptx" of target
  "omptarget.rtl.cuda.nextgen" does not exist.


-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_NM
    CMAKE_OBJCOPY
    CMAKE_READELF

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

my build systems have ROCm installed, which might make a difference.
since i can reproduce the issue locally

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

i would like to request the change be reverted so we can restore the bot to green, and work the issue locally , and push up a patch sometime tomorrow to attempt to resolve for our bot.

are you seeing any other bots failing from this change ?

my build systems have ROCm installed, which might make a difference.
since i can reproduce the issue locally

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

i would like to request the change be reverted so we can restore the bot to green, and work the issue locally , and push up a patch sometime tomorrow to attempt to resolve for our bot.

are you seeing any other bots failing from this change ?

I am ok with a temporary revert.

The first revert was done nearly a month ago but no action was taken to properly fix the issue 88da0de14f135bee034b286cc736dacd6274e77d. I wish that OpenMP folks can fix this properly:)

MaskRay added a comment.EditedMon, Nov 7, 5:05 PM

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

For what's worth, I now see the libLTO failures that @jhuber6 saw above and similar failures in libRemarks when doing a multistage build. I can dig out the cmake commands tomorrow if a standard one does not reproduce it.

[3825/4195] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[3826/4195] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /mnt/nvme/tmp/build/llvm/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -w -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/mnt/nvme/tmp/build/llvm/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/mnt/nvme/tmp/build/llvm/stage2/profiles"   -Wl,--gc-sections -Wl,--version-script,"/mnt/nvme/tmp/build/llvm/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

IIRC this was a somewhat weird scenario. I only noticed these errors after doing a re-build. If I deleted everything and did a fresh build it would work fine but would start failing again once I started trying to recompile without starting from scratch. Makes me thing it's a CMake problem.

FWIW I fixed all the issues libomptarget had. libomp had the issues I outlined above and I didn't get a solid recommendation on what to do about it.

What's the reproduce instruction? My two-stage build using LLVM_ENABLE_LLD=on works fine: ninja -C /tmp/out/s2-custom lib/libLTO.so.16git lib/libRemarks.so.16git clang

This set of options and commands reproduces the issue for me. This is the first stage building the second (instrumented) stage of a PGO build.

$ cmake \
-B build/stage1 \
-G Ninja \
-DCMAKE_AR=$(command -v llvm-ar) \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=$(command -v clang) \
-DCMAKE_CXX_COMPILER=$(command -v clang++) \
-DCMAKE_RANLIB=$(command -v llvm-ranlib) \
-DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
-DCOMPILER_RT_BUILD_CRT=OFF \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_BUILD_SANITIZERS=OFF \
-DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_USE_LINKER=lld \
-S llvm >/dev/null

$ ninja -C build/stage1

$ cmake \
-B build/stage2 \
-G Ninja \
-DCLANG_TABLEGEN=$PWD/build/stage1/bin/clang-tblgen \
-DCMAKE_AR=$PWD/build/stage1/bin/llvm-ar \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=$PWD/build/stage1/bin/clang \
-DCMAKE_CXX_COMPILER=$PWD/build/stage1/bin/clang++ \
-DCMAKE_RANLIB=$PWD/build/stage1/bin/llvm-ranlib \
-DLLVM_BINUTILS_INCDIR=/usr/include \
-DLLVM_BUILD_INSTRUMENTED=IR \
-DLLVM_BUILD_RUNTIME=OFF \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_ENABLE_PLUGINS=ON \
-DLLVM_ENABLE_PROJECTS="clang;lld" \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_TABLEGEN=$PWD/build/stage1/bin/llvm-tblgen \
-DLLVM_USE_LINKER=$PWD/build/stage1/bin/ld.lld \
-DLLVM_VP_COUNTERS_PER_SITE=6 \
-S llvm >/dev/null

$ ninja -C build/stage2
...
[2802/2803] Linking CXX shared library lib/libRemarks.so.16git
FAILED: lib/libRemarks.so.16git
: && /home/nathan/cbl/src/llvm-project/build/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/home/nathan/cbl/src/llvm-project/build/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles"   -Wl,--gc-sections  -Wl,--version-script,"/home/nathan/cbl/src/llvm-project/build/stage2/tools/remarks-shlib/Remarks.exports" -shared -Wl,-soname,libRemarks.so.16git -o lib/libRemarks.so.16git tools/remarks-shlib/CMakeFiles/Remarks.dir/libremarks.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
clang-16: warning: '-fuse-ld=' taking a path is deprecated; use '--ld-path=' instead [-Wfuse-ld-path]
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetData' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkStringGetLen' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceFilePath' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceLine' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkDebugLocGetSourceColumn' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetKey' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetValue' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkArgGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetType' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetPassName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetRemarkName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFunctionName' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetDebugLoc' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetHotness' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNumArgs' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetFirstArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkEntryGetNextArg' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateYAML' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMRemarkParserCreateBitstream' failed: symbol not defined
ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
[2803/2803] Linking CXX shared library lib/libLTO.so.16git
FAILED: lib/libLTO.so.16git
: && /home/nathan/cbl/src/llvm-project/build/stage1/bin/clang++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles" -Xclang -mllvm -Xclang -vp-counters-per-site=6 -O3 -DNDEBUG  -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=/home/nathan/cbl/src/llvm-project/build/stage1/bin/ld.lld -Wl,--color-diagnostics -fprofile-generate="/home/nathan/cbl/src/llvm-project/build/stage2/profiles"   -Wl,--gc-sections  -Wl,--version-script,"/home/nathan/cbl/src/llvm-project/build/stage2/tools/lto/LTO.exports" -shared -Wl,-soname,libLTO.so.16git -o lib/libLTO.so.16git tools/lto/CMakeFiles/LTO.dir/LTODisassembler.cpp.o tools/lto/CMakeFiles/LTO.dir/lto.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVM-16git.so && :
clang-16: warning: '-fuse-ld=' taking a path is deprecated; use '--ld-path=' instead [-Wfuse-ld-path]
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasm' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPU' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmDispose' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMDisasmInstruction' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMSetDisasmOptions' failed: symbol not defined
ld.lld: error: version script assignment of 'LLVM_16' to symbol 'LLVMCreateDisasmCPUFeatures' failed: symbol not defined
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Please revert this change, so our bots can go green.
we are struggling to figure out how to fix openmp/runtime

we will keep working it , but a broken bot leaves us vulnerable to other changes.
also, we would like the authors assistance to help fix the issue so the patch can reland.

This looks conclusive to me. It's a sensible change to LLD which has found things that look broken in LLVM and in OpenMP, both of which should be fixed, but we can't leave multistage builds and openmp dead in the meantime. I'm going to revert. https://lab.llvm.org/buildbot/#/builders/193 green immediately following the revert, thankfully.

For openmp defining different symbols in release and debug, that's nasty - suggest we define functions that do nothing for the release build. I don't recognise lto.exports but the fix is likely to be removing those lines from the version script, or having multiple version scripts, or patching the source to always define those symbols.

edit: I see this was reverted and then relanded two days ago, which explains why the bots went red then. The most recent commit is eedbe44b8755, reverted in b1fdeeec1a5a570be

Looking at the buildbot logs, the openmp failures were along the lines of:

ld.lld: error: openmp/runtime/src/CMakeFiles/omp.dir/kmp_csupport.cpp.o: symbol ompc_set_affinity_format@@VERSION has undefined version VERSION

which seems to be something that is broken in a way that doesn't matter...

So I'm inclined to say we should pass -Wl,--undefined-version through CMake, optimistically hoping that ld will ignore flags it doesn't understand and/or there is some mechanism to pass different flags to different linkers in CMake, at which point the openmp side of things will stop falling over. Maybe the same workaround is legitimate for lto export

So I'm inclined to say we should pass -Wl,--undefined-version through CMake, optimistically hoping that ld will ignore flags it doesn't understand and/or there is some mechanism to pass different flags to different linkers in CMake, at which point the openmp side of things will stop falling over. Maybe the same workaround is legitimate for lto export

That was the original approach, but this didn't work out as the GNU linker doesn't ignore those and I don't think there's a way to configure CMake based off of the linker. Users are free to configure clang to use whatever linker by default, and we don't necessarily know if the compiler is clang. I'm not very familiar with the libomp code, but we could potentially just export a wildcard and accept a few new symbols leaking out in different configuraitons.