Page MenuHomePhabricator

[Proof of concept] Add an opt-in way to statically link a hermetic libxml2
Needs ReviewPublic

Authored by thakis on Jul 13 2022, 9:44 AM.



When LLVM_ENABLE_LIBXML2 is set to HERMETIC, we now download libxml2
at cmake time, build it at build time, and statically link to it
in llvm-mt.

This reverts and uses
LLVM_ENABLE_LIBXML2=HERMETIC instead for the Windows installer.

This works for llvm-mt, but c-index-test and lldb also pick up the hermetic libxml2, and since the built libxml2 is so minimal, fail to link.

But even if that worked, I'm not sure if this is worth it. It looks kind of janky: The external project build step runs on every build; every target depending on libxml needs some extra code to force the install step to run, cmake is a bit temperamental about this, and so on. Maybe this isn't all that much easier than what D129571 did.

Or maybe there's a better way for doing this that I can't see.

Maybe this should be WindowsManifest-specific and WindowsManifest should optionally depend on the hermetic stuff, and HermeticLibXML2.cmake shouldn't try to pretend its find_package()-compatible. Then lldb and c-index-test wouldn't pick up the hermetic lib.

I've spent more time on this than I should've, so uploading as-is. Maybe someone has an opinion on this.

Diff Detail

Unit TestsFailed

60,040 msx64 debian > MLIR.Examples/standalone::test.toy
Script: -- : 'RUN: at line 1'; /usr/bin/cmake /var/lib/buildkite-agent/builds/llvm-project/mlir/examples/standalone -G "Ninja" -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_C_COMPILER=/usr/bin/clang -DLLVM_ENABLE_LIBCXX=OFF -DMLIR_DIR=/var/lib/buildkite-agent/builds/llvm-project/build/lib/cmake/mlir -DLLVM_USE_LINKER=lld
60,040 msx64 debian > ThreadSanitizer-x86_64.ThreadSanitizer-x86_64::restore_stack.cpp
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang --driver-mode=g++ -fsanitize=thread -Wall -m64 -msse4.2 -gline-tables-only -I/var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/tsan/../ -std=c++11 -I/var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/tsan/../ -O1 /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/tsan/restore_stack.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/tsan/X86_64Config/Output/restore_stack.cpp.tmp && not /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/tsan/X86_64Config/Output/restore_stack.cpp.tmp 2>&1 | FileCheck /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/tsan/restore_stack.cpp
60,050 msx64 debian > libFuzzer.libFuzzer::fuzzer-leak.test
Script: -- : 'RUN: at line 3'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/var/lib/buildkite-agent/builds/llvm-project/compiler-rt/lib/fuzzer -m64 /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/fuzzer/LeakTest.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/fuzzer-leak.test.tmp-LeakTest
60,040 msx64 debian > libFuzzer.libFuzzer::minimize_crash.test
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang --driver-mode=g++ -O2 -gline-tables-only -fsanitize=address,fuzzer -I/var/lib/buildkite-agent/builds/llvm-project/compiler-rt/lib/fuzzer -m64 /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/fuzzer/NullDerefTest.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/minimize_crash.test.tmp-NullDerefTest
60,020 msx64 debian > libFuzzer.libFuzzer::out-of-process-fuzz.test
Script: -- : 'RUN: at line 2'; rm -rf /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/out-of-process-fuzz.test.tmp && mkdir /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/fuzzer/X86_64DefaultLinuxConfig/Output/out-of-process-fuzz.test.tmp
View Full Test Results (6 Failed)

Event Timeline

thakis created this revision.Jul 13 2022, 9:44 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 13 2022, 9:44 AM
thakis requested review of this revision.Jul 13 2022, 9:44 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 13 2022, 9:44 AM
hans added a comment.Jul 14 2022, 2:58 AM

It certainly simplifies build_llvm_release.bat a lot :-)

nit: Calling this hermetic seems counter-intuitive here, since it's actually pulling in stuff from way outside the build -- the opposite of hermetic really. Maybe the flag value should be "DOWNLOAD" or something like that?

But yeah, also not sure whether this is worth it. I kind of wish llvm-mt didn't have this dependency and could use something simpler, but I don't know whether that's realistic.