This is an archive of the discontinued LLVM Phabricator instance.

[runtimes] Remove support for standalone builds
ClosedPublic

Authored by ldionne on Feb 8 2022, 8:40 AM.

Details

Reviewers
dim
Group Reviewers
Restricted Project
Restricted Project
Restricted Project
Commits
rG6f17768e1148: [runtimes] Remove support for standalone builds
Summary

Standalone build have been deprecated for some time now, so this
commit removes support for those builds entirely from libc++, libc++abi
and libunwind.

This, along with the removal of other legacy ways to build, will allow
for major build system simplifications.

Diff Detail

Event Timeline

ldionne created this revision.Feb 8 2022, 8:40 AM
Herald added a project: Restricted Project. · View Herald TranscriptFeb 8 2022, 8:40 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
ldionne requested review of this revision.Feb 8 2022, 8:40 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptFeb 8 2022, 8:40 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
Herald added a reviewer: Restricted Project. · View Herald Transcript
ldionne added subscribers: thakis, Restricted Project.Feb 8 2022, 8:41 AM

Heads up libc++ vendors, if you still depend on this, you should move to the supported ways of building.

Also @thakis I think you were using this at some point unless I'm mistaken (you might want to add yourself to libc++ vendors to be automatically notified about these kinds of things).

Quuxplusone added inline comments.
libcxxabi/www/index.html
76

This sentence is missing the "why". Is the intent something like "If you'd like to get involved with the project, and/or learn how to build libc++ from source, please see the libc++ documentation"? And if so, are there specific "getting started" and/or "getting involved" sections of the libc++ documentation to which you should link, specifically?

Mordante added inline comments.
libcxx/docs/ReleaseNotes.rst
56

Can you make the link a .rst link?

ldionne updated this revision to Diff 406934.Feb 8 2022, 12:31 PM
ldionne marked 2 inline comments as done.

Address review comments

libcxxabi/www/index.html
76

Right -- basically this documentation as a whole is out of date. I'll be more precise.

emaste added a subscriber: dim.Feb 8 2022, 12:38 PM
dim accepted this revision.Feb 8 2022, 12:40 PM
dim added a subscriber: emaste.

LGTM (@emaste I've never used the standalone build, always as part of the 'full monty' all llvm components)

ldionne accepted this revision as: Restricted Project, Restricted Project, Restricted Project.Feb 8 2022, 3:57 PM

I will land this tomorrow morning (EST) and monitor potential failures throughout the day. If you get here through a failure caused by this commit, please ping @ldionne on libc++'s Discord channel.

This revision is now accepted and ready to land.Feb 8 2022, 3:57 PM
This revision was automatically updated to reflect the committed changes.
ldionne added a subscriber: simoll.Feb 9 2022, 6:59 AM

@simoll You are marked as responsible for the clang-ve-ninja built bot that broke because of this change here: https://lab.llvm.org/buildbot/#/builders/91/builds/3480.

Are you able to move that bot to the normal runtimes build? Right now the bot is doing:

cd "/scratch/buildbot/bothome/clang-ve-ninja/build/build_libunwind_ve" && cmake /scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/libunwind -G Ninja \
    -DLIBUNWIND_TARGET_TRIPLE=""ve-linux"" \
    -DCMAKE_C_COMPILER="""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/clang" \
    -DCMAKE_CXX_COMPILER="""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/clang++" \
    -DCMAKE_AR=""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/llvm-ar \
    -DCMAKE_RANLIB=""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/llvm-ranlib \
    -DCMAKE_C_COMPILER_TARGET=""ve-linux"" \
    -DCMAKE_CXX_COMPILER_TARGET=""ve-linux"" \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_INSTALL_PREFIX=""""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/lib/clang/15.0.0"" \
    -DLIBUNWIND_LIBDIR_SUFFIX="/linux/ve" \
    -DCMAKE_CXX_FLAGS="-nostdlib" \
    -DCMAKE_CXX_FLAGS_RELEASE="-O2" \
    -DCMAKE_C_FLAGS="-nostdlib" \
    -DCMAKE_C_FLAGS_RELEASE="-O2" \
    -DLIBUNWIND_LIBCXX_PATH=/scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/libcxx \
    -DLLVM_PATH=/scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/llvm

Instead, you want to be doing something like this:

cmake -S llvm-project/runtimes -B <BUILD_DIR> -G Ninja \
    -DLIBUNWIND_TARGET_TRIPLE=""ve-linux"" \
    -DCMAKE_C_COMPILER="<whatever>" \
    -DCMAKE_CXX_COMPILER="<whatever>" \
    -DCMAKE_AR="<whatever>" \
    -DCMAKE_RANLIB=""<whatever>" \
    -DCMAKE_C_COMPILER_TARGET="ve-linux" \
    -DCMAKE_CXX_COMPILER_TARGET="ve-linux" \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_INSTALL_PREFIX="<whatever>" \
    -DLIBUNWIND_LIBDIR_SUFFIX="/linux/ve" \
    -DCMAKE_CXX_FLAGS="-nostdlib" \
    -DCMAKE_CXX_FLAGS_RELEASE="-O2" \
    -DCMAKE_C_FLAGS="-nostdlib" \
    -DCMAKE_C_FLAGS_RELEASE="-O2" \
    -DLLVM_ENABLE_RUNTIMES="libunwind"

So basically, use <monorepo>/runtimes as the root of the CMake tree, drop LIBUNWIND_LIBCXX_PATH and LLVM_PATH, and add -DLLVM_ENABLE_RUNTIMES="libunwind".

FWIW, I think we should add a cmake warning in the 14.x release if building with the old standalone setup, telling the user that the configuration is deprecated, so users not reading the release notes get a better chance of noticing.

FWIW, I think we should add a cmake warning in the 14.x release if building with the old standalone setup, telling the user that the configuration is deprecated, so users not reading the release notes get a better chance of noticing.

That's a good point, see D119341.

No more issues so far -- the buildbot issues should have been shaken out by now, so it seems like the clang-ve-ninja failure might be the only one.

I'll stop actively monitoring this now. If you get here and want support, please reach out on the libc++ Discord channel -- you can also reply here but it's possible that I won't see it amid the flood of Phabricator emails.

@simoll You are marked as responsible for the clang-ve-ninja built bot that broke because of this change here: https://lab.llvm.org/buildbot/#/builders/91/builds/3480.

Are you able to move that bot to the normal runtimes build? Right now the bot is doing:

cd "/scratch/buildbot/bothome/clang-ve-ninja/build/build_libunwind_ve" && cmake /scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/libunwind -G Ninja \
    -DLIBUNWIND_TARGET_TRIPLE=""ve-linux"" \
    -DCMAKE_C_COMPILER="""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/clang" \
    -DCMAKE_CXX_COMPILER="""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/clang++" \
    -DCMAKE_AR=""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/llvm-ar \
    -DCMAKE_RANLIB=""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/bin/llvm-ranlib \
    -DCMAKE_C_COMPILER_TARGET=""ve-linux"" \
    -DCMAKE_CXX_COMPILER_TARGET=""ve-linux"" \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_INSTALL_PREFIX=""""/scratch/buildbot/bothome/clang-ve-ninja/build/build_llvm""/lib/clang/15.0.0"" \
    -DLIBUNWIND_LIBDIR_SUFFIX="/linux/ve" \
    -DCMAKE_CXX_FLAGS="-nostdlib" \
    -DCMAKE_CXX_FLAGS_RELEASE="-O2" \
    -DCMAKE_C_FLAGS="-nostdlib" \
    -DCMAKE_C_FLAGS_RELEASE="-O2" \
    -DLIBUNWIND_LIBCXX_PATH=/scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/libcxx \
    -DLLVM_PATH=/scratch/buildbot/bothome/clang-ve-ninja/build/../llvm-project/llvm

Instead, you want to be doing something like this:

cmake -S llvm-project/runtimes -B <BUILD_DIR> -G Ninja \
    -DLIBUNWIND_TARGET_TRIPLE=""ve-linux"" \
    -DCMAKE_C_COMPILER="<whatever>" \
    -DCMAKE_CXX_COMPILER="<whatever>" \
    -DCMAKE_AR="<whatever>" \
    -DCMAKE_RANLIB=""<whatever>" \
    -DCMAKE_C_COMPILER_TARGET="ve-linux" \
    -DCMAKE_CXX_COMPILER_TARGET="ve-linux" \
    -DCMAKE_BUILD_TYPE="Release" \
    -DCMAKE_INSTALL_PREFIX="<whatever>" \
    -DLIBUNWIND_LIBDIR_SUFFIX="/linux/ve" \
    -DCMAKE_CXX_FLAGS="-nostdlib" \
    -DCMAKE_CXX_FLAGS_RELEASE="-O2" \
    -DCMAKE_C_FLAGS="-nostdlib" \
    -DCMAKE_C_FLAGS_RELEASE="-O2" \
    -DLLVM_ENABLE_RUNTIMES="libunwind"

So basically, use <monorepo>/runtimes as the root of the CMake tree, drop LIBUNWIND_LIBCXX_PATH and LLVM_PATH, and add -DLLVM_ENABLE_RUNTIMES="libunwind".

Thanks. We are testing this now and will switch to a runtimes build soon.

For the record, I just want to say that this sucks a lot for Gentoo and I'm really unhappy with the recent trend of ignoring other people's effort and destroying their use cases for one's own convenience. But I guess I have to live with it. It's just sad how much effort I've wasted fixing things every time standalone builds were broken, and in the end the working code is being broken for the sake of preventing others from maintaining standalone builds. In the end, we'll manage but probably at the cost of preventing our users from installing GNU libunwind and libc++ simultaneously but I guess people using binary distros just don't care that could be a better way.

Standalone build have been deprecated for some time now,

@ldionne : can you point me to the thread / documentation that states so? I can find https://discourse.llvm.org/t/standalone-build-for-clang-lld-fix-or-remove but this thread doesn't lead me to the same conclusion.

ldionne added a subscriber: phosek.Feb 10 2022, 1:05 PM

Thanks. We are testing this now and will switch to a runtimes build soon.

Sure! Let me know if you run into issues with this. Remember, you want to be rooting your CMake invocation at <monorepo>/runtimes, not <monorepo>/llvm. The former is going to build similarly to the previous standalone build, and the latter would cause Clang to be bootstrapped first, which is a bigger change from what you were doing previously.

For the record, I just want to say that this sucks a lot for Gentoo and I'm really unhappy with the recent trend of ignoring other people's effort and destroying their use cases for one's own convenience. But I guess I have to live with it. It's just sad how much effort I've wasted fixing things every time standalone builds were broken, and in the end the working code is being broken for the sake of preventing others from maintaining standalone builds. In the end, we'll manage but probably at the cost of preventing our users from installing GNU libunwind and libc++ simultaneously but I guess people using binary distros just don't care that could be a better way.

I'm going to stay nice despite the accusatory tone of your message.

This change has been cooking for a long time, just like the upcoming removal of LLVM_ENABLE_PROJECTS for libcxx, libcxxabi and libunwind. This is not about "convenience" or breaking people just for the sake of it, as this commit might make you think. The reason why this commit is just adding message(FATAL_ERROR is because I wanted to keep the cleanup that will follow separate, in case this needed to be reverted.

Standalone builds are a pain to maintain. We need to tie the various projects together with a bunch of CMake variables, which adds a bunch of complexity. Instead, it's much easier if we can assume that things are laid out in a specific way in the repository. We'll be able to reuse a bunch of code for all the runtimes, and as a result make things simpler, easier to maintain and even more customizable in the future. I already have a bunch of ideas that can't be implemented right now because we'd have to triplicate the code in libcxx, libcxxabi and libunwind, which is just terrible.

Also, there is a common misconception about the removal of standalone builds. Removing standalone builds DOES NOT MEAN that we remove any customizability in your builds. You can still build just one project, nothing changes here. Please go and read our documentation on the supported ways to build: https://libcxx.llvm.org/BuildingLibcxx.html#the-default-build. If you want to build just libcxx, you can always do this:

cmake -G Ninja -S <monorepo>/runtimes -B build -DLLVM_ENABLE_RUNTIMES="libcxx"
ninja -C build cxx

(You'll need to pass the usual options for specifying the ABI library to use unless you also add libcxxabi to LLVM_ENABLE_RUNTIMES)

This is basically the same as a standalone build -- it doesn't build Clang, it doesn't build libc++abi, it doesn't build libunwind. It builds libc++ only, and gives you exactly the same control you had before on your build. If you want to build libc++abi or libunwind separately, you can run three of these builds with a different LLVM_ENABLE_RUNTIMES=<foo> each time, like you do for standalone builds. It's cumbersome to do that, but nobody's going to stop you. If there is some customizability you think you are losing with the new way to build, please let me know concretely what it is, and we'll try to support it. But I don't think there should be anything that you could do with standalone builds that you can't do anymore, except laying out the projects in arbitrary ways (but dropping support for that is the whole point, since it's hard to support and provides virtually no benefit).

Standalone build have been deprecated for some time now,

@ldionne : can you point me to the thread / documentation that states so? I can find https://discourse.llvm.org/t/standalone-build-for-clang-lld-fix-or-remove but this thread doesn't lead me to the same conclusion.

There was discussion about this in 2020 here, and then various patches by @phosek, myself and others over the last 1.5 years to work towards what we're finally doing today. The direction we ended up taking for the "unified standalone build" is basically what you do when you root the CMake invocation at <monorepo>/runtimes.

I'm sorry about my tone and thank you for the explanation.

ldionne added a comment.EditedFeb 11 2022, 6:40 AM

I'm sorry about my tone and thank you for the explanation.

No worries. Migrations tend to cause tensions by their very nature. I saw you opened a couple of patches to fix some quirks with the new way to build. Thanks -- I'm looking forward to collaborating to make sure we're still addressing Gentoo's use cases (and others) in a satisfactory way.