Utility for checking out llvm, clang, and associated tools and configuring a build folder

[lefticus]

Summary

Based on Chandler's talk at Pacific++ 2017

This script is intended for Unix-like hosts. It checks out llvm, clang, lld and utilities from svn and configures a build directory using ninja. It is intended to be very simple, providing only minimal options.

[joerg]

utils/checkout_llvm.sh

1

Add an explicit #!/bin/sh

13

Please just use cd llvm/tools and cd .././ below. Same for the rest.

40

I'd suggest putting $2 into a variable, shifting twice and then appending any remaining arguments. Makes the script a bit more useful and allows overriding undesirable settings.

[MatzeB]

This whole script is pretty opinionated:

• Use of svn vs. the git mirrors
• picking a specific set of llvm projects
• picking a specific set of cmake options (BUILD_TYPE=Release, ENABLE_ASSERTIONS=OFF, ninja...)

How can we avoid giving the impression that these are somehow recommended? (Because really I could start 3 longish bikeshed-like discussions now about what choices would be better here...)

[chandlerc]

In D41957#973990, @MatzeB wrote:

This whole script is pretty opinionated:

• Use of svn vs. the git mirrors
• picking a specific set of llvm projects
• picking a specific set of cmake options (BUILD_TYPE=Release, ENABLE_ASSERTIONS=OFF, ninja...)

How can we avoid giving the impression that these are somehow recommended? (Because really I could start 3 longish bikeshed-like discussions now about what choices would be better here...)

I think this is an important question.

Fundamentally, I *like* this script precisely because it is a bit opinionated. For some users, the fact that there are options doesn't matter and they'd just like a "reasonable" checkout (for some highly subjective definition). But I totally see your point and am not sure how to avoid this becoming some tacit endorsement -- which it should not be!

Maybe we could give it a name and prefix the script with that name so that it was very clear that this is just *one* way to check out LLVM. My suggestion for this would be something along the lines of toolchain-svn and the script could be toolchain_svn_checkout.sh or some such. The idea being, that this includes all the LLVM subprojects that are needed to build a complete LLVM toolchain. Maybe there are better names for this slice of opinionated options though. The goal (I think) should be to capture the motivation that leads to the opinionated parts of the script, or if it is arbitrary, the choice. So the subproject set is probably based on a particular motivation / use case, but it's an arbitrary choice to use svn vs. git.

utils/checkout_llvm.sh

25–35

If you want these three, you probably also want openmp.

[MatzeB]

In D41957#974820, @chandlerc wrote:

In D41957#973990, @MatzeB wrote:

This whole script is pretty opinionated:

• Use of svn vs. the git mirrors
• picking a specific set of llvm projects
• picking a specific set of cmake options (BUILD_TYPE=Release, ENABLE_ASSERTIONS=OFF, ninja...)

How can we avoid giving the impression that these are somehow recommended? (Because really I could start 3 longish bikeshed-like discussions now about what choices would be better here...)

I think this is an important question.

Fundamentally, I *like* this script precisely because it is a bit opinionated. For some users, the fact that there are options doesn't matter and they'd just like a "reasonable" checkout (for some highly subjective definition). But I totally see your point and am not sure how to avoid this becoming some tacit endorsement -- which it should not be!

Maybe we could give it a name and prefix the script with that name so that it was very clear that this is just *one* way to check out LLVM. My suggestion for this would be something along the lines of toolchain-svn and the script could be toolchain_svn_checkout.sh or some such. The idea being, that this includes all the LLVM subprojects that are needed to build a complete LLVM toolchain. Maybe there are better names for this slice of opinionated options though. The goal (I think) should be to capture the motivation that leads to the opinionated parts of the script, or if it is arbitrary, the choice. So the subproject set is probably based on a particular motivation / use case, but it's an arbitrary choice to use svn vs. git.

get_started.sh? Well git vs. svn is really personal preference so okay, the set of projects is also fine in that it gives you a typical clang based toolchain which should be the most popular use of llvm. I'm not sure why you would enable release mode and turn of assertions, I would expect someone that bothers to checkout from version control wants to do development and should have at least assertions turned on...

[probinson]

There's a medium-term goal of migrating to git, so an svn-based script seems like the wrong direction.
I've been using the experimental git mono-repo for a while and I'm finding it pretty convenient. A sample configuration script for setting up a useful subset of projects and canonical CMake parameters seems like a good helpful starting point.

utils/checkout_llvm.sh

+if [ $# -gt 1 ]
+  then
+    echo "Checking out LLVM '$1' branch from svn into '`pwd`/llvm' and setting install prefix to '$2'"
+    echo "Press Return To Continue"
+    read $VAR
+  else
+    echo "Usage: $0 <branch name> <install prefix>"
+    exit 
+fi
+
+svn co http://llvm.org/svn/llvm-project/llvm/$1 llvm
+
+pushd llvm/tools
+svn co http://llvm.org/svn/llvm-project/lld/$1 lld
+popd
+
+pushd llvm/tools
+svn co http://llvm.org/svn/llvm-project/cfe/$1 clang
+popd
+
+pushd llvm/tools/clang/tools
+svn co http://llvm.org/svn/llvm-project/clang-tools-extra/$1 extra
+popd
+
+pushd llvm/projects
+svn co http://llvm.org/svn/llvm-project/compiler-rt/$1 compiler-rt
+popd
+
+pushd llvm/projects
+svn co http://llvm.org/svn/llvm-project/libcxx/$1 libcxx
+popd
+
+pushd llvm/projects
+svn co http://llvm.org/svn/llvm-project/libcxxabi/$1 libcxxabi
+popd
+
+
+mkdir -p llvm-build
+pushd llvm-build
+CXX=$COMPILER cmake -G "Ninja" ../llvm -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_INSTALL_PREFIX:PATH=$2 -DCOMPILER_RT_INCLUDE_TESTS:BOOL=OFF -DLLVM_ENABLE_ASSERTIONS:BOOL=OFF -DBUILD_SHARED_LIBS:BOOL=ON -DLLVM_OPTIMIZED_TABLEGEN:BOOL=ON
+popd
+
+