test-release.sh: Add -openmp option.

[hans]

Summary

This adds an -openmp option (off by default) for building the OpenMP run-time.

The run-time is still not well integrated with the regular LLVM build, but this allows building the run-time "on the side".

I'm thinking that we could provide separate pre-builts of the run-time (e.g. for x86_64 and Mac) at http://llvm.org/releases/download.html as a convenience for users who wish to experiment with Clang's OpenMP run-time.

Let me know what you think.

[hans]

Ping?

[rengolin]

Hi Hans,

Sorry, this fell out of my radar. I think it's an important addition, I just think it should be part of the main build and behave like the others. I have left some comments inline.

Also, about the tar ball, are we going to start uploading both of them? Or is it maybe better to bundle the OpenMP tar inside the LLVM+Clang one? If we make this part of the main build, it may be easier to bundle all together.

Either way is fine for me, but would be good if all target testers / code owners would agree on one for a reason. :)

cheers,
--renato

utils/release/test-release.sh

440 ↗

(On Diff #30601)

"set -x" considered harmful. :)

442 ↗

(On Diff #30601)

This better stay in PhaseM/Release/... like the rest.

446 ↗

(On Diff #30601)

Hasn't this moved out of /usr/local?

454 ↗

(On Diff #30601)

Better not to override a global variable with the same name, it's confusing.

Call it OpenMPPackage or something.

455 ↗

(On Diff #30601)

I think this should be inside the main build process, so that we use the same standard openmp-3.7.0-install and then rename / tar like llvmCore-3.7.0-install, etc.

[hans]

In D11494#213401, @rengolin wrote:

Hi Hans,

Sorry, this fell out of my radar. I think it's an important addition, I just think it should be part of the main build and behave like the others. I have left some comments inline.

Also, about the tar ball, are we going to start uploading both of them? Or is it maybe better to bundle the OpenMP tar inside the LLVM+Clang one? If we make this part of the main build, it may be easier to bundle all together.

Either way is fine for me, but would be good if all target testers / code owners would agree on one for a reason. :)

I'm hoping that for the next release, the openmp builds and tests as any other LLVM module, by simply putting it under tools/ or projects. Currently it builds on the side and I'm not sure how to test it.

We could of course still work around that in the script and bake it into the regular LLVM+Clang tarball. However, I'm kind of hesitant to do so this late in the release process when none of us have really tested it.

My plan was to let those testers who wish try to build it, and then we'd upload the tarballs on the side, as a convenience for those who want to experiment with the openmp support.

utils/release/test-release.sh

440 ↗

(On Diff #30601)

Removed.

442 ↗

(On Diff #30601)

Done.

446 ↗

(On Diff #30601)

No, but we make sure the /usr/local prefix doesn't go in the tarball.

454 ↗

(On Diff #30601)

Done.

455 ↗

(On Diff #30601)

I've commented on that below. I'm not sure we're ready yet.

[hans]

Addressing Renato's comments.

[rengolin]

LGTM, but we better add some comment saying it's an experimental feature and not meant for official testing the release (which then should be done at every stage).

llvm/trunk/utils/release/test-release.sh

 use_gzip="no"
 do_checkout="yes"
 do_debug="no"
 do_asserts="no"
 do_compare="yes"
 do_rt="yes"
 do_libs="yes"
 do_test_suite="yes"
+do_openmp="no"
 BuildDir="`pwd`"
 use_autoconf="no"
 ExtraConfigureFlags=""
 ExportBranch=""
 
 function usage() {
     echo "usage: `basename $0` -release X.Y.Z -rc NUM [OPTIONS]"
     echo ""
function usage() {
     echo " -use-gzip            Use gzip instead of xz."
     echo " -configure-flags FLAGS  Extra flags to pass to the configure step."
     echo " -use-autoconf        Use autoconf instead of cmake"
     echo " -svn-path DIR        Use the specified DIR instead of a release."
     echo "                      For example -svn-path trunk or -svn-path branches/release_37"
     echo " -no-rt               Disable check-out & build Compiler-RT"
     echo " -no-libs             Disable check-out & build libcxx/libcxxabi/libunwind"
     echo " -no-test-suite       Disable check-out & build test-suite"
+    echo " -openmp              Check out and build the OpenMP run-time (experimental)"
 }
 
 if [ `uname -s` = "Darwin" ]; then
   # compiler-rt doesn't yet build with CMake on Darwin.
   use_autoconf="yes"
 fi
 
 while [ $# -gt 0 ]; do
case $1 in
             do_rt="no"
             ;;
         -no-libs )
             do_libs="no"
             ;;
         -no-test-suite )
             do_test_suite="no"
             ;;
+        -openmp )
+            do_openmp="yes"
+            ;;
         -help | --help | -h | --h | -\? )
             usage
             exit 0
             ;;
         * )
             echo "unknown option: $1"
             usage
             exit 1
if [ $do_rt = "yes" ]; then
   projects="$projects compiler-rt"
 fi
 if [ $do_libs = "yes" ]; then
   projects="$projects libcxx libcxxabi libunwind"
 fi
 if [ $do_test_suite = "yes" ]; then
   projects="$projects test-suite"
 fi
+if [ $do_openmp = "yes" ]; then
+  projects="$projects openmp"
+fi
 
 # Go to the build directory (may be different from CWD)
 BuildDir=$BuildDir/$RC
 mkdir -p $BuildDir
 cd $BuildDir
 
 # Location of log files.
 LogDir=$BuildDir/logs
if [ "$use_gzip" = "yes" ]; then
       tar cfz $BuildDir/$Package.tar.gz $Package
     else
       tar cfJ $BuildDir/$Package.tar.xz $Package
     fi
     mv $Package llvmCore-$Release-$RC.install/usr/local
     cd $cwd
 }
 
+# Build and package the OpenMP run-time. This is still experimental and not
+# meant for official testing in the release, but as a way for providing
+# binaries as a convenience to those who want to try it out.
+function build_OpenMP() {
+    cwd=`pwd`
+
+    rm -rf $BuildDir/Phase3/openmp
+    rm -rf $BuildDir/Phase3/openmp.install
+    mkdir -p $BuildDir/Phase3/openmp
+    cd $BuildDir/Phase3/openmp
+    clang=$BuildDir/Phase3/Release/llvmCore-$Release-$RC.install/usr/local/bin/clang
+
+    echo "#" cmake -DCMAKE_C_COMPILER=${clang} -DCMAKE_CXX_COMPILER=${clang}++ \
+            -DCMAKE_BUILD_TYPE=Release -DLIBOMP_MICRO_TESTS=on \
+            $BuildDir/openmp.src/runtime
+    cmake -DCMAKE_C_COMPILER=${clang} -DCMAKE_CXX_COMPILER=${clang}++ \
+            -DCMAKE_BUILD_TYPE=Release -DLIBOMP_MICRO_TESTS=on \
+            $BuildDir/openmp.src/runtime
+
+    echo "# Building OpenMP run-time"
+    echo "# ${MAKE} -j $NumJobs VERBOSE=1"
+    ${MAKE} -j $NumJobs VERBOSE=1
+    echo "# ${MAKE} libomp-micro-tests VERBOSE=1"
+    ${MAKE} libomp-micro-tests VERBOSE=1
+    echo "# ${MAKE} install DESTDIR=$BuildDir/Phase3/openmp.install"
+    ${MAKE} install DESTDIR=$BuildDir/Phase3/openmp.install
+
+    OpenMPPackage=OpenMP-$Triple
+    mv $BuildDir/Phase3/openmp.install/usr/local $BuildDir/$OpenMPPackage
+    cd $BuildDir
+    tar cvfJ $BuildDir/$OpenMPPackage.tar.xz $OpenMPPackage
+    mv $OpenMPPackage $BuildDir/Phase3/openmp.install/usr/local
+    cd $cwd
+}
+
 # Exit if any command fails
 # Note: pipefail is necessary for running build commands through
 # a pipe (i.e. it changes the output of ``false | tee /dev/null ; echo $?``)
 set -e
 set -o pipefail
 
 if [ "$do_checkout" = "yes" ]; then
     export_sources
for Flavor in $Flavors ; do
     echo "  Build:       $Flavor"
     echo "  System Info: "
     echo "    `uname -a`"
     echo "********************************************************************************"
     echo ""
 
     c_compiler="$CC"
     cxx_compiler="$CXX"
-
     llvmCore_phase1_objdir=$BuildDir/Phase1/$Flavor/llvmCore-$Release-$RC.obj
     llvmCore_phase1_destdir=$BuildDir/Phase1/$Flavor/llvmCore-$Release-$RC.install
 
     llvmCore_phase2_objdir=$BuildDir/Phase2/$Flavor/llvmCore-$Release-$RC.obj
     llvmCore_phase2_destdir=$BuildDir/Phase2/$Flavor/llvmCore-$Release-$RC.install
 
     llvmCore_phase3_objdir=$BuildDir/Phase3/$Flavor/llvmCore-$Release-$RC.obj
     llvmCore_phase3_destdir=$BuildDir/Phase3/$Flavor/llvmCore-$Release-$RC.install
if [ "$do_compare" = "yes" ]; then
             # sed adds a newline to the output, so pass $p3 through sed too.
             if ! cmp -s <(sed -e 's,Phase2,Phase3,g' $p2) <(sed -e '' $p3) \
                     16 16 ; then
                 echo "file `basename $p2` differs between phase 2 and phase 3"
             fi
         done
     fi
 done
+
+if [ $do_openmp = "yes" ]; then
+  build_OpenMP
+fi
+
 ) 2>&1 | tee $LogDir/testing.$Release-$RC.log
 
 package_release
 
 set +e
 
 # Woo hoo!
 echo "### Testing Finished ###"