[zorg] Run test-suite with flang-new

[rovka]

Summary

At the moment, clang-aarch64-full-2stage is running the
llvm-test-suite with flang-to-external-fc, which is a wrapper script
involving both flang and gfortran.

This patch switches to using the flang-new driver, therefore removing
our dependence on gfortran.

However, support for generating executables with flang-new is still
experimental and needs a few flags to help it along.

Furthermore, we need an environment variable when running the
test-suite (https://flang.llvm.org/docs/FortranLLVMTestSuite.html#:~:text=to%20set%20the-,NO_STOP_MESSAGE,-environment%20variable%20to).

[DavidSpickett]

so it would be nice if we could source this from the worker instead (I don't know how to do that).

So that you don't have to make a change to zorg each time one is fixed? I'm sure it's possible but it's probably more trouble than it's worth vs. waiting for a change to zorg to make it's way to the bot.

You could make the filter list in the test suite instead. I did one a while back that was based on a flag being supported https://github.com/llvm/llvm-test-suite/commit/9fb1916e98466baa2677f32b61626b03afc978a7, maybe you can get a "is llvm flang" check in there.

[DavidSpickett]

Also why exactly is NO_STOP_MESSAGE needed? I guess fortran tests do some stdout comparison and this would otherwise print a line that gets in the way.

[DavidSpickett]

I'm not sure where but it would be worth noting that running the test suite like this requires a build of libpgmath and cmake >= 3.24.

I think we (Linaro) are the primary (if not only) people running it but even we will forget at some point.

[rovka]

@DavidSpickett You're right, it's probably better to put the list in the test-suite. I'll update accordingly.

In D130628#3682032, @DavidSpickett wrote:

Also why exactly is NO_STOP_MESSAGE needed? I guess fortran tests do some stdout comparison and this would otherwise print a line that gets in the way.

Yep, that's exactly it :) This is documented here.

[gkistanova]

Hi Diana,

You decided to put the filter list in the test-suite, so this is just for the sake of completeness.

Since you plan to fix those failing tests in the very near future, you may want to either leave them failing and fix them while the builder is in the staging, or set the LIT_FILTER_OUT envvar for your build-worker account locally on that machine.

Setting it locally for the bot account environment is a straightforward, it will be listed in the envvars in the buildbot, and you will be able to change it easily without circling changes to zorg.

[DavidSpickett]

it will be listed in the envvars in the buildbot

This is good to know, my worry with setting it locally was I thought you'd have to have access to the bot itself to see the value.

[awarzynski]

In D130628#3682115, @rovka wrote:

@DavidSpickett You're right, it's probably better to put the list in the test-suite. I'll update accordingly.

In D130628#3682032, @DavidSpickett wrote:

Also why exactly is NO_STOP_MESSAGE needed? I guess fortran tests do some stdout comparison and this would otherwise print a line that gets in the way.

Yep, that's exactly it :) This is documented here.

Perhaps worth adding a comment somewhere so that the rationale for this is clearer (either in the code or in the commit message).

zorg/buildbot/builders/ClangBuilder.py

512

What's this flag for? libpgmath?

[rovka]

Updated after review comments. Thanks!

zorg/buildbot/builders/ClangBuilder.py

512

No, this is actually not needed anymore, I'll remove it (IIRC when I started experimenting with this, the runtime libraries weren't being properly installed, but I fixed that in the meantime). Thanks for catching this!

[gkistanova]

LGTM

[rovka]

Hi Galina,

I think the staging buildmaster got restarted after this change, but not the production one. Could you please restart the production buildmaster when you get a chance?

Thanks!

buildbot/osuosl/master/config/builders.py

# Clang builders.
     'builddir': "clang-aarch64-full-2stage",
     'factory' : ClangBuilder.getClangCMakeBuildFactory(
                     clean=False,
                     checkout_flang=True,
                     checkout_lld=False,
                     useTwoStage=True,
                     testStage1=False,
                     runTestSuite=True,
+                    env={
+                        'NO_STOP_MESSAGE':'1', # For Fortran test-suite
+                    },
                     testsuite_flags=[
                         '--cppflags', '-mcpu=cortex-a57',
                         '--threads=32', '--build-threads=32'],
                     extra_cmake_args=[
                         "-DCMAKE_C_FLAGS='-mcpu=cortex-a57'",
                         "-DCMAKE_CXX_FLAGS='-mcpu=cortex-a57'",
                         "-DLLVM_LIT_ARGS='-v -j4'"])},
 

zorg/buildbot/builders/ClangBuilder.py

if stage1_upload_directory:
                           use_pixz_compression=use_pixz_compression,
                           xz_compression_factor=xz_compression_factor)
 
     # Compute the cmake define flag to set the C and C++ compiler to clang. Use
     # clang-cl if we used MSVC for stage1.
     if not vs:
         cc = 'clang'
         cxx = 'clang++'
-        fc = 'flang-to-external-fc'
+        fc = 'flang-new'
     else:
         cc = 'clang-cl.exe'
         cxx = 'clang-cl.exe'
-        fc = 'flang.exe'
+        fc = 'flang-new.exe'
 
 
     ############# STAGE 2
     if useTwoStage:
         # We always cleanly build the stage 2. If the compiler has been
         # changed on the stage 1, we cannot trust any of the intermediate file
         # from the old compiler. And if the stage 1 compiler is the same, we
         # should not build in the first place.
if runTestSuite:
                               '--test-suite', test_suite_dir,
                               '--cc', cc,
                               '--cxx', cxx,
                               '--use-lit', lit]
             # Enable fortran if flang is checked out
             if checkout_flang:
                 fortran_flags = [
                         '--cmake-define=TEST_SUITE_FORTRAN:STRING=ON',
+                        '--cmake-define=CMAKE_Fortran_FLAGS:STRING=' +
+                            '-flang-experimental-exec -lpgmath',
                         util.Interpolate(
                             '--cmake-define=CMAKE_Fortran_COMPILER=' +
                             '%(prop:builddir)s/'+compiler_path+'/bin/'+fc)]
                 test_suite_cmd.extend(fortran_flags)
             # Append any option provided by the user
             test_suite_cmd.extend(testsuite_flags)
 
         # Only submit if a URL has been specified
         if submitURL is not None:
             if not isinstance(submitURL, list):