[zorg] Add libomp builders with OMPT

[Hahnfeld]

Summary

Dmitri Gribenko said that it is ok to "add more (reasonably-sized) jobs to [his] buildbot" (http://lists.llvm.org/pipermail/openmp-dev/2016-March/001169.html)

[gribozavr]

LGTM.

[gkistanova]

The patch looks fine.
Except one strange thing with the category.

buildbot/osuosl/master/config/builders.py

822 ↗

(On Diff #52996)

The category gets overwritten to 'openmp' in get_builders().
Why setting it here explicitly?

[Hahnfeld]

buildbot/osuosl/master/config/builders.py

822 ↗

(On Diff #52996)

Looks like I did some copy&paste from the other configs. Will update the patch and remove the category from all related buildes

[Hahnfeld]

Remove category because it is set in get_builders anyway

[gkistanova]

buildbot/osuosl/master/config/builders.py

827 ↗

(On Diff #54035)

The env value looks specific to a particular buildslave rather than the builder. Do you expect each of the slaves be setup exactly the same way, including the user account name and such?

This is not a blocker for this patch, though. Just something that caught my eye.

zorg/buildbot/builders/Libiomp5Builder.py

93 ↗

(On Diff #54035)

Could you use the NinjaCommand from zorg.buildbot.commands.NinjaCommand instead, please?
Something like this would do:

f.addStep(NinjaCommand(name="build_llgo",
                       targets=["FileCheck"],
                       haltOnFailure=True,
                       description=["make ompt test utils"],
                       workdir=llvm_builddir,
                       env=merged_env))

98 ↗

(On Diff #54035)

You are missing ) here.

[Hahnfeld]

Add missing )

[Hahnfeld]

zorg/buildbot/builders/Libiomp5Builder.py

93 ↗

(On Diff #54164)

Can this be done in a separate patch? This should then be adjusted in the whole file...

[gkistanova]

LGTM, assuming the patch with NinjaCommand will follow shortly.

zorg/trunk/buildbot/osuosl/master/config/builders.py

 def _get_openmp_builders():
     return [
         {'name': "libomp-gcc-x86_64-linux-debian",
          'slavenames':["gribozavr4"],
          'builddir':"libomp-gcc-x86_64-linux-debian",
          'factory' : Libiomp5Builder.getLibompCMakeBuildFactory(
                          c_compiler="gcc",
                          cxx_compiler="g++",
-                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'}),
-         'category' : 'libomp'},
+                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'})},
 
         {'name': "libomp-clang-x86_64-linux-debian",
          'slavenames':["gribozavr4"],
          'builddir':"libomp-clang-x86_64-linux-debian",
          'factory' : Libiomp5Builder.getLibompCMakeBuildFactory(
                          c_compiler="clang",
                          cxx_compiler="clang++",
-                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'}),
-         'category' : 'libomp'},
+                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'})},
+
+        {'name': "libomp-ompt-gcc-x86_64-linux-debian",
+         'slavenames':["gribozavr4"],
+         'builddir':"libomp-ompt-gcc-x86_64-linux-debian",
+         'factory' : Libiomp5Builder.getLibompCMakeBuildFactory(
+                         c_compiler="gcc",
+                         cxx_compiler="g++",
+                         ompt=True,
+                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'})},
+
+        {'name': "libomp-ompt-clang-x86_64-linux-debian",
+         'slavenames':["gribozavr4"],
+         'builddir':"libomp-ompt-clang-x86_64-linux-debian",
+         'factory' : Libiomp5Builder.getLibompCMakeBuildFactory(
+                         c_compiler="clang",
+                         cxx_compiler="clang++",
+                         ompt=True
+                         env={'PATH':'/home/llvmbb/bin/clang-latest/bin:/home/llvmbb/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin'})},
         ]
 
 def _get_libcxx_builders():
     return [
         {'name': 'libcxx-libcxxabi-x86_64-linux-debian',
          'slavenames': ['gribozavr4'],
          'builddir': 'libcxx-libcxxabi-x86_64-linux-debian',
          'factory': LibcxxAndAbiBuilder.getLibcxxAndAbiBuilder(

zorg/trunk/zorg/buildbot/builders/Libiomp5Builder.py

 import os
 
 import buildbot
 import buildbot.process.factory
 from buildbot.steps.source import SVN
 from buildbot.steps.shell import ShellCommand
 from buildbot.steps.shell import WarningCountingShellCommand
 from buildbot.process.properties import WithProperties
 
 import zorg.buildbot.commands as commands
 import zorg.buildbot.commands.LitTestCommand as lit_test_command
 
 
-def getLibompCMakeBuildFactory(clean=True, env=None, test=True, c_compiler="gcc", cxx_compiler="g++"):
+def getLibompCMakeBuildFactory(clean=True, env=None, ompt=False, test=True, c_compiler="gcc", cxx_compiler="g++"):
 
     # Prepare environmental variables. Set here all env we want everywhere.
     merged_env = {
         'TERM' : 'dumb' # Make sure Clang doesn't use color escape sequences.
                  }
     if env is not None:
         # Overwrite pre-set items with the given ones, so user can set anything.
         merged_env.update(env)
def getLibompCMakeBuildFactory(clean=True, env=None, ompt=False, test=True, c_compiler="gcc", cxx_compiler="g++"):
     else:
         f.addStep(WarningCountingShellCommand(name='make llvm utils build',
                                               command=['ninja', 'LLVMX86Utils'],
                                               haltOnFailure=True,
                                               description='make llvm utils build',
                                               workdir=llvm_builddir,
                                               env=merged_env))
 
+    if ompt:
+        f.addStep(WarningCountingShellCommand(name='make ompt test utils',
+                                              command=['ninja', 'FileCheck'],
+                                              haltOnFailure=True,
+                                              description='make ompt test utils',
+                                              workdir=llvm_builddir,
+                                              env=merged_env))
+
     # Add clang/llvm-lit to PATH
     merged_env.update( { 'PATH' : WithProperties("%(workdir)s/" + llvm_builddir + "/bin:" + "${PATH}")} )
 
     # CMake libomp
-    f.addStep(ShellCommand(name='cmake libomp',
-                           command=["cmake", "../"+openmp_srcdir,
-                                    "-DCMAKE_C_COMPILER="+c_compiler,
-                                    "-DCMAKE_CXX_COMPILER="+cxx_compiler],
+             "-DCMAKE_CXX_COMPILER="+cxx_compiler]
+    if ompt:
+        command.append("-DLIBOMP_OMPT_SUPPORT=ON")
+    f.addStep(ShellCommand(name='cmake libomp',
+                           command=command,
                            haltOnFailure=True,
                            description='cmake libomp',
                            workdir=openmp_builddir,
                            env=merged_env))
 
     # Make libomp
     f.addStep(WarningCountingShellCommand(name='make libomp build',
                                           command=['make'],