[zorg] Windows/Linux CMake builders for Hexagon using common getClangCMakeBuildFactory

[rfoos]

Summary

This depends on updates to getClangCMakeBuildFactory.

The builder definitions are examples of Linux and Windows(MSVC) builds using a single build factory.

[rengolin]

buildbot/osuosl/master/config/builders.py

401

Will we have to add this to the current builders, too?

[rfoos]

buildbot/osuosl/master/config/builders.py

401

Yes the builder definition for a non-ninja build must pass everything in the command line.

This looks more difficult on the surface, but in fact it removes all the special case code from the builder. It is a much simpler way when you look at adding code to cover both windows and linux builds.

The default is correctly formed ninja that works on Linux. This goes with the original intent of the builder.

To use make or MSBuild or anything else the whole cmd line for each step is passed.

This eliminates the complexity of the MSVC factories.

The msbuild factories hardcoded or passed by argument the additional information for configuration (i.e. RelWithDebInfo), compiler version, environment, and targets/project files. (i.e. MSBuild doesn't use targets but rather named project files)

There's no real smart way to map "ninja install" to "MSBuild INSTALL.vcxproj" and many of the others.

Specifying everything in the command line arguments simplified everything.

• additional planned change

I will add jobs and loadaverage to the command line arguments too. These are supposed to be passed as properties defined by the buildslave. The current code has to change anyway.

The reason to move jobs/loadaverage out to the command line arguments are special -j/-l cases for make/ninja, lit, MSBuild, and Windows (warnings on loadaverage).

The logic for jobs/loadaverage brings back special case code, so it will move to the arguments, and the relevant commands will be loaded wrapped in WithProperties/Interpolate to allow any property to be used.

[gkistanova]

buildbot/osuosl/master/config/builders.py

401

The params seem generic enough and the same for all the similar VS builders.
How about defining them once for all the similar builders?

[rfoos]

buildbot/osuosl/master/config/builders.py

401

I'm thinking you mean a wrapper builder that takes an existing VS builder and translates the arguments into a call to the generic cmake builder?

If so Yes!

After the current one is used by a few others, and is running production for 2 days.

There are bound to be a few changes when others start using this, and waiting a short period would make testing a lot easier.

Wrapper builders that demonstrate some more complicated command lines would make it both easy to use, and serve as documentation for some more complicated use cases.

[gkistanova]

buildbot/osuosl/master/config/builders.py

401

Yes, I was thinking of a wrapper builder.
And of course this is something which could wait till later.

The only potential challenge for wrapper builders I see is to make sure they will reload safely and correctly during the reconfig.

[rengolin]

Any progress in this? If not, can we close this review?

[rfoos]

Superseded by D6866

buildbot/osuosl/master/config/builders.py

                                            '--target=hexagon-unknown-elf',
                                            '--enable-targets=hexagon'])},
 
+        {'name': "clang-msvc-x86_64",
+         'slavenames': ["hexagon-build-01"],
+         'builddir': "builddir\\clang-msvc-x86_64",
+         'factory': ClangBuilder.getClangCMakeBuildFactory(
+                    jobs=None,
+                    loadaverage=None,
+                    cmakeGenerator="Visual Studio 12",
+                    cmakeProjectfile="ALL_BUILD.vcxproj",
+                    # MSBuild command with user arguments.
+                    build_cmd=["MSBuild","/p:Configuration=RelWithDebInfo",
+                    "/maxcpucount", "ALL_BUILD.vcxproj"],
+                    build_install_cmd=["MSBuild","/p:Configuration=RelWithDebInfo",
+                    "/maxcpucount", "INSTALL.vcxproj"],
+                    build_check_cmd=["MSBuild","/p:Configuration=RelWithDebInfo",
+                    "/maxcpucount", "check-clang.vcxproj"],
+                    build_check_dir='stage1/tools/clang/test',
+
+                    # Phase 2 build with built Clang.
+                    cmakeGenerator2='Ninja',     
+                    cmakeProjectfile2='build.ninja',
+                    build_cmd2=['ninja'],
+                    build_install_cmd2=['ninja', 'install'],
+                    build_check_cmd2=['ninja', 'check-all'],
+
+                    # Multi-stage compilation
+                    useTwoStage=False,
+                    testStage1=True,
+                    stage1_config='RelWithDebInfo',
+                    stage2_config='RelWithDebInfo', 
+                    slave_envCmd=r""""%VS120COMNTOOLS%\vsvars32.bat" %PROCESSOR_ARCHITECTURE% & set""",
+                    slave_envCmd2=r"""set""",
+                    extra_cmake_args=[
+                    '-DLLVM_BUILD_RUNTIME:BOOL=OFF',
+                    '-DLIBCLANG_BUILD_STATIC=ON',
+                    '-DLLVM_ENABLE_PIC:BOOL=ON',
+                    '-DLLVM_MINIMAL_INSTALL:BOOL=ON',
+                    '-DLLVM_ENABLE_ASSERTIONS:BOOL=ON',
+                    '-DLLVM_ENABLE_STATS:BOOL=ON',
+                    '-DLLVM_LIT_ARGS:=--verbose --no-progress-bar --param build_config=Win32',
+                    '-DCMAKE_BUILD_TYPE=RelWithDebInfo',
+                    '-DLLVM_APPEND_VC_REV:BOOL=ON',
+                    "-DLLVM_TARGETS_TO_BUILD='Hexagon'"                     ],
+                     )},
+        {'name': "clang-cmake-hexagon-linux",
+         'slavenames':["hexagon-build-02"],
+         'builddir':"clang-cmake-hexagon-linux",
+         'factory' : ClangBuilder.getClangCMakeBuildFactory(
+                      jobs=32,
+                      loadaverage=32,
+                      clean=False,
+                      checkout_compiler_rt=False,
+                      useTwoStage=False,
+                      testStage1=True,
+                      env={'PATH':'/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'},
+                      extra_cmake_args=["-DLLVM_TARGETS_TO_BUILD='Hexagon'"])},
+
         {'name' : "clang-aarch64-lnt",
          'slavenames' :["aarch64-qemu-lnt"],
          'builddir' :"clang-aarch64-lnt",