This is an archive of the discontinued LLVM Phabricator instance.

[Bazel] Update bazel toolchains workflow to remove deprecated rbe_autoconfig.
Needs RevisionPublic

Authored by rubensf on Aug 11 2021, 12:35 AM.

Details

Summary

All the files under re_configs are auto generated by rbe_configs_gen,
which effectively copies them from the bazel auto generated
configurations of the chosen docker container.

Note that this will not work on windows until a similar command is
ran with a windows docker image on a windows operating system.

Diff Detail

Event Timeline

rubensf created this revision.Aug 11 2021, 12:35 AM
rubensf requested review of this revision.Aug 11 2021, 12:35 AM
Herald added a project: Restricted Project. · View Herald Transcript
rubensf updated this revision to Diff 365692.Aug 11 2021, 12:39 AM
rubensf retitled this revision from Update bazel toolchains workflow to remove deprecated rbe_autoconfig. to [Bazel] Update bazel toolchains workflow to remove deprecated rbe_autoconfig..
rubensf edited the summary of this revision. (Show Details)

Update commit message.

GMNGeoffrey requested changes to this revision.Aug 16 2021, 4:29 PM

Hmmm I think adding this much configuration, especially with a different license, is likely to be a problem. Do we have other options? This might be a deal breaker for using RBE

This revision now requires changes to proceed.Aug 16 2021, 4:29 PM

Oh and it also appears to break the Bazel build?

ERROR: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/b236419cf6ea18fac6e67230694da07b/external/llvm-project/llvm/BUILD.bazel:145:11: Compiling llvm/lib/Demangle/MicrosoftDemangle.cpp [for host] failed: (Exit 1): clang failed: error executing command /usr/local/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 63 argument(s) skipped)
 
Use --sandbox_debug to see verbose messages from the sandbox clang failed: error executing command /usr/local/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 63 argument(s) skipped)
 
Use --sandbox_debug to see verbose messages from the sandbox
src/main/tools/process-wrapper-legacy.cc:80: "execvp(/usr/local/bin/clang, ...)": No such file or directory

https://buildkite.com/llvm-project/premerge-checks/builds/52126

Note that all this configuration is auto-generated. "All" that rbe_configs_gen does is run bazel inside of the specified docker container (that'll run the action in the remote environment), and copy the auto-generated toolchains that bazel creates into the target folder. These toolchains can be written by hand as well, potentially even in a much more simplified way (eg maybe there's no need to specify features such as "pic", "fission-support", etc), with the caveat that they have to be maintained if the spec changes through bazel versions.

In fairness, maybe this doesn't need to be included at all. Users can set their own global home bazelrc or separate repo with set of bazel configurations for their desired execution environment.

Yeah so I think putting this in the LLVM monorepo is not a good idea. How about one of the other options, like hosting it as a tarball somewhere: https://github.com/bazelbuild/bazel-toolchains/tree/463914fa8f1359e51458d11de8112815f6a035c5#option-3-remote-tarball-archive ? Although the fact that the build fails on this suggests that maybe it doesn't really work at all?

utils/bazel/re_configs/linux/cc/cc_wrapper.sh