This is an archive of the discontinued LLVM Phabricator instance.

[Bazel] Add bzlmod support
Changes PlannedPublic

Authored by aaronmondal on Oct 28 2022, 10:55 PM.

Details

Reviewers
GMNGeoffrey
csigg
Summary

Introduces a build configuration for bzlmod.

The version introduced in this module is importable in MODULE.bazel files via

python
bazel_dep(name="llvm-project-overlay", version="16.0.0-bcr.0")

llvm_project_overlay = use_extension(
    "@llvm-project-overlay//utils/bazel:extensions.bzl",
    "llvm_project_overlay",
)

use_repo(llvm_project_overlay, "llvm-project")

Optionally, users may specify custom commits, targets and patches by overriding llvm_project_overlay.configure:

python
llvm_project_overlay.configure(
    targets = ["X86"],
    commit = "<some_commit>",
    sha256 = "<sha_of_commit_archive>",
    patches = ["<some_patch>"],
)

Diff Detail

Event Timeline

aaronmondal created this revision.Oct 28 2022, 10:55 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 28 2022, 10:55 PM
aaronmondal requested review of this revision.Oct 28 2022, 10:55 PM

A few notes:

The --registry=https://github.com/eomii/bazel-eomii-registry/commit/04d3d96f0fa35e38b056853db3bae8dd903a4bbd line in the example is only meant for testing and should not be part of the final version of this diff.

I "delayed" the custom commit example to https://reviews.llvm.org/D137009 since it will require an LLVM commit that contains the changes in this commit.

The`utils/bazel/MODULE.bazel` file is the file used if someone (most likely a CI system) would run a bzlmod build from within a cloned version of LLVM. The MODULE.bazel file that most users will actually interact with is the one patched in by bzlmod_compatibility_patch.patch.

Something I'm still figuring out is how to add WORKSPACE, MODULE.bazel and BUILD.bazel dynamically when building in-tree. When using the http_archive variant the files can just be patched in, and bazel registries will have to contain a copy of bzlmod_compatibility_patch.patch anyways (unless we add these files to the real top level llvm-project directory, which is probably not possible). When building from within utils/bazel, patching may hurt reproducibility as @fmeum mentioned in a previous discussion. I'll investigate this further.

Remove that hacky patch. It is not necessary and will usually be supplied by a bazel registry.

Also fixes the logic to determine whether or not to use a custom commit or the default sources.

matts1 added a subscriber: matts1.May 16 2023, 6:36 PM
matts1 added inline comments.
utils/bazel/extensions.bzl
34

FYI, since I wasn't aware of your implementation when I was working on this yesterday, I wrote my own implementation. I'll add the code below, so feel free to steal whatever seems good from it.

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "new_git_repository")
load("//repos:non_bcr_deps.bzl", "non_bcr_deps")
load("//:configure.bzl", "llvm_disable_optional_support_deps", "llvm_use_system_support_deps", _llvm_configure_repo = "llvm_configure")
load("//:terminfo.bzl", "llvm_terminfo_disable", "llvm_terminfo_system")
load("//:vulkan_sdk.bzl", "vulkan_sdk_setup")
load("//:zlib.bzl", "llvm_zlib_disable", "llvm_zlib_external", "llvm_zlib_system")

_ZLIB_NAME = "llvm_zlib"
_TERMINFO_NAME = "llvm_terminfo"

def _llvm_configure_impl(module_ctx):
    root = module_ctx.modules[0]
    tag = root.tags.configure[0]

    non_bcr_deps()

    if tag.zlib_strategy == "external":
        llvm_zlib_external(name = _ZLIB_NAME, external_zlib = "@zlib")
    elif tag.zlib_strategy == "system":
        llvm_zlib_system(name = _ZLIB_NAME)
    elif tag.zlib_strategy == "disabled":
        llvm_zlib_disable(name = _ZLIB_NAME)
    else:
        fail("zlib_strategy must be one of 'external', 'system', and 'disabled'")

    if tag.terminfo_strategy == "system":
        llvm_terminfo_system(name = _TERMINFO_NAME)
    elif tag.terminfo_strategy == "disabled":
        llvm_terminfo_disable(name = _TERMINFO_NAME)
    else:
        fail("terminfo_strategy must be one of 'disabled' and 'system'")

    vulkan_sdk_setup(name = "vulkan_sdk")

    _llvm_configure_repo(name = tag.name)

_llvm_configure_tag = tag_class(attrs = dict(
    name = attr.string(mandatory = True),
    zlib_strategy = attr.string(mandatory = True, default = "disabled"),
    terminfo_strategy = attr.string(default = "disabled"),
))

llvm_configure = module_extension(
    implementation = _llvm_configure_impl,
    tag_classes = dict(configure = _llvm_configure_tag),
)
matts1 added inline comments.May 16 2023, 6:37 PM
utils/bazel/examples/bzlmod/.bazelrc
4

I'll add a comment here so you don't forget to delete before submitting

aaronmondal planned changes to this revision.May 19 2023, 3:15 PM
aaronmondal added inline comments.
utils/bazel/extensions.bzl
34

Thanks a lot for this! After the changes to zlib we con't need the custom zlib logic anymore, and if we remove terminfo from the external deps entirely we can also get rid of that. I feel like your patch at https://reviews.llvm.org/D150641 was the missing piece to make all of this fall into place quite nicely :) I propose that we first remove terminfo, then get you patch in and then move forward with the patch here to finally add bzlmod support natively.