This is an archive of the discontinued LLVM Phabricator instance.

Get Bazel building `//clang` on Windows with clang-cl.
ClosedPublic

Authored by chandlerc on Oct 24 2021, 10:10 PM.

Details

Summary

This required substantially more invasive changes.

We need to handle some of the LLVM config.h changes differently from
the old pattern. These aren't always safe on the commandline, and the
Windows ones specifically break Clang. Instead, use conditional defines
in the header itself. This more closely matches how CMake builds see the
definitions. I think this is also just cleaner and we should maybe move
more of the macros out of Bazel.

The config defines for Windows that I've kept in Bazel are the ones that
LLVM's CMake does at the commandline as well. I've also added numerous
ones that CMake uses and we didn't replicate in Bazel.

I also needed a different approach to get libclang working well. This,
IMO, improves things on all platforms. Now we build the plugin and
actually wrap it back up with cc_import. We have to use a collection
of manually tagged cc_binary rules to get the naming to work out the
right way, but this isn't too different from the prior approach. By
directly having a cc_binary rule for each platform spelling of
libclang, we can actually extract the interface library from it and
correctly depend on it with cc_import. I think the result now is much
closer to the intent and to the CMake build for libclang.

Sadly, some tests also needed disabling. This is actually narrower than
what CMake does. The issue isn't indicative of anything serious -- the
test just assumes Unix-style paths.

I also have cleaned up the Windows flags in .bazelrc to much more
closely match what CMake does.

Diff Detail

Event Timeline

chandlerc created this revision.Oct 24 2021, 10:10 PM
chandlerc requested review of this revision.Oct 24 2021, 10:10 PM
GMNGeoffrey accepted this revision.Oct 25 2021, 1:39 PM
GMNGeoffrey added inline comments.
utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
78

Hmm I think we probably should have a change-detector for this config as well as the LLVM ones

utils/bazel/llvm-project-overlay/llvm/cc_plugin_library.bzl
33

What about this makes binary_alias no longer work?

This revision is now accepted and ready to land.Oct 25 2021, 1:39 PM
thakis added a subscriber: thakis.Oct 25 2021, 6:29 PM
thakis added inline comments.
clang/include/clang/Basic/Builtins.def
1059

Why do we need this with bazel but not with other windows builds?

chandlerc added inline comments.Oct 25 2021, 7:06 PM
clang/include/clang/Basic/Builtins.def
1059

I don't know how this never was hit by other builders. I don't usually develop on Windows so I have very little experience with different builds there.

My guess is that it is the particular way that Bazel+clang-cl compile on Windows causes sligthtly more to be transitively included with #include <windows.h> and that grows the number of functions hit with this. I thought about trying to "fix" that, but the existing #undef lines made me think it wouldn't be completely successful.

Are you worried about the change? Looking for a different fix?

utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
78

Yeah, but didn't want to try to add that in this change.

IMO, the better thing would be for this to go away as it is almost entirely redundant already.... But agin, not this change....

utils/bazel/llvm-project-overlay/llvm/cc_plugin_library.bzl
33

The output_group in the subsequent filegroup. It requires the actions in the sources to have a correctly named output_group, which cc_binary with linkshared does.

It's possible that I could just rename things enough by adding multiple layers of file groups and maybe some added "copy this file" rules... But I'm worried that wouldn't work well long term. For example, it seems reasonable for the .lib file to *name* the .dll file that should be loaded. And so if we build it with cc_binary that has the wrong name, I'm worried things won't work as expected. I have higher confidence in this arrangement where the cc_binary directly produces the desired name.

Something like that actually happened when I tried a different way of wiring this up that still used the binary_alias on Linux -- it ended up actually loading liblibclang_impl.so and not libclang.so. That was a different setup, so maaaybe I wouldn't hit that exact situation, but its the kind of thing that inclined me towards this model.

GMNGeoffrey added inline comments.Oct 26 2021, 2:28 PM
clang/include/clang/Basic/Builtins.def
1059

Yeah I want to note that my review is basically only for the Bazel stuff. This looked fine to me based on the existing undefs, but mostly trusting Chandler to determine that this is OK and isn't in the territory of the Bazel build requiring unsavory things in the core project.

utils/bazel/llvm-project-overlay/llvm/cc_plugin_library.bzl
33

That's sufficiently convincing. Hopefully some layer will figure out these are all the same and not build the same thing 3 times 🤞

thakis added inline comments.Oct 28 2021, 8:43 AM
clang/include/clang/Basic/Builtins.def
1059

Worried is a strong word :) It just feels off. The redundant build system config changes shouldn't need changes to LLVM's code itself imho. If there's some difference in build behavior here, it feels like we should fix that on the build config side of things, else we'll have weird one-off fixes like this in other places potentially. I feel we should at least understand what's going on.

(This isn't a terribly strong opinion fwiw.)

rnk added a subscriber: rnk.Oct 28 2021, 3:12 PM
rnk added inline comments.
clang/include/clang/Basic/Builtins.def
1059

Does Bazel build clang with modules? That could lead to windows.h proliferation.

If you are motivated to investigate, you can re-run the failing compile command with /showIncludes to work out where the windows.h include comes in. Take the corresponding object file, build it with cmake ninja, extract the compile command there, run it with showincludes, and compare the output.

utils/bazel/.bazelrc
113–116

It is true that MSVC's -Wall is Clang's -Weverything, so this needs a prefix. -Werror could be -WX, but it's just cosmetic.

125

You shouldn't need to prefix warning flags with /clang:, those are supported out of the box.

utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
81

Unrelated to your change, but is this stale?

GMNGeoffrey added inline comments.Oct 28 2021, 3:22 PM
utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
81

Yes and we don't currently have a good way to keep it up to date. The overall problem of how to make sure we keep up to date with CMake configure knobs is unsolved. I wonder if we could run CMake in some limited capacity, but that's a whole can of worms...

chandlerc updated this revision to Diff 383634.Oct 30 2021, 9:17 PM
chandlerc marked 5 inline comments as done.
chandlerc edited the summary of this revision. (Show Details)
chandlerc added a reviewer: rnk.

Major update to better fix some of the issues here. No longer requires any changes outside of Bazel.

chandlerc requested review of this revision.Oct 30 2021, 9:18 PM

PTAL, and thanks for feedback so far!

clang/include/clang/Basic/Builtins.def
1059

I figured this all out. It was right in front of me. New patch fixes.

utils/bazel/.bazelrc
113–116

Yeah, I just figured it was more readable in this section to consistently use /clang:-W.

125

Again, this just seemed more consistent. I can change if you think the other is better though.

utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
81

I'll just commit an update to this separately. I assume that doesn't need separate review? ;]

rnk added inline comments.Nov 1 2021, 11:01 AM
utils/bazel/.bazelrc
82

Try adding /permissive- to get more conforming behavior from clang-cl. If that doesn't work, try /Zc:twoPhase-, aka -fno-delayed-template-parsing.

85

This is only necessary for MSVC. LLVM MC auto-detects when bigobj is needed. IMO, less flags is always better.

utils/bazel/llvm-project-overlay/clang/BUILD.bazel
1830

Enabling warnings is good, but what made this change necessary? The -Wno- flag should have worked with clang-cl. If this change isn't necessary, maybe go ahead and delete this copt in a separate commit.

utils/bazel/llvm-project-overlay/clang/include/clang/Config/config.h
81

Nope, ship it.

GMNGeoffrey accepted this revision.Nov 1 2021, 2:17 PM
GMNGeoffrey added inline comments.
utils/bazel/llvm-project-overlay/clang/BUILD.bazel
364

Seems like this should be in clang/config.h?

utils/bazel/llvm-project-overlay/clang/unittests/BUILD.bazel
484

Probably good to note here that these are also disabled in the CMake build

utils/bazel/llvm-project-overlay/llvm/include/llvm/Config/config.h
375 ↗(On Diff #383634)

Oh nice, yeah this seems like a reasonable approach. I can't believe we didn't think of this before... The less Bazel goop the better. We're still going to need some way to deal with the things that are more complicated than just platform and some way to handle keeping this up to date. That might push us back to doing it with Bazel. IDK

This revision is now accepted and ready to land.Nov 1 2021, 2:17 PM

Thanks, making suggested changes and then landing!

utils/bazel/llvm-project-overlay/clang/BUILD.bazel
364

Ah, yeah. Might as well.

utils/bazel/llvm-project-overlay/clang/unittests/BUILD.bazel
484

Good idea, will do.

utils/bazel/llvm-project-overlay/llvm/include/llvm/Config/config.h
375 ↗(On Diff #383634)

Well, *all* of Abseil's configuration is now done this way I think... so I've some optimism. Anyways, I'll send subsequent patches when I get some time.

chandlerc added inline comments.Nov 1 2021, 7:55 PM
utils/bazel/llvm-project-overlay/clang/BUILD.bazel
1830

Restored.

This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptNov 1 2021, 7:58 PM