Makes sense

@phosek, does LLVM’s runtime directory do this correctly?

LGTM

LGTM

It is expected that CMake will process all projects put under llvm/tools unless explicitly told not to. So, it is expected that it would process clang there. If you don't want it to process clang you need to set LLVM_TOOL_CLANG_BUILD=Off when configuring.

LGTM.

Sorry for being behind on this, but I don't think this is the right solution to the problem.

My $0.02 on the yaml discussion for tests:

From an Apple perspective this almost certainly needs @mtrent's eyes, as he owns our binary tools.

How are you instructing lit to not run the tests that rely on these tools? Doesn't seem like you are, so I would expect check-llvm to fail on a clean build configuration due to missing tools.

LGTM.

LGTM!

One slightly nitpicky comment.

LGTM

I can certainly foresee other LLVM-based projects needing entitlements, so I am very much in favor of adding ENTITLEMENTS and FORCE options onto llvm_codesign.

Why not just add entitlement support to ‘llvm_codesign’? Or feed through extra arguments?

I’m very confused how this could happen with Ninja. All the CMake invocations are done in targets with ‘USES_TERMINAL’ set on them which puts them in the ‘console’ pool which only allows one task at a time. So there shouldn’t be a way to have more than one Ninja invocation running at a time.

My biggest argument against changing how clang does things is that we want to get the policy failures there too so that we know to fix them. We do support building clang out of tree, and hiding the warnings for in-tree builds will just make it harder to find and fix them.

LGTM

Honestly, I don't know that we need to specify a working directory on this command at all. Since we're specifying the target file using a generator expression everything else should work because we use explicit not relative paths.

Adding -bundle_loader in add_llvm_loadable_module doesn't work because LLVM modules can be loaded by lots of different tools, so they actually need to be fully self-contained. Obviously if a specific distribution is only supporting the module from one tool that distribution can do things differently, but as a general rule we can't make that assumption.

I'm not really sure this is the best solution. This will result in bloating the clang binary substantially. An alternative approach would be to use linker order precedence, and pass the LLVM executable first, then LLVM static archives later on the link line. That would allow minimal tool binaries, and link the LLVM bits into your plugin.

Yes, this is the preferred way to do this. No, we don’t need a big comment about it. The solution I described is how you define command-line configurable options in CMake that are not Boolean (for Boolean options CMake has the option function which is really just a wrapper around this pattern).

I don't disagree with the intent of the patch, but this patch is *way* more complicated than it needs to be to accomplish the goal. All you should need to do is add CACHE STRING "" to the set command in line 137. The set command's CACHE option doesn't override existing values, it only sets them if they were not previously set.

LGTM.

@lebedev.ri, I told him he could commit the changes without review because I saw them in this patch, and in my mind, as an experienced developer in this area of LLVM, I saw those changes as obvious and good. That is not in violation of the developer policy, further that discussion is right here in this review, so if you have a problem with it you should point it at me, not @DiamondLovesYou.

If you add this to your patch here, the Clang patch should work as I've suggested:

diff --git a/cmake/modules/LLVM-Config.cmake b/cmake/modules/LLVM-Config.cmake
index 8eabddc7377..a306e41f26d 100644
--- a/cmake/modules/LLVM-Config.cmake
+++ b/cmake/modules/LLVM-Config.cmake
@@ -244,6 +244,9 @@ function(llvm_map_components_to_libnames out_libs)
           list(APPEND expanded_components "LLVM${t}Info")
         endif()
       endforeach(t)
+    elseif( c STREQUAL "Polly")
+      # LLVMPolly is the Polly loadable module target, the static archive is just Polly
+      list(APPEND expanded_components "${c}")
     else( NOT idx LESS 0 )
       # Canonize the component name:
       string(TOUPPER "${c}" capitalized)

It is unfortunate that Polly doesn't match the naming conventions of other LLVM components, but we have a lot of special case handling for this kind of thing anyways.

I think this should be split into multiple patches. The places you changed add_library to add_llvm_library should be split out. Also you can break out the unittests/Passes/CmakeLists.txt into its own patch. All of those are good cleanup, and you can commit them without review.

I agree that changing the naming scheme probably makes sense, but I also think that you can probably make a fairly well contained change to the LLVM CMake module that maps components to libnames to special case Polly and make this all work.

After line 18 in this file you could do something like:

I don’t think this is the right solution. The build system knows what components are in the dylib and should remove them from the list of libraries linked individually. You should be able to make Polly behave like an LLVM component, then tools don’t need to care if the dylib is used or not.

LGTM.

Unfortunately I can't make this kind of policy decision about whether or not this would be acceptable for Darwin. That would probably need to be @dexonsmith.

LGTM! Didn't look back through enough context.

One small comment below. Otherwise LGTM.

Looping in @phosek.

LGTM. This is consistent with how we setup these targets for other projects.

Heh... yea, didn't notice this review. I removed the policy setting (which will default to NEW). @chapuni fixed the underlying issue that required us to have this enabled *years* ago.

LGTM as-is

Just want to bring an IRC conversation that I had with @phosek into the proper code review.

LGTM!

lgtm

Please check and see if that works for you. In general I prefer to avoid adding new features that aren't used in-tree unless it is unavoidable or would be difficult to work around out-of-tree. This one seems pretty straightforward to me.

I’m not opposed to this patch, but is there a real need for this? Have you considered just running clang with the -print-resource-dir flag?

Adding @jfb since this is his domain now too.

Looks good. Sorry for the extended delay in review.

Looks fine. Sorry for the extended delay in review.

@mtrent and I talked offline, and he explained how the sphinx documentation covers the manpage and program directives, and I agree with his assertion that using manpage is correct.

One last small fix

Given how mach-o is radically different from ELF, it might make sense to sort or group ELF and mach-o specific options. Right now we only call out options that only work on Mach-o, but I assume some of the options only work on ELF and/or COFF. Might make sense to document that behavior correctly too.

Looping in @davide and @JDevlieghere. LLDB has some hand-rolled goop in debug server's build that could potentially be replaced by this.

LGTM

LGTM!

I prefer DEPS to COMPONENTS because I've tried to explicitly use the term components to mean targets that have paired build and install targets. Otherwise looks good.

Looping in @bruno in case he has feedback.

This all looks reasonable to me. I had a few small comments below. There are a few isolated bits of code cleanup that you included in this patch, and I'd prefer to see those land separately.

@dexonsmith is often super busy, so @bruno may be able to weigh in instead.

Historically we've duplicated cmake_policy calls on a per-project basis which we needed to support standalone builds. That said, it would be nice if we had LLVM vend some CMake modules that encapsulated this stuff better. In general I think that cleanup would be ideal to do after we move LLVM to the GitHub mono-repo because then we can create a cmake/modules folder at the root of the mono-repo that has common modules useable by all sub-projects.

LGTM

LGTM

This looks good to me. Please correct me if I'm wrong, but it also looks like you've written this patch so that it will work correctly even if it is landed before the llvm-readobj changes are landed. If that is the case go ahead and land this.

For reference this relates to D41673. Looking in @kubamracek.

LGTM

I like @phosek's idea of moving the files into /share, but that needs some accompanying changes to clang to find those files in their new location.

You should split the CMake cache file you created into two files, (1) a CMake Cache to manage the build configuration and (2) a tool chain file for targeting Linux. As @semeenai pointed out we absolutly want the behavior of the toolchain file being loaded multiple times. That is the correct way this build should work.

LGTM

I'm going to nitpick here a little. The patch is fine, but one of two things should change. Either line 29 should use list(APPEND...) instead of set, or you should do something more like:

A few inline comments. Overall I like the direction.

@phosek not sure if there is an equivalent. Probably strip -x is the closest.

@phosek On darwin ld64 supports the flags -all_load and -force_load which work similarly to --whole-archive. The ld64 manpage has the documentation for those options.

This patch LGTM.

Abstracting check for whether or not it is safe to remove an instruction as per the feedback from davide.

LGTM

LGTM

LGTM.

LGTM.

LGTM

This is a workaround predating CMake 3.1 when they fixed the underlying issues with the expansion of variables in STREQUAL. See: https://cmake.org/cmake/help/v3.1/policy/CMP0054.html.

LGTM.

LGTM

LGTM.

Yes, that comment is very out of date. The compiler-rt standalone build is essential to many of the users. It is quite common to build and use compiler-rt without LLVM (like sanitizer support in gcc).

The docs changes look good to me. I think if we want to deprecate llvm/projects for runtime libraries, we should have a discussion on llvm-dev & cfe-dev. I think too few people are using llvm/runtimes today as part of their regular builds for me to feel comfortable deprecating llvm/projects without giving some notice first and getting more people to try the runtimes directory.

This change isn't safe. Compiler-RT is buildable without LLVM's modules as long as you disable the tests, so you can't use an AddLLVM function inside AddCompilerRT unless it is only used when tests are disabled.

LGTM.