This is an archive of the discontinued LLVM Phabricator instance.

[CMake] Ensure `CLANG_RESOURCE_DIR` is respected
AcceptedPublic

Authored by paperchalice on Jan 17 2023, 2:29 AM.

Details

Summary

This patch ensures all resources are installed into CLANG_RESOURCE_DIR.
Fixes: https://github.com/llvm/llvm-project/issues/59726

Diff Detail

Event Timeline

paperchalice created this revision.Jan 17 2023, 2:29 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 17 2023, 2:29 AM
Herald added a subscriber: Enna1. · View Herald Transcript
paperchalice requested review of this revision.Jan 17 2023, 2:29 AM
Herald added projects: Restricted Project, Restricted Project, Restricted Project, Restricted Project, Restricted Project. · View Herald TranscriptJan 17 2023, 2:29 AM

Use LLVM_LIBRARY_OUTPUT_INTDIR.

While the general direction seems sound, I suspect this may be a real PITA given that for us:

-DCLANG_RESOURCE_DIR="../../../../lib/clang/${LLVM_MAJOR}"

I'm going to try and see later throughout the week if it's possible to keep that working with this patch.

Ok, this turned out to be surprisingly painless, at least within our packaging.

However, I'd prefer if someone more experienced looked at the CMake changes.

MaskRay added a comment.EditedJan 31 2023, 1:41 PM

The CMake part of this patch improves the matter. Manually constructed resource dir (many duplicates) string is replace with a library function call.

However, the introduced conditions in C++ code like

std::string path_to_clang_dir = std::string(CLANG_RESOURCE_DIR).empty()
                                    ? "/foo/bar/" LLDB_INSTALL_LIBDIR_BASENAME
                                      "/clang/" CLANG_VERSION_MAJOR_STRING
                                    : "/foo/bar/bin/" CLANG_RESOURCE_DIR;

makes me uncomfortable.

I wish that we can just replace all manually constructed CLANG_INSTALL_LIBDIR_BASENAME "/clang/" CLANG_VERSION_MAJOR_STRING with a simple CLANG_RESOURCE_DIR which is guaranteed to be non-empty.

edeaf16f2c2f02d6e43312d48d26d354d87913f3 (2011) added the CMake variable CLANG_RESOURCE_DIR but did not explain why.
In the long term, the CMake variable CLANG_RESOURCE_DIR probably should be removed.

The CMake part of this patch improves the matter. Manually constructed resource dir (many duplicates) string is replace with a library function call.

However, the introduced conditions in C++ code like

std::string path_to_clang_dir = std::string(CLANG_RESOURCE_DIR).empty()
                                    ? "/foo/bar/" LLDB_INSTALL_LIBDIR_BASENAME
                                      "/clang/" CLANG_VERSION_MAJOR_STRING
                                    : "/foo/bar/bin/" CLANG_RESOURCE_DIR;

makes me uncomfortable.

I wish that we can just replace all manually constructed CLANG_INSTALL_LIBDIR_BASENAME "/clang/" CLANG_VERSION_MAJOR_STRING with a simple CLANG_RESOURCE_DIR which is guaranteed to be non-empty.

edeaf16f2c2f02d6e43312d48d26d354d87913f3 (2011) added the CMake variable CLANG_RESOURCE_DIR but did not explain why.
In the long term, the CMake variable CLANG_RESOURCE_DIR probably should be removed.

clang::driver::Driver::GetResourcesPath would be a better choice.

phosek added inline comments.Feb 2 2023, 1:07 AM
cmake/Modules/GetClangResourceDir.cmake
18

This variable is only used once on the following line, I'd just inline it which is more idiomatic.

19

It's more idiomatic to parse arguments at the start of the function.

19

This is just a nit, but we usually use a prefix like ARGS to make it clear that these are input arguments.

  • Use clang::driver::Driver::GetResourcesPath
  • Use ARG as prefix

Ping.

lldb/source/Plugins/ExpressionParser/Clang/ClangHost.cpp
56

Should be "${CMAKE_INSTALL_BINDIR}/lldb" here, but need another variable in config.h.

tstellar added inline comments.May 4 2023, 12:37 PM
cmake/Modules/GetClangResourceDir.cmake
13

Why is the bin prefix here?

paperchalice added inline comments.May 4 2023, 5:01 PM
cmake/Modules/GetClangResourceDir.cmake
13

See https://clang.llvm.org/doxygen/classclang_1_1driver_1_1Driver.html#acda8dfdf4f80efa84df98157e1779152
GetResourcesPath will return <prefix>/lib/<version> when CLANG_RESOURCE_DIR is empty, <prefix>/bin/CLANG_RESOURCE_DIR otherwise.

tstellar added inline comments.May 4 2023, 5:17 PM
cmake/Modules/GetClangResourceDir.cmake
13

GetResourcesPath calls sys::path::parent_path twice on the path to the clang executable which is going to give you /usr on most Linux systems, so it's returning <prefix>/CLANG_RESOURCE_DIR not <prefix>/bin/CLANG_RESOURCE_DIR.

tstellar added inline comments.May 4 2023, 5:28 PM
cmake/Modules/GetClangResourceDir.cmake
13

Sorry, you are correct. I was looking at the wrong branch. It's really strange that it does that.

tstellar added inline comments.May 5 2023, 7:12 AM
cmake/Modules/GetClangResourceDir.cmake
13

@paperchalice Can you update the description of the CLANG_RESOURCE_DIR cache variable in clang/CMakeLists.txt to mention that the path should be relative to the directory with the clang executable.

paperchalice added inline comments.May 5 2023, 5:22 PM
cmake/Modules/GetClangResourceDir.cmake
13

Already in codebase:

set(CLANG_RESOURCE_DIR "" CACHE STRING
  "Relative directory from the Clang binary to its resource files.")
tstellar accepted this revision.May 5 2023, 7:18 PM

Thanks for the patch. I've tested this and it works for my build configuration. LGTM.

This revision is now accepted and ready to land.May 5 2023, 7:18 PM

@paperchalice Do you need someone to commit this for you?

@paperchalice Do you need someone to commit this for you?

Sure, I don't have the commit access.

@paperchalice Do you need someone to commit this for you?

Sure, I don't have the commit access.

What name / email address do you want me to use for the commit author field?

paperchalice added a comment.EditedJun 2 2023, 11:44 PM

@paperchalice Do you need someone to commit this for you?

Sure, I don't have the commit access.

What name / email address do you want me to use for the commit author field?

Sorry for the late, the information in phabricator is OK, and GetClangResourceDir.cmake is not landed correctly. Ping @tstellar

paperchalice closed this revision.Jun 5 2023, 2:59 AM
protze.joachim reopened this revision.Jun 5 2023, 11:01 PM
protze.joachim added a subscriber: protze.joachim.

The review should not be closed, since the patch was reverted and should be recommitted (this time possibly with a reference to this review?)

This revision is now accepted and ready to land.Jun 5 2023, 11:01 PM

The review should not be closed, since the patch was reverted and should be recommitted (this time possibly with a reference to this review?)

This is recommitted as 0beffb854209a41f31beb18f9631258349a99299, but break some lldb builds, so I created https://reviews.llvm.org/D152225 .

sebastian-ne added inline comments.
cmake/Modules/GetClangResourceDir.cmake
15

This fails in our downstream project.
We use add_subdirectory() to include LLVM and our parent project already sets PACKAGE_VERSION to something that is not the LLVM version.

Parsing PACKAGE_VERSION only if CLANG_VERSION_MAJOR is not already known seems to work:

if (NOT CLANG_VERSION_MAJOR)
  string(REGEX MATCH "^[0-9]+" CLANG_VERSION_MAJOR ${PACKAGE_VERSION})
endif()

[...]
edeaf16f2c2f02d6e43312d48d26d354d87913f3 (2011) added the CMake variable CLANG_RESOURCE_DIR but did not explain why.
In the long term, the CMake variable CLANG_RESOURCE_DIR probably should be removed.

My feeling stays the same. In the long term, we should try removing the CMake variable CLANG_RESOURCE_DIR.
Customizing the variable in a special way will make some clang/test/Driver tests fail, I don't know it's healthy for contributors to be aware of this yet-another configure variable for clang/test/Driver tests. The CLANG_RESOURCE_DIR users should be served by specifying -resource-dir= in a configuration file (https://clang.llvm.org/docs/UsersManual.html#configuration-files)

[...]
edeaf16f2c2f02d6e43312d48d26d354d87913f3 (2011) added the CMake variable CLANG_RESOURCE_DIR but did not explain why.
In the long term, the CMake variable CLANG_RESOURCE_DIR probably should be removed.

My feeling stays the same. In the long term, we should try removing the CMake variable CLANG_RESOURCE_DIR.
Customizing the variable in a special way will make some clang/test/Driver tests fail, I don't know it's healthy for contributors to be aware of this yet-another configure variable for clang/test/Driver tests. The CLANG_RESOURCE_DIR users should be served by specifying -resource-dir= in a configuration file (https://clang.llvm.org/docs/UsersManual.html#configuration-files)

I agree with this

[...]
edeaf16f2c2f02d6e43312d48d26d354d87913f3 (2011) added the CMake variable CLANG_RESOURCE_DIR but did not explain why.
In the long term, the CMake variable CLANG_RESOURCE_DIR probably should be removed.

My feeling stays the same. In the long term, we should try removing the CMake variable CLANG_RESOURCE_DIR.
Customizing the variable in a special way will make some clang/test/Driver tests fail, I don't know it's healthy for contributors to be aware of this yet-another configure variable for clang/test/Driver tests. The CLANG_RESOURCE_DIR users should be served by specifying -resource-dir= in a configuration file (https://clang.llvm.org/docs/UsersManual.html#configuration-files)

I think I agree with you here. The other problem I see with CLANG_RESOURCE_DIR is that it depends on the value of LLVM_LIBDIR_SUFFIX, so it's still configurable even without the CLANG_RESOURCE_DIR variable. I think it would make sense to standardize it to /usr/lib/clang rather than /usr/lib${LLVM_LBDIR_SUFFIX}/clang. This is how gcc works and it also ensures that clang can find the 32-bit compiler-rt libraries.

This is the final version that was pushed, correct? (the commits don't link the revision)

I've raised some concerns about the resulting API in https://reviews.llvm.org/D156270#4557902. I'd appreciate if you could take a look.