Page MenuHomePhabricator

[libc++] Add custom clang-tidy checks
AcceptedPublic

Authored by philnik on Aug 16 2022, 6:34 AM.

Details

Reviewers
jdoerfert
ldionne
Group Reviewers
Restricted Project

Diff Detail

Unit TestsFailed

TimeTest
3,240 mslibcxx CI No locale > llvm-libc++-shared-cfg-in.libcxx::double_include.sh.cpp
Script: -- : 'RUN: at line 11'; /usr/bin/clang++-16 -c /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/libcxx/double_include.sh.cpp -o /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/test/libcxx/Output/double_include.sh.cpp.dir/t.tmp.first.o --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings
3,210 mslibcxx CI No locale > llvm-libc++-shared-cfg-in.libcxx::min_max_macros.compile.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-16 /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/libcxx/min_max_macros.compile.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fsyntax-only
3,470 mslibcxx CI No locale > llvm-libc++-shared-cfg-in.libcxx::modules_include.sh.cpp
Script: -- : 'RUN: at line 44'; rm -rf /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/test/libcxx/Output/modules_include.sh.cpp.dir/t.tmp
3,130 mslibcxx CI No locale > llvm-libc++-shared-cfg-in.libcxx::nasty_macros.compile.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-16 /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/libcxx/nasty_macros.compile.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fsyntax-only
3,100 mslibcxx CI No locale > llvm-libc++-shared-cfg-in.libcxx::no_assert_include.compile.pass.cpp
Script: -- : 'COMPILED WITH'; /usr/bin/clang++-16 /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/libcxx/no_assert_include.compile.pass.cpp --target=x86_64-unknown-linux-gnu -nostdinc++ -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/build/generic-no-localization/include/c++/v1 -I /home/libcxx-builder/.buildkite-agent/builds/d8762bc9c633-1/llvm-project/libcxx-ci/libcxx/test/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typedef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -fcoroutines-ts -Werror=thread-safety -Wuser-defined-warnings -fsyntax-only
View Full Test Results (107 Failed)

Event Timeline

philnik created this revision.Aug 16 2022, 6:34 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 16 2022, 6:34 AM

This is really interesting! If you can get me more details about what FindClang.cmake looks like (I wasn't able to find after searching quickly), I might be able to provide more guidance on how to integrate it.

libcxx/test/configs/cmake-bridge.cfg.in
33

Instead, maybe we could introduce a directory where we install build tools like this and use --load=%{build-tools}/libcxx-tidy.*?

libcxx/test/libcxx/clang_tidy_checks/CMakeLists.txt
16 ↗(On Diff #452985)

This relies on the fact that we used AddLLVM in the top-level CMake. This is like a weird hybrid between using the system-installed clang-tidy and the one we have in the monorepo. I think we should do the former. I would expect something like this (pseudo code):

// In the Dockerfile
sudo apt-get install clang-devel # that installs development headers and stuff for clang/clang-tidy/etc

// In the CMake
find_package(Clang)

add_library(libcxx_tidy SHARED ${SOURCES})
target_link_libraries(libcxx_tidy PUBLIC Clang::clang-tidy-lib)

I actually expect that something like this should be all that's needed if Clang does provide a working FindClang.cmake (IDK whether it does, though).

libcxx/utils/libcxx/test/features.py
160

Instead, has-clang-tidy should represent whether we have clang-tidy *AND* the development headers have been installed. That way, we can check for has-clang-tidy only in the tests.

philnik updated this revision to Diff 455594.Aug 25 2022, 8:18 AM
philnik marked 2 inline comments as done.
  • Address some comments
ldionne published this revision for review.Aug 25 2022, 8:34 AM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added a reviewer: Restricted Project. · View Herald Transcript
ldionne requested changes to this revision.Aug 25 2022, 9:13 AM

I think we have to options here:

  1. Use the clang-tidy in the LLVM monorepo.
  2. Use a system-provided clang-tidy if there is one, and disable these tests otherwise.

I think (1) is a non-starter, because we can't start depending on clang-tidy (which depends on clang, LLVM, etc.) inside libc++. That would require doing a bootstrapping build every time we want to run the tests.

So I think we have to to (2), and simply make sure that our Docker image in the CI does have the development packages for clang-tidy.

libcxx/CMakeLists.txt
316–318 ↗(On Diff #455594)

Let's remove this and make this automatic based on whether we have the dev packages. Basically, I don't see a reason to disable this if we can have them enabled.

libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
6

Does this render our other tests for robustness against ADL obsolete? If so, we should take one of those, make it regress, and confirm that this clang-tidy check will catch it. If it doesn't, we should understand why.

Basically, I'd like us to have a clear understanding of whether we need to write manual tests for ADL robustness or not.

10

Instead, let's do

if(NOT Clang_FOUND OR ${LLVM_VERSION_MAJOR} LESS 15)
  message(STATUS "Could not find a suitable version of the Clang development package, custom libc++ clang-tidy checks will not be available.")
  return()
endif()

And in fact, we need to also bail out if we *do* find Clang, but if the development package is not installed. Otherwise, the includes and .a files needed below won't be found.

11–12
12

I would like to make it really clear that we're not using the in-tree version of headers/libraries. I suggest we add a target like this (since find_package(Clang) does not seem to do that):

add_library(clang-tidy-devel IMPORTED)
set_target_properties(clang-tidy-devel PROPERTIES IMPORTED_LOCATION "${?????}/libclangTidy.a")
target_include_directories(clang-tidy-devel INTERFACE "${LLVM_INCLUDE_DIRS}" 
                                                 "${CLANG_INCLUDE_DIRS}"
                                                 "${LLVM_MAIN_INCLUDE_DIR}/../../clang-tools-extra")
target_compile_options(clang-tidy-devel INTERFACE -fno-rtti)

Then, we can link cxx-tidy against clang-tidy-devel.

14–16

I'm not sure why we'd need those. We already set that at the top-level.

19

This can be removed if we have the INTERFACE flag suggested above.

25

Based on the CMake documentation, this should already be the default. I don't understand why this is needed -- can we try removing it and see how it fails?

26

Also, how about using .plugin instead? It's more platform agnostic.

libcxx/utils/libcxx/test/features.py
24

That doesn't work. You're manually expanding the substitution, but substitutions can contains substitutions themselves and so on. We should instead do:

int(re.search('[0-9]+', commandOutput(cfg, ['clang-tidy --version'])).group()) >= 13 and
runScriptExitCode(['stat %{test-tools}/clang_tidy_checks/libcxx-tidy.plugin']) == 0
This revision now requires changes to proceed.Aug 25 2022, 9:13 AM
philnik updated this revision to Diff 455921.Aug 26 2022, 8:34 AM
philnik marked 7 inline comments as done.
  • Rebased
  • Address comments
libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
6

I think we should do that in a follow-up where we remove the current ADL tests and make sure clang-tidy catches all the cases. If clang-tidy doesn't catch something we should find out why and fix the clang-tidy check.

14–16

We use

set_target_properties(${target} PROPERTIES
    CXX_STANDARD 20
    CXX_STANDARD_REQUIRED YES
    CXX_EXTENSIONS NO)

at the top level. Should I do the same here?

25

When I remove it the output file is set as /home/nikolask/llvm-projects/default/build/lib/libcxx-tidy.plugin. I don't know why it doesn't put it in the normal location.

Could this be done with clang-query instead of a clang-tidy plugin?

libcxx/include/charconv
296

Why is this code moved? I am working on this header so the changes probably cause merge conflicts.

In general I dislike moving code in these kind of patches since they may cause unneeded merge conflicts.

Would it be possible to exclude this file and add a TODO so we can fix this after the consexpr version of charconv is done?

libcxx/utils/libcxx/test/features.py
24

I think it would be better when the user specifies the path to clang-tidy in CMake. Our Docker image contains multiple versions of clang-tidy; one for main and one for the release branch. To be able to use both branches the build bot configuration should be able to select the proper version.

It is already intended to remove the symlinks from the Docker file for that reason.

Could this be done with clang-query instead of a clang-tidy plugin?

No. With a clang-tidy plugin we can get all the information in the AST, which we can't through clang-query. See AST_MATCHER(...) in robust_against_adl.cpp for examples. Using a clang-tidy plugin also has the benefit of getting proper error messages and (potentially) IDE integration. If we're ambitious maybe even fixit hints.

libcxx/include/charconv
296

It's moved because it calls the __to_chars_itoa in line 295. This wasn't checked before until the functions is called I think.

libcxx/utils/libcxx/test/features.py
24

Ok, but what has that to do with this change?

is there a reason why we add our own clang-tidy plugin instead of upstreaming this to clang-tidy itself? I could imagine that other library developers could also benefit from this clang-tidy check, if we make the list of allowed customization points a configuration option

is there a reason why we add our own clang-tidy plugin instead of upstreaming this to clang-tidy itself? I could imagine that other library developers could also benefit from this clang-tidy check, if we make the list of allowed customization points a configuration option

We can definitely do this for some checks, but I don't think it makes sense for all checks. I'd like to extend our _LIBCPP_HIDE_FROM_ABI check for example, but that check probably doesn't make sense for any other project. Upstreaming a check would also mean that we'd have to wait up to six months before being able to enforce it.

is there a reason why we add our own clang-tidy plugin instead of upstreaming this to clang-tidy itself? I could imagine that other library developers could also benefit from this clang-tidy check, if we make the list of allowed customization points a configuration option

We can definitely do this for some checks, but I don't think it makes sense for all checks. I'd like to extend our _LIBCPP_HIDE_FROM_ABI check for example, but that check probably doesn't make sense for any other project. Upstreaming a check would also mean that we'd have to wait up to six months before being able to enforce it.

I too prefer to upstream useful changes; _LIBCPP_HIDE_FROM_ABI is indeed not one of them. I disagree with your assessment. We can do both, implement it locally and upstream it at the same time. That way we can use it directly and remove the local version half a year later.

libcxx/include/charconv
296

Can we exclude this file as asked above?

libcxx/utils/libcxx/test/features.py
24

I thought it changed the clang-tidy version use.

philnik marked 2 inline comments as done.Aug 31 2022, 12:05 PM

is there a reason why we add our own clang-tidy plugin instead of upstreaming this to clang-tidy itself? I could imagine that other library developers could also benefit from this clang-tidy check, if we make the list of allowed customization points a configuration option

We can definitely do this for some checks, but I don't think it makes sense for all checks. I'd like to extend our _LIBCPP_HIDE_FROM_ABI check for example, but that check probably doesn't make sense for any other project. Upstreaming a check would also mean that we'd have to wait up to six months before being able to enforce it.

I too prefer to upstream useful changes; _LIBCPP_HIDE_FROM_ABI is indeed not one of them. I disagree with your assessment. We can do both, implement it locally and upstream it at the same time. That way we can use it directly and remove the local version half a year later.

I didn't mean that we should never upstream it. I meant that if we didn't add a clang-tidy plugin that we'd have to wait up to six months before enabling a check. I'd also rather wait to upstream this check, since it's not very well tested yet. I'd like to check if it catches all ADL that the current ADL tests catch and if not improve the check in a follow-up.

libcxx/include/charconv
296

I'll do that if this lands before your patches, which I don't really expect. There is a reason I didn't consider the comment done yet.

tschuett added a subscriber: tschuett.EditedAug 31 2022, 10:52 PM

The LLVM libc has "official" clang tidy checks:
https://github.com/llvm/llvm-project/tree/main/clang-tools-extra/clang-tidy/llvmlibc
I guess there is a buildbot that builds libc and clang-tools-extra.
Even with their own documentation:
https://libc.llvm.org/clang_tidy_checks.html

ldionne added inline comments.Fri, Sep 2, 9:11 AM
libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
3

You should be able to skip the version check below that way.

We'll also have to fix the ClangConfig.cmake file, I don't see a good way of moving forward without that.

14–16

Yes, that seems like the correct thing to do to me.

33
libcxx/utils/ci/run-buildbot
507 ↗(On Diff #455921)

This looks like a leftover.

philnik updated this revision to Diff 461070.Sun, Sep 18, 6:17 AM
philnik marked 5 inline comments as done.
  • Rebased
  • Address comments
Herald added a project: Restricted Project. · View Herald TranscriptSun, Sep 18, 6:17 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript
philnik updated this revision to Diff 462180.Thu, Sep 22, 8:26 AM
  • Rebased
  • Try to fix CI
ldionne accepted this revision.Fri, Sep 23, 8:33 AM
ldionne added subscribers: smeenai, beanz.

I'd be fine with this as-is if it works in the CI.

IIUC, the current blocker for this is that the ClangConfig.cmake installed by LLVM is not robust to the dev packages missing. If you do find_package(Clang 16.0), it will error out if the dev packages are not present, since it will try to reference static libraries (and other artifacts) that don't exist and try to create IMPORTED targets for those. @smeenai @beanz do you know more about that, or do you know someone that does? Do you know who set up the CMake config files so that find_package(Clang) would work in the first place? I'd also welcome your input on the ClangConfigVersion.cmake.in changes.

Just for the context, what we're trying to do here is simply use clang-tidy's development packages from the libc++ build to add our own custom clang-tidy checks.

Accepting because this LGTM, although we do have some stuff to resolve before this can be shipped.

libcxx/test/tools/clang_tidy_checks/CMakeLists.txt
40
libcxx/test/tools/clang_tidy_checks/robust_against_adl.cpp
23

This is funny, I actually saw a LWG issue about that go by a few weeks ago. I'll try to get more details.

This revision is now accepted and ready to land.Fri, Sep 23, 8:33 AM
Mordante added inline comments.
libcxx/test/tools/clang_tidy_checks/robust_against_adl.cpp
23
jwakely added inline comments.Sun, Sep 25, 9:41 AM
libcxx/test/tools/clang_tidy_checks/robust_against_adl.cpp
23

They're definitely customization points. If they weren't, specializing is_error_code_enum and is_error_condition_enum would be completely useless and serve no purpose.

LWG confirmed that and LWG 3629 is Tentatively Ready now.

jwakely added inline comments.Fri, Sep 30, 4:55 AM
libcxx/test/tools/clang_tidy_checks/robust_against_adl.cpp
23

I created https://reviews.llvm.org/D134943 to fix those customization points.