Fixes GH58547.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential D136533
[clang] Fix missing diagnostic of declaration use when accessing TypeDecls through typename access mizvekov on Oct 22 2022, 10:59 AM. Authored by
Details
Fixes GH58547. Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Diff Detail
Unit Tests Event TimelineComment Actions Looks straightforward to me with one suggestion. Is the CI fail related?
Comment Actions Yeah, the CI fail is because, while there is a change in libcxx diagnostics and we fix it, the same tests are run in different pipelines using stock, released clangs. So we need a way to handle the difference in expectations. I have pinged libcxx devs about that.
Comment Actions @philnik it seems that didn't work. It seems those pipelines are already running clang-16, according to the cmake logs, they are just not bootstrapping with this patch. So gating this on clang version will not work.
Comment Actions This breaks compiling many things on macOS, including compiler-rt: /tmp/llvm2/obj/bin/clang++ --target=aarch64-apple-darwin -I/tmp/llvm2/compiler-rt/lib/fuzzer/../../include -Wall -Wno-unused-parameter -O3 -DNDEBUG -arch arm64 -isysroot /tmp/MacOSX11.3.sdk -stdlib=libc++ -mmacosx-version-min=10.10 -isysroot /tmp/MacOSX11.3.sdk -fPIC -fno-builtin -fno-exceptions -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -Wthread-safety -Wthread-safety-reference -Wthread-safety-beta -O3 -g -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -fno-omit-frame-pointer -std=c++17 -MD -MT lib/fuzzer/CMakeFiles/RTfuzzer.osx.dir/FuzzerFork.cpp.o -MF lib/fuzzer/CMakeFiles/RTfuzzer.osx.dir/FuzzerFork.cpp.o.d -o lib/fuzzer/CMakeFiles/RTfuzzer.osx.dir/FuzzerFork.cpp.o -c /tmp/llvm2/compiler-rt/lib/fuzzer/FuzzerFork.cpp In file included from /tmp/llvm2/compiler-rt/lib/fuzzer/FuzzerFork.cpp:11: In file included from /tmp/llvm2/compiler-rt/lib/fuzzer/FuzzerCommand.h:15: In file included from /tmp/llvm2/compiler-rt/lib/fuzzer/FuzzerDefs.h:18: In file included from /tmp/MacOSX11.3.sdk/usr/include/c++/v1/memory:667: /tmp/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:1672:66: error: 'type' is unavailable: introduced in macOS 10.15 typedef _LIBCPP_NODEBUG_TYPE typename remove_reference<_Tp>::type _Up; ^ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:603:47: note: in instantiation of template class 'std::decay<std::filesystem::path>' requested here template <class _Source, class _DS = typename decay<_Source>::type, ^ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:648:31: note: in instantiation of default argument for '__is_pathable_char_array<std::filesystem::path>' required here bool _IsCharIterT = __is_pathable_char_array<_Tp>::value, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:741:26: note: in instantiation of default argument for '__is_pathable<std::filesystem::path, false>' required here typename enable_if<__is_pathable<_SourceOrIter>::value, _Tp>::type; ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:861:29: note: in instantiation of template type alias '_EnableIfPathable' requested here _LIBCPP_INLINE_VISIBILITY _EnableIfPathable<_Source> ^ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:965:19: note: while substituting deduced template arguments into function template 'operator/=' [with _Source = path] return (*this /= __replacement); ^ /tmp/MacOSX11.3.sdk/usr/include/c++/v1/filesystem:738:24: note: 'path' has been explicitly marked unavailable here class _LIBCPP_TYPE_VIS path { ^ etc. Comment Actions pinging @kcc for LibFuzzer, I believe this needs to be fixed there, if we don't want to bump the MacOS minimum version. Comment Actions +1, this breaks building LLVM on macOS. This isn't limited to libFuzzer. See the many hits for "error:" on https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8799329305964156353/+/u/package_clang/stdout?format=raw Let's revert this until LLVM can build again. Comment Actions I don't have access to a macOS machine, and I don't believe any of the pre-commit CI machines are running it either. How do you think we shall proceed? Comment Actions One thing you might try is seeing if this is a libcxx-specific thing instead, and try doing a self-build using libcxx. Comment Actions We tested this on the libc++ CI, as this patch touches a test case in there, and I also tested it on my machine on both windows and linux. I believe this is a macOS specific issue, and I don't believe the problem is in this patch. In either case, this seems to be an issue with libc++ that is out there in the wild. There are likely some cases where folks were using unsupported features for their macOS version, and this bug that we fixed prevented us from correctly diagnosing that. If I understood everything correctly so far, we can't change shipped libc++.
libc++ gated that feature on a minimum OS version, if you try to use it otherwise, this doesn't compile, as intended. Would it even be right to add some kind of libc++ specific workaround, if that configuration was declared as unsupported? Comment Actions I don't believe that the libc++ CI does 'self build with libcxx as the host library', so it would not have caught this. That said, this could definitely just be an issue in the version that ships with macos, and we cannot really release something that doesn't self-build on one of our supported platforms. Comment Actions
I don't think it is, actually. I think it's doing something legitimate. That is, it's declaring in its headers that some things are only available on some versions of macos, which is fine if you don't use them, or if you use them in a block guarded with __builtin_available. The problem this patch introduces is that it makes it an error to even have those declarations. Comment Actions (And specifically, FuzzerFork.cpp doesn't use any of those declared things. Its only sin is to #include <fstream>, which happens to have /some/ methods (std::filesystem-related) only available on macos 10.15+.) Edit: Corollary: anything that #include <fstream>, builds with C++17 or more, and wants to support macos < 10.15, is broken by this patch. Comment Actions Nonetheless, it seems libc++ uses a deprecated declaration from it's own headers. @ldionne FYI, we may have a libc++ issue here. Comment Actions It's the opposite of deprecated, it's (optionally) using something that is only available in newer versions of macos. Comment Actions Right, I mis-typed, but this is probably using the 'availability' attribute, which works on the same mechanism as 'deprecated', except that it gives an error. Comment Actions Is it possible that it would be one of those two bugs? https://github.com/llvm/llvm-project/issues?q=is%3Aopen+is%3Aissue+author%3Aldionne+availability+ Comment Actions I don't think so. It does look like libc++ makes some uses within itself of types which were marked unavailable. Ie if you see the included test case: struct A { using ta [[deprecated]] = int; }; using t1 = typename A::ta; // expected-warning {{'ta' is deprecated}} We missed diagnosing that before, when performing a 'typename' member access. This is diagnosed by GCC as well, and I thought libc++ was also tested with it. Is that maybe not the case for those old Apple systems? In any case, let me know if you think otherwise, or if we will need to implement some kind of workaround... Comment Actions Ah, I understand. Thanks for explaining and sorry, I had only taken a quick look. So -- the reason why this isn't found by the libc++ CI is that it only triggers when we build on macOS, and our CI does not perform a bootstrapping build on macOS. In other words, we're never using your new modified Clang to run our tests on macOS in the libc++ CI. Regarding the bug itself, I am not sure I understand why this is triggering because the path class itself *is* marked as unavailable, which means that all of its methods should be marked as unavailable. I'll have to pull your patch down, build my Clang and play around with that. In terms of the next steps:
Comment Actions I guess the interesting question here would be: @thakis, is there a reason why you are using the SDK-provided libc++ but the tip-of-trunk Clang for building Chrome? (I feel like we've talked about this before but I don't remember). Comment Actions Because that's what you recommended :) We used to use the just-built libc++ but that had other issues. Previous discussions that I found in a hurry: https://reviews.llvm.org/D128927#3670288 , and https://reviews.llvm.org/D82702#2153627 and onward. Comment Actions Ugh, OK. Yeah, your previous setup was even much weirder in fact, it used the trunk libc++ headers but linked against the system dylib IIRC. I stand by my previous recommendation, but your setup is still unusual: you're using the SDK libc++ with a ToT clang, which is technically not a supported combination. The best would be to use the Apple-provided Clang on macOS as well, but I understand you might have reasons to want to use the ToT clang instead. I investigated the issue and the problem reproduces for me with: cat <<EOF | xcrun <path-to-clang-with-this-patch-applied>/clang++ -xc++ - -mmacosx-version-min=10.12 -std=c++17 -v -nostdinc++ -isystem <path-to-just-built-LLVM-libc++> #include <filesystem> #include <type_traits> int main() { } EOF It does not reproduce with trunk -- I bisected the fix to: commit 5fab33af7f083a0043112742027172e9f297c07f Author: Nikolas Klauser <nikolasklauser@berlin.de> Date: Tue Sep 6 00:33:34 2022 +0200 [libc++] Avoid instantiating type_trait classes Use `using` aliases to avoid instantiating lots of types Reviewed By: ldionne, #libc Spies: libcxx-commits, miyuki Differential Revision: https://reviews.llvm.org/D132785 Looking at that commit, which had nothing to do with fixing availability markup, I think this Clang patch might still be missing some diagnostics? Should it diagnose when the typename is accessed through an alias? If that's the case, libc++ would fail with trunk as well, since 5fab33af7f083 would not silence these issues. And if that's the case, then I am not sure how to fix libc++ at all. I suspect the problem here is really https://github.com/llvm/llvm-project/issues/40340, since we do mark the filesystem classes as unavailable as we should. Comment Actions It's quite possible it's missing other cases, there isn't a general helper in clang to access a declaration which would 'automatically' perform the use-checking, so I expect a lot of places to be repeating code / calling low level accessors which do not. I haven't come across an issue with type aliases, though I am not on a personal crusade to fix all these right now, this just came across while I was working on something else. Now that you mention it, looking at the code, I think we don't diagnose an use of a type alias itself, if that is what you mean? Ie, clang doesn't, GCC does, MSVC doesn't: https://godbolt.org/z/WEo4enjbz Would fixing that be a problem for libc++? Otherwise, typename accesses through the type alias pattern itself should work like from any context, except that if the access happens through a dependent entity, we should be diagnosing it when we instantiate the type alias. Does what you are talking about fall into any of that? Otherwise, do you have a short example? Otherwise, if current libc++ is fine and building older libc++with newer clang is not supported, is everyone fine with merging this back, as is? Comment Actions
Where is it specified that it's not a supported combination? Why should it not be supported? You don't even get libc++ from the llvm tree unless you explicitly enable it. Comment Actions It might be a problem, but I would argue we should still do it after fixing any problematic cases. It seems like Clang's current behavior is broken, as it basically ignores the [[deprecated]] attribute on aliases?
Sorry, I'm a bit lost. Let's take it back to the errors we're actually seeing with my reproducer above: In file included from <stdin>:1: In file included from SDK/usr/include/c++/v1/filesystem:245: In file included from SDK/usr/include/c++/v1/__filesystem/directory_entry.h:14: In file included from SDK/usr/include/c++/v1/__chrono/time_point.h:13: In file included from SDK/usr/include/c++/v1/__chrono/duration.h:14: In file included from SDK/usr/include/c++/v1/limits:107: In file included from SDK/usr/include/c++/v1/type_traits:421: In file included from SDK/usr/include/c++/v1/__functional/invoke.h:17: In file included from SDK/usr/include/c++/v1/__type_traits/decay.h:13: SDK/usr/include/c++/v1/__type_traits/add_pointer.h:26:50: error: 'type' is unavailable: introduced in macOS 10.15 _IsSame<typename remove_cv<_Tp>::type, void>::value> ^ SDK/usr/include/c++/v1/__type_traits/add_pointer.h:33:39: note: in instantiation of default argument for '__add_pointer_impl<std::filesystem::file_status>' required here {typedef _LIBCPP_NODEBUG typename __add_pointer_impl<_Tp>::type type;}; ^~~~~~~~~~~~~~~~~~~~~~~ SDK/usr/include/c++/v1/__type_traits/decay.h:44:40: note: in instantiation of template class 'std::add_pointer<std::filesystem::file_status>' requested here typename add_pointer<_Up>::type, ^ SDK/usr/include/c++/v1/__type_traits/decay.h:56:38: note: in instantiation of template class 'std::__decay<std::filesystem::file_status, true>' requested here typedef _LIBCPP_NODEBUG typename __decay<_Up, __is_referenceable<_Up>::value>::type type; ^ SDK/usr/include/c++/v1/__filesystem/path.h:142:47: note: in instantiation of template class 'std::decay<std::filesystem::file_status>' requested here template <class _Source, class _DS = typename decay<_Source>::type, ^ SDK/usr/include/c++/v1/__filesystem/path.h:195:31: note: in instantiation of default argument for '__is_pathable_char_array<std::filesystem::file_status>' required here bool _IsCharIterT = __is_pathable_char_array<_Tp>::value, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SDK/usr/include/c++/v1/__filesystem/path.h:445:26: note: in instantiation of default argument for '__is_pathable<std::filesystem::file_status, false>' required here typename enable_if<__is_pathable<_SourceOrIter>::value, _Tp>::type; ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ SDK/usr/include/c++/v1/__filesystem/path.h:480:36: note: in instantiation of template type alias '_EnableIfPathable' requested here template <class _Source, class = _EnableIfPathable<_Source, void> > ^ SDK/usr/include/c++/v1/__filesystem/path.h:482:3: note: in instantiation of default argument for 'path<std::filesystem::file_status>' required here path(const _Source& __src, format = format::auto_format) { ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ SDK/usr/include/c++/v1/__filesystem/operations.h:99:75: note: while substituting deduced template arguments into function template 'path' [with _Source = file_status, $1 = (no value)] inline _LIBCPP_HIDE_FROM_ABI bool exists(const path& __p) { return exists(__status(__p)); } ^ SDK/usr/include/c++/v1/__filesystem/file_status.h:28:24: note: 'file_status' has been explicitly marked unavailable here class _LIBCPP_TYPE_VIS file_status { ^ file_status, exists and path::path are all marked as unavailable. Why do we diagnose the use of an unavailable type wayyy lower in the stack when we instantiate remove_cv? I am not seeing what libc++ is doing wrong.
See below for what I meant by "unsupported". Another way to think about this would be to say that Clang has the burden of keeping a slightly-older libc++ compiling. If that's the case, then one could argue that this change should be held off until Clang no longer cares about being able to compile that version of libc++. This is not really my decision to make, but I would argue that there's probably value in keeping things working (that would bite people not only on OS X, but anyone using an older libc++ with a newer Clang). If so, then perhaps it would make sense to wait a few months before we merge this, perhaps at least until LLVM 16 is out so there's an officially-released version of libc++ that doesn't break with this change. But I think that's a decision for the Clang folks to make, not libc++. I guess I should reword it that way: If you're using an older libc++ with a newer Clang, there is literally no way for that old libc++ to guarantee that it can work with that not-known-yet Clang. It would imply being forward compatible with potentially arbitrary changes in Clang, which is nonsensical. IOW, we can't guard against issues that we don't know exist yet. Does that make sense? Comment Actions Yep. I expect we will find lots of similar cases.
I would be completely fine with doing a targeted workaround for libc++. The longer we don't patch this and warn on new code, the less likely we will ever be able to get rid of the bug. The problem is that we don't fully understand what is going on here, and how narrow this workaround can be. Comment Actions We're happy to switch to any setup that allows us to do a bootstrap build of clang on macOS (ie one where we build clang with whatever host compiler, and then build it again with just-built clang). We've switched to the currently recommended method several times; if there's a new recommended method we can switch to that. We just need to know what it is :) From what I understand, using newer libc++ headers and linking against system libc++.dylib is explicitly supported (because ABI compat). Comment Actions FYI just another data point, I haven't taken a closer look yet, but now we start diagnosing some deprecated uses of std::experimental::pmr::polymorphic_allocator on current libc++ linux CI. |
I find it hard to understand this enum since it does not correlate with wordings well and need to read code in other places to understand what it is used for. How about using two parameters: bool DiagCtor (replace AK != TypeAccessKind::Explicit) and bool IsImplicitTypename (replace AK == TypeAccessKind::Typename) ?