This is an archive of the discontinued LLVM Phabricator instance.

Rewording the "static_assert" to static assertion
ClosedPublic

Authored by Codesbyusman on Jul 3 2022, 10:59 AM.

Details

Summary

This patch is basically the rewording of the static assert statement's output(error) on screen after failing. Failing a _Static_assert in C should not report that static_assert failed. It’d probably be better to reword the diagnostic to be more like GCC and say “static assertion” failed in both C and C++.

consider a c file having code

_Static_assert(0, "oh no!");

In clang the output is like:

<source>:1:1: error: static_assert failed: oh no!
_Static_assert(0, "oh no!");
^              ~
1 error generated.
Compiler returned: 1

Thus here the "static_assert" is not much good, it will be better to reword it to the "static assertion failed" to more generic. as the gcc prints as:

<source>:1:1: error: static assertion failed: "oh no!"
    1 | _Static_assert(0, "oh no!");
      | ^~~~~~~~~~~~~~
Compiler returned: 1

The above can also be seen here. This patch is about rewording the static_assert to static assertion.

Diff Detail

Event Timeline

Codesbyusman created this revision.Jul 3 2022, 10:59 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 3 2022, 10:59 AM
Codesbyusman requested review of this revision.Jul 3 2022, 10:59 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 3 2022, 10:59 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript

Three test cases are failing - Please see: https://buildkite.com/llvm-project/premerge-checks/builds/100784#0181c539-bcd1-4d01-a2db-f27a63e2f17e/6-15379
But I think you want to actually modify https://github.com/llvm/llvm-project/blob/93d6fdfc232c59975d52146532693178def5ad16/clang/include/clang/Basic/DiagnosticSemaKinds.td#L11133 and https://github.com/llvm/llvm-project/blob/93d6fdfc232c59975d52146532693178def5ad16/clang/include/clang/Basic/DiagnosticSemaKinds.td#L11142.

Please also add more description to the patch as I said before like this -
Failing a _Static_assert in C should not report that static_assert failed. It’d probably be better to reword the diagnostic to be more like GCC and say “static assertion” failed in both C and C++.
For example -
cat test.c
_Static_assert(0, "oh no!");

clang prints
<source>:1:1: error: static_assert failed: oh no!
_Static_assert(0, "oh no!");
^ ~
1 error generated.

while gcc
<source>:1:1: error: static assertion failed: "oh no!"

1 | _Static_assert(0, "oh no!");
Compiler returned: 1

clang/test/CXX/expr/expr.prim/expr.prim.req/nested-requirement.cpp
49

a 'newline' is missing here.

clang/test/CXX/temp/temp.constr/temp.constr.constr/partial-specializations.cpp
86

same

Updating the files as required

Codesbyusman edited the summary of this revision. (Show Details)Jul 4 2022, 11:58 PM

All changes in one place

all changes in one patch revision

new changes as required -- final

Thank you for getting a great start on this diagnostic rewording!

I think we should also update DiagnosticParseKinds.td at the same time, but because that's about parsing rather than semantics, not all of the diagnostics should be reworded to use "static assertion".

err_expected_semi_after_static_assert should switch to use %0 and pass in the actual parsed token -- I am pretty sure (but not 100% sure because I didn't try this myself) that it will print the actual spelling used for the token in that case.

warn_cxx98_compat_static_assert should get single quotes around the static_assert in the message, but is otherwise fine.

ext_ms_static_assert, ext_cxx_static_assert_no_message, ext_c_static_assert_no_message, warn_cxx14_compat_static_assert_no_message, warn_c17_compat_static_assert_no_message, and err_templated_invalid_declaration are all fine as-is.

clang/include/clang/Basic/DiagnosticSemaKinds.td
11133

This is a case where the original text was slightly better -- this diagnostic is only issued for C++ code (modules don't exist in C), so using static_assert was fine.

11142

Same is true here.

Codesbyusman marked 2 inline comments as done.Jul 5 2022, 6:21 AM

updating some to the back strings becaue the orginal were more better

updating the test files for the last update

Precommit CI failures currently look to be unrelated. I think the only thing remaining is this bit:

I think we should also update DiagnosticParseKinds.td at the same time...

Yes I am looking into the DiagnosticParseKinds.td

Yes I am looking into the DiagnosticParseKinds.td. Kindly also look another error encountered in testing while updating the test. Although not having in my system but this is giving .
Look here

Yes I am looking into the DiagnosticParseKinds.td

Excellent, thank you!

Yes I am looking into the DiagnosticParseKinds.td. Kindly also look another error encountered in testing while updating the test. Although not having in my system but this is giving .
Look here

I noticed that one earlier -- I think it's unrelated to your changes. (I've seen that test spuriously fail before and there's nothing in the test files that uses a static assertion, so it seems highly unlikely that changes to the diagnostic message caused that issue; I expect it will go away on its own when you push up another patch in the future.) I think it's safe for you to ignore.

the parser updates maked for the static asserion

I think this is getting pretty close! I did have some style guideline changes, and at least one test addition I'd like to see though. Also, it looks like precommit CI caught a test that still needs to be updated:

FAIL: Clang :: SemaCXX/cxx98-compat.cpp (16448 of 68726)

******************** TEST 'Clang :: SemaCXX/cxx98-compat.cpp' FAILED ********************

Script:

--

: 'RUN: at line 1';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++11 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp

: 'RUN: at line 2';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++14 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp -DCXX14COMPAT

: 'RUN: at line 3';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++17 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp -DCXX14COMPAT -DCXX17COMPAT

--

Exit Code: 1



Command Output (stderr):

--

error: 'warning' diagnostics expected but not seen: 

  File /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp Line 155: static_assert declarations are incompatible with C++98

error: 'warning' diagnostics seen but not expected: 

  File /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp Line 155: 'static_assert' declarations are incompatible with C++98

2 errors generated.



--



********************
clang/include/clang/Basic/DiagnosticParseKinds.td
285

I think we're missing some test coverage, because I would have expected at least one test to fail because of this change. I think you should add a test to clang/test/SemaCXX/static-assert.cpp like this:

static_assert(1, "")
_Static_assert(1, "")

where the semi colons are missing, and add the correct expected-error lines to each so that we have coverage for your changes.

clang/include/clang/Parse/Parser.h
1046
clang/lib/Parse/ParseDeclCXX.cpp
932
996–1000

You should also make sure this fits in the usual 80-col limit and if it doesn't, clang-format the patch (https://clang.llvm.org/docs/ClangFormat.html#script-for-patch-reformatting).

clang/lib/Parse/Parser.cpp
156–175

updated the styles and other small things

Codesbyusman marked 3 inline comments as done.Jul 13 2022, 7:10 AM

I think this is getting pretty close! I did have some style guideline changes, and at least one test addition I'd like to see though. Also, it looks like precommit CI caught a test that still needs to be updated:

FAIL: Clang :: SemaCXX/cxx98-compat.cpp (16448 of 68726)

******************** TEST 'Clang :: SemaCXX/cxx98-compat.cpp' FAILED ********************

Script:

--

: 'RUN: at line 1';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++11 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp

: 'RUN: at line 2';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++14 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp -DCXX14COMPAT

: 'RUN: at line 3';   /var/lib/buildkite-agent/builds/llvm-project/build/bin/clang -cc1 -internal-isystem /var/lib/buildkite-agent/builds/llvm-project/build/lib/clang/15.0.0/include -nostdsysteminc -fsyntax-only -std=c++17 -Wc++98-compat -verify /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp -DCXX14COMPAT -DCXX17COMPAT

--

Exit Code: 1



Command Output (stderr):

--

error: 'warning' diagnostics expected but not seen: 

  File /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp Line 155: static_assert declarations are incompatible with C++98

error: 'warning' diagnostics seen but not expected: 

  File /var/lib/buildkite-agent/builds/llvm-project/clang/test/SemaCXX/cxx98-compat.cpp Line 155: 'static_assert' declarations are incompatible with C++98

2 errors generated.



--



********************

I just make some changes in the cxx98-compact.cpp and test was passed I was missing to update the expected-error comment for this.

the tests for the coverage of changes being made for the static assertion

Codesbyusman marked an inline comment as done.Jul 13 2022, 8:44 AM

A couple of comment suggestions, our comments use a full-stop.

erichkeane added inline comments.Jul 13 2022, 8:55 AM
clang/lib/Parse/ParseDeclCXX.cpp
931
997–999

updated the comment styles suggested

Codesbyusman marked an inline comment as done.Jul 13 2022, 9:40 AM
aaron.ballman added inline comments.Jul 13 2022, 10:04 AM
clang/lib/Parse/ParseDeclCXX.cpp
931
997–999

My suggestion is slightly different from Erich's (mine has whitespace and capitalization fixes), also, this is definitely over the 80 col limit, so you should format the patch.

clang/test/CXX/expr/expr.prim/expr.prim.req/nested-requirement.cpp
49

Still missing the newline at the end of the file here.

clang/test/CXX/temp/temp.constr/temp.constr.constr/partial-specializations.cpp
86

Same here

erichkeane added inline comments.Jul 13 2022, 10:11 AM
clang/lib/Parse/ParseDeclCXX.cpp
997–999

Ah, missed that <3

updated the format issues

Codesbyusman marked an inline comment as done.Jul 13 2022, 10:25 AM
Codesbyusman added inline comments.
clang/test/CXX/expr/expr.prim/expr.prim.req/nested-requirement.cpp
49

what here is mean by the missing line I am adding the newline but no changes to the comment

No newline at end of file

aaron.ballman added inline comments.Jul 13 2022, 10:30 AM
clang/test/CXX/expr/expr.prim/expr.prim.req/nested-requirement.cpp
49

C++ used to have a requirement that every file end with a newline (C still has that requirement), so Phabricator is reminding you that the file should still end in a newline. The fix is to put a newline after the closing curly brace on line 47 (so the newline is line 48).

updating new lines

Please format your patch with clang-format. See the command at the end of this: https://clang.llvm.org/docs/ClangFormat.html

git diff -U0 --color=never | clang-format-diff.py -i -p1

run the clang-format

Please format your patch with clang-format. See the command at the end of this: https://clang.llvm.org/docs/ClangFormat.html

git diff -U0 --color=never | clang-format-diff.py -i -p1

Done

aaron.ballman accepted this revision.Jul 13 2022, 12:00 PM

The changes here LGTM aside from one thing (which I'll take care of when I land the changes on your behalf). ParseDeclCXX.cpp still has a bunch of unrelated whitespace changes in it which should be backed out. We don't want unrelated changes sneaking in to a commit because that makes a future git blame that much harder on folks. But, rather than have you fiddle with it, I can strip them easily enough on my end, so this is just something to keep in mind for next time.

Thank you for the cleanup here!

This revision is now accepted and ready to land.Jul 13 2022, 12:00 PM

ok. I will take care of this next time. Thank you.

This revision was landed with ongoing or failed builds.Jul 14 2022, 4:47 AM
This revision was automatically updated to reflect the committed changes.
hctim added a subscriber: hctim.Jul 14 2022, 10:58 AM

Looks like unfortunately this breaks all lots of libcxx tests - which were picked up by our sanitizer buildbots:

https://lab.llvm.org/buildbot/#/builders/85/builds/9157

Should be reproducible with just cmake -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" && make check-cxx.

An example:

******************** TEST 'llvm-libc++-shared.cfg.in :: std/utilities/format/format.formatter/format.parse.ctx/ch
eck_arg_id.verify.cpp' FAILED ********************                                                               
Script:                                                                                                          
--                                                                                                               
: 'COMPILED WITH';  /llvm-build/opt/./bin/clang++ /llvm
/libcxx/test/std/utilities/format/format.formatter/format.parse.ctx/check_arg_id.verify.cpp  --target=x86_64-unkn
own-linux-gnu -nostdinc++ -I /llvm-build/opt/include/c++/v1 -I /llvm-build/opt/include/x86_64-unknown-linux-gnu/c++/v1 -I /llvm/libcxx/test
/support -std=c++2b -Werror -Wall -Wextra -Wshadow -Wundef -Wno-unused-command-line-argument -Wno-attributes -Wno
-pessimizing-move -Wno-c++11-extensions -Wno-noexcept-type -Wno-atomic-alignment -Wno-user-defined-literals -Wno-
tautological-compare -Wsign-compare -Wunused-variable -Wunused-parameter -Wunreachable-code -Wno-unused-local-typ
edef -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER -D_LIBCPP_ENABLE_EXPERIMENTAL -D_LIBCPP_DISABLE_AVAILABILITY -fcorouti
nes-ts -Werror=thread-safety -Wuser-defined-warnings  -fsyntax-only -Wno-error -Xclang -verify -Xclang -verify-ig
nore-unexpected=note -ferror-limit=0                                                                             
--                                                                                                               
Exit Code: 1                                                                                                     
                                                                                                                 
Command Output (stdout):                                                                                         
--                                                                                                               
$ ":" "COMPILED WITH"                                                                                            
$ "/llvm-build/opt/./bin/clang++" "/llvm/libcxx/test/st
d/utilities/format/format.formatter/format.parse.ctx/check_arg_id.verify.cpp" "--target=x86_64-unknown-linux-gnu"
 "-nostdinc++" "-I" "/llvm-build/opt/include/c++/v1" "-I" "/llvm-build/opt/include/x86_64-unknown-linux-gnu/c++/v1" "-I" "/llvm/libcxx/test
/support" "-std=c++2b" "-Werror" "-Wall" "-Wextra" "-Wshadow" "-Wundef" "-Wno-unused-command-line-argument" "-Wno
-attributes" "-Wno-pessimizing-move" "-Wno-c++11-extensions" "-Wno-noexcept-type" "-Wno-atomic-alignment" "-Wno-u
ser-defined-literals" "-Wno-tautological-compare" "-Wsign-compare" "-Wunused-variable" "-Wunused-parameter" "-Wun
reachable-code" "-Wno-unused-local-typedef" "-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER" "-D_LIBCPP_ENABLE_EXPERIMENTA
L" "-D_LIBCPP_DISABLE_AVAILABILITY" "-fcoroutines-ts" "-Werror=thread-safety" "-Wuser-defined-warnings" "-fsyntax
-only" "-Wno-error" "-Xclang" "-verify" "-Xclang" "-verify-ignore-unexpected=note" "-ferror-limit=0"             
# command stderr:                                                                                                
error: 'error' diagnostics expected but not seen:                                                                
  File llvm/libcxx/test/std/utilities/format/format.formatter/format.parse.ctx/chec
k_arg_id.verify.cpp Line 29: static assertion expression is not an integral constant expression                  
2 errors generated.                                                                                              
                                                                                                                 
error: command failed with exit status: 1                                                                        
                                                                                                                 
--

Looks like unfortunately this breaks all lots of libcxx tests - which were picked up by our sanitizer buildbots:

Thanks for letting us know. Do you happen to know if there's any particular reason why no email was sent out for the build failure?

Looks like unfortunately this breaks all lots of libcxx tests - which were picked up by our sanitizer buildbots:

Thanks for letting us know. Do you happen to know if there's any particular reason why no email was sent out for the build failure?

No, sorry. Did it not get sent to codesbyusman@gmail.com? I see you committed it, but their address is the author of the commit.

Looks like unfortunately this breaks all lots of libcxx tests - which were picked up by our sanitizer buildbots:

Thanks for letting us know. Do you happen to know if there's any particular reason why no email was sent out for the build failure?

No, sorry. Did it not get sent to codesbyusman@gmail.com? I see you committed it, but their address is the author of the commit.

That's possible, but I recall getting build failure emails before when I've committed on someone's behalf. Oh well, thanks for letting us know and sorry for the breakage!

Most of the libc++ CI testing happens on BuildKite, and we only trigger the libc++ buildkite pipeline if files in libcxx/ (and a few others) have changed. This wasn't the case here. If you want to re-land without breaking libc++, you'd have to go through libc++'s .verify.cpp tests and change the regex that we look for.

xgupta reopened this revision.Jul 19 2022, 7:20 AM
This revision is now accepted and ready to land.Jul 19 2022, 7:20 AM

the libcxx updates but not tested. working on them

Herald added a project: Restricted Project. · View Herald TranscriptJul 19 2022, 7:36 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
This revision now requires review to proceed.Jul 19 2022, 7:36 AM

the libcxx updates but not tested. working on them

I just run the check-cxx on local system. It seems to pass all testcases after applying the current patch.

Testing Time: 2147.75s

Unsupported      :  307
Passed           : 7184
Expectedly Failed:   41

the libcxx updates but not tested. working on them

I just run the check-cxx on local system. It seems to pass all testcases after applying the current patch.

Testing Time: 2147.75s

Unsupported      :  307
Passed           : 7184
Expectedly Failed:   41

Thanks @xgupta

Tests ran clean for me as well. Looking at the CI, it appears that the libcxx tests are improperly configured, and are being run against an 'older' version of the compiler (not including this patch).

This revision was not accepted when it landed; it landed in state Needs Review.Jul 21 2022, 6:34 AM
This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.

Tests ran clean for me as well. Looking at the CI, it appears that the libcxx tests are improperly configured, and are being run against an 'older' version of the compiler (not including this patch).

The libc++ CI isn't misconfigured. libc++ supports older clang versions. Please revert this and fix the tests. You should be able to use a regex.

Also, please wait for #libc approval next time.

Also, please wait for #libc approval next time.

This, x1000.

We go through the trouble of having excellent pre-commit testing and automatic review groups assigned to reviews, please don't bypass that.

Also, please wait for #libc approval next time.

This, x1000.

We go through the trouble of having excellent pre-commit testing and automatic review groups assigned to reviews, please don't bypass that.

We certainly will keep this in mind for the future, thanks for pointing it out! However, the precommit CI behavior confused multiple Clang contributors (I also thought the bot was misconfigured because Clang tests are never run against old Clang versions), the output did not clearly identify what was wrong or how to address it, and the difficulties with testing this locally are all things the libc++ maintainers should also keep in mind.

Also, please wait for #libc approval next time.

This, x1000.

We go through the trouble of having excellent pre-commit testing and automatic review groups assigned to reviews, please don't bypass that.

We certainly will keep this in mind for the future, thanks for pointing it out! However, the precommit CI behavior confused multiple Clang contributors (I also thought the bot was misconfigured because Clang tests are never run against old Clang versions), the output did not clearly identify what was wrong or how to address it, and the difficulties with testing this locally are all things the libc++ maintainers should also keep in mind.

What can we do to make it easier for Clang contributors to understand the libc++ CI?
Maybe we can discuss it on Discord.

libcxx/test/libcxx/atomics/atomics.types.operations/atomics.types.operations.req/atomic_fetch_add.verify.cpp
49 ↗(On Diff #446469)

All libc++ tests need a regex accepting both the old and new diagnostic. Since we support older Clang versions.

Also, please wait for #libc approval next time.

This, x1000.

We go through the trouble of having excellent pre-commit testing and automatic review groups assigned to reviews, please don't bypass that.

We certainly will keep this in mind for the future, thanks for pointing it out! However, the precommit CI behavior confused multiple Clang contributors (I also thought the bot was misconfigured because Clang tests are never run against old Clang versions), the output did not clearly identify what was wrong or how to address it, and the difficulties with testing this locally are all things the libc++ maintainers should also keep in mind.

What can we do to make it easier for Clang contributors to understand the libc++ CI?
Maybe we can discuss it on Discord.

That's a great question to be asking, so thank you! I'm not on Discord (still on IRC though), but I'm happy to chat about it via whatever means works for us.

At a high-level, I think it boils down to familiarity. If we can get the libc++ CI to run as part of precommit CI (assuming precommit CI can be made to run with reliable results, which is a bit of a question mark), then I think that will help get all clang contributors more familiar with the libc++ testing behavior over time, because we'll be far more used to seeing it when we cause a failure. It would have also helped to catch the cause for the initial revert where everyone thought the patch was ready to land. Another thing that would help would be to get the libc++ test bots into the LLVM lab (https://lab.llvm.org/buildbot/#/builders) and ensure that they're sending failure emails to everyone on the blame list including people who land a patch on behalf of others. It looks like we have one builder for libc++, but it doesn't appear to be working recently: https://lab.llvm.org/buildbot/#/builders/156.

Also, please wait for #libc approval next time.

This, x1000.

We go through the trouble of having excellent pre-commit testing and automatic review groups assigned to reviews, please don't bypass that.

We certainly will keep this in mind for the future, thanks for pointing it out! However, the precommit CI behavior confused multiple Clang contributors (I also thought the bot was misconfigured because Clang tests are never run against old Clang versions), the output did not clearly identify what was wrong or how to address it, and the difficulties with testing this locally are all things the libc++ maintainers should also keep in mind.

What can we do to make it easier for Clang contributors to understand the libc++ CI?
Maybe we can discuss it on Discord.

That's a great question to be asking, so thank you! I'm not on Discord (still on IRC though), but I'm happy to chat about it via whatever means works for us.

At a high-level, I think it boils down to familiarity. If we can get the libc++ CI to run as part of precommit CI (assuming precommit CI can be made to run with reliable results, which is a bit of a question mark), then I think that will help get all clang contributors more familiar with the libc++ testing behavior over time, because we'll be far more used to seeing it when we cause a failure. It would have also helped to catch the cause for the initial revert where everyone thought the patch was ready to land. Another thing that would help would be to get the libc++ test bots into the LLVM lab (https://lab.llvm.org/buildbot/#/builders) and ensure that they're sending failure emails to everyone on the blame list including people who land a patch on behalf of others. It looks like we have one builder for libc++, but it doesn't appear to be working recently: https://lab.llvm.org/buildbot/#/builders/156.

As @ldionne said we mainly use a pre-commit CI instead of a post-commit CI. I agree that it's probably a lack of familiarity, so let's try to remedy that. That will make everybody happier.

I'll reach out to you on IRC either tomorrow or next week. Then we can discuss how we can get the clang contributors more familiar with libc++'s way of testing.

The libc++ CI runs pre-commit only, and it runs on all commits that touch something under libcxx/, libcxxabi/, runtimes/ and libunwind/. In particular, it won't trigger if you make changes to clang/ only -- that would create too much traffic and concretely it wouldn't help much, as we're rarely broken by Clang changes (this is an exception). Once the tests under libcxx/ were touched, the libc++ CI triggered, I found this build for instance: https://buildkite.com/llvm-project/libcxx-ci/builds/12210. I do think it boils down to familiarity -- while the Clang pre-commit CI is sometimes overlooked because it fails spuriously, the libc++ CI usually doesn't, so when it fails, there's usually a good reason. I do understand why someone mostly familiar with the Clang workflow could think they needed to be overlooked, though.

Concretely, libc++ supports the two latest releases of Clang, and so the CI tests against those. In fact, we run most of the tests using pre-installed Clangs since it allows us to bypass building Clang itself, which is a necessary optimization in the current setup. Regarding the tests themselves, I am able to find the following kind of output in the CI job that failed:

FAIL: llvm-libc++-shared.cfg.in :: std/numerics/numbers/illformed.verify.cpp (4388 of 7532)
[...]
error: 'error' diagnostics expected but not seen:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line * (directive at /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/libcxx/test/std/numerics/numbers/illformed.verify.cpp:15): static assertion failed{{.*}}A program that instantiates a primary template of a mathematical constant variable template is ill-formed.
error: 'error' diagnostics seen but not expected:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line 83: static_assert failed due to requirement '__false<int>' "A program that instantiates a primary template of a mathematical constant variable template is ill-formed."
2 errors generated.

IMO this is reasonable output, in fact it's pretty much as good as we could have had. So I'm not taking any action item regarding improving the output of our CI, but I'm entirely open to suggestions if somebody has some.

At a high-level, I think it boils down to familiarity. If we can get the libc++ CI to run as part of precommit CI (assuming precommit CI can be made to run with reliable results, which is a bit of a question mark),

It is pretty reliable. However, I don't think it'll ever become reasonable to run the full libc++ CI on every Clang patch due to resource limitations.

It would have also helped to catch the cause for the initial revert where everyone thought the patch was ready to land.

100% agreed, however this is blocked on the resource limitations mentioned above. I guess one thing we could do here is trigger a simple bootstrapping build of Clang and run the libc++ tests in a single configuration even on changes to clang/. That might catch most issues and perhaps that would be small enough to be scalable. I'll experiment with that after the release.

Another thing that would help would be to get the libc++ test bots into the LLVM lab (https://lab.llvm.org/buildbot/#/builders)

Libc++ CI is pre-commit, the lab is post-commit only AFAICT. I also don't know whether there's a way to bridge between BuildKite and BuildBot, and what that would look like. So while I share the desire to have a single place to look for all LLVM wide CI, I'm not sure it is possible given the variety of technologies used.

and ensure that they're sending failure emails to everyone on the blame list including people who land a patch on behalf of others.

100% agreed here -- this is one problem with our current setup.

It looks like we have one builder for libc++, but it doesn't appear to be working recently: https://lab.llvm.org/buildbot/#/builders/156.

Yeah, this one needs to be removed, it's a remnant of when we used to have libc++ CI on BuildBot.

Okay, so to summarize:

  1. I'll experiment with a simple job that runs the libc++ pre-commit CI on every patch to Clang. We'll see if that scales.
  2. I'll look into sending blame emails from the ongoing (every 3h) job that runs the whole libc++ CI. This is basically equivalent to the buildbot functionality and it's arguably something that we're missing here. I have no idea how to do that, though, but it must be possible.
  3. Usually, for these types of failures, we actually notice and fix it on our end without requesting a revert. For this kind of change, my opinion is that the burden of fixing the code should have been on us, kind of like when the LLDB data formatters break because we made a totally legal change in libc++. Nobody jumped in to fix it on the libc++ side this time because (I assume) we're racing for the release, but normally I or someone else would have chimed in to fix this without annoying Clang.
  4. That being said, let's try to spread the word that when the libc++ tests do fail on a Clang patch (which implies that libcxx/ will have been touched until (1) is done), they must not be ignored.

The libc++ CI runs pre-commit only, and it runs on all commits that touch something under libcxx/, libcxxabi/, runtimes/ and libunwind/. In particular, it won't trigger if you make changes to clang/ only -- that would create too much traffic and concretely it wouldn't help much, as we're rarely broken by Clang changes (this is an exception). Once the tests under libcxx/ were touched, the libc++ CI triggered, I found this build for instance: https://buildkite.com/llvm-project/libcxx-ci/builds/12210. I do think it boils down to familiarity -- while the Clang pre-commit CI is sometimes overlooked because it fails spuriously, the libc++ CI usually doesn't, so when it fails, there's usually a good reason. I do understand why someone mostly familiar with the Clang workflow could think they needed to be overlooked, though.

Thanks for the explanation!

FWIW, the specific reason why I overloooked it was because of running against old versions of Clang -- I saw the diagnostic was correct in the test and presumed that the old clang was a bot configuration issue. It definitely doesn't help that Clang's precommit CI tends to be spuriously broken quite often; it's trained some of us reviewers to be more dismissive of failing precommit CI than we should.

Concretely, libc++ supports the two latest releases of Clang, and so the CI tests against those. In fact, we run most of the tests using pre-installed Clangs since it allows us to bypass building Clang itself, which is a necessary optimization in the current setup. Regarding the tests themselves, I am able to find the following kind of output in the CI job that failed:

Yeah, which makes a lot of sense for libc++!

FAIL: llvm-libc++-shared.cfg.in :: std/numerics/numbers/illformed.verify.cpp (4388 of 7532)
[...]
error: 'error' diagnostics expected but not seen:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line * (directive at /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/libcxx/test/std/numerics/numbers/illformed.verify.cpp:15): static assertion failed{{.*}}A program that instantiates a primary template of a mathematical constant variable template is ill-formed.
error: 'error' diagnostics seen but not expected:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line 83: static_assert failed due to requirement '__false<int>' "A program that instantiates a primary template of a mathematical constant variable template is ill-formed."
2 errors generated.

IMO this is reasonable output, in fact it's pretty much as good as we could have had. So I'm not taking any action item regarding improving the output of our CI, but I'm entirely open to suggestions if somebody has some.

I don't have a concrete suggestion for the CI output itself -- the failure was clear in retrospect, this was more a matter of not being familiar with the testing practices used.

At a high-level, I think it boils down to familiarity. If we can get the libc++ CI to run as part of precommit CI (assuming precommit CI can be made to run with reliable results, which is a bit of a question mark),

It is pretty reliable. However, I don't think it'll ever become reasonable to run the full libc++ CI on every Clang patch due to resource limitations.

The resource limitations are certainly a pragmatic thing for us to be concerned with, but I hope we're able to find some happy middle ground.

It would have also helped to catch the cause for the initial revert where everyone thought the patch was ready to land.

100% agreed, however this is blocked on the resource limitations mentioned above. I guess one thing we could do here is trigger a simple bootstrapping build of Clang and run the libc++ tests in a single configuration even on changes to clang/. That might catch most issues and perhaps that would be small enough to be scalable. I'll experiment with that after the release.

Thanks, that would be really beneficial. It's not just diagnostic wording changes that cause Clang developers problems with libc++ reverts; I've also seen C++20 work reverted because it broke libc++ without anyone in Clang knowing about it until after it landed (most recently, it's with the deferred concept checking work done by @erichkeane).

Another thing that would help would be to get the libc++ test bots into the LLVM lab (https://lab.llvm.org/buildbot/#/builders)

Libc++ CI is pre-commit, the lab is post-commit only AFAICT.

Correct, but right now we have no community testing for libc++ whatsoever from the perspective of Clang folks. We only find out about failures when libc++ folks start reverting or pointing them out.

I also don't know whether there's a way to bridge between BuildKite and BuildBot, and what that would look like. So while I share the desire to have a single place to look for all LLVM wide CI, I'm not sure it is possible given the variety of technologies used.

I don't know enough about the technology to know if it's easy or hard, unfortunately.

and ensure that they're sending failure emails to everyone on the blame list including people who land a patch on behalf of others.

100% agreed here -- this is one problem with our current setup.

It looks like we have one builder for libc++, but it doesn't appear to be working recently: https://lab.llvm.org/buildbot/#/builders/156.

Yeah, this one needs to be removed, it's a remnant of when we used to have libc++ CI on BuildBot.

Okay, so to summarize:

  1. I'll experiment with a simple job that runs the libc++ pre-commit CI on every patch to Clang. We'll see if that scales.

Thanks, that would be awesome if it works out!

  1. I'll look into sending blame emails from the ongoing (every 3h) job that runs the whole libc++ CI. This is basically equivalent to the buildbot functionality and it's arguably something that we're missing here. I have no idea how to do that, though, but it must be possible.

Thank you!!

  1. Usually, for these types of failures, we actually notice and fix it on our end without requesting a revert. For this kind of change, my opinion is that the burden of fixing the code should have been on us, kind of like when the LLDB data formatters break because we made a totally legal change in libc++. Nobody jumped in to fix it on the libc++ side this time because (I assume) we're racing for the release, but normally I or someone else would have chimed in to fix this without annoying Clang.

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.) That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help? (For reference, Usman is a Google Summer of Code mentee and this was his first patch for Clang. I think this has been a really good way for him to see how open source collaborations work in practice but I also don't want him to get bogged down too much if I can help it. Normally I'd step in to help directly, but I've been in WG14 meetings all week and so I am pretty swamped at the moment.)

  1. That being said, let's try to spread the word that when the libc++ tests do fail on a Clang patch (which implies that libcxx/ will have been touched until (1) is done), they must not be ignored.

Absolutely!

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.)

Yes, exactly. It's always possible to write tests that depend on implementation details of another component in subtle ways, and the fact that such brittle tests exist doesn't create a responsibility on that component to avoid breaking those downstream users. Here, it's about Clang making a valid change and libc++ containing brittle tests, but it happens quite often where libc++ changes something valid and a downstream consumer is broken in some way. The LLVM revert culture has some benefits to keep things working in a post-commit CI world, however I've noticed that it also creates a lot of friction between projects and sometimes makes important work much more difficult to land than it should. For example, we're currently in the middle of D128146 where LLDB reverted an important patch for LLVM 15 because one of their tests broke. This is not something that comes up super often, but it's extremely disruptive and frustrating when it does, and I think it would be worth trying to address. Anyway, I'm digressing now, but I'll try to talk to a few people at the LLVM Dev Meeting to see if this is a shared concern and to think about potential solutions to address this.

That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

Agreed.

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help?

Replacing static_assert by (static_assert|static assertion) should do the trick. See the patch attached to this comment, I think it should satisfy the CI @Codesbyusman.

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.)

Yes, exactly. It's always possible to write tests that depend on implementation details of another component in subtle ways, and the fact that such brittle tests exist doesn't create a responsibility on that component to avoid breaking those downstream users. Here, it's about Clang making a valid change and libc++ containing brittle tests, but it happens quite often where libc++ changes something valid and a downstream consumer is broken in some way. The LLVM revert culture has some benefits to keep things working in a post-commit CI world, however I've noticed that it also creates a lot of friction between projects and sometimes makes important work much more difficult to land than it should. For example, we're currently in the middle of D128146 where LLDB reverted an important patch for LLVM 15 because one of their tests broke. This is not something that comes up super often, but it's extremely disruptive and frustrating when it does, and I think it would be worth trying to address. Anyway, I'm digressing now, but I'll try to talk to a few people at the LLVM Dev Meeting to see if this is a shared concern and to think about potential solutions to address this.

That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

Agreed.

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help?

Replacing static_assert by (static_assert|static assertion) should do the trick. See the patch attached to this comment, I think it should satisfy the CI @Codesbyusman.

I just want to say: I really appreciate the insight you gave into the libc++ testing in this thread. From the CFE's perspective, our view of precommit is "it is basically always misconfigured or broken in some way", so knowing that you guys keep yours running well is nice to hear! We'll be more cognizant in the future.

I DO think there IS a sizable difference between patches like this one, and patches like my deferred-concepts (which, btw, the libc++ tests have been great for coming up with new tests... though it was a pain to figure out how to run them in the first place!). My patches DID deserve the reverts it received so far, so don't see Aaron's mention as an issue with that.

I appreciate your suggestion/patch file, that IS a much better regex than we'd come up with, and looks like it should work. So thank you! We'll commit that once @Codesbyusman can get it integrated into his patch (and we'll make sure CI passes too!).

Thanks again Louis!

The libc++ CI runs pre-commit only, and it runs on all commits that touch something under libcxx/, libcxxabi/, runtimes/ and libunwind/. In particular, it won't trigger if you make changes to clang/ only -- that would create too much traffic and concretely it wouldn't help much, as we're rarely broken by Clang changes (this is an exception). Once the tests under libcxx/ were touched, the libc++ CI triggered, I found this build for instance: https://buildkite.com/llvm-project/libcxx-ci/builds/12210. I do think it boils down to familiarity -- while the Clang pre-commit CI is sometimes overlooked because it fails spuriously, the libc++ CI usually doesn't, so when it fails, there's usually a good reason. I do understand why someone mostly familiar with the Clang workflow could think they needed to be overlooked, though.

Thanks for the explanation!

FWIW, the specific reason why I overloooked it was because of running against old versions of Clang -- I saw the diagnostic was correct in the test and presumed that the old clang was a bot configuration issue. It definitely doesn't help that Clang's precommit CI tends to be spuriously broken quite often; it's trained some of us reviewers to be more dismissive of failing precommit CI than we should.

Concretely, libc++ supports the two latest releases of Clang, and so the CI tests against those. In fact, we run most of the tests using pre-installed Clangs since it allows us to bypass building Clang itself, which is a necessary optimization in the current setup. Regarding the tests themselves, I am able to find the following kind of output in the CI job that failed:

Yeah, which makes a lot of sense for libc++!

FAIL: llvm-libc++-shared.cfg.in :: std/numerics/numbers/illformed.verify.cpp (4388 of 7532)
[...]
error: 'error' diagnostics expected but not seen:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line * (directive at /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/libcxx/test/std/numerics/numbers/illformed.verify.cpp:15): static assertion failed{{.*}}A program that instantiates a primary template of a mathematical constant variable template is ill-formed.
error: 'error' diagnostics seen but not expected:
  File /home/libcxx-builder/.buildkite-agent/builds/2b1c77dd9e90-1/llvm-project/libcxx-ci/build/generic-cxx2b/include/c++/v1/numbers Line 83: static_assert failed due to requirement '__false<int>' "A program that instantiates a primary template of a mathematical constant variable template is ill-formed."
2 errors generated.

IMO this is reasonable output, in fact it's pretty much as good as we could have had. So I'm not taking any action item regarding improving the output of our CI, but I'm entirely open to suggestions if somebody has some.

I don't have a concrete suggestion for the CI output itself -- the failure was clear in retrospect, this was more a matter of not being familiar with the testing practices used.

At a high-level, I think it boils down to familiarity. If we can get the libc++ CI to run as part of precommit CI (assuming precommit CI can be made to run with reliable results, which is a bit of a question mark),

It is pretty reliable. However, I don't think it'll ever become reasonable to run the full libc++ CI on every Clang patch due to resource limitations.

The resource limitations are certainly a pragmatic thing for us to be concerned with, but I hope we're able to find some happy middle ground.

It would have also helped to catch the cause for the initial revert where everyone thought the patch was ready to land.

100% agreed, however this is blocked on the resource limitations mentioned above. I guess one thing we could do here is trigger a simple bootstrapping build of Clang and run the libc++ tests in a single configuration even on changes to clang/. That might catch most issues and perhaps that would be small enough to be scalable. I'll experiment with that after the release.

Thanks, that would be really beneficial. It's not just diagnostic wording changes that cause Clang developers problems with libc++ reverts; I've also seen C++20 work reverted because it broke libc++ without anyone in Clang knowing about it until after it landed (most recently, it's with the deferred concept checking work done by @erichkeane).

Another thing that would help would be to get the libc++ test bots into the LLVM lab (https://lab.llvm.org/buildbot/#/builders)

Libc++ CI is pre-commit, the lab is post-commit only AFAICT.

Correct, but right now we have no community testing for libc++ whatsoever from the perspective of Clang folks. We only find out about failures when libc++ folks start reverting or pointing them out.

I also don't know whether there's a way to bridge between BuildKite and BuildBot, and what that would look like. So while I share the desire to have a single place to look for all LLVM wide CI, I'm not sure it is possible given the variety of technologies used.

I don't know enough about the technology to know if it's easy or hard, unfortunately.

and ensure that they're sending failure emails to everyone on the blame list including people who land a patch on behalf of others.

100% agreed here -- this is one problem with our current setup.

It looks like we have one builder for libc++, but it doesn't appear to be working recently: https://lab.llvm.org/buildbot/#/builders/156.

Yeah, this one needs to be removed, it's a remnant of when we used to have libc++ CI on BuildBot.

Okay, so to summarize:

  1. I'll experiment with a simple job that runs the libc++ pre-commit CI on every patch to Clang. We'll see if that scales.

Thanks, that would be awesome if it works out!

  1. I'll look into sending blame emails from the ongoing (every 3h) job that runs the whole libc++ CI. This is basically equivalent to the buildbot functionality and it's arguably something that we're missing here. I have no idea how to do that, though, but it must be possible.

Thank you!!

  1. Usually, for these types of failures, we actually notice and fix it on our end without requesting a revert. For this kind of change, my opinion is that the burden of fixing the code should have been on us, kind of like when the LLDB data formatters break because we made a totally legal change in libc++. Nobody jumped in to fix it on the libc++ side this time because (I assume) we're racing for the release, but normally I or someone else would have chimed in to fix this without annoying Clang.

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.) That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help? (For reference, Usman is a Google Summer of Code mentee and this was his first patch for Clang. I think this has been a really good way for him to see how open source collaborations work in practice but I also don't want him to get bogged down too much if I can help it. Normally I'd step in to help directly, but I've been in WG14 meetings all week and so I am pretty swamped at the moment.)

Yes for me as a complete be beginner was great to see how really open source works. I am working on the libc++ and used regex for the passing of test of previous versions. That is almost complete.

  1. That being said, let's try to spread the word that when the libc++ tests do fail on a Clang patch (which implies that libcxx/ will have been touched until (1) is done), they must not be ignored.

Absolutely!

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.)

Yes, exactly. It's always possible to write tests that depend on implementation details of another component in subtle ways, and the fact that such brittle tests exist doesn't create a responsibility on that component to avoid breaking those downstream users. Here, it's about Clang making a valid change and libc++ containing brittle tests, but it happens quite often where libc++ changes something valid and a downstream consumer is broken in some way. The LLVM revert culture has some benefits to keep things working in a post-commit CI world, however I've noticed that it also creates a lot of friction between projects and sometimes makes important work much more difficult to land than it should. For example, we're currently in the middle of D128146 where LLDB reverted an important patch for LLVM 15 because one of their tests broke. This is not something that comes up super often, but it's extremely disruptive and frustrating when it does, and I think it would be worth trying to address. Anyway, I'm digressing now, but I'll try to talk to a few people at the LLVM Dev Meeting to see if this is a shared concern and to think about potential solutions to address this.

That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

Agreed.

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help?

Replacing static_assert by (static_assert|static assertion) should do the trick. See the patch attached to this comment, I think it should satisfy the CI @Codesbyusman.

I just want to say: I really appreciate the insight you gave into the libc++ testing in this thread. From the CFE's perspective, our view of precommit is "it is basically always misconfigured or broken in some way", so knowing that you guys keep yours running well is nice to hear! We'll be more cognizant in the future.

I DO think there IS a sizable difference between patches like this one, and patches like my deferred-concepts (which, btw, the libc++ tests have been great for coming up with new tests... though it was a pain to figure out how to run them in the first place!). My patches DID deserve the reverts it received so far, so don't see Aaron's mention as an issue with that.

I appreciate your suggestion/patch file, that IS a much better regex than we'd come up with, and looks like it should work. So thank you! We'll commit that once @Codesbyusman can get it integrated into his patch (and we'll make sure CI passes too!).

yes will update soon

Thanks again Louis!

FWIW, I've convinced myself that I agree with you here that the burden probably should have been on libc++ maintainers in this case. libc++ almost feels more like a downstream consumer of Clang in terms of testing needs, so I think when tests break because Clang diagnostic wording or behavior changes in a standards conforming way, libc++ folks should address it, but if Clang changes in a way that's not standards conforming, then Clang folks should address it. (Roughly speaking.)

Yes, exactly. It's always possible to write tests that depend on implementation details of another component in subtle ways, and the fact that such brittle tests exist doesn't create a responsibility on that component to avoid breaking those downstream users. Here, it's about Clang making a valid change and libc++ containing brittle tests, but it happens quite often where libc++ changes something valid and a downstream consumer is broken in some way. The LLVM revert culture has some benefits to keep things working in a post-commit CI world, however I've noticed that it also creates a lot of friction between projects and sometimes makes important work much more difficult to land than it should. For example, we're currently in the middle of D128146 where LLDB reverted an important patch for LLVM 15 because one of their tests broke. This is not something that comes up super often, but it's extremely disruptive and frustrating when it does, and I think it would be worth trying to address. Anyway, I'm digressing now, but I'll try to talk to a few people at the LLVM Dev Meeting to see if this is a shared concern and to think about potential solutions to address this.

That said, I absolutely think we all need to continue to collaborate closely with one another as best we can when issues arise, and I really appreciate the discussion on how we can do that!

Agreed.

For this particular issue, I'd like @Codesbyusman to continue to try to fix the libc++ testing issues (it's good experience), but if that takes significantly longer (say, more than 8 hours of his effort), perhaps @ldionne or someone else from libc++ will have a moment to step in to help?

Replacing static_assert by (static_assert|static assertion) should do the trick. See the patch attached to this comment, I think it should satisfy the CI @Codesbyusman.

thanks

Codesbyusman reopened this revision.Jul 22 2022, 9:00 AM

updating of the libcxx with the regex in the diagnostic messages, haven't test them yet locally

updated libcxx ready to check again

updated the libcxx all files

Mordante accepted this revision.Jul 24 2022, 12:56 PM

updated the libcxx all files

FYI for libcxx you can ignore the red format CI step. So the libcxx build is green :-)

I had a look at your libc++ changes and they look good, thanks.
So LGTM for the libc++ part.

This revision is now accepted and ready to land.Jul 24 2022, 12:56 PM

updated the libcxx all files

FYI for libcxx you can ignore the red format CI step. So the libcxx build is green :-)

I had a look at your libc++ changes and they look good, thanks.
So LGTM for the libc++ part.

That's great :)

This revision was landed with ongoing or failed builds.Jul 25 2022, 4:23 AM
This revision was automatically updated to reflect the committed changes.

updated the libcxx all files

FYI for libcxx you can ignore the red format CI step. So the libcxx build is green :-)

I had a look at your libc++ changes and they look good, thanks.
So LGTM for the libc++ part.

That's great :)

Thank you everyone for the great collaboration on this! I just landed the changes and will watch the build bots for any surprise fallout.

thakis added a subscriber: thakis.Aug 1 2022, 5:06 PM

This change caused a ton of churn, for what I understand fix the diag in C mode when assert.h is not included.

IMHO, it would've been better to change the diag to say either static_assert as before in c++ mode or when assert.h is included (i.e. almost always), and _Static_assert else. You can use PP.getLastMacroWithSpelling() to (effectively) detect if assert.h was included.

Is this something y'all had considered?

This change caused a ton of churn, for what I understand fix the diag in C mode when assert.h is not included.

Which is not that uncommon of a scenario: https://sourcegraph.com/search?q=context:global+file:.*%5C.c+_Static_assert&patternType=literal&case=yes

IMHO, it would've been better to change the diag to say either static_assert as before in c++ mode or when assert.h is included (i.e. almost always), and _Static_assert else. You can use PP.getLastMacroWithSpelling() to (effectively) detect if assert.h was included.

Is this something y'all had considered?

Yes, and that approach is actually more complicated to get correct. Part of the trouble is MSVC compatibility where static_assert is allowed in C mode even without including assert.h and that kind of thing. Ultimately, we decided the principle is: if the diagnostic is about parsing the syntax we'll use the actual token name, otherwise we'll use "static assertion" as a prose term instead to avoid having the diagnostic be wrong in edge cases.