This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Categorize mdspan assertions, and move assertions tests
ClosedPublic

Authored by crtrott on Jul 24 2023, 3:56 PM.

Details

Reviewers
var-const
Group Reviewers
Restricted Project
Summary

Categorize most of the mdspan related assertions as valid_element_access, since almost all of them in some way or another prevent direct, or later undetectable out of bounds accesses. For the vast majority of assertions I added comments explaining how they could lead to out of bounds access issues.

Also moved all the assertions tests from libcxx/test/std to libcxx/test/libcxx

Diff Detail

Event Timeline

crtrott created this revision.Jul 24 2023, 3:56 PM
Herald added a project: Restricted Project. · View Herald TranscriptJul 24 2023, 3:56 PM
Herald added a subscriber: arphaman. · View Herald Transcript
crtrott requested review of this revision.Jul 24 2023, 3:56 PM
Herald added a reviewer: Restricted Project. · View Herald TranscriptJul 24 2023, 3:56 PM
var-const accepted this revision.Jul 24 2023, 5:25 PM
var-const added a subscriber: var-const.

LGTM with one question (and a green CI). Thanks!

libcxx/include/__mdspan/extents.h
177

Nit: s/matchgin/matching/.

178

Nit: s/things/thinks/ (I think).

libcxx/include/__mdspan/layout_left.h
103

Nit: s/rank 1:/rank 1,/.

libcxx/include/__mdspan/mdspan.h
191–194

Is there any case where if we don't do the check inside layout_left/layout_right, this assertion in mdspan would not catch it?

This revision is now accepted and ready to land.Jul 24 2023, 5:25 PM
crtrott added inline comments.Jul 24 2023, 5:47 PM
libcxx/include/__mdspan/mdspan.h
191–194

Only if one would use layout_left/right on for their own data structure. For example mdarray will use reuse the layouts, and a user implemented version of that, or for example a user implemented "matrix_type" may not itself check. Generally I do expect the layouts to be used in such types, since its useful machinery beyond mdspan. The reason not to use mdspan in that case is that users may want value semantics of a matrix type, so they just store say a double[16] and a layout_left::mapping<extents<char, 4, 4> or so.

So the philosophical question is whether we should make the machinery for implementing such types access safe in hardened mode or not. I do believe there is value to doing so and promising essentially that stuff like the standard layouts are safe in hardened mode to be used on their own. But I am not dead-set on it.

var-const added inline comments.Jul 24 2023, 5:56 PM
libcxx/include/__mdspan/mdspan.h
191–194

Thank you for explaining. I'm a little concerned about imposing a penalty on the main use case (using mdspan with the default layout types) for the sake of, IIUC, a rather niche use case in the hardened mode. In addition, users that reuse the layout types can do their own checking if they want, so I think covering that case is a lesser priority. Also, while we can go both ways if necessary, I'd avoid removing assertions from the hardened mode if possible, so I'd rather start small and grow than vice versa. So that being said, I'd avoid double-checking in this release, with the potential to revisit this for LLVM 18. IIUC, this means that in the hardened mode:

  • users of mdspan will get all the benefits of checking without paying a potential extra cost for the double check (however small that cost might be);
  • users who implement their own data structures using layout_left/layout_right would need to do their own element access checking.

If so, I'd avoid double-checking, at least for this release.

Sounds good!

crtrott updated this revision to Diff 543771.Jul 24 2023, 6:11 PM

Apply review comments: specifically don't check index bounds in layout_left/right since it's double checking when used in mdspan.

crtrott marked 3 inline comments as done.Jul 24 2023, 6:12 PM
crtrott marked 2 inline comments as done.
crtrott updated this revision to Diff 543804.Jul 24 2023, 8:36 PM
crtrott updated this revision to Diff 543937.Jul 25 2023, 5:41 AM

Will ignore the CI failure of the "Module build with local submodule visibility": known common issue with the module cache producing over 1000 errors in the mold of:

bk;t=1690294997920/home/libcxx-builder/.buildkite-agent/builds/google-libcxx-builder-49573bbe06a2-1/llvm-project/libcxx-ci/build/generic-modules-lsv/include/c++/v1/__concepts/regular.h:13:2: fatal error: module file '/home/libcxx-builder/.cache/clang/ModuleCache/1BZ5PZK31RHUI/std_private_concepts_semiregular-1920JD1R2UQK3.pcm' is out of date and needs to be rebuilt

_bk;t=1690294997920   13 | #include <__concepts/semiregular.h>

Mark concurred.

Will also ignore the CI failure for Bootstrapping build based on Mark's input, that this test has issues and will be removed. It failed too here for example: https://buildkite.com/llvm-project/libcxx-ci/builds/28580

Will ignore the CI failure of the "Module build with local submodule visibility": known common issue with the module cache producing over 1000 errors in the mold of:

bk;t=1690294997920/home/libcxx-builder/.buildkite-agent/builds/google-libcxx-builder-49573bbe06a2-1/llvm-project/libcxx-ci/build/generic-modules-lsv/include/c++/v1/__concepts/regular.h:13:2: fatal error: module file '/home/libcxx-builder/.cache/clang/ModuleCache/1BZ5PZK31RHUI/std_private_concepts_semiregular-1920JD1R2UQK3.pcm' is out of date and needs to be rebuilt

_bk;t=1690294997920   13 | #include <__concepts/semiregular.h>

Mark concurred.

Indeed, we've seen this several times and it seems flaky.

Will also ignore the CI failure for Bootstrapping build based on Mark's input, that this test has issues and will be removed. It failed too here for example: https://buildkite.com/llvm-project/libcxx-ci/builds/28580

The test is already disabled for clang-18, which is what the bootstrap uses. The proposed removal is D156242.