This is an archive of the discontinued LLVM Phabricator instance.

[libc++][hardening][NFC] Add macros to enable hardened mode.
ClosedPublic

Authored by var-const on Jun 27 2023, 11:06 AM.

Details

Summary

This patch only adds new configuration knobs -- the actual assertions
will be added in follow-up patches.

Diff Detail

Event Timeline

var-const created this revision.Jun 27 2023, 11:06 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 27 2023, 11:06 AM
var-const requested review of this revision.Jun 27 2023, 11:06 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 27 2023, 11:06 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript

Note: this patch is split from https://reviews.llvm.org/D151637 (which is a draft but helps see the state towards which this patch is working).

Since this is not a draft, I looked more in detail. So some additional remarks.

libcxx/include/__config
215

_LIBCPP_ENABLE_HARDENED_MODE_DEFAULT is undefined, or is that in one another patch?
The same for _LIBCPP_ENABLE_HARDENED_DEBUG_MODE_DEFAULT.

229

Maybe enabled isn't the greatest word too. However defined is wrong; you made sure both are defined.

233

The comment on line 238 // TODO(hardening): more checks to be added here... seem to suggest we get quite a bit of duplication in the #if and #elif branch. So I propose

#if _LIBCPP_ENABLE_HARDENED_MODE || _LIBCPP_ENABLE_HARDENED_DEBUG_MODE
// do stuff for hardened and hardened debug
#endif // _LIBCPP_ENABLE_HARDENED_MODE || _LIBCPP_ENABLE_HARDENED_DEBUG_MODE

#if _LIBCPP_ENABLE_HARDENED_DEBUG_MODE
// do stuff only for hardened debug
#endif // _LIBCPP_ENABLE_HARDENED_DEBUG_MODE
260

Do we really want to disable things? If I only want to define _LIBCPP_ABI_BOUNDED_ITERATORS is that prohibited?

ldionne requested changes to this revision.Jun 27 2023, 2:54 PM
ldionne added a subscriber: ldionne.
ldionne added inline comments.
libcxx/include/__config
196

We don't have a good story for splitting up the whole __config header, however we could pretty easily add all the hardening configuration macros to something like __hardening or __config_dir/hardening.h. IMO it would make sure that this configuration is nicely contained in a single place.

You'll want to make sure to include __config in that header since that includes __config_site, and that's where you get some of the defaults from.

221

IMO _LIBCPP_ENABLE_HARDENED_DEBUG_MODE is fine as a temporary name until we remove the legacy debug mode, but it's pretty confusing so we want to rename it to _LIBCPP_ENABLE_DEBUG_MODE when we can.

233

Considering there will be only a small number of categories (at least at first), IMO it will end up being easier to read by listing all of them explicitly in each mode. We can also cross that bridge once we get to it in an upcoming patch.

250–253

I think the separation between hardening/debug mode selection and ABI flags should be kept. Otherwise, we lose the property of having a simple and consistent usage model for the mode selection. For example, we can't claim anymore that you can change modes between translation units (since that would affect ABI).

Similarly, I think it would be sensible (although a bit weird) for someone to enable the debug mode without wanting to break the ABI. For example you'd get the unspecified behavior randomization but you wouldn't get bounded iterators. That sounds like something we shouldn't preclude from the get go.

So essentially I'd just remove these lines and we can figure out how to enable the bounded iterator ABI later (soon).

260

If I only want to define _LIBCPP_ABI_BOUNDED_ITERATORS is that prohibited?

No, that's allowed. ABI flags are disjoint from the hardening checks, so you could enable bounded iterators but if you don't enable any hardening, the in-bounds assertions inside __bounded_iter would be disabled. The intent is that vendors would decide on those ABI flags at configuration time, since users can't change them on a per-TU basis.

libcxx/include/__config_site.in
30

We'll want to add a release note explaining that we have these two new options and what they do.

38

This needs to be tied into CMake and into the CI as well. For now I'll suggest we add one configuration for each of them, however that will be offset by the fact that we're removing the legacy debug mode, and we can also remove the With Assertions Enabled CI configuration in a separate patch since it'll be covered by those two.

Tying into CMake

You can look for this in libcxx/CMakeLists.txt

if (LIBCXX_ENABLE_ASSERTIONS)
  config_define(1 _LIBCPP_ENABLE_ASSERTIONS_DEFAULT)
else()
  config_define(0 _LIBCPP_ENABLE_ASSERTIONS_DEFAULT)
endif()
Tying into the CI

You'll want to add new CMake caches similar to libcxx/cmake/caches/Generic-assertions.cmake and then modify run-buildbot and also buildkite-pipeline.yml to add new jobs.

This revision now requires changes to proceed.Jun 27 2023, 2:54 PM
ldionne added inline comments.Jun 27 2023, 2:57 PM
libcxx/include/__config
188

We should also add a .rst document that explains what this mode is. This can be similar to DebugMode.rst but it should explain the various hardening modes instead (there will be TODOs at first).

var-const marked 5 inline comments as done.

Partially address feedback.

var-const added inline comments.Jun 29 2023, 10:03 AM
libcxx/include/__config
233

I'm open to doing that in the future if we have enough categories that duplication becomes a concern. For now, I'd prefer to have each mode separate (I find it convenient to be able to focus on one mode in isolation).

260

@Mordante I'm doing this only to avoid compiler warnings -- it's to make sure every macro is defined and set to either 0 or 1. Otherwise, the compiler would issue a warning each time one of these macros is referred to. I can't set them all to 0 beforehand since that would produce a warning about redefining macros.

var-const added inline comments.Jun 29 2023, 10:45 AM
libcxx/include/__config
196

I think the original idea for the split was that __config will include everything from __config_dir, so that other files can simply include __config and get everything. If that's the case, __config_dir/hardening.h cannot include __config. It could include __config_site directly, but I'm concerned it will need other includes in the future (for other macros it might rely upon), necessitating more granularization, and I'm not sure we want to go that way right now.

OTOH, it's straightforward to include __config from __config_dir/hardening.h, but then all the other files that use assertions would need to include __config_dir/hardening.h (or I guess we could include it in __assert and then those files would just include __assert?). What do you think?

Fix the CI.

var-const marked an inline comment as done.

Address more feedback.

var-const marked an inline comment as done.

Address feedback.

var-const added inline comments.Jun 29 2023, 11:58 AM
libcxx/include/__config
188

Took a stab. Should the doc be under docs/, or docs/DesignDocs/? (DebugMode.rst is under the latter)

215

It comes from __config_site.in.

libcxx/include/__config_site.in
30

Added a very short note. I'm not sure we want to expand it in this patch given that these variables don't currently do anything. What do you think?

38

Thanks for explaining the steps!

ldionne requested changes to this revision.Jun 29 2023, 12:56 PM

This is starting to look really good!

libcxx/CMakeLists.txt
65–76

Do we want to make this a multi-valued option instead? Like LIBCXX_HARDENING_MODE="unsafe|hardened|debug" and then tie that into the __config_site using the same macros you added in this patch? This would make it easier to add a new mode and also it avoids the issue of mutual exclusion of CMake options. It also mirrors a bit more closely what we're trying to do with e.g. the ABI library choice.

libcxx/docs/DesignDocs/HardenedMode.rst
23–25 ↗(On Diff #535946)

Can you mention how users can enable it on a per-TU basis? Basically, the setting that vendors select is only used as the *default* value for user code (and also used for building the dylib itself). We also need to make sure that we're clearly identifying what is a CMake option and what is a compiler define, otherwise casual users pointed to this documentation will be extremely confused. I generally use some wording like "the LIBCXX_ENABLE_HARDENED_MODE option at CMake configuration-time", or something like that. And when I talk about a macro, I generally say "the _LIBCPP_ENABLE_FOO macro". That makes things less ambiguous.

27–31 ↗(On Diff #535946)

You're talking about LIBCXX_ENABLE_ASSERTIONS in specific terms here, and I think that adds a lot of complexity if you want to be precise. Instead, I would say:

The hardened and debug modes require assertions <LINK TO THE DOCUMENTATION FOR ASSERTIONS> to work. If assertions were explicitly disabled, none of the checks of the hardened/debug mode will trigger assertions.

(also note that I wouldn't document that we enable assertions automatically cause we might not want to maintain that, we only do that to be nice to early adopters)

With a wording like this, you don't have to be precise about LIBCXX_ENABLE_ASSERTIONS vs _LIBCPP_ENABLE_ASSERTIONS=1.

33 ↗(On Diff #535946)

We want to expand a bit more on the relationship between ABI settings and these modes. Basically, you can say that the "mode" is separate from the ABI, i.e. it never modifies the ABI. However, libc++ provides alternate ABIs under which checks from the hardened/debug mode become a lot more potent (for example bounded iterators).

libcxx/docs/ReleaseNotes.rst
74–76
libcxx/include/__config
188

I would move to docs/. I'm not really fond of DesignDocs/ anymore, since we do most of our design work in Discourse / Discord nowadays.

196

Yeah, I didn't really think about that. I think it's fine to leave as-is for this patch. I might take a stab at moving it to another header at some point, but for now this seems fine.

libcxx/utils/ci/buildkite-pipeline.yml
523–540

I think this is a leftover

libcxx/utils/libcxx/test/params.py
298

Using a multi-valued option here would also be nicer!

306

You'll need to make sure that we pass --param enable_hardened_mode=True when LIBCXX_ENABLE_HARDENED_MODE=ON. We do this for assertions in libcxx/test/CMakeLists.txt for example:

if (LIBCXX_ENABLE_ASSERTIONS)
  serialize_lit_param(enable_assertions True)
endif()
This revision now requires changes to proceed.Jun 29 2023, 12:56 PM
var-const updated this revision to Diff 536021.Jun 29 2023, 3:05 PM
var-const marked 6 inline comments as done.

Partially address feedback

var-const updated this revision to Diff 536459.Jun 30 2023, 3:32 PM
var-const marked 4 inline comments as done.

Address feedback

var-const updated this revision to Diff 536461.Jun 30 2023, 3:34 PM

Syntax fix.

var-const updated this revision to Diff 536465.Jun 30 2023, 3:41 PM

Fix tests.

var-const updated this revision to Diff 536482.Jun 30 2023, 5:03 PM

Slightly better formatting in an error message.

In general quite happy with the patch, but some minor issues.

libcxx/docs/HardenedMode.rst
3

Is this name future-proof with all the other pre-condition violation checks you want to do?

11–12
16–17

I feel "by *security-conscious projects" is opinionated.
Even if I would not care about security since my device has no connection with the outside world I still may want to avoid undefined behavior.

I think the text is better without the last part. It already mentions "security" and "undefined behavior" before giving the reader enough information to whether or not they like this feature.

26

Maybe list the CMake options with a short description of what they do.

libcxx/docs/ReleaseNotes.rst
75

crash has a negative meaning.

libcxx/docs/index.rst
42

Please add HardenedMode here.

187

This file is not in design docs, so it does not belong here.

ldionne added inline comments.Jul 5 2023, 12:46 PM
libcxx/CMakeLists.txt
67–70
808–809

This check is duplicated with above, I don't think it is needed.

libcxx/docs/ReleaseNotes.rst
90–93
libcxx/include/__algorithm/comp_ref_type.h
68–69

The TODO comment isn't relevant anymore.

libcxx/include/__algorithm/three_way_comp_ref_type.h
61

The TODO comment isn't relevant anymore.

74

There's a similar TODO comment in __hash_table I think, look for _LIBCPP_DEBUG_ASSERT after rebasing onto main.

libcxx/test/support/container_debug_tests.h
17

Let's rename libcpp-has-debug-mode to libcpp-has-legacy-debug-mode (preferably in a separate patch before this one). That'll clear up a lot of the confusion.

libcxx/utils/ci/buildkite-pipeline.yml
496

Let's add ENABLE_CLANG_TIDY: "On" here and below.

libcxx/utils/libcxx/test/params.py
290

We need tests for these macros along these lines: https://godbolt.org/z/sEKa1W3sn

var-const updated this revision to Diff 537782.Jul 6 2023, 10:30 AM
var-const marked 15 inline comments as done.

Address most feedback and rebase.

var-const updated this revision to Diff 537798.Jul 6 2023, 10:58 AM

Address feedback.

libcxx/docs/HardenedMode.rst
3

Pretty much. Hardening is the name of the whole effort, and I expect that the name of the main mode will always be "hardened".

11–12

Thanks!

16–17

This is very useful feedback, thanks!

libcxx/docs/index.rst
187

Thanks!

libcxx/test/support/container_debug_tests.h
17

Pushed e61764c32ff77ae85509e56234a13a47b2d36cec.

libcxx/utils/libcxx/test/params.py
290

Added some tests -- do we want any additional test cases?

ldionne added inline comments.Jul 6 2023, 11:26 AM
libcxx/docs/ReleaseNotes.rst
90–93
libcxx/test/libcxx/assertions/modes/debug.pass.cpp
15–18 ↗(On Diff #537798)
libcxx/test/libcxx/assertions/modes/debug_no_assertions.pass.cpp
9–10 ↗(On Diff #537798)
libcxx/test/libcxx/assertions/modes/hardened_and_debug_mutually_exclusive.verify.cpp
13–17 ↗(On Diff #537798)

No need to define a main function! It makes it more obvious that we're not running this test.

libcxx/test/libcxx/assertions/modes/hardened_no_assertions.pass.cpp
15–16 ↗(On Diff #537798)
libcxx/test/libcxx/assertions/modes/unchecked.pass.cpp
15 ↗(On Diff #537798)

I would guard this with #if !_LIBCPP_ENABLE_ASSERTIONS because the test will fail otherwise. Also add a TODO comment to remove this once we "remove" _LIBCPP_ENABLE_ASSERTIONS (in reality we'll keep it just for backwards compat).

15–16 ↗(On Diff #537798)
18 ↗(On Diff #537798)

Leftover!

var-const updated this revision to Diff 537815.Jul 6 2023, 11:27 AM

Forgotten tests.

ldionne added inline comments.Jul 6 2023, 11:33 AM
libcxx/test/libcxx/assertions/modes/debug.pass.cpp
1 ↗(On Diff #537815)

I think we should add a test like this:

// This test ensures that we can disable the debug mode on a per-TU basis regardless of how the library was built.

// ADDITIONAL_COMPILE_FLAGS: -D_LIBCPP_ENABLE_DEBUG_MODE=0

// now check that _LIBCPP_ASSERT_UNCATEGORIZED doesn't fire.

We should do the same for the hardened mode too.

3 ↗(On Diff #537815)

We also need one like this:

// This test ensures that we can enable the debug mode on a per-TU basis regardless of how the library was built.

// ADDITIONAL_COMPILE_FLAGS: -D_LIBCPP_ENABLE_DEBUG_MODE=1

// now check that _LIBCPP_ASSERT_UNCATEGORIZED fires.

We should do the same for the hardened mode too.

8–9 ↗(On Diff #537815)

Similar comments should also be added to the hardened mode tests.

libcxx/test/libcxx/assertions/modes/debug_mode_not_1_or_0.verify.cpp
13–17 ↗(On Diff #537815)
var-const updated this revision to Diff 538220.Jul 7 2023, 11:49 AM
var-const marked 12 inline comments as done.

Address feedback.

ldionne accepted this revision.Jul 7 2023, 12:36 PM
This revision is now accepted and ready to land.Jul 7 2023, 12:36 PM
var-const updated this revision to Diff 538296.Jul 7 2023, 4:59 PM

Fix the CI and rebase.

var-const updated this revision to Diff 538310.Jul 7 2023, 8:05 PM

Partially fix CI.

Mordante accepted this revision.Jul 8 2023, 4:04 AM

LGTM, thanks for working on this!

libcxx/test/libcxx/algorithms/alg.sorting/assert.sort.invalid_comparator.pass.cpp
36 ↗(On Diff #538310)

Shouldn't this be XFAIL since we always expect the test to fail.

var-const updated this revision to Diff 538834.Jul 10 2023, 2:43 PM
var-const marked an inline comment as done.

Fix the CI.

The CI is red because a couple of back-deployment configurations timed out. I don't think it can be related to this patch, so I'll go ahead and merge this.

This revision was landed with ongoing or failed builds.Jul 12 2023, 10:13 AM
This revision was automatically updated to reflect the committed changes.