This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Add check for building with picolibc
ClosedPublic

Authored by domin144 on Jun 30 2023, 11:42 AM.

Details

Summary

This is intended to identify changes that would fail to build on
embedded platforms e.g. D152382

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ldionne added inline comments.Jul 3 2023, 1:29 PM
libcxx/utils/ci/run-buildbot
624

I think we should just use a pragma to disable the warning in that function.

632

We're not running the tests? I would expect

echo "+++ Running the libc++ tests"
${NINJA} -vC "${BUILD_DIR}" check-cxx check-cxxabi check-unwind
675

Should we add something to libcxx/docs/index.rst in the platform support table that mentions that picolibc is supported?

michaelplatings marked 2 inline comments as done.

Apply @ldionne's suggestions

michaelplatings added inline comments.Jul 4 2023, 3:53 AM
libcxx/utils/ci/run-buildbot
624

I've now found that the warning pops up in all the __c11_atomic_... invocations so I've surrounded all the invocations with a pragma disable.

632

Yes I'd also like to get the tests passing or xfailed but that's going to require a fair bit of work. Off the top of my head:

  • Add linker scripts.
  • Figure out how to run the tests via emulation in the Docker containers.
  • Xfail all the tests that rely on features that picolibc doesn't support.

All of these have been done already with various hacks in various downstream repositories, but finding a suitable way to do them properly in the libcxx sources is something I anticipate could take months.

For now I just want to set a baseline of keeping libcxx+picolibc building. Once that's in place we can start ratcheting up the quality bar.

DavidSpickett added inline comments.Jul 5 2023, 5:47 AM
libcxx/utils/ci/build-picolibc.sh
70

clang -> cc so it can work on our container.

DavidSpickett added a comment.EditedJul 5 2023, 9:17 AM

I got this to build on Linaro's container.

clang 15.0.0 (our current compiler) fails with errors about not finding types that should be in locale.h. The files are all in the right place, and moving to clang 16 fixes this. So I assume some driver changes did that.

clang 16.0.4 failed with this on one part of the build:

clang-16: error: argument unused during compilation: '-G R-' [-Werror,-Wunused-command-line-argument]

Turns out that -GR- is the equivalent of /GR- which is the msvc option to disable RTTI (https://learn.microsoft.com/en-us/cpp/build/reference/gr-enable-run-time-type-information?view=msvc-170).

My guess is that some part of clang checked for this -GR- or -fno-rtti and forgot to claim the other one. This was fixed in https://github.com/llvm/llvm-project/commit/cd18efb61d759405956dbd30e4b5f2720d8e1783 but we will have to wait until 17.0.0 to have that, and I don't want to be building a custom clang for Linaro's bots.

So I think the sensible thing to do right now is add -Wno-unused-command-line-argument to the build flags and I will update the container's clang to 16.0.4. Which we would have to do eventually anyway.

I'll let you know once that's done so you can update this diff and see a green CI build before landing.

libcxx/utils/ci/run-buildbot
623

flags="--sysroot=${INSTALL_DIR} -Wno-unused-command-line-argument"

For reasons explained in the top level comment.

clang-16: error: argument unused during compilation: '-G R-' [-Werror,-Wunused-command-line-argument]

Turns out that -GR- is the equivalent of /GR- which is the msvc option to disable RTTI

That's familiar – I ran into that in another context last month. libc++ contains this in its cmake files:

function(cxx_add_rtti_flags target)
  if (NOT LIBCXX_ENABLE_RTTI)
    target_add_compile_flags_if_supported(${target} PUBLIC -GR-)
    target_add_compile_flags_if_supported(${target} PUBLIC -fno-rtti)
  endif()
endfunction()

and that "add_compile_flags_if_supported" is somehow misidentifying whether it's supported. Possibly because -GR- is possible to re-parse in a non-msvc-style command line as the completely different option -G with argument word R-. The fix might be to explicitly query the command-line flavour, via if (CMAKE_C_COMPILER_ID MATCHES "MSVC" OR CMAKE_C_COMPILER_FRONTEND_VARIANT MATCHES "MSVC") or something along those lines, and use that to condition the use of that MSVC option.

Aha, even better, check LIBCXX_TARGETING_MSVC which the same CMakeLists has already defined.

michaelplatings marked 2 inline comments as done.

clang -> $CC

Production containers are updated.

I've done a build on one, with the cc setting fixed, and it works. So in theory, update this one more time and we'll get a green build.

libcxx/utils/ci/build-picolibc.sh
70

Needs to literally be cc. As /usr/local/bin/cc is linked to clang-16.0.4 in our images, we don't set any env vars.

I know it's not great for anyone reproducing, and an env var would probably be better but cross that road when it's needed.

Default to compiling picolibc with cc

michaelplatings marked an inline comment as done.Jul 6 2023, 10:38 AM
michaelplatings added inline comments.
libcxx/utils/ci/build-picolibc.sh
70

Thanks for explaining. Now if CC isn't set then cc will be used.

michaelplatings marked an inline comment as done.Jul 6 2023, 10:39 AM

We got a green build - https://buildkite.com/llvm-project/libcxx-ci/builds/27739#_

All the infrastructure changes are done, so please libcxx folks proceed with the review as usual.

libcxx/utils/ci/build-picolibc.sh
70

Nice, that's much more convenient.

Mordante added inline comments.Jul 10 2023, 8:34 AM
libcxx/docs/index.rst
135

The CI is green. Is this work-in-progress or working?

libcxx/utils/ci/build-picolibc.sh
10

Can you add a short comment what this file does?

michaelplatings marked 2 inline comments as done.

Add summary comment to build-picolibc.sh

libcxx/docs/index.rst
135

At present all this job will do is check that libc++ can build against picolibc. It doesn't yet run any tests so there's not yet any guarantee that the build output actually works. Therefore work-in-progress. Once tests are running and green this note should be removed.

ldionne added inline comments.Jul 10 2023, 1:25 PM
libcxx/utils/ci/run-buildbot
632

For now I just want to set a baseline of keeping libcxx+picolibc building. Once that's in place we can start ratcheting up the quality bar.

Ratcheting the quality bar is usually done by first setting up the test suite and adding a bunch of XFAIL and UNSUPPORTED markup that is then gradually removed. In this case, we're not running the test suite at all, and that's insufficient to claim support for it. I hate to be that person, but this patch needs to include whatever's necessary to build and run the test suite before it is merged.

One thing we could discuss is actually only compiling and linking all the tests, but not running them right away since that might require more complicated setup. I'm not sure we want to set that precedent, but I'd definitely be open to the discussion if we had everything compiling and linking. If we were to do that, we'd replace the %{exec} substitution by something like true and that would cause the tests not to be executed for real.

Mordante added inline comments.Jul 14 2023, 9:56 AM
libcxx/utils/ci/run-buildbot
632

I too would like to see XFAIL and UNSUPPORTED. That gives us more insights in which parts work and which parts don't work. This also helps with future patches to decide how much effort we should spend on getting it to work with picolib. For example getting locales tested properly may take quite a bit of effort; at least that has been my experience. When adding a new locale test it's easier to justify to add a new XFAIL/UNSUPPORTED.

Compile and link tests

Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptJul 14 2023, 10:49 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript
Herald added a reviewer: Restricted Project. · View Herald Transcript
michaelplatings marked 3 inline comments as done.Jul 14 2023, 10:52 AM
michaelplatings added inline comments.
libcxx/utils/ci/run-buildbot
632

OK I've updated it to compile & link all the tests, using true for %{exec} as suggested.

philnik added inline comments.Jul 14 2023, 11:00 AM
libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

I guess this is caused by a missing libatomic?

libcxx/utils/ci/run-buildbot
615

Personally I'm not super concerned about building picolibc every time. I don't expect it to be a long build (probably similar to libc++), and testing always the latest version seems like a nice thing to me.

arichardson added inline comments.Jul 14 2023, 1:40 PM
libcxx/utils/libcxx/test/features.py
280

Why pipe? Is that function actually called inside libc++?

libunwind/test/configs/armv7m-libunwind.cfg.in
20 ↗(On Diff #540496)

There should probably be a comment here such as todo: run using qemu-system-arm?

michaelplatings marked 3 inline comments as done.Jul 14 2023, 2:26 PM
michaelplatings added inline comments.
libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

Yes, there's no libatomic or equivalent. Perhaps something like REQUIRES: atomic would be better here.

libcxx/utils/ci/run-buildbot
615

Yes, the time to build picolibc is seconds - negligible compared to the testing time. The revision of picolibc is pinned, but doing it this way does make it trivial to update to the latest version.

libcxx/utils/libcxx/test/features.py
280

check_assertion.h uses pipe, fork, exec etc. Picolibc declares these in unistd.h but doesn't implement them. Therefore this feature check was previously passing with picolibc, but now fails at link time.

libunwind/test/configs/armv7m-libunwind.cfg.in
20 ↗(On Diff #540496)

Agreed

michaelplatings marked 5 inline comments as done.Jul 14 2023, 3:11 PM
michaelplatings added inline comments.
libcxx/utils/libcxx/test/features.py
280

(Despite the name "has-unix-headers", the check is mostly (always?) used to xfail tests that use check_assertion.h)

I want to have a closer look when the CI is green. But it seems the number of disabled tests is quite small :-)

libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

When there are no plans to add that to picolib soon I think we indeed should add something like //UNSUPPRTED: no-libatomic (maybe needs a better name). The LIBCXX-PICOLIBC-FIXME should be used for things that can be fixed.

michaelplatings marked 2 inline comments as done.

Add "atomics" feature, run git-clang-format, and other minor tweaks

libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

For consistency with the existing non-lockfree-atomics feature I've gone with the name atomics.

philnik accepted this revision as: philnik.Jul 17 2023, 5:12 PM

LGTM. Leaving final approval to @ldionne, since he knows the CI best.

libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

I would just go with // REQUIRES: atomics. That's a lot less confusing that the double negation.

michaelplatings marked an inline comment as done.

XFAIL: !atomics -> REQUIRES: atomics

libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

I went with XFAIL for consistency with non-lockfree-atomics. REQUIRES means the tests won't be run at all, but I'm OK with that.

libcxx/test/std/atomics/atomics.types.generic/integral.pass.cpp
9

The CI results with XFAIL: !atomics suggest that the code to detect atomics support was incorrect. REQUIRES wouldn't have picked that up so I feel XFAIL is the better choice.

Change how atomics feature is detected and change atomic REQUIRES back to negated XFAIL

Use 16MB area of RAM to enable running bigger tests

arichardson added inline comments.Jul 18 2023, 11:13 PM
libcxx/utils/libcxx/test/features.py
172

This is effectively checking for availability of 64-bit atomics (since uintmax_t essentially has to be the same as uint64_t for ABI reasons). Maybe something like has-64-bit-atomics is more accurate? I believe this will fail for 32 bit arm picolibc since that needs a libatomic runtime function. The other larger libatomic requirements should already be handled by non-lockfree-atomics.

michaelplatings marked an inline comment as done.

atomics -> has-64-bit-atomics

Rebase after -Wcast-qual error was fixed

Is this ready for review? I see that the CI is failing. This would need to be rebased and the CI to be green.

Once that's done, please ping me on Discord. We're trying to drain the review queue here on Phabricator as part of our transition to Github PRs.

Is this ready for review?

Yes this is rebased, ready for review and CI is looking good now. The same goes for D155521. I've pinged you on Discord as requested.

ldionne requested changes to this revision.Sep 28 2023, 11:41 AM

This looks pretty clean, a lot cleaner than I would have expected for an embedded c library. I have a few questions and comments but I don't see major blockers at this point.

libcxx/include/__atomic/cxx_atomic_impl.h
318–323 ↗(On Diff #557024)

I am not certain we want to disable this warning here. At the end of the day, it seems like the compiler is warning us about something legitimate, no? We could perhaps silence that warning in the armv7m testing configuration instead?

libcxx/test/configs/armv7m-libc++.cfg.in
1–2 ↗(On Diff #557024)

This comment seems copy-pasted (same for the libc++abi one and probably the libunwind one too).

12 ↗(On Diff #557024)

Why is that include needed?

libcxx/test/libcxx/system_reserved_names.gen.py
106

What is the relationship between picolibc and _NEWLIB_VERSION?

libcxx/utils/ci/run-buildbot
622

It's weird that you're passing this here but also hardcoding it in your Armv7M-picolibc.cmake cache file. IMO it would make more sense *not* to hardcode it in the cache.

libcxx/utils/libcxx/test/features.py
178

Otherwise this test will fail in -ffreestanding mode, where main() isn't special.

280

Yeah it's pretty lame, we need to change that.

This revision now requires changes to proceed.Sep 28 2023, 11:41 AM

Make changes requested by @ldionne and rebase.

michaelplatings marked 5 inline comments as done.Sep 29 2023, 4:22 AM
michaelplatings added inline comments.
libcxx/test/configs/armv7m-libc++.cfg.in
12 ↗(On Diff #557024)

Added comment explaining it.

libcxx/test/libcxx/system_reserved_names.gen.py
106

Picolibc is derived from Newlib, so defines _NEWLIB_VERSION. I've updated the comment to make it clear that it applies to both libraries.

libcxx/utils/ci/run-buildbot
622

Yes I'd prefer not to put the target in the CMake cache file, but that I think that would cause confusion because it would be inconsistent with all the other CMake cache files.

Would very much like to see this land soon. Two small comments

libcxx/test/configs/armv7m-libc++.cfg.in
17 ↗(On Diff #557476)

Doesn't this mean the library can't be used without adding this include to any build system that is trying to use libc++ with picolibc? I think it would be a lot cleaner if we could just add the required includes to the libc++ source code?

As far as I recall from the last time I tried to use libc++ with picolibc this was not needed but I may have been building a slightly different configuration

38 ↗(On Diff #557476)

Could this be autodetected instead by checking for the presence of the picolibc header? Then it would also work out of the box with the default config file.

michaelplatings marked 2 inline comments as done.Oct 2 2023, 2:37 AM
michaelplatings added inline comments.
libcxx/test/configs/armv7m-libc++.cfg.in
17 ↗(On Diff #557476)

Doesn't this mean the library can't be used without adding this include to any build system that is trying to use libc++ with picolibc?

Most libc++ headers already indirectly include picolibc.h so most of the time this isn't a problem in practice. It's a minority of tests that fail without this include.

I think it would be a lot cleaner if we could just add the required includes to the libc++ source code?

I agree with you that it would be cleaner to remove this line and instead change the libc++ headers. However I can imagine that adding extra includes could cause undesirable knock-on effects. I think it's best to limit the scope of this change to only adding the test setup and not touching libc++ itself.

38 ↗(On Diff #557476)

I don't think we can infer that we're targeting picolibc from the presence of the header. It might also happen to be on an include path in a scenario in which another platform is targeted. I think it's best to be explicit here.

arichardson added inline comments.Oct 30 2023, 10:28 PM
libcxx/test/configs/armv7m-libc++.cfg.in
17 ↗(On Diff #557476)

Do you have a list of tests that fail? If it's just a few maybe the better solution is to add the fixme to REQUIRES?

38 ↗(On Diff #557476)

I meant trying to compile something like the following:

#include <stdio.h>
#ifndef __PICOLIBC__
#error not picolibc
#endif
domin144 commandeered this revision.Nov 2 2023, 1:57 AM
domin144 added a reviewer: michaelplatings.
domin144 updated this revision to Diff 557982.Nov 2 2023, 2:05 AM
This comment was removed by domin144.
domin144 updated this revision to Diff 557983.Nov 2 2023, 2:08 AM

rebase (minor conflict in doc formatting)
move LIBCXX-PICOLIBC-FIXME derivation to features.py (as suggested in comment)

domin144 added inline comments.Nov 2 2023, 2:21 AM
libcxx/test/configs/armv7m-libc++.cfg.in
17 ↗(On Diff #557476)

There are 138 tests that fail without the ' -include picolibc.h'.
I haven't reviewed them all, but most of them fall into two categories:

  • use of __input reserved name in stdlib.h (e.g. algorithm.compile.pass.cpp)
  • ld.lld: error: undefined symbol: std::__1::basic_streambuf<char, std::__1::char_traits<char>>::seekoff(long long, std::__1::ios_base::seekdir, unsigned int) (e.g. eof.pass.cpp)

I would prefer to solve it in a separate patch as this might need modification to libc++ headers.

38 ↗(On Diff #557476)

Thanks! Applied in the new diff.

One suggestion otherwise looking good to me

libcxx/test/configs/armv7m-libc++.cfg.in
17 ↗(On Diff #557476)

If it's so many tests then yes that sounds good to me.

libcxx/test/libcxx/system_reserved_names.gen.py
106

So it seems this header needs the picolibc.h include? Could we just add something like #include <stdlib.h> to the top of this file?

libcxxabi/test/configs/armv7m-libc++abi.cfg.in
29 ↗(On Diff #557983)

I believe this is not needed anymore since it's in features.oy now?

domin144 updated this revision to Diff 558024.Nov 6 2023, 4:40 AM

Applied suggestions from review:

  • remove redundant LIBCXX-PICOLIBC-FIXME definition
  • add "#include <string.h>" to make sure the _NEWLIB_VERSION is available
domin144 marked 3 inline comments as done.Nov 6 2023, 4:45 AM
domin144 added inline comments.
libcxx/test/libcxx/system_reserved_names.gen.py
106

Done.

libcxxabi/test/configs/armv7m-libc++abi.cfg.in
29 ↗(On Diff #557983)

Good spot, I removed this part in the libc++ version and missed it here. Thanks.

Not quite sure what is going on with the CI here, it sounds like GCC is happy with those tests even though the has-64-bit-atomics feature check fails. Should be good to land once that has been fixed and once @ldionne has approved this review.

libcxx/test/libcxx/selftest/dsl/dsl.sh.py
10

To be pedantic, this is not really a problem with picolibc but rather with using an executor stub. Maybe XFAIL: libcxx-fake-executor and defining that feature in the .cfg?

libcxx/test/libcxx/selftest/pass.cpp/run-error.pass.cpp
9

Same here

libcxx/test/libcxx/selftest/pass.mm/run-error.pass.mm
10

And here

domin144 updated this revision to Diff 558036.Nov 7 2023, 6:52 AM
domin144 marked 2 inline comments as done.

Applied comments and fixed gcc-13 failure:

  • add libcxx-fake-executor feature,
  • move "has-64-bit-atomics" after "gcc" feature
domin144 marked 3 inline comments as done.Nov 7 2023, 6:56 AM

The gcc-13 error in "has-64-bit-atomics" detection is:

/home/domwoj01/sources/llvm-project/builds/libcxx-gcc-13-test/build/include/c++/v1/limits:529:69: error: ‘float_denorm_style’ is deprecated [-Werror=deprecated-declarations]
  529 |     _LIBCPP_CONSTEXPR const float_denorm_style numeric_limits<_Tp>::has_denorm;
      |                                                                     ^~~~~~~~~~

The tests are run with the flag below, while the "has-64-bit-atomics" are not.

AddCompileFlag("-D_LIBCPP_DISABLE_DEPRECATION_WARNINGS"),

I moved the "has-64-bit-atomics" check to the end, so it also gets the flag.

domin144 updated this revision to Diff 558105.Nov 15 2023, 2:55 AM

Rebase to resolve conflict
Update picolibc, so XFAIL for stdio.h is no longer needed.

domin144 updated this revision to Diff 558141.Nov 20 2023, 11:42 PM

resolve conflict

domin144 updated this revision to Diff 558169.Nov 24 2023, 12:33 PM

Rebase, add XFAIL for new tests of floating point atomics as they need support for 64-bit atomic types.

ldionne requested changes to this revision.Nov 27 2023, 8:51 AM
ldionne added inline comments.
libcxx/test/configs/armv7m-libc++.cfg.in
3 ↗(On Diff #558169)

This doesn't have to be a Lit substitution. You could just do this instead:

libc_linker_script = '@CMAKE_INSTALL_PREFIX@/lib/picolibcpp.ld'

# and then below:

' -T {}'.format(libc_linker_script)
17 ↗(On Diff #558169)

Let's name the configuration files armv6m-picolibc-libc++.cfg.in since this is not a generic armv7m configuration, it's tied to picolibc.

libcxx/test/libcxx/system_reserved_names.gen.py
23–24

We shouldn't be including a file here at all, since this breaks the intent of the test (which is to check that every individual header doesn't use poisoned identifiers).

For example if <string.h> has #undef __allocator, we'd now fail to have coverage for e.g. vector using __allocator as a name, since <string.h> would always undef it.

I think we should rely on the -include picolibc.h part for now. It seems better to only affect the picolibc configuration with this "workaround" than affect the correctness of this test for everyone else.

libcxx/test/std/iterators/iterator.primitives/iterator.traits/cxx20_iterator_traits.compile.pass.cpp
17

I'm pretty sure this is a real bug in our test suite. Fine with this being a FIXME for now, but we should really investigate and fix it for everyone.

libcxx/utils/libcxx/test/features.py
291–292

This comment becomes unnecessary once I move the compiler detection features above. Still not great to rely on implicit ordering but 🤷🏼‍♂️

294

Just to record what we're going to do: we're going to leave this feature as-is and then change non-lockfree-atomics to be something like

Feature(
    name="has-128-bits-atomics",
    when=lambda cfg: sourceBuilds(
        cfg,
        """
        #include <atomic>
        struct Large { __int128_t storage; };
        std::atomic<Large> x;
        int main(int, char**) { (void)x.load(); return 0; }
      """,
    ),
),
This revision now requires changes to proceed.Nov 27 2023, 8:51 AM
domin144 updated this revision to Diff 558176.Nov 28 2023, 4:51 AM

Applied changes from comments.

ldionne accepted this revision.Nov 29 2023, 6:55 AM

LGTM, thanks! It's nice to have official support for a more embedded libc.

This revision is now accepted and ready to land.Nov 29 2023, 6:55 AM
This revision was landed with ongoing or failed builds.Nov 29 2023, 7:43 AM
This revision was automatically updated to reflect the committed changes.