This is an archive of the discontinued LLVM Phabricator instance.

[N4917] Add freestanding C++ support for libc++ for support C++23 freestanding
AbandonedPublic

Authored by trcrsired on Oct 13 2022, 12:56 PM.

Details

Reviewers
philnik
ldionne
trcrsired
Group Reviewers
Restricted Project
Summary

Current GNU libstdc++ supports --disable-hosted-libstdcxx to support freestanding and allow hosted toolchain to use -ffreestanding flag to use freestanding C++. However LLVM libc++ does not provide these features. Leaving C++ extremely difficult for writing operating systems or using libcxx to bootstrap libc (MLIBC for example)

This patch adds a new toggle -DLIBCXX_FREESTANDING to cmake that would allow using libcxx with freestanding. Users can also use native toolchain with libcxx to deploy freestanding one.

Besides what N4917, we also provide extra headers like what libstdc++ does.

//===----------------------------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//

// Test that _LIBCPP_FREESTANDING is not defined when -ffreestanding is not passed
// to the compiler but defined when -ffreestanding is passed to the compiler.

// RUN: %{cxx} %{flags} %{compile_flags} -fsyntax-only %s
// RUN: %{cxx} %{flags} %{compile_flags} -fsyntax-only -ffreestanding %s

#include <__config>

#if defined(__has_feature)
#  if __has_feature(modules)
#    define _LIBCPP_FREESTANDING_NO_TEST_MODULES
#  endif
#elif defined(__cpp_modules)
#  define _LIBCPP_FREESTANDING_NO_TEST_MODULES
#endif

#if defined(_LIBCPP_FREESTANDING) && !defined(_LIBCPP_FREESTANDING_NO_TEST_MODULES)
#  include <cstddef>
#  include <limits>
#  include <cfloat>
#  include <version>
#  include <cstdint>
#  include <cstdlib>
#  include <new>
#  include <typeinfo>
#  if __has_include(<source_location>)
#    include <source_location>
#  endif
#  include <exception>
#  include <initializer_list>
#  include <compare>
#  include <coroutine>
#  include <cstdarg>
#  include <concepts>
#  include <type_traits>
#  include <bit>
#  include <atomic>
#  include <utility>
#  include <tuple>
#  include <memory>
#  include <functional>
#  include <ratio>
#  include <iterator>
#  include <ranges>
#  include <typeinfo>

/*
We tested these headers are for preventing build issues
*/
#  include <iosfwd>
#  include <cstdio>
#  include <cerrno>
#  include <cstring>
#  include <cmath>
#  ifndef _LIBCPP_HAS_NO_WIDE_CHARACTERS
#    include <cwchar>
#  endif
/*
 * libc++ libstdc++ like extensions
 */
#  include <array>
#  include <span>
#  include <algorithm>
#  include <bitset>
#  include <variant>
#  include <optional>
#  include <numbers>
#  include <scoped_allocator>
#  include <typeindex>
#  include <string_view>
#  include <numeric>
#  include <complex>
#  include <chrono>
#  include <charconv>
#  include <expected>
#  include <random>
#  include <any>

/*
 * libc++ Containers Extensions
 */
#  include <vector>
#  include <deque>
#  include <string>
#  include <stack>
#  include <queue>
#  include <list>
#  include <forward_list>
#  include <map>
#  include <set>
#  include <unordered_set>
#  include <unordered_map>
#  if __has_include(<mdspan>)
#    include <mdspan>
#  endif
#  include <valarray>

#  if _LIBCPP_STD_VER < 20
#    include <ciso646>
#    if __has_include(<cstdalign>)
#      include <cstdalign>
#    endif
#    include <cstdbool>
#  endif
#endif

source_location header is missing in libcxx currently and I know it would add in the future by other commits, I won't put them here. We provide all freestanding headers libstdc++ provides for freestanding. We provide one extra header which is string_view. However their functionalities may work very differently in the hosted one.

Diff Detail

Event Timeline

trcrsired created this revision.Oct 13 2022, 12:56 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 13 2022, 12:56 PM
Herald added a subscriber: mstorsjo. · View Herald Transcript
trcrsired requested review of this revision.Oct 13 2022, 12:56 PM
philnik requested changes to this revision.Oct 13 2022, 1:06 PM

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

This revision now requires changes to proceed.Oct 13 2022, 1:06 PM
trcrsired updated this revision to Diff 467581.Oct 13 2022, 1:09 PM
ldionne requested changes to this revision.Oct 13 2022, 1:12 PM
ldionne added a subscriber: ldionne.

Thanks for your interest in making libc++ work better on freestanding environments.

The path libc++ has taken so far is to carve parts of libc++ out based on CMake configuration time toggles, like LIBCXX_ENABLE_FILESYSTEM or LIBCXX_ENABLE_LOCALIZATION. Those are general in nature and can be cherry-picked to craft a specific configuration of libc++ that suits the exact need of a platform. In a way, this is more flexible than just a single "freestanding" toggle, and we've used it with great success in various scenarios. Is there a reason why that doesn't suit your needs?

Before pouring too much time into this, I would suggest agreeing on a design first. In particular, I think it would be interesting to have a single "Freestanding" toggle that implements what the C++ Standard describes as "Freestanding", however that should ideally be a composition of more general options (like the ones we have today, e.g. LIBCXX_ENABLE_LOCALIZATION). Also, we need to consider what changes are required in the test suite to support this.

This revision now requires changes to proceed.Oct 13 2022, 1:12 PM
trcrsired added a comment.EditedOct 13 2022, 1:13 PM

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.
  5. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

I will ask libstdc++ dev Jonathan Wakely for helping reviewing this patch. Thank you

See: N4917

https://github.com/cplusplus/draft/releases/download/n4917/n4917.pdf

It requires a lot of more features to support for freestanding now. I am fixing this part to support C++23 freestanding.

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

Libc++ is a hosted implementation, and so doesn't have to meet the requirements of a freestanding implementation. There is no ISO C++ compliance problem here.

If libc++ doesn't advertise itself as a freestanding implementation, then the requirements on freestanding implementations are irrelevant to it.

I will ask libstdc++ dev Jonathan Wakely for helping reviewing this patch. Thank you

I'm not interested in doing that.

ldionne added a comment.EditedOct 13 2022, 1:22 PM

To be clear -- I agree there is value in pursuing this as a matter of ISO C++ conformance (so we can claim that we support a Freestanding implementation). However, concretely, it is generally more useful to be able to disable only the parts that a platform doesn't support (or doesn't want to support) instead of picking the "Freestanding" subset as a whole, which may be excluding things that a platform could (and may want to) support.

I still think we should support Freestanding as a matter of ISO C++ conformance, however we should approach solving that problem from the stance that we've taken so far, i.e. by defining general-purpose carve-outs in the library that can be combined to create a specific version of the library that suits a platform's needs.

trcrsired added a comment.EditedOct 13 2022, 1:31 PM

To be clear -- I agree there is value in pursuing this as a matter of ISO C++ conformance (so we can claim that we support a Freestanding implementation). However, concretely, it is generally more useful to be able to disable only the parts that a platform doesn't support (or doesn't want to support) instead of picking the "Freestanding" subset as a whole, which may be excluding things that a platform could (and may want to) support.

I still think we should support Freestanding as a matter of ISO C++ conformance, however we should approach solving that problem from the stance that we've taken so far, i.e. by defining general-purpose carve-outs in the library that can be combined to create a specific version of the library that suits a platform's needs.

Then what are you going to do for supporting things like writing libc with C++? Or someone tries to use C++ to write their own operating system kernels?

Libc like mlibc they have to recreate their own non standard compliant C++ headers to write libc which adds tons of portability trap. The C++ freestanding headers usually tie specifically to the features of specific compilers. It just adds headache for everyone.

There are no headers at all. Not even cstddef exists. clang and GCC they by default provide freestanding C headers but no freestanding C++ headers.

C++ freestanding has been a long term issue (over 20 years) due to lack of proper support from toolchain. Let's just not make the situation even more deteriorating.

trcrsired updated this revision to Diff 467594.Oct 13 2022, 1:47 PM

Support missed numeric header

trcrsired edited the summary of this revision. (Show Details)Oct 13 2022, 1:48 PM
trcrsired updated this revision to Diff 467600.Oct 13 2022, 2:09 PM
trcrsired edited the summary of this revision. (Show Details)

Add test to freestanding.

Ensure -ffreestanding would work for all headers for hosted toolchain.

Add headers to tests in freestanding.sh

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

I don't think it makes sense to pursue building libc++ without a freestanding C library. We'd effectively have to implement it anyways, so we might as well require a freestanding libc. What exactly does libc++ not conform to as a freestanding library? IOW what can't be disabled that requires a hosted environment?

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.

These seem to be generally good ideas, but doesn't answer my question: What problem do you run into when you try to do any of this?

  1. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

I don't see why we should mimic what libstdc++ does.

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

Assuming you mean _LIBCPP_FREESTANDING, it indeed looks like it doesn't do anything and should be removed. What part of the freestanding library relies on hosted features?

trcrsired added a comment.EditedOct 13 2022, 2:41 PM

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

I don't think it makes sense to pursue building libc++ without a freestanding C library. We'd effectively have to implement it anyways, so we might as well require a freestanding libc. What exactly does libc++ not conform to as a freestanding library? IOW what can't be disabled that requires a hosted environment?

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.

These seem to be generally good ideas, but doesn't answer my question: What problem do you run into when you try to do any of this?

  1. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

I don't see why we should mimic what libstdc++ does.

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

Assuming you mean _LIBCPP_FREESTANDING, it indeed looks like it doesn't do anything and should be removed. What part of the freestanding library relies on hosted features?

I am sure libcxx does not work with freestanding C headers as base either.
https://en.cppreference.com/w/c/language/conformance

Before C23 (which is not what C++23 accepts),
<float.h> Limits of floating-point types
<iso646.h> (C95) Alternative operator spellings
<limits.h> Ranges of integer types
<stdalign.h> (C11) alignas and alignof convenience macros
<stdarg.h> Variable arguments
<stdbool.h> (C99) Macros for boolean type
<stddef.h> Common macro definitions
<stdint.h> (C99) Fixed-width integer types
<stdnoreturn.h> (C11) noreturn convenience macro

Partially freestanding standard library headers
<string.h> (C23) String handling
Conditionally fully freestanding standard library headers
<fenv.h> (C23) Floating-point environment
<math.h> (C23) Common mathematics functions
Conditionally partially freestanding standard library headers
<stdlib.h> (C23) General utilities: memory management, program utilities, string conversions, random numbers, algorithms

What libcxx requires is something like a full POSIX C library which just assumes you have an operating system. Which does not makes sense.

Things like iostream, threading etc are clearly not something freestanding supports.

Problems? The code just does not compile or linking correctly. Simple.
You do not have any headers to use, no type_traits, no cstddef. Nothing my friend.

libcxx has a long history of assuming how things would work. Often, they are not working. For example, libcxx does not work for x86_64-w64-mingw32 with win32 or mcf thread model. It does not work for MSDOS either. All the features just assume it is a POSIX like platform.

You may talk about newlib, however newlib is not working all environments either (UEFI for example). And they have licensing issues. Still, it makes assumptions on how the toolchain provides headers and they are usually not portable.

If you are not working in freestanding environment as people like I do, then better just accept my commit so people like I who work in that environment would help maintain this part. Everyone will be happy.

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

I don't think it makes sense to pursue building libc++ without a freestanding C library. We'd effectively have to implement it anyways, so we might as well require a freestanding libc. What exactly does libc++ not conform to as a freestanding library? IOW what can't be disabled that requires a hosted environment?

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.

These seem to be generally good ideas, but doesn't answer my question: What problem do you run into when you try to do any of this?

  1. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

I don't see why we should mimic what libstdc++ does.

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

Assuming you mean _LIBCPP_FREESTANDING, it indeed looks like it doesn't do anything and should be removed. What part of the freestanding library relies on hosted features?

I am sure libcxx does not work with freestanding C headers as base either.
https://en.cppreference.com/w/c/language/conformance

Before C23 (which is not what C++23 accepts),
<float.h> Limits of floating-point types
<iso646.h> (C95) Alternative operator spellings
<limits.h> Ranges of integer types
<stdalign.h> (C11) alignas and alignof convenience macros
<stdarg.h> Variable arguments
<stdbool.h> (C99) Macros for boolean type
<stddef.h> Common macro definitions
<stdint.h> (C99) Fixed-width integer types
<stdnoreturn.h> (C11) noreturn convenience macro

Partially freestanding standard library headers
<string.h> (C23) String handling
Conditionally fully freestanding standard library headers
<fenv.h> (C23) Floating-point environment
<math.h> (C23) Common mathematics functions
Conditionally partially freestanding standard library headers
<stdlib.h> (C23) General utilities: memory management, program utilities, string conversions, random numbers, algorithms

What libcxx requires is something like a full POSIX C library which just assumes you have an operating system. Which does not makes sense.

Things like iostream, threading etc are clearly not something freestanding supports.

As @ldionne already mentioned, we have flags to disable parts of libc++. Specifically, you can disable threads, monotonic clock, filesystem, random device, localization and wide characters.

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

I don't think it makes sense to pursue building libc++ without a freestanding C library. We'd effectively have to implement it anyways, so we might as well require a freestanding libc. What exactly does libc++ not conform to as a freestanding library? IOW what can't be disabled that requires a hosted environment?

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.

These seem to be generally good ideas, but doesn't answer my question: What problem do you run into when you try to do any of this?

  1. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

I don't see why we should mimic what libstdc++ does.

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

Assuming you mean _LIBCPP_FREESTANDING, it indeed looks like it doesn't do anything and should be removed. What part of the freestanding library relies on hosted features?

I am sure libcxx does not work with freestanding C headers as base either.
https://en.cppreference.com/w/c/language/conformance

Before C23 (which is not what C++23 accepts),
<float.h> Limits of floating-point types
<iso646.h> (C95) Alternative operator spellings
<limits.h> Ranges of integer types
<stdalign.h> (C11) alignas and alignof convenience macros
<stdarg.h> Variable arguments
<stdbool.h> (C99) Macros for boolean type
<stddef.h> Common macro definitions
<stdint.h> (C99) Fixed-width integer types
<stdnoreturn.h> (C11) noreturn convenience macro

Partially freestanding standard library headers
<string.h> (C23) String handling
Conditionally fully freestanding standard library headers
<fenv.h> (C23) Floating-point environment
<math.h> (C23) Common mathematics functions
Conditionally partially freestanding standard library headers
<stdlib.h> (C23) General utilities: memory management, program utilities, string conversions, random numbers, algorithms

What libcxx requires is something like a full POSIX C library which just assumes you have an operating system. Which does not makes sense.

Things like iostream, threading etc are clearly not something freestanding supports.

As @ldionne already mentioned, we have flags to disable parts of libc++. Specifically, you can disable threads, monotonic clock, filesystem, random device, localization and wide characters.

The problem is that they are not enough. A lot of more you have to disable. Like stdio.h, string.h, stdlib.h, time.h ETC. None of them is freestanding.

trcrsired updated this revision to Diff 467623.Oct 13 2022, 3:44 PM

Fix all potential formating issues and build failure.

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

I don't think it makes sense to pursue building libc++ without a freestanding C library. We'd effectively have to implement it anyways, so we might as well require a freestanding libc. What exactly does libc++ not conform to as a freestanding library? IOW what can't be disabled that requires a hosted environment?

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.

These seem to be generally good ideas, but doesn't answer my question: What problem do you run into when you try to do any of this?

  1. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

I don't see why we should mimic what libstdc++ does.

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

Assuming you mean _LIBCPP_FREESTANDING, it indeed looks like it doesn't do anything and should be removed. What part of the freestanding library relies on hosted features?

I am sure libcxx does not work with freestanding C headers as base either.
https://en.cppreference.com/w/c/language/conformance

Before C23 (which is not what C++23 accepts),
<float.h> Limits of floating-point types
<iso646.h> (C95) Alternative operator spellings
<limits.h> Ranges of integer types
<stdalign.h> (C11) alignas and alignof convenience macros
<stdarg.h> Variable arguments
<stdbool.h> (C99) Macros for boolean type
<stddef.h> Common macro definitions
<stdint.h> (C99) Fixed-width integer types
<stdnoreturn.h> (C11) noreturn convenience macro

Partially freestanding standard library headers
<string.h> (C23) String handling
Conditionally fully freestanding standard library headers
<fenv.h> (C23) Floating-point environment
<math.h> (C23) Common mathematics functions
Conditionally partially freestanding standard library headers
<stdlib.h> (C23) General utilities: memory management, program utilities, string conversions, random numbers, algorithms

What libcxx requires is something like a full POSIX C library which just assumes you have an operating system. Which does not makes sense.

Things like iostream, threading etc are clearly not something freestanding supports.

As @ldionne already mentioned, we have flags to disable parts of libc++. Specifically, you can disable threads, monotonic clock, filesystem, random device, localization and wide characters.

The problem is that they are not enough. A lot of more you have to disable. Like stdio.h, string.h, stdlib.h, time.h ETC. None of them is freestanding.

I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

trcrsired updated this revision to Diff 467627.Oct 13 2022, 3:59 PM

This patch merge last twos and it should work now

trcrsired added a comment.EditedOct 13 2022, 4:01 PM

I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

This is exactly what I am talking about that libcxx always makes wrong assumptions. Even those headers exist, you still cannot guarantee they are working headers. There is no reason to make any assumption on the implementations.

Just avoid including those headers in first place. You do not need them tbh.

BTW. you shouldn't need to implement those features either. GCC and clang always assumes memcpy, memmove, memset exist. However any other headers, unless WG14 and WG21 say something, we must avoid them at all costs.

In fact, you cannot even guarantee a correct implementation for memcpy. memcpy is tied to systems too. despite you can easily implement them. However, to implement them efficiently, only users know how to do that.

trcrsired added a comment.EditedOct 13 2022, 4:10 PM

I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

This is exactly what I am talking about that libcxx always makes wrong assumptions. Even those headers exist, you still cannot guarantee they are working headers. There is no reason to make any assumption on the implementations.

Just avoid including those headers in first place. You do not need them tbh.

BTW. you shouldn't need to implement those features either. GCC and clang always assumes memcpy, memmove, memset exist. However any other headers, unless WG14 and WG21 say something, we must avoid them at all costs.

In fact, you cannot even guarantee a correct implementation for memcpy. memcpy is tied to systems too. despite you can easily implement them. However, to implement them efficiently, only users know how to do that.

I recently encounter an example on why detecting headers won't work. pdp11-aout provides its own libc. However that libc was written in 1960s probably, the code just could not compile with a C++ compiler since they use K&R style function declaration. None of those headers can be used. Making any assumptions on how libc works would create all sorts of issues.

That is why a LIBCXX_FREESTANDING toggle would be extremely useful since they avoid any assumptions on how libc is implemented.

! In D135911#3857157, @trcrsired wrote:

! In D135911#3857150, @trcrsired wrote:
I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

This is exactly what I am talking about that libcxx always makes wrong assumptions. Even those headers exist, you still cannot guarantee they are working headers. There is no reason to make any assumption on the implementations.

How is there no reason to assume that the C implementation is conforming? libc++ isn't aiming to be a C library, it aiming to be a conforming C++ library.

Just avoid including those headers in first place. You do not need them tbh.

We do need them to provide a quality implementation.

BTW. you shouldn't need to implement those features either. GCC and clang always assumes memcpy, memmove, memset exist. However any other headers, unless WG14 and WG21 say something, we must avoid them at all costs.

We definitely shouldn't avoid them at all costs. I see no reason it's not acceptable to require LLVM libc in freestanding if the functionality isn't provided by the native libc.

In fact, you cannot even guarantee a correct implementation for memcpy. memcpy is tied to systems too. despite you can easily implement them. However, to implement them efficiently, only users know how to do that.

We cannot guarantee it, but we can require it. That's not an unreasonable requirement on a C library.

I recently encounter an example on why detecting headers won't work. pdp11-aout provides its own libc. However that libc was written in 1960s probably, the code just could not compile with a C++ compiler since they use K&R style function declaration. None of those headers can be used. Making any assumptions on how libc works would create all sorts of issues.

I don't think it's reasonable to build a 2022 libc++ on a C library from the 1960s. That's not a use-case we aim to support.

That is why a LIBCXX_FREESTANDING toggle would be extremely useful since they avoid any assumptions on how libc is implemented.

If you can't assume anything about the C library it's impossible to implement a conforming freestanding C++ library. As said before, it's absolutely reasonable to assume a conforming C library.

trcrsired added a comment.EditedOct 13 2022, 4:43 PM

What exactly is the problem you are trying to solve? libc++ is used in lots of embedded environments, just with parts that aren't possible to implement in that environment disabled.

To allow libcxx to build without libc and to satisfy ISO C++ standard.

This will allow a lot of things:

  1. Writing windows kernel drivers without tweaking headers to get a toolchain.
  2. To allow libcs like mlibc that uses C++ to write libc itself then build a full hosted one. Or you get trouble into chicken egg problem.
  3. Allow writing custom operating systems that usually have no libc support.
  4. To support truly freestanding environment. Embedded does not necessarily mean freestanding. What we mean is something like GNU or OS kernel.
  5. To match the behavior of GNU libstdc++.

See here:
https://github.com/gcc-mirror/gcc/commits/master?after=ab8477af9949a7e6fcaf89c5f1dcf32788accf88+34&branch=master&path%5B%5D=libstdc%2B%2B-v3&qualified_name=refs%2Fheads%2Fmaster

Shortly. I need freestanding toolchains without libc support to make it more flexible.

BTW. libcxx has bugs of freestanding. The LIBCXX_FREESTANDING macro exists but it does not do anything. Many freestanding headers will rely on hosted C++ features which are not standard compliant.

I will ask libstdc++ dev Jonathan Wakely for helping reviewing this patch. Thank you

See: N4917

https://github.com/cplusplus/draft/releases/download/n4917/n4917.pdf

It requires a lot of more features to support for freestanding now. I am fixing this part to support C++23 freestanding.

The point of freestanding C++ is exactly THERE IS NO libc. Then why bother talking about libc? I do not understand know. In fact freestanding C headers are not a subset freestanding C++ headers by WG21. Why bother talking about that?

! In D135911#3857157, @trcrsired wrote:

! In D135911#3857150, @trcrsired wrote:
I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

This is exactly what I am talking about that libcxx always makes wrong assumptions. Even those headers exist, you still cannot guarantee they are working headers. There is no reason to make any assumption on the implementations.

How is there no reason to assume that the C implementation is conforming? libc++ isn't aiming to be a C library, it aiming to be a conforming C++ library.

Just avoid including those headers in first place. You do not need them tbh.

We do need them to provide a quality implementation.

BTW. you shouldn't need to implement those features either. GCC and clang always assumes memcpy, memmove, memset exist. However any other headers, unless WG14 and WG21 say something, we must avoid them at all costs.

We definitely shouldn't avoid them at all costs. I see no reason it's not acceptable to require LLVM libc in freestanding if the functionality isn't provided by the native libc.

In fact, you cannot even guarantee a correct implementation for memcpy. memcpy is tied to systems too. despite you can easily implement them. However, to implement them efficiently, only users know how to do that.

We cannot guarantee it, but we can require it. That's not an unreasonable requirement on a C library.

I recently encounter an example on why detecting headers won't work. pdp11-aout provides its own libc. However that libc was written in 1960s probably, the code just could not compile with a C++ compiler since they use K&R style function declaration. None of those headers can be used. Making any assumptions on how libc works would create all sorts of issues.

I don't think it's reasonable to build a 2022 libc++ on a C library from the 1960s. That's not a use-case we aim to support.

That is why a LIBCXX_FREESTANDING toggle would be extremely useful since they avoid any assumptions on how libc is implemented.

If you can't assume anything about the C library it's impossible to implement a conforming freestanding C++ library. As said before, it's absolutely reasonable to assume a conforming C library.

Quality implementation =/= avoid implementation as a whole. What you are doing is basically preventing people for using freestanding C++ for doing what it supposes to do.

Yes it can be implemented. Or it is a bug in the standard.

trcrsired updated this revision to Diff 467643.Oct 13 2022, 5:27 PM

! In D135911#3857157, @trcrsired wrote:

! In D135911#3857150, @trcrsired wrote:
I think it would make sense to go through the list of C headers and decide on a case-by-case basis whether we want to rely on that header.
For example, it would probably be a good idea to remove any dependencies on stdio.h, but it doesn't make sense to remove dependencies on string.h. It's a very fundamental header and very simple to implement at least a basic version. IMO the C standard should actually require at least a subset of the header in freestanding mode. If there is any environment that doesn't provide the functions we use from that, I think it's reasonable to require the use of LLVM libc (or similar), which provides platform-agnostic implementations for these functions. The implementation there is probably a lot faster than anything we could provide in libc++ without basically duplicating LLVM libc.

This is exactly what I am talking about that libcxx always makes wrong assumptions. Even those headers exist, you still cannot guarantee they are working headers. There is no reason to make any assumption on the implementations.

How is there no reason to assume that the C implementation is conforming? libc++ isn't aiming to be a C library, it aiming to be a conforming C++ library.

Just avoid including those headers in first place. You do not need them tbh.

We do need them to provide a quality implementation.

BTW. you shouldn't need to implement those features either. GCC and clang always assumes memcpy, memmove, memset exist. However any other headers, unless WG14 and WG21 say something, we must avoid them at all costs.

We definitely shouldn't avoid them at all costs. I see no reason it's not acceptable to require LLVM libc in freestanding if the functionality isn't provided by the native libc.

In fact, you cannot even guarantee a correct implementation for memcpy. memcpy is tied to systems too. despite you can easily implement them. However, to implement them efficiently, only users know how to do that.

We cannot guarantee it, but we can require it. That's not an unreasonable requirement on a C library.

I recently encounter an example on why detecting headers won't work. pdp11-aout provides its own libc. However that libc was written in 1960s probably, the code just could not compile with a C++ compiler since they use K&R style function declaration. None of those headers can be used. Making any assumptions on how libc works would create all sorts of issues.

I don't think it's reasonable to build a 2022 libc++ on a C library from the 1960s. That's not a use-case we aim to support.

That is why a LIBCXX_FREESTANDING toggle would be extremely useful since they avoid any assumptions on how libc is implemented.

If you can't assume anything about the C library it's impossible to implement a conforming freestanding C++ library. As said before, it's absolutely reasonable to assume a conforming C library.

Tons of building issues with libcxx with GCC now. Probably I will commit a patch to fix build for libcxx on GCC first.

trcrsired edited the summary of this revision. (Show Details)

A new patch to try to support C++23 freestanding.

charconv is no longer supported for freestanding since std::errc is not freestanding

trcrsired updated this revision to Diff 482039.Dec 12 2022, 2:05 AM

Make feature testing macros reflect freestanding settings.

Trivialize the implementation by implementing part of libc that would not break libcxx.

Add more headers to freestanding.

trcrsired updated this revision to Diff 482041.Dec 12 2022, 2:32 AM

Fix warnings and format

trcrsired updated this revision to Diff 482060.Dec 12 2022, 3:43 AM

Remove _LIBCPP_NO_THREADS error for all headers since it breaks the clang module header system.

trcrsired updated this revision to Diff 482065.Dec 12 2022, 3:50 AM

format issue

trcrsired edited the summary of this revision. (Show Details)Dec 12 2022, 3:51 AM
trcrsired updated this revision to Diff 482071.Dec 12 2022, 4:21 AM

disable freestanding test for modules since clang does not understand the semantics correctly.

trcrsired updated this revision to Diff 482079.Dec 12 2022, 4:53 AM

barrier should not be defined under hosted environment

trcrsired updated this revision to Diff 482083.Dec 12 2022, 5:08 AM

no longer define _LIBCPP_NO_THREADS when _LIBCPP_FREESTANDING is defined with a hosted toolchain

trcrsired updated this revision to Diff 482086.Dec 12 2022, 5:17 AM

libcxx still does not support module with freestanding due to incorrect assumptions on std module. Disable it

trcrsired updated this revision to Diff 482089.Dec 12 2022, 5:26 AM

Fix incorrect tooltip in cmake

trcrsired accepted this revision.Dec 12 2022, 6:43 AM

Done!!
Finally got all tests passed

Can you review the changes here? It should be done. Now libcxx supports freestanding mode. The headers can be installed and deployed without a libc now. This should match libstdc++ now.
-ffreestanding flag will also follow the standard behavior.

@jwakely Does this match the behavior of libstdc++? Do you think I need more changes? libcxx support much more headers than libstdc++ for freestanding.

jwakely resigned from this revision.Dec 12 2022, 8:20 AM

I already said I'm not interested in reviewing this, stop asking me to.

@ldionne @philnik Can you help review this? Many people (like mlibc devs) and I really want to see libcxx work for freestanding. I've added a hosted setting in the generate_feature_test_macro_components.py.

Thank you so much!

trcrsired updated this revision to Diff 483395.Dec 15 2022, 5:07 PM

Update for today's update to libcxx.

Adding more headers to freestanding test.

trcrsired edited the summary of this revision. (Show Details)Dec 15 2022, 5:14 PM
trcrsired edited the summary of this revision. (Show Details)
trcrsired resigned from this revision.Dec 15 2022, 5:26 PM
trcrsired removed a reviewer: jwakely.
trcrsired updated this revision to Diff 483960.Dec 19 2022, 7:17 AM
trcrsired updated this revision to Diff 485241.Dec 25 2022, 1:51 PM

Re add patch for recent patch

trcrsired updated this revision to Diff 485243.Dec 25 2022, 1:59 PM
trcrsired updated this revision to Diff 485244.Dec 25 2022, 2:08 PM

Fix format issue

trcrsired accepted this revision.Dec 26 2022, 6:14 AM

Finished. Please merge.