Support tests in freestanding

[jfb]

Summary

Freestanding is *weird*. The standard allows it to differ in a bunch of odd
manners from regular C++, and the committee would like to improve that
situation. I'd like to make libc++ behave better with what freestanding should
be, so that it can be a tool we use in improving the standard. To do that we
need to try stuff out, both with "freestanding the language mode" and
"freestanding the library subset".

Let's start with the super basic: run the libc++ tests in freestanding, using
clang as the compiler, and see what works. The easiest hack to do this:

In utils/libcxx/test/config.py add:

self.cxx.compile_flags += ['-ffreestanding']

Run the tests and they all fail.

Why? Because in freestanding main isn't special. This "not special" property
has two effects: main doesn't get mangled, and main isn't allowed to omit its
return statement. The first means main gets mangled and the linker can't
create a valid executable for us to test. The second means we spew out warnings
(ew) and the compiler doesn't insert the return we omitted, and main just
falls of the end and does whatever undefined behavior (if you're luck, ud2
leading to non-zero return code).

Let's start my work with the basics. This patch changes all libc++ tests to mark
main as extern "C" so it mangles properly, and adds return 0; to all places
where it was missing. This touches 6118 files, and I apologize.

The former was done with The Magic Of Sed:

git grep "\bmain\b" ./test/ | grep -v 'extern "C"' | grep "\.cpp:" | cut -d: -f1 | xargs -n1 -I{} -P20 gsed -i 's/int main *(/extern "C" int main(/g' {}

The later was done with a (not quite correct but decent) clang tool:

https://gist.github.com/jfbastien/793819ff360baa845483dde81170feed

This works for most tests, though I did have to adjust a few places when e.g.
the test runs with -x c, macros are used for main (such as for the filesystem
tests), etc.

Once this is in we can create a freestanding bot which will prevent further
regressions. After that, we can start the real work of supporting C++
freestanding fairly well in libc++.

rdar://problem/47754795

[jfb]

Apologies for the lack of diff context, this patch is 34MB with context which is 2MB bigger than Phabricator's limit.

[EricWF]

This feels like a fix at the wrong level.

[jfb]

In D57624#1382293, @EricWF wrote:

This feels like a fix at the wrong level.

Here's a few alternatives which I don't think work:

• We can change LLVM but have no idea what changing isMain's effects will be (this behavior has been there for 10 years).
• We can link in another main when in freestanding mode and call the mangled main, and hope we only ever write int main() so it's always the same mangled name. This doesn't fix falling off the end.
• We can add a header which every test includes (so add the include where missing) which declares main as extern "C" int main(). That again forbids using another main signature, and doesn't fix falling off the end.
• We can write linker scripts for every platform we support to rewrite main to the proper mangling for that platform.
• We can add a flag to LLVM which makes falling off the end in int functions return 0 automatically.

There's a bunch of other ways to fix this. It sounds like you have another idea in mind. At which level are you suggesting the responsibility to create valid freestanding source files lies, and how should that level address the issue?

[bcraig]

For what it's worth, when I ran these tests with MSVC in the Windows kernel, I just called main, and falling off the end of main happened to work. I can't say I'm thrilled with that approach though.

I did attempt some very rough sed script renames of "main" to something else. I think I even tried re-writing the signature to "void test_case_name_main()". I vaguely recall that some tests will do a return some_int though, so my plans were foiled there as well.

For context, I was _also_ attempting to build things in such a way that I could build all the tests into one binary. So maybe a few of those techniques will work fine if you aren't also trying to make one massive executable.

[jfb]

In D57624#1383165, @bcraig wrote:

For what it's worth, when I ran these tests with MSVC in the Windows kernel, I just called main, and falling off the end of main happened to work. I can't say I'm thrilled with that approach though.

It doesn't on the platform I've been testing, and I wouldn't rely on it working forever.

I did attempt some very rough sed script renames of "main" to something else. I think I even tried re-writing the signature to "void test_case_name_main()". I vaguely recall that some tests will do a return some_int though, so my plans were foiled there as well.

For context, I was _also_ attempting to build things in such a way that I could build all the tests into one binary. So maybe a few of those techniques will work fine if you aren't also trying to make one massive executable.

[ldionne]

FWIW, I'm not a big fan of this change (especially since it means we'll have to do this weirdness whenever we add a test), but I don't see other ways of making the tests work in freestanding besides what JF has already suggested. @EricWF it would be nice if you could explain at what level you think the right fix should be applied and we can look into that.

[mclow.lists]

My take on this is

• adding return 0 to the end of all the tests, and changing the signature of main to always be int main(int, char**) is a simple mechanical transformation, and once we decide that we want to do that, doesn't really need review

• I strongly believe that different freestanding systems will have different setup/teardown requirements that we should not try to address in the libc++ test suite, and so @jfb 's suggestion of having a different entry point that calls main(int, char**) is a good way to go.

[ldionne]

In D57624#1383452, @mclow.lists wrote:

My take on this is

• adding return 0 to the end of all the tests, and changing the signature of main to always be int main(int, char**) is a simple mechanical transformation, and once we decide that we want to do that, doesn't really need review

Agreed.

• I strongly believe that different freestanding systems will have different setup/teardown requirements that we should not try to address in the libc++ test suite, and so @jfb 's suggestion of having a different entry point that calls main(int, char**) is a good way to go.

I like this too, however I don't know how this interacts with tests like test/libcxx/language.support/support.dynamic/new_faligned_allocation.sh.cpp that specify the build command-line explicitly:

// RUN: %build -faligned-allocation

We'd make sure that %build links in the additional entry point?

[jfb]

• Don't do extern "C", go with consistent main mangling isntead as suggested by Marshall.

[jfb]

• I strongly believe that different freestanding systems will have different setup/teardown requirements that we should not try to address in the libc++ test suite, and so @jfb 's suggestion of having a different entry point that calls main(int, char**) is a good way to go.

OK I updated the patch with this.

I like this too, however I don't know how this interacts with tests like test/libcxx/language.support/support.dynamic/new_faligned_allocation.sh.cpp that specify the build command-line explicitly:

// RUN: %build -faligned-allocation

We'd make sure that %build links in the additional entry point?

Agreed that seems problematic, but I think we can fix it after this patch lands. :)

[mclow.lists]

@jfb, I am not going to look at all of these changes. If you assure me that they all look like this: https://reviews.llvm.org/differential/changeset/?ref=1324002
and that the test suite still passes on the desktops, I'm fine with you applying this.

[jfb]

In D57624#1383555, @mclow.lists wrote:

@jfb, I am not going to look at all of these changes. If you assure me that they all look like this: https://reviews.llvm.org/differential/changeset/?ref=1324002
and that the test suite still passes on the desktops, I'm fine with you applying this.

Yeah, that's all there should be to it (and some whitespace fixes). I did re-run the tests before uploading the patch. Thanks!

[ldionne]

Like Marshall said, LGTM if all changes look like https://reviews.llvm.org/differential/changeset/?ref=1324002. What's the plan for actually supporting freestanding though? We'll need a separate entry point? Do we know that it's feasible?

[jfb]

In D57624#1383576, @ldionne wrote:

Like Marshall said, LGTM if all changes look like https://reviews.llvm.org/differential/changeset/?ref=1324002. What's the plan for actually supporting freestanding though? We'll need a separate entry point? Do we know that it's feasible?

Yeah we'll need to force-link a TU which defines extern "C" int main(int argc, char **argv) { return _whatevermainmanglesto(argc, argv); }

As you said above it might make some of the tests sad, I'll disentangle that separately.

[bcraig]

I like this too, however I don't know how this interacts with tests like test/libcxx/language.support/support.dynamic/new_faligned_allocation.sh.cpp that specify the build command-line explicitly:

// RUN: %build -faligned-allocation

We'd make sure that %build links in the additional entry point?

Agreed that seems problematic, but I think we can fix it after this patch lands. :)

I didn't run into any of these for the freestanding things I was working on. Admittedly, I skipped the things that I want removed, and that includes any allocating form of new and delete. It was also in a snapshot of libcxx from a bit more than a year ago.

[• Quuxplusone]

Hi @jfb, this commit broke /utils/generate_feature_test_macro_components.py in the libc++ repo by replacing two curly braces with one curly brace:

diff --git a/utils/generate_feature_test_macro_components.py b/utils/generate_feature_test_macro_components.py
index 2bd80a8..d923208 100755
--- a/utils/generate_feature_test_macro_components.py
+++ b/utils/generate_feature_test_macro_components.py
@@ -865,7 +865,7 @@ def produce_tests():
 
 #endif // TEST_STD_VER > 17
 
-int main() {{}}
+int main(int, char**) { return 0; }
 """.format(script_name=script_name,
            header=h,
            test_tags=test_tags,

The correct line here would have been

int main(int, char**) {{ return 0; }}

Do you think you could push that fix? Thanks!

[jfb]

In D57624#1384644, @Quuxplusone wrote:

Hi @jfb, this commit broke /utils/generate_feature_test_macro_components.py in the libc++ repo by replacing two curly braces with one curly brace:

diff --git a/utils/generate_feature_test_macro_components.py b/utils/generate_feature_test_macro_components.py
index 2bd80a8..d923208 100755
--- a/utils/generate_feature_test_macro_components.py
+++ b/utils/generate_feature_test_macro_components.py
@@ -865,7 +865,7 @@ def produce_tests():
 
 #endif // TEST_STD_VER > 17
 
-int main() {{}}
+int main(int, char**) { return 0; }
 """.format(script_name=script_name,
            header=h,
            test_tags=test_tags,

The correct line here would have been

int main(int, char**) {{ return 0; }}

Do you think you could push that fix? Thanks!

Passed all the tests!

Fixed in r353140.