Page MenuHomePhabricator

[libc] Extends the testing framework to support typed test
ClosedPublic

Authored by gchatelet on Apr 16 2021, 2:37 AM.

Details

Summary

This patch provides TYPED_TEST and TYPED_TEST_F (similar in functionnality to gtest).
This is needed to extensively test building blocks for memory functions.

Example for TYPED_TEST_F:

template <typename T> class LlvmLibcMyTestFixture : public testing::Test {};

using Types = testing::TypeList<char, int, long>;

TYPED_TEST_F(LlvmLibcMyTestFixture, Simple, Types) {
  EXPECT_LE(sizeof(ParamType), 8UL);
}

Example for TYPED_TEST:

using Types = testing::TypeList<char, int, long>;

TYPED_TEST(LlvmLibcMyTest, Simple, Types) {
  EXPECT_LE(sizeof(ParamType), 8UL);
}

ParamType is displayed as fully qualified canonical type which can be difficult to read, the user can provide a more readable name by using the REGISTER_TYPE_NAME macro.

I noticed that Fuchsia is using its own testing framework. We can either add the same functionality to zxtest or disable them on Fuchsia. WDYT?

Diff Detail

Event Timeline

gchatelet created this revision.Apr 16 2021, 2:37 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 16 2021, 2:37 AM
gchatelet requested review of this revision.Apr 16 2021, 2:37 AM
gchatelet edited the summary of this revision. (Show Details)Apr 16 2021, 2:41 AM
gchatelet updated this revision to Diff 338040.Apr 16 2021, 2:45 AM
  • Fix typo and static assert message.
gchatelet updated this revision to Diff 338041.Apr 16 2021, 2:48 AM
  • Fixed base revision

While I am not opposed to this, I want to learn the usefulness of this feature. IIUC, this limits to a single ParamType - as in a uni-dimensional type variance. We have examples in math functions where there is two-dimensional type variance. I am not sure the feature added in this patch can help with it. For such cases, have used template test classes and helper macros to list the various combination of tests: https://github.com/llvm/llvm-project/blob/main/libc/test/src/math/RoundToIntegerTest.h.

While I am not opposed to this, I want to learn the usefulness of this feature. IIUC, this limits to a single ParamType - as in a uni-dimensional type variance. We have examples in math functions where there is two-dimensional type variance. I am not sure the feature added in this patch can help with it. For such cases, have used template test classes and helper macros to list the various combination of tests: https://github.com/llvm/llvm-project/blob/main/libc/test/src/math/RoundToIntegerTest.h.

Yes I've spotted these tests but I found that it scaled poorly as you define more tests since you have to define the tests into a macro which makes for poor editing.
You ended up moving the body of the test to an external function to help with this but I think it's not ideal (especially for failure location).

In the case of multi-dimensional types you can pack them into a tuple and unpack them in the test fixture. I'll try to implement RoundToIntegerTest with TYPED_TEST and let you know how it looks like. My particular use case is for 19 types and 4 tests (but more to come).

While I am not opposed to this, I want to learn the usefulness of this feature. IIUC, this limits to a single ParamType - as in a uni-dimensional type variance. We have examples in math functions where there is two-dimensional type variance. I am not sure the feature added in this patch can help with it. For such cases, have used template test classes and helper macros to list the various combination of tests: https://github.com/llvm/llvm-project/blob/main/libc/test/src/math/RoundToIntegerTest.h.

Yes I've spotted these tests but I found that it scaled poorly as you define more tests since you have to define the tests into a macro which makes for poor editing.
You ended up moving the body of the test to an external function to help with this but I think it's not ideal (especially for failure location).

There is no problem with failure location - you will get the correct file and line number. With math tests we do not have a problem with scaling also - we want tests for each type combination to be in different files as they correspond to different functions.
Also, I am not sure what you mean by "define tests in a macro". The tests are member functions of the template class. The listing part is done with a macro, which is minor.

In the case of multi-dimensional types you can pack them into a tuple and unpack them in the test fixture. I'll try to implement RoundToIntegerTest with TYPED_TEST and let you know how it looks like. My particular use case is for 19 types and 4 tests (but more to come).

If you find it useful, I am totally fine with it. Lets wait to hear from phosek about adding this to zxtest.

I'll check with zxtest maintainers.

I'll check with zxtest maintainers.

Looks like we already have a few use cases for TYPED_TEST in Fuchsia, this is tracked as fxbug.dev/62890.

sivachandra accepted this revision.EditedApr 16 2021, 1:21 PM

Accepting this as it looks like zxtest can be extended to have this feature. Irrespective though, as I think more about it, it is definitely of value when listing the differently-typed tests that can all be listed in a single file/single test.

This revision is now accepted and ready to land.Apr 16 2021, 1:21 PM
gchatelet updated this revision to Diff 338226.Apr 16 2021, 1:49 PM
  • Recover ParamType from PRETTY_FUNCTION instead of RTTI or user provided type name.
gchatelet updated this revision to Diff 338229.Apr 16 2021, 1:53 PM
  • Added documentation

I'll check with zxtest maintainers.

Looks like we already have a few use cases for TYPED_TEST in Fuchsia, this is tracked as fxbug.dev/62890.

Thx @phosek ! I'll ping on the bug to see how I can bring this to zxtest.

Thx @sivachandra for the LGTM.

gchatelet edited the summary of this revision. (Show Details)Apr 16 2021, 2:09 PM

Note that Fuchsia does not block us from using this feature in LLVM libc source tree as only ctype tests from LLVM libc run in Fuchsia.

This revision was landed with ongoing or failed builds.Apr 16 2021, 2:21 PM
This revision was automatically updated to reflect the committed changes.

Note that Fuchsia does not block us from using this feature in LLVM libc source tree as only ctype tests from LLVM libc run in Fuchsia.

Ha thx I thought it was a blocker.