Page MenuHomePhabricator

[libc++] Integrate the PSTL into libc++
Needs ReviewPublic

Authored by ldionne on Apr 9 2019, 12:23 PM.

Details

Reviewers
rodgert
EricWF
Summary

This commit allows specifying LIBCXX_ENABLE_PARALLEL_ALGORITHMS when configuring libc++ in CMake. When that option is enabled, libc++ will assume that the PSTL can be found somewhere on the CMake module path, and it will provide the C++17 parallel algorithms based on the PSTL (that is assumed to be available).

The commit also adds support for running the PSTL tests as part of the libc++ test suite.

Event Timeline

ldionne created this revision.Apr 9 2019, 12:23 PM
ldionne marked 2 inline comments as done.Apr 9 2019, 12:30 PM

The purpose of this review is to expose how I plan to integrate the parallel STL into libc++, and to discuss some changes that are desirable in pstl for this to be possible. This is a work in progress, obviously, and it's meant as a concrete support for having discussions.

Concrete changes I think we need to make in pstl:

  • Rename all the files to double underscores.
  • Move everything that's meant to be included directly by standard library implementations out of internal/, since that doesn't make sense. It's not internal, it's the API of PSTL (which is meant to be consumed by standard libraries).
  • We could provide the glue definitions and declarations as .ipp files that are meant to be included by standard libraries as-is. The Standard library could then define stuff like the namespace in which those declarations are injected, etc.

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

WDYT @rodgert ?

libcxx/include/algorithm
5683

The same pattern is repeated in other files. Basically, we check whether <execution> has been included, and if so, then we provide all the definitions. Otherwise, we just provide forward declarations to avoid the compile-time overhead if you just #include <algorithm> but don't care about the parallel algorithms.

libcxx/lib/CMakeLists.txt
125 ↗(On Diff #194378)

The basic idea is that when ParallelSTL is installed on the system (or if it's in the tree), we just find it with find_package and then declare a dependency on it and everything should work.

zoecarver removed a subscriber: zoecarver.
zoecarver added a subscriber: zoecarver.

How does this include pattern work with modules? It's important the libc++ headers maintain their modularity.

libcxx/include/algorithm
5684

We normally use _ONE_UNDERSCORE_UPPER or __two_underscore_lower

@ldionne

There's a circular dependency between libc++ and libc++abi, which causes normal CMake dependencies not to work for PSTL. Specifically, libc++abi will complain about not having access to pstl/internal/XXX when building.

Would this be fixed if libc++ and libc++abi configured as a single project instead of two?

The purpose of this review is to expose how I plan to integrate the parallel STL into libc++, and to discuss some changes that are desirable in pstl for this to be possible. This is a work in progress, obviously, and it's meant as a concrete support for having discussions.

Concrete changes I think we need to make in pstl:

  • Rename all the files to double underscores.

This isn't a libstdc++ convention, but I'm not opposed.

  • Move everything that's meant to be included directly by standard library implementations out of internal/, since that doesn't make sense. It's not internal, it's the API of PSTL (which is meant to be consumed by standard libraries).

I would like to maintain a clear separation between the bits that implement PSTL and the glue bits that tie into a given library. Currently everything in 'internal/' goes to libstdc++v3/include/pstl when I integrate with libstdc++v3. I don't want to be left guessing what those files are, I want them to move as a cohesive unit into the library's implementation detail. We can name whatever we want.

  • We could provide the glue definitions and declarations as .ipp files that are meant to be included by standard libraries as-is. The Standard library could then define stuff like the namespace in which those declarations are injected, etc.

I would be interested in exploring this, subject to the caveat that whole of what's currently in 'internal' is an obvious package of related implementation detail that I can poke into libstdc++ as a unit.

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

    WDYT @rodgert ?

If we remove them, then we sort of lose the ability to test pstl outside of standard library, no? I'm not going to be testing or packaging libc++ most likely, but I am going to be contributing changes here and running tests within the context of libstdc++. Having the ability to test changes to the library independent of any integration with another standard library is still useful for me.

rodgert added a comment.EditedApr 9 2019, 10:24 PM

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

libc++ code should follow libc++ conventions.

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

But also thank you very much for your work :-)

The purpose of this review is to expose how I plan to integrate the parallel STL into libc++, and to discuss some changes that are desirable in pstl for this to be possible. This is a work in progress, obviously, and it's meant as a concrete support for having discussions.

Concrete changes I think we need to make in pstl:

  • Rename all the files to double underscores.

This isn't a libstdc++ convention, but I'm not opposed.

The problem with not doing that is that our headers could collide with user's header files.

  • Move everything that's meant to be included directly by standard library implementations out of internal/, since that doesn't make sense. It's not internal, it's the API of PSTL (which is meant to be consumed by standard libraries).

I would like to maintain a clear separation between the bits that implement PSTL and the glue bits that tie into a given library. Currently everything in 'internal/' goes to libstdc++v3/include/pstl when I integrate with libstdc++v3. I don't want to be left guessing what those files are, I want them to move as a cohesive unit into the library's implementation detail. We can name whatever we want.

Right, I agree. What I meant (but that's not what I wrote) is that the glue headers and everything that is expected to be used from standard libraries should be out of internal. The backend and the config stuff can (and should) stay in internal. Do we agree here?

  • We could provide the glue definitions and declarations as .ipp files that are meant to be included by standard libraries as-is. The Standard library could then define stuff like the namespace in which those declarations are injected, etc.

I would be interested in exploring this, subject to the caveat that whole of what's currently in 'internal' is an obvious package of related implementation detail that I can poke into libstdc++ as a unit.

Well, so only the glue part would be .ipps. And I don't care if they're headers or .ipps, the idea is just that I want to reverse the model from "libc++ includes pstl headers that define the parallel algorithms" to "pstl provides declarations that a standard library can copy-paste into their own headers via #include".

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

    WDYT @rodgert ?

If we remove them, then we sort of lose the ability to test pstl outside of standard library, no? I'm not going to be testing or packaging libc++ most likely, but I am going to be contributing changes here and running tests within the context of libstdc++. Having the ability to test changes to the library independent of any integration with another standard library is still useful for me.

I believe those headers should be provided by a dummy standard library that we use only to run the tests. Basically we'd have a dummy standard library in pstl/test that would define these headers, and that's what we would run the tests against.

Lastly, about the __PSTL_XXX vs _PSTL_XXX: This patch is using what's in PSTL today, that's all. As a separate change, I thought about changing all __PSTL_XXX macros to be _PSTL_XXX to follow the more common convention (underscore-capital or double-underscore-lowercase), but this is a change separate from the integration into libc++.

The purpose of this review is to expose how I plan to integrate the parallel STL into libc++, and to discuss some changes that are desirable in pstl for this to be possible. This is a work in progress, obviously, and it's meant as a concrete support for having discussions.

Concrete changes I think we need to make in pstl:

  • Rename all the files to double underscores.

This isn't a libstdc++ convention, but I'm not opposed.

The problem with not doing that is that our headers could collide with user's header files.

I understand the complication. libstdc++ follows the convention of everything 'internal' being in bits/ and then the std headers #include <bits/foo.h>, and this is something that's
on my list to address for PSTL (as what is currently in libstdc++ could still clash with user headers). I don't have any real objection to the underscore naming convention. I can mechanically
translate them to #include <pstl/non_underscore_named_file.h>.

  • Move everything that's meant to be included directly by standard library implementations out of internal/, since that doesn't make sense. It's not internal, it's the API of PSTL (which is meant to be consumed by standard libraries).

I would like to maintain a clear separation between the bits that implement PSTL and the glue bits that tie into a given library. Currently everything in 'internal/' goes to libstdc++v3/include/pstl when I integrate with libstdc++v3. I don't want to be left guessing what those files are, I want them to move as a cohesive unit into the library's implementation detail. We can name whatever we want.

Right, I agree. What I meant (but that's not what I wrote) is that the glue headers and everything that is expected to be used from standard libraries should be out of internal. The backend and the config stuff can (and should) stay in internal. Do we agree here?

Ok, yeah, I'm good with that.

  • We could provide the glue definitions and declarations as .ipp files that are meant to be included by standard libraries as-is. The Standard library could then define stuff like the namespace in which those declarations are injected, etc.

I would be interested in exploring this, subject to the caveat that whole of what's currently in 'internal' is an obvious package of related implementation detail that I can poke into libstdc++ as a unit.

Well, so only the glue part would be .ipps. And I don't care if they're headers or .ipps, the idea is just that I want to reverse the model from "libc++ includes pstl headers that define the parallel algorithms" to "pstl provides declarations that a standard library can copy-paste into their own headers via #include".

I'm also good with this.

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

    WDYT @rodgert ?

If we remove them, then we sort of lose the ability to test pstl outside of standard library, no? I'm not going to be testing or packaging libc++ most likely, but I am going to be contributing changes here and running tests within the context of libstdc++. Having the ability to test changes to the library independent of any integration with another standard library is still useful for me.

I believe those headers should be provided by a dummy standard library that we use only to run the tests. Basically we'd have a dummy standard library in pstl/test that would define these headers, and that's what we would run the tests against.

That would work for me...AND...we can probably remove the per-test #ifdef for the standalone checks, centralize it in the dummy standard headers in 'pstl/test' which would be much cleaner than what is currently there.

Lastly, about the __PSTL_XXX vs _PSTL_XXX: This patch is using what's in PSTL today, that's all. As a separate change, I thought about changing all __PSTL_XXX macros to be _PSTL_XXX to follow the more common convention (underscore-capital or double-underscore-lowercase), but this is a change separate from the integration into libc++.

JF and I discussed possibly some tooling for this sort of thing. I'm still interested in looking at that, I just don't see a lot of value in changing it by hand just to comply with conflicting conventions between standard library implementations.

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

    WDYT @rodgert ?

If we remove them, then we sort of lose the ability to test pstl outside of standard library, no? I'm not going to be testing or packaging libc++ most likely, but I am going to be contributing changes here and running tests within the context of libstdc++. Having the ability to test changes to the library independent of any integration with another standard library is still useful for me.

I believe those headers should be provided by a dummy standard library that we use only to run the tests. Basically we'd have a dummy standard library in pstl/test that would define these headers, and that's what we would run the tests against.

That would work for me...AND...we can probably remove the per-test #ifdef for the standalone checks, centralize it in the dummy standard headers in 'pstl/test' which would be much cleaner than what is currently there.

Ok, I'll submit a patch for that.

Lastly, about the __PSTL_XXX vs _PSTL_XXX: This patch is using what's in PSTL today, that's all. As a separate change, I thought about changing all __PSTL_XXX macros to be _PSTL_XXX to follow the more common convention (underscore-capital or double-underscore-lowercase), but this is a change separate from the integration into libc++.

JF and I discussed possibly some tooling for this sort of thing. I'm still interested in looking at that, I just don't see a lot of value in changing it by hand just to comply with conflicting conventions between standard library implementations.

I don't think the conventions used in PSTL should conform to the coding standard in any one standard library implementation. It's a library on its own. I think it should conform to its own coding standards, and in this case, I think those coding standard should be that of LLVM. Does this point of view make sense?

jfb added a subscriber: jfb.Apr 10 2019, 10:01 AM

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

libc++ code should follow libc++ conventions.

Just to be clear: PSTL is neither libc++ nor libstdc++. Louis kinda said so above, and Tom is making that point too, but I want to reiterate it. It's overall a silly bikeshedding point, but it's critical to keep everyone happy. PSTL isn't more "ours" than "theirs" (or rather, it's "ours" in that we're all in the same C++ team). This point was critical in getting PSTL to be a shared resource in the first place, and I don't want to walk back this commitment.

As Tom mentions above, we've talked about having tooling to massage PSTL to libc++'s and libstdc++'s liking on integration. Without volunteers to write said tooling, we should just accept slight ache when integrating PSTL, as we see here when integrating it into libc++.

I take back my comments about libc++ style. I didn't understand what was going on here. my apologies.

My only remaining concern with this patch is that our headers stay modular and continue to work with clangs modules.

I also suggest:

  • Removing the <algorithm> & friends headers from PSTL itself. Those don't belong there, they belong with the standard library.

    WDYT @rodgert ?

If we remove them, then we sort of lose the ability to test pstl outside of standard library, no? I'm not going to be testing or packaging libc++ most likely, but I am going to be contributing changes here and running tests within the context of libstdc++. Having the ability to test changes to the library independent of any integration with another standard library is still useful for me.

I believe those headers should be provided by a dummy standard library that we use only to run the tests. Basically we'd have a dummy standard library in pstl/test that would define these headers, and that's what we would run the tests against.

That would work for me...AND...we can probably remove the per-test #ifdef for the standalone checks, centralize it in the dummy standard headers in 'pstl/test' which would be much cleaner than what is currently there.

Ok, I'll submit a patch for that.

Lastly, about the __PSTL_XXX vs _PSTL_XXX: This patch is using what's in PSTL today, that's all. As a separate change, I thought about changing all __PSTL_XXX macros to be _PSTL_XXX to follow the more common convention (underscore-capital or double-underscore-lowercase), but this is a change separate from the integration into libc++.

JF and I discussed possibly some tooling for this sort of thing. I'm still interested in looking at that, I just don't see a lot of value in changing it by hand just to comply with conflicting conventions between standard library implementations.

I don't think the conventions used in PSTL should conform to the coding standard in any one standard library implementation. It's a library on its own. I think it should conform to its own coding standards, and in this case, I think those coding standard should be that of LLVM. Does this point of view make sense?

At the end of the day, I'd rather invest in the tooling to make a uniform convention, than have to do this exercise again...ever. The more we can agree a consistent set of rules (ideally enforced by clang-format) for this library the better.

In D60480#1461555, @jfb wrote:

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

libc++ code should follow libc++ conventions.

Just to be clear: PSTL is neither libc++ nor libstdc++. Louis kinda said so above, and Tom is making that point too, but I want to reiterate it. It's overall a silly bikeshedding point, but it's critical to keep everyone happy. PSTL isn't more "ours" than "theirs" (or rather, it's "ours" in that we're all in the same C++ team). This point was critical in getting PSTL to be a shared resource in the first place, and I don't want to walk back this commitment.

As Tom mentions above, we've talked about having tooling to massage PSTL to libc++'s and libstdc++'s liking on integration. Without volunteers to write said tooling, we should just accept slight ache when integrating PSTL, as we see here when integrating it into libc++.

The process of applying this kind of change is very error prone. I still have a patch coming that fixes a few cases I missed. That's the bigger argument IMO for tooling than the specific format.

In D60480#1461555, @jfb wrote:

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

libc++ code should follow libc++ conventions.

Just to be clear: PSTL is neither libc++ nor libstdc++. Louis kinda said so above, and Tom is making that point too, but I want to reiterate it. It's overall a silly bikeshedding point, but it's critical to keep everyone happy. PSTL isn't more "ours" than "theirs" (or rather, it's "ours" in that we're all in the same C++ team). This point was critical in getting PSTL to be a shared resource in the first place, and I don't want to walk back this commitment.

As Tom mentions above, we've talked about having tooling to massage PSTL to libc++'s and libstdc++'s liking on integration. Without volunteers to write said tooling, we should just accept slight ache when integrating PSTL, as we see here when integrating it into libc++.

The process of applying this kind of change is very error prone. I still have a patch coming that fixes a few cases I missed. That's the bigger argument IMO for tooling than the specific format.

I think I'm missing yours and JF's point. I don't understand why you need any tooling. All I'm saying is that PSTL being under the LLVM umbrella should conform to the usual LLVM code style, which used single underscore + capitals for macros unless I'm mistaken. libstdc++ and libc++ wouldn't apply any code transformation on the PSTL sources after that, it would just use it as a separate library and there wouldn't be very many PSTL macros used inside libstdc++/libc++. Only those macros that are part of the PSTL's interface, and yes, those would be using the PSTL's naming convention.

IOW, this is something that we simply missed in the initial check-in.

In D60480#1461555, @jfb wrote:

Not sure what is up with phabricator, but it won't let me respond inline to EricWF, specifically -

__FIRST_MOVER_ADVANTAGE

libc++ and libstdc++ have different 'uglification' protocols. As I said to mclow early on in this process, If I'm the one that has to do the grunt work of *all* of the uglification (very much has been case, BTW), it's going to follow libstdc++ convention as much as is possible.

libc++ code should follow libc++ conventions.

Just to be clear: PSTL is neither libc++ nor libstdc++. Louis kinda said so above, and Tom is making that point too, but I want to reiterate it. It's overall a silly bikeshedding point, but it's critical to keep everyone happy. PSTL isn't more "ours" than "theirs" (or rather, it's "ours" in that we're all in the same C++ team). This point was critical in getting PSTL to be a shared resource in the first place, and I don't want to walk back this commitment.

As Tom mentions above, we've talked about having tooling to massage PSTL to libc++'s and libstdc++'s liking on integration. Without volunteers to write said tooling, we should just accept slight ache when integrating PSTL, as we see here when integrating it into libc++.

The process of applying this kind of change is very error prone. I still have a patch coming that fixes a few cases I missed. That's the bigger argument IMO for tooling than the specific format.

I think I'm missing yours and JF's point. I don't understand why you need any tooling. All I'm saying is that PSTL being under the LLVM umbrella should conform to the usual LLVM code style, which used single underscore + capitals for macros unless I'm mistaken. libstdc++ and libc++ wouldn't apply any code transformation on the PSTL sources after that, it would just use it as a separate library and there wouldn't be very many PSTL macros used inside libstdc++/libc++. Only those macros that are part of the PSTL's interface, and yes, those would be using the PSTL's naming convention.

IOW, this is something that we simply missed in the initial check-in.

So, I hear you volunteering to apply those changes :)

ldionne added a comment.EditedApr 10 2019, 10:39 AM

So, I hear you volunteering to apply those changes :)

Yes, it's never been my intention to push that work onto someone else. I just wanted to make sure we agreed before going ahead and doing it. https://reviews.llvm.org/D60521

ldionne updated this revision to Diff 205866.Thu, Jun 20, 11:19 AM

Non-draft version of the patch.

Herald added a project: Restricted Project. · View Herald TranscriptThu, Jun 20, 11:19 AM
ldionne retitled this revision from [WIP] integration of pstl into libc++ to [libc++] Integrate the PSTL into libc++.Thu, Jun 20, 11:20 AM
ldionne edited the summary of this revision. (Show Details)
EricWF requested changes to this revision.Fri, Jun 21, 3:21 AM
EricWF added inline comments.
libcxx/include/algorithm
5683

I really really really dislike this pattern. We shouldn't do it.
I don't think this is how you should structure a library.

Are there other alternatives?

libcxxabi/src/CMakeLists.txt
149–160

Why does libc++abi depend on parallel algorithms? That seems backwards?

pstl/test/pstl/header_inclusion_order_algorithm_0.pass.cpp
11 ↗(On Diff #205866)

Does every test under pstl/test/pstl require "parallel-algorithms"? Because if so there are easier ways to do this.

This revision now requires changes to proceed.Fri, Jun 21, 3:21 AM
ldionne marked 4 inline comments as done.Tue, Jun 25, 1:12 PM

Please note that this review is really about integrating pstl into libc++, not pstl itself. I know there are changed needed before pstl is ready for prime time (and LIBCXX_ENABLE_PARALLEL_ALGORITHMS becomes ON by default).

libcxx/include/algorithm
5683

I dislike it too. The only alternative I can think of is to unconditionally include the definitions, however this may impose a compile-time hit on everyone that's not using the parallel algorithms.

5684

For file names? Please disregard pstl file names for the time being. The goal of this review is to check-in the integration between libc++ and pstl, but it's understood that pstl's structure is going to change in the future. I'm just trying to integrate early to better see how changes in pstl will end up impacting libc++.

libcxxabi/src/CMakeLists.txt
149–160

Yes, it is. It's because libc++abi includes headers from libc++ that pull in the parallel algorithms when they're enabled. So this is needed to support building libc++abi against a libc++ that has the parallel algorithms enabled.

pstl/test/pstl/header_inclusion_order_algorithm_0.pass.cpp
11 ↗(On Diff #205866)

Yes, they all do. I could also use a lit.local.cfg if that's what you're thinking about?

dexonsmith added inline comments.
libcxx/include/algorithm
5683

@bruno, will this pattern cause a problem for Clang modules (esp. with local submodule visibility)?

EricWF added inline comments.Tue, Jun 25, 3:04 PM
libcxxabi/src/CMakeLists.txt
149–160

This is a layering violation and creates a circular dependency. I don't think we can or should structure the project like this.

rsmith added a subscriber: rsmith.Tue, Jun 25, 3:23 PM
rsmith added inline comments.
libcxx/include/algorithm
5683

This won't work with modules (neither Clang modules with local submodule visibility, nor C++20 modules). We will build <algorithm> in a context where <execution> has not been included, and vice versa, and neither of them will contain definitions of the parallel algorithms. More fundamentally, we should not be trying to create a situation where we have two headers and including both of them together provides more content than the union of the content provided by including the two separately -- that's bad header design and inevitably will lead to problems.

One possible alternative design approach:

  • <algorithm> provides definitions of the algorithms that forward to ExecutionPolicy::__<algorithm>(<args>) if they exist, or otherwise forward to the non-execution-policy overload if ExecutionPolicy::<some tag member> exists (indicating that the template argument was a real execution policy that just doesn't happen to specialize that algorithm), and otherwise produce an error that the argument is not an execution policy.
  • <execution> defines the specialized algorithms for each execution policy as static member functions on the execution policy.

This adds minimal cost to code that only includes <algorithm> and not <execution> while keeping both headers modular. It also supports the easy addition of new implementation-specific execution policies, defining one policy as a refinement of another, and gives an easy way for an implementation to be conforming in the absence of a pstl implementation (or to provide an easy non-parallel implementation for single-threaded builds): just define the execution policy objects with no members other than the "I'm a real execution policy" tag member.

Would that work?

ldionne planned changes to this revision.Wed, Jun 26, 3:53 PM
ldionne marked 2 inline comments as done.
ldionne added inline comments.
libcxx/include/algorithm
5683

I'll experiment with that. For now, however, we could always include <glue_algorithm_impl.h>, which will pull in the algorithm definition. That'll take care of the libc++/pstl integration. I can then work on pstl in isolation to make it include the definitions of algorithms only when <execution> is included.

Once we're satisfied with what we have we can turn on the parallel algorithms by default in libc++.

libcxxabi/src/CMakeLists.txt
149–160

To be clear, we already have this problem today. libc++abi requires libc++ in order to build. Now we just need to transitively pull in the pstl.

MikeDvorskiy added inline comments.
libcxx/lib/CMakeLists.txt
125 ↗(On Diff #194378)

ParallelSTL is installed on the system

What does it mean? The fact there is no ParallelSTL as "product/component". ParallelSTL - just a set of headers.

So, the cmake file should find "ParallelSTL package".

ldionne marked an inline comment as done.Thu, Jun 27, 9:16 AM
ldionne added inline comments.
libcxx/lib/CMakeLists.txt
125 ↗(On Diff #194378)

Either it's in the tree and we can find it (like we do here), or it's somewhere on the system and has a ParallelSTLConfig.cmake file installed alongside it (which our current pstl build does install). Realistically, I only expect the former to be used.

ldionne updated this revision to Diff 208065.Thu, Jul 4, 11:38 AM
ldionne edited the summary of this revision. (Show Details)

Use a local lit cfg to disable pstl tests and always include the definition of
algorithms in the headers.

ldionne updated this revision to Diff 208199.Fri, Jul 5, 9:39 AM

Install <execution> alongside other headers

So, as it turns out, we actually have forward declarations for all the algorithms in the pstl, so the integration as-is will not pull any of those in. They're only pulled in when <execution> is included.

MikeDvorskiy added inline comments.Mon, Jul 8, 6:28 AM
libcxx/lib/CMakeLists.txt
125 ↗(On Diff #194378)

Parallel STL is not a separate product/package, so it doesn't make sense to have formal versioning for it. Also it doesn't make sense installation part from CMakeLists.txt.

We won't be able to use find_package(ParallelSTL) (and find_package(ParallelSTL <version>) as well) in order to include Parallel STL into user project. But it is possible to use add_subdirectory(/path/to/parallelstl).

WDYT?

ldionne marked an inline comment as done.Mon, Jul 8, 8:47 AM
ldionne added inline comments.
libcxx/lib/CMakeLists.txt
125 ↗(On Diff #194378)

Parallel STL is not a separate product/package, so it doesn't make sense to have formal versioning for it. Also it doesn't make sense installation part from CMakeLists.txt.

It definitely is a separate package, no? It's a self-standing library that libc++ happens to be using as an implementation detail. I can't see another way to look at this other than saying it's a bunch of sources that we copy-paste into libc++. But that way of looking at things isn't good because it means it doesn't make sense to build/test the PSTL on its own, which we want to do.

We won't be able to use find_package(ParallelSTL)

It does work if the PSTL is installed (which installs a ParallelSTLConfig.cmake file) before building libc++. Otherwise, it also works if the ParallelSTL is enabled as part of the monorepo, because even though the call to find_package(ParallelSTL) will fail, the pstl::ParallelSTL CMake target still exists.

Also, we're commenting on an old diff.

ldionne updated this revision to Diff 208478.Mon, Jul 8, 12:37 PM

Make sure we don't warn when we don't find the ParallelSTL package through CMake. In most if not all cases, this lookup will fail and we'll be using the local PSTL inside the monorepo, so we shouldn't be warning in the common case.

Ping! I think this is good to go and doesn't have the problems mentioned by reviewers.

I'll ship this on Friday if no additional objections are noted.