This is an archive of the discontinued LLVM Phabricator instance.

[LLDB] Do not dereference promise pointer in `coroutine_handle` pretty printer
ClosedPublic

Authored by avogelsgesang on Aug 28 2022, 1:08 PM.

Details

Summary

So far, the pretty printer for std::coroutine_handle internally
dereferenced the contained frame pointer displayed the promise
as a sub-value. As noticed in https://reviews.llvm.org/D132624
by @labath, this can lead to an endless loop in lldb during printing
if the coroutine frame pointers form a cycle.

This commit breaks the cycle by exposing the promise as a pointer
type instead of a value type. The depth to which the frame variable
and the expression commands dereference those pointers can be
controlled using the --ptr-depth argument.

Diff Detail

Event Timeline

avogelsgesang created this revision.Aug 28 2022, 1:08 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 28 2022, 1:08 PM
avogelsgesang requested review of this revision.Aug 28 2022, 1:08 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 28 2022, 1:08 PM
labath added inline comments.Aug 31 2022, 6:10 AM
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
246–248

Have you checked there won't be a use-after-free problem here, given that this data extractor will refer to the stack object?

To create persistent data, you need to use the DataBufferSP constructor, but I'm wondering if we couldn't fix this by creating the (non-pointer) object using the CreateValueObjectFromAddress function, as above, but then actually use valobj->AddressOf as the synthetic child.

I am also somewhat surprised that we need to use the GetAddressOf trick here, as this seems to indicate that the coroutine contains (in the proper C "subobject" kind of way) the promise object. That's not necessarily wrong, but it makes me think we may be "breaking the cycle" at the wrong place.

avogelsgesang added inline comments.Aug 31 2022, 7:04 AM
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
246–248

Thanks for taking a look!

To create persistent data, you need to use the DataBufferSP constructor

good point, I will keep this in mind as a fallback in case we don't decide to follow any of the other directions you hinted at.

wondering if we couldn't fix this by creating the (non-pointer) object using the CreateValueObjectFromAddress function, as above, but then actually use valobj->AddressOf as the synthetic child

Good idea! I will give it a try

[...] as this seems to indicate that the coroutine contains (in the proper C "subobject" kind of way) the promise object. That's not necessarily wrong, but it makes me think we may be "breaking the cycle" at the wrong place.

The physical layout of this is:

// in the standard library
template<typename promise_type>
struct exception_handle<promise_type> {
    __coro_frame<promise_type>* __hdl; // <--- this is the pointer we read with `GetCoroFramePtrFromHandle`
};

// compiler-generated coroutine frame. Generated ad-hoc per coroutine
struct __coro_frame<promise_type> {
     // The ABI guaranteees that hose two pointers are always the first two pointers in the struct.
     void (*resume)(void*); // function pointer for type erasure
     void (*destroy)(void*); // function pointer for type erasure
     // Next comes our promise type. This is under the control of the program author
     promise_type promise;
     // Next comes any compiler-generated, internal state which gets persisted across suspension points. 
     // The functions pointed to by `resume`/`destroy` know how to interpret this part of the coroutine frame.
     int __suspension_point_id;
     double __some_internal_state;
     std::string __some_other_internal_state;
     ....
};

The programmer (i.e., most likely the user of this pretty-printer), wrote only the "promise" explicitly in his code. Everything else is compiler-generated. As such, the lldb-user will usually look for the "promise" first, and I would like to make it easy to find it, by exposing it as a top-level children of the exception_handle instead of hiding it inside a sub-child.

labath added inline comments.Aug 31 2022, 7:36 AM
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
246–248

As such, the lldb-user will usually look for the "promise" first, and I would like to make it easy to find it, by exposing it as a top-level children of the exception_handle instead of hiding it inside a sub-child.

That makes sense. And I think that's a good argument for automatically "dereferencing" that object (i.e., status quo). That said, it's not fully clear to me how do we end up looping here. I take it the promise object contains a (compiler-generated ?) pointer to another __coro_frame object? What would happen if we turned *that* into a pointer?

avogelsgesang added inline comments.Aug 31 2022, 7:56 AM
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
246–248

I take it the promise object contains a [...] pointer to another __coro_frame object?

yes

(compiler-generated ?)

no

We end up looping if the user explicitly puts an coroutine_handle inside promise_type. You can find an example of this in https://clang.llvm.org/docs/DebuggingCoroutines.html#get-the-asynchronous-stack, in particular

struct promise_type {
    // [...] shortened for readability

    std::coroutine_handle<> continuation = std::noop_coroutine();
    int result = 0;
};

In asynchronous programming, it is common to "chain continuations" by putting a coroutine_handle into the promise_type. This coroutine_handle inside the promise_type gives my asynchronous coroutine the answer to the question "where should I continue next, after finishing the asychronous operation modelled by my own coroutine?".

In normal situations (i.e. in the absence of bugs inside the debugged program), those coroutine_handle's should not form cycles. But I guess there could also be other use cases for coroutines where coroutine_handle cycles might be useful...

What would happen if we turned *that* into a pointer?

The promise_type is a user-written struct, so I guess that I have little leverage whether it contains a coroutine_handle by value or by pointer? And turning an object which the user wrote as "by value" into a "by pointer" representation in the debugger would be pretty surprising...

labath added inline comments.Sep 5 2022, 8:02 AM
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
246–248

Ok, thanks for your patience, I think I finally understand what's going on. Given all of the above, the approach in this patch (not dereferencing the promise type) makes sense to me, since the coroutine_handletype has pointer-like semantics. However, do take my opinion with a grain of salt, since I probably not going to be using this code.

I hope that these comments are helpful. If they are not, please feel free to tell me to stop! I appreciated learning from reading through your discussion with @labath !

lldb/packages/Python/lldbsuite/test/lldbtest.py
265

nit: typo Derefence => Dereference

lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
19

Do we want to return a lldb::addr_t? I know that they are likely the same, but do we want it for consistency?

I see (below) several instances where we are passing around "addresses" and they are plain uint64_t so I am obviously being unhelpful. Sorry!

avogelsgesang marked an inline comment as done.

address review comments

wondering if we couldn't fix this by creating the (non-pointer) object using the CreateValueObjectFromAddress function, as above, but then actually use valobj->AddressOf as the synthetic child

yes, that worked quite nicely

avogelsgesang marked 4 inline comments as done.Sep 19 2022, 9:28 AM
avogelsgesang added inline comments.
lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
19

changed to lldb::addr_t in this complete file

aprantl accepted this revision.Sep 22 2022, 3:13 PM

Seems fine if Pavel is happy with it.

lldb/packages/Python/lldbsuite/test/lldbtest.py
249

That looks like a useful addition!

lldb/source/Plugins/Language/CPlusPlus/Coroutines.cpp
21

Some day we should start returning Optional<addr_t> ...

212

.

This revision is now accepted and ready to land.Sep 22 2022, 3:13 PM
labath accepted this revision.Sep 23 2022, 4:29 AM

Seems fine (with the caveat that all I know about coroutines was learned in this review).

This revision was landed with ongoing or failed builds.Nov 20 2022, 2:29 PM
This revision was automatically updated to reflect the committed changes.
avogelsgesang marked an inline comment as done.

I reverted this change for now to unblock the macOS CI bots.

commit a410b34a66accd864b174b6330d3d353e73f54d8
Author: Jason Molenda <jason@molenda.com>
Date:   Fri Nov 25 12:15:47 2022 -0800

    Revert "[LLDB] Do not dereference promise pointer in `coroutine_handle` pretty printer"
    
    This reverts commit cd3091a88f7c55c90d9b5fff372ce1cdfc71948d.
    
    This change crashes on macOS systems in
    formatters::StdlibCoroutineHandleSyntheticFrontEnd when
    it fails to create the `ValueObjectSP promise` and calls
    a method on it.  The failure causes a segfault while running
    TestCoroutineHandle.py on the "LLDB Incremental" CI bot,
    https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/
    
    This change originally landed via https://reviews.llvm.org/D132815

I saw in the description, " As noticed in https://reviews.llvm.org/D132624
by @labath, this can lead to an endless loop in lldb during printing
if the coroutine frame pointers form a cycle." - if we should revert further changes until this can be all addressed, that's fine, we'll see how the bots react.

The issue with this change was that if devirtualization is failing, then in the line

lldb::ValueObjectSP promise = CreateValueObjectFromAddress(
      "promise", frame_ptr_addr + 2 * ptr_size, exe_ctx, promise_type);

the promise_type is void. Creating an object of type void is obviously not possible.

Adding a simple additional !promise_type.isVoid() check will fix this. Waiting for resolution of the test failure on https://reviews.llvm.org/D132735 first, though.

avogelsgesang reopened this revision.Dec 17 2022, 4:26 AM
This revision is now accepted and ready to land.Dec 17 2022, 4:26 AM
avogelsgesang marked an inline comment as done.

fix crash in cases where the promise type is unknown even after an attempt at devirtualization

Since I am still blocked on solving the Apple-specific issues in https://reviews.llvm.org/D132735 and couldn't solve them in time for LLVM-16, I now rebased this change directly on main, so that it no longer depends on the refactorings from the other commit.

My idea is to backport this change to LLVM-16 after it landed successfully on main. I think including this commit in LLVM-16 makes sense, because I see it as a "bug fix": it fixes a potential infinite loop in the debugger.

https://reviews.llvm.org/D132735 on the other hand can wait until LLVM-17, given it's only a usability improvement/new feature.