Page MenuHomePhabricator

[lldb] Fix that SBThread.GetStopDescription is returning strings with uninitialized memory at the end.
ClosedPublic

Authored by teemperor on Jan 2 2020, 7:41 AM.

Details

Summary

SBThread.GetStopDescription is a curious API as it takes a buffer length as a parameter that specifies
how many bytes the buffer we pass has. Then we fill the buffer until the specified length (or the length
of the stop description string) and return the string length. If the buffer is a nullptr however, we instead
return how many bytes we would have written to the buffer so that the user can allocate a buffer with
the right size and pass that size to a subsequent SBThread.GetStopDescription call.

Funnily enough, it is not possible to pass a nullptr via the Python SWIG bindings, so that might be the
first API in LLDB that is not only hard to use correctly but impossible to use correctly. The only way to
call this function via Python is to throw in a large size limit that is hopefully large enough to contain the
stop description (otherwise we only get the truncated stop description).

Currently passing a size limit that is smaller than the returned stop description doesn't cause the
Python bindings to return the stop description but instead the truncated stop description + uninitialized characters
at the end of the string. The reason for this is that we return the result of snprintf from the method
which returns the amount of bytes that *would* have been written (which is larger than the buffer).
This causes our Python bindings to return a string that is as large as full stop description but the
buffer that has been filled is only as large as the passed in buffer size.

This patch fixes this issue by just recalculating the string length in our buffer instead of relying on the wrong
return value. We also have to do this in a new type map as the old type map is also used for all methods
with the given argument pair char *dst, size_t dst_len (e.g. SBProcess.GetSTDOUT`). These methods have
different semantics for these arguments and don't null-terminate the returned buffer (they instead return the
size in bytes) so we can't change the existing typemap without breaking them.

Diff Detail

Event Timeline

teemperor created this revision.Jan 2 2020, 7:41 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 2 2020, 7:41 AM
shafik added a subscriber: shafik.Jan 2 2020, 1:04 PM
shafik added inline comments.
lldb/source/API/SBThread.cpp
337 ↗(On Diff #235875)

The else branch feels inconsistent with the change above. Especially the + 1.

Would it be possible to fix this problem in the swig bindings instead? (i.e. move the std::min stuff into the swig code). That way, the behavior of this function will at least match that of snprintf, which will hopefully be less surprising that what you've done here. (And it will also fix the inconsistency noticed by @shafik).

lldb/source/API/SBThread.cpp
337 ↗(On Diff #235875)

yep

teemperor updated this revision to Diff 236542.Jan 7 2020, 3:04 AM
teemperor edited the summary of this revision. (Show Details)
  • Move change to SWIG type map.
  • Fixed some type in the test.

The API never emulated the snprintf behavior as it includes a NULL byte when we pass a nullptr buffer (and that's why it has the inconsistent + 1 in the else branch). We get this whole thing even more wrong as we add the NULL byte to the snprintf result in the other branch (at the end of the method). Anyway, I changed this in the type map now and just use strlen on the result string which is simpler (and doesn't rely on any of our bogus return values).

I'll make another patch where we can discuss what this function is supposed to return.

labath accepted this revision.Jan 7 2020, 3:43 AM

This is fine. It also sounds like a good idea to solve the +1 vs snprintf inconsistency.

lldb/scripts/Python/python-typemaps.swig
124–125 ↗(On Diff #236542)

This is just StringRef ref = $1, right ?

This revision is now accepted and ready to land.Jan 7 2020, 3:43 AM
teemperor updated this revision to Diff 236790.Jan 8 2020, 3:42 AM
teemperor marked an inline comment as done.
  • Simplify swig type map code.
teemperor marked an inline comment as done.Jan 8 2020, 3:43 AM
teemperor added inline comments.
lldb/scripts/Python/python-typemaps.swig
124–125 ↗(On Diff #236542)

Swig gives $1 a void* type (at least on my system), so I we still need the static_cast (but we can get rid of the rest).

teemperor updated this revision to Diff 236793.Jan 8 2020, 3:56 AM
  • Remove unintended empty line change.
clayborg added inline comments.
lldb/scripts/Python/python-typemaps.swig
124 ↗(On Diff #236793)

This assignment looks a bit goofy IMHO. Just remove result from original code?:

llvm::StringRef ref(static_cast<char*>($1));
teemperor updated this revision to Diff 236986.Jan 9 2020, 1:51 AM
  • Just removed the result parameter.

For reasons that are beyond my understanding this change seems to break SBProcess.GetSTDOUT in random tests like this:

======================================================================
ERROR: test_change_value_dwarf (TestChangeValueAPI.ChangeValueAPITestCase)
   Exercise the SBValue::SetValueFromCString API.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/teemperor/work/ci/llvm/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1732, in test_method
    return attrvalue(self)
  File "/home/teemperor/work/ci/llvm/lldb/packages/Python/lldbsuite/test/decorators.py", line 454, in wrapper
    func(*args, **kwargs)
  File "/home/teemperor/work/ci/llvm/lldb/packages/Python/lldbsuite/test/decorators.py", line 110, in wrapper
    func(*args, **kwargs)
  File "/home/teemperor/work/ci/llvm/lldb/packages/Python/lldbsuite/test/python_api/value/change_values/TestChangeValueAPI.py", line 149, in test_change_value
    stdout = process.GetSTDOUT(1000)
  File "/home/teemperor/work/ci/build/lib/python3.8/site-packages/lldb/__init__.py", line 8020, in GetSTDOUT
    return _lldb.SBProcess_GetSTDOUT(self, dst)
SystemError: <built-in function SBProcess_GetSTDOUT> returned NULL without setting an error
Config=x86_64-/home/teemperor/work/ci/build/bin/clang-10

Anyone got a clue what is going on here? It seems like these tests only fail when run in combination with some other tests as running them alone is fine.

Ah, seems like we match in the type map by function signature and GetSTDOUT matches the signature by accident....

  • Move to use a new typemap for GetStopDescription.
teemperor requested review of this revision.Jan 13 2020, 12:56 AM
teemperor edited the summary of this revision. (Show Details)

I made a new typemap now that is only for GetStopDescription and it's snprintf semantics. This keeps all the other functions working. I kept the old behavior of requiring a >0 buffer size etc. in tact for now.

labath accepted this revision.Jan 13 2020, 2:20 AM

Overall, I think it would be nice to introduce some consistency into the return values of functions which fill out a user-provided buffer, but the thing which convinced me that a separate typemap is needed is utf8 handling. For functions like GetStopDescription, we should hopefully be able to guarantee that the returned value is a valid utf8 string (and so we can return a python string), but for things like GetSTOUT we most certainly cannot (a python bytes object would be more suitable)...

This revision is now accepted and ready to land.Jan 13 2020, 2:20 AM
This revision was automatically updated to reflect the committed changes.