This is an archive of the discontinued LLVM Phabricator instance.

[LLDB] Add unit tests for Editline keyboard shortcuts
Needs ReviewPublic

Authored by nealsid on Dec 5 2021, 2:08 PM.

Details

Reviewers
None
Group Reviewers
Restricted Project
Summary

This patch adds tests that verify keyboard shortcuts are set up for multi-line/single-line mode. Along the way I also added a few other small changes:

  • Removed usage of shared_ptr in a case where object ownership was not shared.
  • Changed a file's open mode to be read/write.
  • Removed non-emacs configuration of keyboard shortcuts, because Editline config is hardcoded to "emacs" in ConfigureEditor.

Diff Detail

Event Timeline

nealsid created this revision.Dec 5 2021, 2:08 PM
nealsid requested review of this revision.Dec 5 2021, 2:08 PM
nealsid retitled this revision from [LLDB} Add unit tests for Editline keyboard shortcuts to [LLDB] Add unit tests for Editline keyboard shortcuts.Dec 5 2021, 2:08 PM
nealsid updated this revision to Diff 391976.Dec 5 2021, 11:51 PM

Fix casing typo in friend class declaration.

labath added a subscriber: labath.Dec 6 2021, 5:08 AM

All of the changes seem fine, but it's not clear to me what is the value of the new test. To me, it just seems like a change-detector forcing the two versions of the keybindings to stay in sync. The need for the friend declarations and everything also enforces the feeling that this test is not checking what it should be. Did you actually run into some kind of a problem/bug here that you think this test would have prevented?

Personally, I think it would be more useful if we verified the actual behavior of our editline code. We have some code here, in TestMultilineNavigation.py, and a couple of other places, but it would definitely be useful to have more of those.

All of the changes seem fine, but it's not clear to me what is the value of the new test. To me, it just seems like a change-detector forcing the two versions of the keybindings to stay in sync. The need for the friend declarations and everything also enforces the feeling that this test is not checking what it should be. Did you actually run into some kind of a problem/bug here that you think this test would have prevented?

Personally, I think it would be more useful if we verified the actual behavior of our editline code. We have some code here, in TestMultilineNavigation.py, and a couple of other places, but it would definitely be useful to have more of those.

Thank you! My thought process on the tests was that the exact binding is user facing, and a stray extra character or accidental deletion in the strings that represent the key sequences should be tested against, as well as any change that caused the shortcut to be registered incorrectly. If I used the same table in the dev & test code, it would not catch the first case, and a lost keyboard shortcut can be a visible bug, although, in this case, is easily fixable through the .editrc file.

The motivation is another change I have in progress that turns the Editline code into a table that stores the command, callback, key sequence, and help text in a table and loops over it to add them (which sounds suspiciously like this patch, I admit ;-)), and I wanted to make sure I didn't break any existing shortcuts when making that change, and some other changes I have planned, such as removing the conditional WCHAR support.

I also was on the fence about adding the test classes as friends. I could have exposed the functionality to retrieve a shortcut through our Editline class, but the test would be the only client today.

Definitely +1 to having keyboard shortcut tests as part of the Python tests. Mind if I take that as a TODO? I see how the coverage of both kinds of tests overlaps, but having the gtest tests can help find bugs earlier. Thanks again.

Random unrelated question, and if I missed docs somewhere, I apologize. Did I forget something in creating the revision that caused Harbormaster to not do arc lint or arc test? https://reviews.llvm.org/B137600. Thanks.

nealsid updated this revision to Diff 392497.Dec 7 2021, 12:15 PM
nealsid edited the summary of this revision. (Show Details)

Testing ARC, no action necessary.

labath added a comment.Dec 8 2021, 2:36 AM

Well.. I would say that the most user-facing aspect is the /action/ that happens after the user pressed the appropriate key. But at the end of the day, that doesn't matter that much -- we can (and do) test non-user-facing interfaces as well. But those tests make most sense when there is some complex logic in the code. Making sure that a table does not change by keeping another copy of the table is the very definition of a change-detector test (https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html). Like, it may have some value while you're doing the refactoring, but that value quickly drops to zero (or even becomes negative) after you complete it.

If it was a clean test, I could say "whatever, we can delete it if it becomes a burden" , but I can see several problems with the test (and the friend declaration is not the biggest of them), and I don't think it's worth trying to fix those. If you really want to work on that, then I'm not going to stop you, but I would be also perfectly happy to approve a patch that turns the keybindings into a table without an additional test.

lldb/unittests/Editline/EditlineTest.cpp
124–130

I find this hard to believe. I think it's more likely that this is working around some problem in the test itself. I can't say what is the precise problem without more investigation, but for example the synchronization around testOutputBuffer seems inadequate. GetEditlineOutput could deadlock if the read thread reads the newline before it reaches the wait call. It also does not handle spurious wakeups. And the accesses to testOutputBuffer are completely thread-unsafe.

230

This isn't right. The return type of fgetc is deliberately not char so that EOF does not conflict with an actual character.

339

You already have a const on the variable declaration. You don't need to put one on every member as well.

346–353

Why not just do the replacements in operator<< ?

448–449

Why not just Contains(kbtv)? It doesn't look like you're making use of the regex functionality.

nealsid added a comment.EditedDec 8 2021, 12:39 PM

Thank you, Pavel - will address the specific code feedback in subsequent diff to this revision.

Well.. I would say that the most user-facing aspect is the /action/ that happens after the user pressed the appropriate key. But at the end of the day, that doesn't matter that much -- we can (and do) test non-user-facing interfaces as well. But those tests make most sense when there is some complex logic in the code. Making sure that a table does not change by keeping another copy of the table is the very definition of a change-detector test (https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html). Like, it may have some value while you're doing the refactoring, but that value quickly drops to zero (or even becomes negative) after you complete it.

I'd disagree completely that the keypresses & key bindings are not user-facing.

The TOTT is interesting, but the changes I'm protecting against don't fall into that category. One reason is that, if the same table were to be used, a stray or inadvertent change could compile, pass tests, and make it to users.

Another reason is that editline's API takes varargs, which means that we rely on editline to handle all error cases correctly. I already ran into one issue where el_set(el, EL_BIND, ...) supports printing all bound keys (rather than making the caller specify which key to return a binding for), but the arguments it expects don't pass initial validation in el_set().

(these links may not be the same version as what ships on macOS or Linux distros)

https://salsa.debian.org/debian/libedit/-/blob/master/src/eln.c#L176
https://salsa.debian.org/debian/libedit/-/blob/master/src/map.c#L1315

I've never actually seen tests like the one in TOTT; I had 8 amazing years at Google, but one thing that always stuck out culturally was how every variation of "hello, world" could be turned into a conference-starting PhD thesis.

If it was a clean test, I could say "whatever, we can delete it if it becomes a burden" , but I can see several problems with the test (and the friend declaration is not the biggest of them), and I don't think it's worth trying to fix those. If you really want to work on that, then I'm not going to stop you, but I would be also perfectly happy to approve a patch that turns the keybindings into a table without an additional test.

I think it's worth it! It may not be the most important thing to LLDB users but there's clearly some code that doesn't get touched very often, and making it easier to make changes if requests come along will be a good thing. Also, for my own reasons, it is giving me some familiarity with the LLDB & LLVM code bases, especially the common code, before moving onto more core parts.

nealsid updated this revision to Diff 393200.Dec 9 2021, 9:20 AM

Patch to address CR feedback.

Thanks, Pavel.

lldb/unittests/Editline/EditlineTest.cpp
124–130

I spent a few hours on this before sending the initial patch; it may be some interaction with the file APIs and the Psuedoterminal class we have in LLVM. I could not debug anything it was doing that was leading to this behavior, but reliably saw EOF being sent before data that had been written to the stream. I also tried various things like flushing it manually, which did not work, but sleeping before closing the secondary FD enabled all data to be read on the test side. If you have some time, I'd be happy to have a gchat or screen sharing session and see if we can brainstorm something.

Regarding the synchronization, good call; I built on top of the existing threading, which did not require synchronizing reading the output of editline, because the actual output was never used. Now, I just call provide a method to call thread::join for tests that need to read the output to ensure it's all been written.

230

Ah, yeah, I changed it back. For some reason I thought EOF was sent as an actual EOF character like CTRL-D, not a special platform-specific integer.

339

I like const on the struct fields since it expresses the intent (a compile-time data structure) better, and, without it, if the array variable decl type was changed to remove the const, modifying the fields would be permitted.

346–353

I started that approach, but doing it at compile time seems easier because the output can be matched to the specific test by looking through the table, and...it's less code at runtime :)

nealsid updated this revision to Diff 393228.Dec 9 2021, 11:19 AM

Remove extra header and using decl

nealsid marked an inline comment as done.Dec 9 2021, 11:21 AM
labath added inline comments.Dec 13 2021, 10:41 AM
lldb/include/lldb/Host/Editline.h
392–393

This also isn't something we normally do. I'd recommend just making a function like DumpKeyBindings() or something. It's true that it would only be used in tests right now, but I can imagine some day adding an lldb command to dump current bindings, so it does not seem completely out of place. If you wanted to, you could even add it right now and do your test that way.

lldb/unittests/Editline/EditlineTest.cpp
86

const on a return value is useless

124–130

Well.. I tried running this locally, and the test passed just fine even with this code deleted. I don't think we should commit code like this without understanding the problem in more detail. (Like, if it's a problem with llvm code, then lets fix that; if it's a problem with libc code (I doubt that), then let's file a bug before working around it.)If you want to discuss this in a more interactive fashion, you can find me on the #lldb channel @ irc.oftc.net.

339

Well, we definitely have have different views on redundancy. I could write a PhD thesis on why I think this is wrong, but right now, let me just say this: it isn't consistent with how we define const structs anywhere else in llvm or lldb. Here's some examples: https://github.com/llvm/llvm-project/blob/e356027016c6365b3d8924f54c33e2c63d931492/lldb/unittests/Process/Utility/RegisterContextTest.cpp#L20, https://github.com/llvm/llvm-project/blob/165545c7a431aa3682fd9468014771c1c5228226/lldb/unittests/Process/gdb-remote/GDBRemoteCommunicationTest.cpp#L55, https://github.com/llvm/llvm-project/blob/e356027016c6365b3d8924f54c33e2c63d931492/llvm/unittests/Support/DJBTest.cpp#L21. (Full disclosure: I wrote the last two; while searching for examples, I also found a lot patterns like this which did not have any const annotation whatsoever.)

346–353

Ok, I can accept that.

Thanks, Pavel. Some comments responded to, will take another look at the buffered/unbuffered I/O issue and address the remaining comments.

lldb/include/lldb/Host/Editline.h
392–393

Can you clarify this? I see a lot of friend class declarations in the LLDB codebase, both prod and test.

A command to dump key bindings could be useful in the future, but the requirement of parsing editline's stdout and manipulating it into some sort of structure for an API sounds like a maintenance tax until that command is implemented.

lldb/unittests/Editline/EditlineTest.cpp
124–130

Cool, I'll take another look..what platform were you running on?

339

I understand it may not be done this way elsewhere, but it still captures the intent of the data structure better. The idea of having a dependency on the declaration to enforce constness, rather than in the definition, doesn't reflect the point of the data structure.

labath added inline comments.Dec 14 2021, 6:22 AM
lldb/include/lldb/Host/Editline.h
392–393

We have a lot (too many) of friend declarations, but I'm not aware of any case where a production class befriends a test class (particularly one with a gtest-generated name). I know we sometimes make things protected (even though they could be private) for the sake of testing. That approach would also work here, but I think the new function would be better.

I didn't mean to parse the generated output. The function basically just be a wrapper for el_set(el, EL_BIND, nullptr) (I'm assuming that output is suitable for human consumption). The lldb command would then just call this function when invoked.

lldb/unittests/Editline/EditlineTest.cpp
124–130

It was x86_64-linux glibc-2.33 kernel-5.10.

339

Well, that's the difference. I don't see the constness as a necessary/intrinsic property of the data structure. I just view it as a glorified container -- a nicer version of std::tuple. For all it cares someone may want to dynamically generate test cases. It's up to the user to say what he wants to do with it. And the user -- the actual variable -- wants to provide a compile time list of test cases, and there it's perfectly natural to be const.

And in general, it's very rare to see const on a data member in c++, so upon seeing that, my mind automatically goes searching for some subtle reason for why it has to be there -- and then gets confused when it doesn't find any.