This is an archive of the discontinued LLVM Phabricator instance.

[lldb] Remove forward-connect ability from lldb-server tests
Changes PlannedPublic

Authored by labath on Jan 18 2022, 6:26 AM.

Details

Summary

Forward connections are impossible to set up reliably, which is why the
existing implementation was relying on retries (and crossing fingers).
Reverse connections are reliable and can be used in all situations that
the test suite is currently being used.

To compensate for the loss of forward-connect test coverage, I create a
dedicated test using the forward connection method. It still relies on
retries, but I've tried to make it as robust as I can, and at least it's
just one test instead of the whole test suite.

Diff Detail

Event Timeline

labath requested review of this revision.Jan 18 2022, 6:26 AM
labath created this revision.
Herald added a project: Restricted Project. · View Herald TranscriptJan 18 2022, 6:26 AM

Omair can comment on how useful this is for SVE testing.

I do use this to run the memory tagging tests on qemu, we don't have real hardware yet. Let me see what extra steps I need to run them within qemu instead.

Thanks for the info. If you are using it then I think we can come up with something would support your use case. TBH, I am kinda surprised this even works (are you sure you're running lldb-*server* tests and not just the "normal" lldb ones?) -- I haven't seen any patches in this area, so I expected it to have bitrotted.

The main thing which is in my way me is the (flaky) forward-connection mode. Would it be ok if we used reverse connections here? I am assuming we don't have to worry about firewalls with qemu..

DavidSpickett added a comment.EditedJan 19 2022, 3:08 AM

I'm running lldb/test/API/tools/lldb-server/memory-tagging/TestGdbRemoteMemoryTagging.py.

If the details matter any...

In qemu:

$ ./build-cross/bin/lldb-server platform --server --listen 0.0.0.0:54321

On the host:

$ ./bin/lldb-dotest --platform-name remote-linux --platform-url connect://<VM IP>:54321 --platform-working-dir /tmp/test_lldb -p TestGdbRemoteMemoryTagging.py --arch aarch64

I have qemu setup so that any port lldb-server picks should be accessible from the host and the VM is just running the usual background linux stuff plus lldb-server. Not a lot to use up ports.

Can you remind me what forward/reverse mean here? I guess that forward is us telling the remote the port to use, which might already be taken on the remote itself. Then reverse is the remote launching the lldb-server on a port it chooses, then telling us what it chose. The latter should be fine for my setup.

This doesnt interfere with LLDB tests for SVE. Rather I have not used it in a while for SVE tests I ahve using setup similar to to one David has explained above.

@labath Do you know if android adb based tests are being run anywhere? Are we still maintaining this support and does it work?

I'm running lldb/test/API/tools/lldb-server/memory-tagging/TestGdbRemoteMemoryTagging.py.

If the details matter any...

In qemu:

$ ./build-cross/bin/lldb-server platform --server --listen 0.0.0.0:54321

On the host:

$ ./bin/lldb-dotest --platform-name remote-linux --platform-url connect://<VM IP>:54321 --platform-working-dir /tmp/test_lldb -p TestGdbRemoteMemoryTagging.py --arch aarch64

Got it. Thanks.

I have qemu setup so that any port lldb-server picks should be accessible from the host and the VM is just running the usual background linux stuff plus lldb-server. Not a lot to use up ports.

The tricky part is that even individual parallel test runs (if you run the whole test suite, not just a single test like you did above) can race with each other.

Can you remind me what forward/reverse mean here? I guess that forward is us telling the remote the port to use, which might already be taken on the remote itself.

Correct.

Then reverse is the remote launching the lldb-server on a port it chooses, then telling us what it chose.

Not quite. In "reverse mode" it is the client (in this case -- the test suite) who is choosing a port and listening on it. lldb-server gets an address and it connects to it.

We also have the method which you described (we probably have more connection methods than we should). That one is also reliable (though it gets fuzzy with IPv6, as then the port number alone does not uniquely identify an endpoint), but it has more moving parts, and it is not implemented in the test suite -- just in the client, which we don't use for these tests.

One of the goals I am trying to achieve here is actually to be able to test the various connection methods at the lldb-server level, but for that I need to untangle the connection code from the rest of the test suite setup. And ideally I would only support one method (the simplest one) for the remote connections and have mark other tests @skipIfRemote -- the reason for that is that we'd need different code to (e.g.) read the server-selected port from a remote machine than we would for the local one, so this would be more of a test of the test suite than the server itself.

This doesnt interfere with LLDB tests for SVE. Rather I have not used it in a while for SVE tests I ahve using setup similar to to one David has explained above.

If you're using the setup that David mentioned, then this would break that.

I am going to try to make a patch to use reverse connect for these tests instead.

@labath Do you know if android adb based tests are being run anywhere? Are we still maintaining this support and does it work?

I would guess "no", but I don't know, really. I created this patch to find that out myself. :)

labath updated this revision to Diff 415120.Mar 14 2022, 9:35 AM

Remove just the forward-connect logic instead.

Herald added a project: Restricted Project. · View Herald TranscriptMar 14 2022, 9:35 AM
labath retitled this revision from [lldb] Remove remote testing ability from lldb**-server** tests to [lldb] Remove forward-connect ability from lldb-server tests.Mar 14 2022, 9:36 AM
labath edited the summary of this revision. (Show Details)
labath added a reviewer: DavidSpickett.

Sorry about the delay. Here's the version which switches to reverse connect everywhere. Please try it out.

FYI I am testing this locally, didn't work straight away but I think this might be networking issues on my part. Taking some time to check that.

I was able to use the commands I showed before successfully but only after making these changes:

--- a/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py
+++ b/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py
@@ -231,14 +231,15 @@ class GdbRemoteTestCaseBase(Base):
         return target.GetByteOrder()

     def launch_debug_monitor(self, attach_pid=None, logfile=None):
-        family, type, proto, _, addr = socket.getaddrinfo("localhost", 0, proto=socket.IPPROTO_TCP)[0]
+        info = socket.getaddrinfo("0.0.0.0", 0, proto=socket.IPPROTO_TCP)
+        family, type, proto, _, addr = info[0]
         sock = socket.socket(family, type, proto)
         sock.settimeout(self.DEFAULT_TIMEOUT)

         sock.bind(addr)
         sock.listen(1)
         addr = sock.getsockname()
-        self.connect_address = "[{}]:{}".format(*addr)
+        self.connect_address = "[{}]:{}".format(socket.getfqdn(), addr[1])

This is a hack obviously but it fixes the issues my particular setup has:

  • By asking for "localhost" we get the IPV6 interface first and the QEMU VM doesn't have IPV6 to the host. (I'm sure it could with some extra setup)
  • "0.0.0.0" fixes that issue since it's an IPV4 special address. (and means the port can be connected to externally?)
  • Then we were telling lldb-server in the VM to connect to a local port, not to the port on the host. getfqdn fixes that.

So I end up with the host side listening to 0.0.0.0:<port> and lldb-server in the VM connecting to <host's hostname>:<port> and that works.

If we want this scenario to work (and it's not a dodgy network setup on my part) perhaps we could take a hint from how we're connected to the initial platform?

Connecting to remote platform 'remote-linux' at 'connect://<VM IPv4 address>:54321'

So here we could assume that IPv4 is ok and since the hostname isn't any sort of localhost we need to tell the lldb-server to connect to the hostname of the machine running the tests.
(though that assumes you can in fact connect to that from the VM, with what I have, you can)

(Thanks for trying this out, I'm still thinking about the best way to move this forward.)

labath planned changes to this revision.Mar 30 2022, 8:43 AM