[platform process list] add a flag for showing the processes of all users

[wallace]

Summary

For context: https://reviews.llvm.org/D68293

We need a way to show all the processes on android regardless of the user id.
When you run platform process list, you only see the processes with the same user as the user that launched lldb-server. However, it's quite useful to see all the processes, though, and it will lay a foundation for full apk debugging support from lldb.

Before:

PID    PARENT USER       TRIPLE                   NAME
====== ====== ========== ======================== ============================
3234   1                 aarch64-unknown-linux-android adbd
8034   3234              aarch64-unknown-linux-android sh
9096   3234              aarch64-unknown-linux-android sh
9098   9096              aarch64-unknown-linux-android lldb-server
(lldb) ^D

Now:

(lldb) platform process list -x
205 matching processes were found on "remote-android"
PID    PARENT USER       TRIPLE                   NAME
====== ====== ========== ======================== ============================
1      0                                          init
524    1                                          init
525    1                                          init
531    1                                          ueventd
568    1                                          logd
569    1                 aarch64-unknown-linux-android servicemanager
570    1                 aarch64-unknown-linux-android hwservicemanager
571    1                 aarch64-unknown-linux-android vndservicemanager
577    1                 aarch64-unknown-linux-android qseecomd
580    577               aarch64-unknown-linux-android qseecomd
...
23816  979                                        com.android.providers.calendar
24600  979                                        com.verizon.mips.services
27888  979                                        com.hualai
28043  2378                                       com.android.chrome:sandboxed_process0
31449  979                                        com.att.shm
31779  979                                        com.samsung.android.authfw
31846  979                                        com.samsung.android.server.iris
32014  979                                        com.samsung.android.MtpApplication
32045  979                                        com.samsung.InputEventApp

[wallace]

Btw, I couldn't find a better short command than 'x', but if you have a better suggestion, i'd be happy to accept it :)

[labath]

I think the fact that all of our options *must* have a short version is a pretty serious deficiency in our option parsing system. It results in a lot of unintuitive options that nobody uses because they can't guess what they stand for. But anyway, -x seems as good as any, as all the good alternatives are already taken.

For testing this, I think the best option would be something similar to the gdb-client suite (test/testcases/functionalities/gdb_remote_client), which allows you to mock the remote end of a connection. It is currently geared toward talking to the gdb server (and not the platform), but as the underlying protocol is the same, I am hoping that it can be repurposed to this use case fairly easily. I am imagining a flow where you create a mock server, configure it to respond appropriately to any packet that lldb issues, and then run e.g. "platform select remote-linux", "platform connect connect://localhost:%d", "platform proces list --all-users".

Setting that up is a bit overkill for a simple patch as this one, but as it looks like you're going to be doing a lot of work in this area in the near future, I think it makes sense to set up some testing infrastructure early.

[wallace]

added a basic python test

It was a nice learning experience and fortunately there was no need to add any intrusive code

[labath]

LG, thanks for the test.

lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestPlatformClient.py

31

Maybe also check that some of the values from the mock response made it through? (e.g., pid, process name)

[wallace]

lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestPlatformClient.py

31

I'm adding that in the other diffs, but i'll make stricter checks anyway

[shafik]

@wallace it looks like TestPlatformClient.py is failing on osx see green dragon log here: http://lab.llvm.org:8080/green/view/LLDB/job/lldb-cmake/2317/testReport/junit/lldb-Suite/functionalities_gdb_remote_client/TestPlatformClient_py/

======================================================================
FAIL: test_process_list_with_all_users (TestPlatformClient.TestPlatformClient)
   Test connecting to a remote linux platform
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestPlatformClient.py", line 30, in test_process_list_with_all_users
    startstr="1 matching process was found", endstr="process" + os.linesep)
  File "/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2309, in expect
    inHistory=inHistory)
AssertionError: False is not True : Command 'platform process list -x
Error output:
error: no processes were found on the "remote-linux" platform
' returns successfully

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestPlatformClient.py", line 34, in test_process_list_with_all_users
    self.runCmd("platform disconnect")
  File "/Users/buildslave/jenkins/workspace/lldb-cmake/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2068, in runCmd
    msg if (msg) else CMD_MSG(cmd))
AssertionError: False is not True : Command 'platform disconnect
Error output:
error: not connected to 'remote-linux'
' returns successfully
Config=x86_64-/Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/bin/clang-10
----------------------------------------------------------------------
Ran 1 test in 0.043s

RESULT: FAILED (0 passes, 1 failures, 0 errors, 0 skipped, 0 expected failures, 0 unexpected successes)

[wallace]

damn, i'll rebase and try to fix it

[stella.stamenova]

The windows LLDB bot also has the same failure: http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja/builds/9600

Please make sure your fix works for both.

[shafik]

I reverted this change b/c the green dragon bots have been red for too long.

When you fix the problem please make sure you add

Differential Revision: https://reviews.llvm.org/D68354

to your commit message.

[wallace]

this was reverted

[wallace]

now it should work, the test pass

[wallace]

• update

[wallace]

another try

[wallace]

adding some logging for exceptions in the mock server

[wallace]

after the logging finally found the bug, it's was an encoding issue due to a different python version

[wallace]

i think i finally wrote something that works python 2 and 3

lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestPlatformClient.py

+import lldb
+import binascii
+import os
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test.decorators import *
+from gdbclientutils import *
+
+
+class TestPlatformClient(GDBRemoteTestBase):
+
+    def test_process_list_with_all_users(self):
+        """Test connecting to a remote linux platform"""
+
+        class MyResponder(MockGDBServerResponder):
+            def qfProcessInfo(self, packet):
+                if "all_users:1" in packet:
+                    return "pid:10;ppid:1;uid:1;gid:1;euid:1;egid:1;name:" + binascii.hexlify("/a/test_process".encode()).decode() + ";"
+                else:
+                    return "E04"
+
+        self.server.responder = MyResponder()
+
+        self.runCmd("platform select remote-linux")
+
+        try:
+            self.runCmd("platform connect connect://localhost:%d" %
+                        self.server.port)
+            self.assertTrue(self.dbg.GetSelectedPlatform().IsConnected())
+            self.expect("platform process list -x",
+                        substrs=["1 matching process was found", "test_process"])
+            self.expect("platform process list",
+                        error=True,
+                        substrs=["error: no processes were found on the \"remote-linux\" platform"])
+        finally:
+            self.dbg.GetSelectedPlatform().DisconnectRemote()

lldb/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py

 import os
 import os.path
 import threading
 import socket
 import lldb
+import binascii
+import traceback
 from lldbsuite.support import seven
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbtest_config
 
 
 def checksum(message):
     """
     Calculate the GDB server protocol checksum of the message.
def respond(self, packet):
             threadnum = int (packet[15:], 16)
             return self.threadStopInfo(threadnum)
         if packet == "QThreadSuffixSupported":
             return self.QThreadSuffixSupported()
         if packet == "QListThreadsInStopReply":
             return self.QListThreadsInStopReply()
         if packet.startswith("qMemoryRegionInfo:"):
             return self.qMemoryRegionInfo()
+        if packet == "qQueryGDBServer":
+            return self.qQueryGDBServer()
+        if packet == "qHostInfo":
+            return self.qHostInfo()
+        if packet == "qGetWorkingDir":
+            return self.qGetWorkingDir()
+        if packet == "qsProcessInfo":
+            return self.qsProcessInfo()
+        if packet.startswith("qfProcessInfo"):
+            return self.qfProcessInfo(packet)
 
         return self.other(packet)
 
+    def qsProcessInfo(self):
+        return "E04"
+
+    def qfProcessInfo(self, packet):
+        return "E04"
+
+    def qGetWorkingDir(self):
+        return "2f"
+
+    def qHostInfo(self):
+        return "ptrsize:8;endian:little;"
+
+    def qQueryGDBServer(self):
+        return "E04"
+
     def interrupt(self):
         raise self.UnexpectedPacketException()
 
     def cont(self):
         raise self.UnexpectedPacketException()
 
     def vCont(self, packet):
         raise self.UnexpectedPacketException()
-    
     def readRegisters(self):
         return "00000000" * self.registerCount
 
     def readRegister(self, register):
         return "00000000"
 
     def writeRegisters(self, registers_hex):
         return "OK"
def _run(self):
         data = None
         while True:
             try:
                 data = seven.bitcast_to_string(self._client.recv(4096))
                 if data is None or len(data) == 0:
                     break
                 self._receive(data)
             except Exception as e:
+                print("An exception happened when receiving the response from the gdb server. Closing the client...")
+                traceback.print_exc()
                 self._client.close()
                 break
 
     def _receive(self, data):
         """
         Collects data, parses and responds to as many packets as exist.
         Any leftover data is kept for parsing the next time around.
         """
def _handlePacket(self, packet):
             self._client.sendall(seven.bitcast_to_bytes(framed))
 
     PACKET_ACK = object()
     PACKET_INTERRUPT = object()
 
     class InvalidPacketException(Exception):
         pass
 
-
 class GDBRemoteTestBase(TestBase):
     """
     Base class for GDB client tests.
 
     This class will setup and start a mock GDB server for the test to use.
     It also provides assertPacketLogContains, which simplifies the checking
     of packets sent by the client.
     """

lldb/source/Commands/CommandObjectPlatform.cpp

Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_arg,
       case 'A':
         show_args = true;
         break;
 
       case 'v':
         verbose = true;
         break;
 
+      case 'x':
+        match_info.SetMatchAllUsers(true);
+        break;
+
       default:
         llvm_unreachable("Unimplemented option");
       }
 
       return error;
     }
 
     void OptionParsingStarting(ExecutionContext *execution_context) override {

lldb/source/Commands/Options.td

def platform_process_list_egid : Option<"egid", "G">, GroupRange<2, 6>,
     Arg<"UnsignedInteger">, Validator<"&posix_validator">,
     Desc<"Find processes that have a matching effective group ID.">;
   def platform_process_list_arch : Option<"arch", "a">, GroupRange<2, 6>,
     Arg<"Architecture">,
     Desc<"Find processes that have a matching architecture.">;
   def platform_process_list_show_args : Option<"show-args", "A">,
     GroupRange<1, 6>,
     Desc<"Show process arguments instead of the process executable basename.">;
+  def platform_process_list_all_users: Option<"all-users", "x">,
+    GroupRange<1,6>,
+    Desc<"Show processes matching all user IDs.">;
   def platform_process_list_verbose : Option<"verbose", "v">, GroupRange<1, 6>,
     Desc<"Enable verbose output.">;
 }
 
 let Command = "platform process attach" in {
   def platform_process_attach_plugin : Option<"plugin", "P">, Arg<"Plugin">,
     Desc<"Name of the process plugin you want to use.">;
   def platform_process_attach_pid : Option<"pid", "p">, Group<1>, Arg<"Pid">,

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationClient.cpp

if (!match_info.MatchAllProcesses()) {
       if (match_info.GetProcessInfo().GroupIDIsValid())
         packet.Printf("gid:%u;", match_info.GetProcessInfo().GetGroupID());
       if (match_info.GetProcessInfo().EffectiveUserIDIsValid())
         packet.Printf("euid:%u;",
                       match_info.GetProcessInfo().GetEffectiveUserID());
       if (match_info.GetProcessInfo().EffectiveGroupIDIsValid())
         packet.Printf("egid:%u;",
                       match_info.GetProcessInfo().GetEffectiveGroupID());
-      if (match_info.GetProcessInfo().EffectiveGroupIDIsValid())
-        packet.Printf("all_users:%u;", match_info.GetMatchAllUsers() ? 1 : 0);
       if (match_info.GetProcessInfo().GetArchitecture().IsValid()) {
         const ArchSpec &match_arch =
             match_info.GetProcessInfo().GetArchitecture();
         const llvm::Triple &triple = match_arch.GetTriple();
         packet.PutCString("triple:");
         packet.PutCString(triple.getTriple());
         packet.PutChar(';');
       }