Add target.xml support for qXfer request.

[PatriosTheGreat]

Summary

Requesting registers one by one takes a while in our project.
We want to get rid of it by using target.xml.

[jarin]

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

2819

As discussed offline, this should not be hard-coded, register context provides register count.

2825

Fill all the relevant fields, please (everything that the qRegisterInfo handler provides).

[labath]

Yeah, we should fill in the other attributes too. As that will probably noticeably increase the amount of code, it would be good to put this stuff into a separate function (or two..). For a test, you could create some kind of a lldb-server test (packages/Python/lldbsuite/test/tools/lldb-server/). I am imagining something that will check that this parses as valid xml, and then does some kind of basic validation on the contents.

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

2832

"target.xml" would be a better buffer name than __FUNCTION__.

[labath]

I think this is looking pretty good overall. I just have a bunch of stylistic remarks...

lldb/packages/Python/lldbsuite/test/tools/lldb-server/registers-target-xml-reading/main.cpp

1 ↗

(On Diff #243579)

I don't think the choice of inferior really matters here. This code could just be like int main(){}

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerCommon.cpp

849

You can put this outside the #ifdef

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

380–459

These should return a llvm::StringRef to avoid copies, and take the argument as a reference to convey non-nullness.

2789–2913

Please move this into a separate function

2791–2795

This check could be factored to the front of the function (the svr4 branch does currently not have it, but it most likely should).

2844–2846

We generally don't put curly braces around statements which fit on a single line.

2874–2897

Make a utility function (lambda?) for this?

[labath]

Just a couple of comments to make this code more llvm-y. Without those, we have just moved the place which constructs the temporary std::string. I haven't highlighted all the uses, but overall I'd recommend using the Format function or something similar instead of Printf, as the latter can be error-prone (because of the need to pass the right PRIx macro to get the integer size right) and/or inefficient (need to do the .str().c_str() dance on StringRefs).

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

1719

Despite the name, PutCString actually takes a StringRef, so you can just drop the .str().c_str() thingy. In fact, I'd consider just writing response << "encoding:" << encoding << ';';

2791

no braces

2801

Similarly, response << "encoding='" << encoding << "' ", or response.Format("encoding='{0}'", encoding) would be shorter, and avoid string copying.

[labath]

Just one more thing I didn't catch in the previous round :(.

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

461–481

Sorry I wasn't clear about this, but my idea was to make a _single_ utility function for this. If you pass in the uint32_t* to the function instead of the whole RegisterInfo struct, then you can use the same code in both places..

[labath]

Looks good. Thanks for doing this.

[PatriosTheGreat]

Hi Pavel,
Thank you for review.
Could you also commit this to master, since I don't have commit access?

[labath]

Committed as aedc196101e33bd58f7443c5b93398418ce55edf. I've updated the commit message to include a better description of what is being changed.

[omjavaid]

This breaks LLDB AArch64 Linux buildbot http://lab.llvm.org:8011/builders/lldb-aarch64-ubuntu/builds/1713

[omjavaid]

I have reverted this change temporarily. Please look at the test failure. I ll also do the same at my end. Thanks!

[jarin]

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

465

I think this is not correct: target.xml expects this to be decimal. (This is different from qRegisterInfo, which expects hex.)

[jarin]

lldb/packages/Python/lldbsuite/test/tools/lldb-server/registers-target-xml-reading/TestGdbRemoteTargetXmlPacket.py

69 ↗

(On Diff #244934)

Why don't you test all the fields here?

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

2801

Nit: Now it is a funny mixture of operator<<, Printfs and PutCString. Is there a reason not to use << for everything? (I guess PutHex8 can't be easily done with <<, but everything else can?)

[jarin]

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

2833

I know this is copy&paste from qRegisterInfo, but should not this be if (reg_info->invalidate_regs && reg_info->invalidate_regs[0] != LLDB_INVALID_REGNUM)?

[labath]

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

2801

Yeah, I'm all for using anything else than printf (operator<<, the Format function, etc.). I just didn't feel like raising this point at that time. :/

[PatriosTheGreat]

Hi Muhammad,

Thank you for finding this issue.
After I fix that, could I update current review or should I create new one?

[labath]

Please update the current one to keep the context together.

[PatriosTheGreat]

In this revision I fix value_regnums and invalidate_regnums serialization in target.xml.
However I'm not so sure that this is the cause of problem with ARM tests.

From tests log it seems like there are some unexpected value in register.
Is it possible to get logs output from this test run http://lab.llvm.org:8011/builders/lldb-aarch64-ubuntu/builds/1713/steps/test/logs/stdio that would help us a lot to fix it.

[omjavaid]

I have tested this not sure why it actually failed on the buildbot. The test seems to pass independently. Lets see if the new version fixes the issue.

[PatriosTheGreat]

Hi Muhammad,
Thank you for review.
Could you or Pavel commit this to master, since I don't have commit access?

[omjavaid]

In D74217#1882539, @PatriosTheGreat wrote:

Hi Muhammad,
Thank you for review.
Could you or Pavel commit this to master, since I don't have commit access?

Hi Patrios,

There was some problem with our buildbot and it went down overnight. I have put it back in place and will commit your patch today.

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerCommon.cpp

GDBRemoteCommunicationServerCommon::Handle_qSupported(
 
   response.PutCString(";QStartNoAckMode+");
   response.PutCString(";QThreadSuffixSupported+");
   response.PutCString(";QListThreadsInStopReply+");
   response.PutCString(";qEcho+");
 #if defined(__linux__) || defined(__NetBSD__)
   response.PutCString(";QPassSignals+");
   response.PutCString(";qXfer:auxv:read+");
+  response.PutCString(";qXfer:features:read+");
   response.PutCString(";qXfer:libraries-svr4:read+");
 #endif
 
   return SendPacketNoLock(response.GetString());
 }
 
 GDBRemoteCommunication::PacketResult
 GDBRemoteCommunicationServerCommon::Handle_QThreadSuffixSupported(

lldb/source/Plugins/Process/gdb-remote/GDBRemoteCommunicationServerLLGS.cpp

if (swap) {
     for (i = buf_size - 1; i >= 0; i--)
       response.PutHex8(buf[i]);
   } else {
     for (i = 0; i < buf_size; i++)
       response.PutHex8(buf[i]);
   }
 }
 
 static void WriteRegisterValueInHexFixedWidth(
     StreamString &response, NativeRegisterContext &reg_ctx,
     const RegisterInfo &reg_info, const RegisterValue *reg_value_p,
     lldb::ByteOrder byte_order) {
   RegisterValue reg_value;
   if (!reg_value_p) {
     Status error = reg_ctx.ReadRegister(&reg_info, reg_value);
     if (error.Success())
       reg_value_p = &reg_value;
     // else log.
   }
 
   if (reg_value_p) {
     AppendHexValue(response, (const uint8_t *)reg_value_p->GetBytes(),
                    reg_value_p->GetByteSize(),
                    byte_order == lldb::eByteOrderLittle);
   } else {
     // Zero-out any unreadable values.
     if (reg_info.byte_size > 0) {
       std::basic_string<uint8_t> zeros(reg_info.byte_size, '\0');
       AppendHexValue(response, zeros.data(), zeros.size(), false);
     }
   }
 }
 
 static llvm::Expected<json::Object>
 GetRegistersAsJSON(NativeThreadProtocol &thread) {
   Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_THREAD));
 
   NativeRegisterContext& reg_ctx = thread.GetRegisterContext();
 
   json::Object register_object;
 
 #ifdef LLDB_JTHREADSINFO_FULL_REGISTER_SET
   // Expedite all registers in the first register set (i.e. should be GPRs)
   // that are not contained in other registers.
   const RegisterSet *reg_set_p = reg_ctx_sp->GetRegisterSet(0);
   if (!reg_set_p)
     return llvm::make_error<llvm::StringError>("failed to get registers",
                                                llvm::inconvertibleErrorCode());
   for (const uint32_t *reg_num_p = reg_set_p->registers;
        *reg_num_p != LLDB_INVALID_REGNUM; ++reg_num_p) {
     uint32_t reg_num = *reg_num_p;
 #else
   // Expedite only a couple of registers until we figure out why sending
   // registers is expensive.
   static const uint32_t k_expedited_registers[] = {
       LLDB_REGNUM_GENERIC_PC, LLDB_REGNUM_GENERIC_SP, LLDB_REGNUM_GENERIC_FP,
       LLDB_REGNUM_GENERIC_RA, LLDB_INVALID_REGNUM};
 
   for (const uint32_t *generic_reg_p = k_expedited_registers;
        *generic_reg_p != LLDB_INVALID_REGNUM; ++generic_reg_p) {
     uint32_t reg_num = reg_ctx.ConvertRegisterKindToRegisterNumber(
         eRegisterKindGeneric, *generic_reg_p);
     if (reg_num == LLDB_INVALID_REGNUM)
       continue; // Target does not support the given register.
 #endif
 
     const RegisterInfo *const reg_info_p =
         reg_ctx.GetRegisterInfoAtIndex(reg_num);
     if (reg_info_p == nullptr) {
       LLDB_LOGF(log,
                 "%s failed to get register info for register index %" PRIu32,
                 __FUNCTION__, reg_num);
       continue;
     }
 
     if (reg_info_p->value_regs != nullptr)
       continue; // Only expedite registers that are not contained in other
                 // registers.
 
     RegisterValue reg_value;
     Status error = reg_ctx.ReadRegister(reg_info_p, reg_value);
     if (error.Fail()) {
       LLDB_LOGF(log, "%s failed to read register '%s' index %" PRIu32 ": %s",
                 __FUNCTION__,
                 reg_info_p->name ? reg_info_p->name : "<unnamed-register>",
                 reg_num, error.AsCString());
       continue;
     }
 
     StreamString stream;
     WriteRegisterValueInHexFixedWidth(stream, reg_ctx, *reg_info_p,
                                       &reg_value, lldb::eByteOrderBig);
 
     register_object.try_emplace(llvm::to_string(reg_num),
                                 stream.GetString().str());
   }
 
   return register_object;
 }
 
 static const char *GetStopReasonString(StopReason stop_reason) {
   switch (stop_reason) {
   case eStopReasonTrace:
     return "trace";
   case eStopReasonBreakpoint:
     return "breakpoint";
   case eStopReasonWatchpoint:
     return "watchpoint";
   case eStopReasonSignal:
     return "signal";
   case eStopReasonException:
     return "exception";
   case eStopReasonExec:
     return "exec";
   case eStopReasonInstrumentation:
   case eStopReasonInvalid:
   case eStopReasonPlanComplete:
   case eStopReasonThreadExiting:
case eEncodingIEEE754:
     break;
   case eEncodingVector:
     response.PutCString("encoding:vector;");
     break;
   default:
     break;
   }
 
   switch (reg_info->format) {
   case eFormatBinary:
     response.PutCString("format:binary;");
     break;
   case eFormatDecimal:
     response.PutCString("format:decimal;");
     break;
   case eFormatHex:
     response.PutCString("format:hex;");
for (auto const &library : *library_list) {
       response.Printf("lm=\"0x%" PRIx64 "\" ", library.link_map);
       response.Printf("l_addr=\"0x%" PRIx64 "\" ", library.base_addr);
       response.Printf("l_ld=\"0x%" PRIx64 "\" />", library.ld_addr);
     }
     response.Printf("</library-list-svr4>");
     return MemoryBuffer::getMemBufferCopy(response.GetString(), __FUNCTION__);
   }
 
+  if (object == "features" && annex == "target.xml") {
+    // Make sure we have a valid process.
+    if (!m_debugged_process_up ||
+        (m_debugged_process_up->GetID() == LLDB_INVALID_PROCESS_ID)) {
+      return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                     "No process available");
+    }
+
+    // Ensure we have a thread.
+    NativeThreadProtocol *thread = m_debugged_process_up->GetThreadAtIndex(0);
+    if (!thread)
+      return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                     "No thread available");
+
+    // Get the register context for the first thread.
+    NativeRegisterContext &reg_context = thread->GetRegisterContext();
+
+    StreamString response;
+
+    response.Printf("<?xml version=\"1.0\"?>");
+    response.Printf("<target version=\"1.0\">");
+
+    response.Printf("<architecture>%s</architecture>",
+                    m_debugged_process_up->GetArchitecture()
+                        .GetTriple()
+                        .getArchName()
+                        .str()
+                        .c_str());
+
+    response.Printf("<feature>");
+    const int registersCount = 128;
+    for (int reg_index = 0; reg_index < registersCount; reg_index++) {
+      const RegisterInfo *reg_info =
+          reg_context.GetRegisterInfoAtIndex(reg_index);
+
+      response.Printf(
+          "<reg name=\"%s\" bitsize=\"%u\" offset=\"%u\" regnum=\"%d\" />",
+          reg_info->name, reg_info->byte_size * 8, reg_info->byte_offset,
+          reg_index);
+    }
+
+    response.Printf("</feature>");
+    response.Printf("</target>");
+    return MemoryBuffer::getMemBufferCopy(response.GetString(), __FUNCTION__);
+  }
+
   return llvm::make_error<PacketUnimplementedError>(
       "Xfer object not supported");
 }
 
 GDBRemoteCommunication::PacketResult
 GDBRemoteCommunicationServerLLGS::Handle_qXfer(
     StringExtractorGDBRemote &packet) {
   SmallVector<StringRef, 5> fields;
   // The packet format is "qXfer:<object>:<action>:<annex>:offset,length"
   StringRef(packet.GetStringRef()).split(fields, ':', 4);
   if (fields.size() != 5)
     return SendIllFormedResponse(packet, "malformed qXfer packet");
   StringRef &xfer_object = fields[1];
   StringRef &xfer_action = fields[2];
   StringRef &xfer_annex = fields[3];
   StringExtractor offset_data(fields[4]);
   if (xfer_action != "read")
     return SendUnimplementedResponse("qXfer action not supported");
   // Parse offset.
   const uint64_t xfer_offset =
       offset_data.GetHexMaxU64(false, std::numeric_limits<uint64_t>::max());
   if (xfer_offset == std::numeric_limits<uint64_t>::max())
     return SendIllFormedResponse(packet, "qXfer packet missing offset");
   // Parse out comma.
   if (offset_data.GetChar() != ',')
     return SendIllFormedResponse(packet,
                                  "qXfer packet missing comma after offset");
   // Parse out the length.
   const uint64_t xfer_length =
       offset_data.GetHexMaxU64(false, std::numeric_limits<uint64_t>::max());
   if (xfer_length == std::numeric_limits<uint64_t>::max())
     return SendIllFormedResponse(packet, "qXfer packet missing length");
 
   // Get a previously constructed buffer if it exists or create it now.
   std::string buffer_key = (xfer_object + xfer_action + xfer_annex).str();
   auto buffer_it = m_xfer_buffer_map.find(buffer_key);
   if (buffer_it == m_xfer_buffer_map.end()) {
     auto buffer_up = ReadXferObject(xfer_object, xfer_annex);
     if (!buffer_up)
       return SendErrorResponse(buffer_up.takeError());
     buffer_it = m_xfer_buffer_map
                     .insert(std::make_pair(buffer_key, std::move(*buffer_up)))
                     .first;
   }
 
   // Send back the response
   StreamGDBRemote response;
   bool done_with_buffer = false;
   llvm::StringRef buffer = buffer_it->second->getBuffer();
   if (xfer_offset >= buffer.size()) {
     // We have nothing left to send.  Mark the buffer as complete.
     response.PutChar('l');
     done_with_buffer = true;
   } else {
     // Figure out how many bytes are available starting at the given offset.
     buffer = buffer.drop_front(xfer_offset);
     // Mark the response type according to whether we're reading the remainder
     // of the data.
     if (xfer_length >= buffer.size()) {
       // There will be nothing left to read after this
       response.PutChar('l');
       done_with_buffer = true;
     } else {
       // There will still be bytes to read after this request.
       response.PutChar('m');
       buffer = buffer.take_front(xfer_length);
     }
     // Now write the data in encoded binary form.
     response.PutEscapedBytes(buffer.data(), buffer.size());
   }
 
   if (done_with_buffer)
     m_xfer_buffer_map.erase(buffer_it);
 
   return SendPacketNoLock(response.GetString());
 }
 
 GDBRemoteCommunication::PacketResult
 GDBRemoteCommunicationServerLLGS::Handle_QSaveRegisterState(
     StringExtractorGDBRemote &packet) {
   Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_THREAD));
 
   // Move past packet name.
   packet.SetFilePos(strlen("QSaveRegisterState"));
 
   // Get the thread to use.
   NativeThreadProtocol *thread = GetThreadFromSuffix(packet);