[lldb] Add --all option to "memory region"

[DavidSpickett]

Summary

This adds an option to the memory region command
to print all regions at once. Like you can do by
starting at address 0 and repeating the command
manually.

memory region [-a] [<address-expression>]

(lldb) memory region --all
[0x0000000000000000-0x0000000000400000) ---
[0x0000000000400000-0x0000000000401000) r-x <...>/a.out PT_LOAD[0]
<...>
[0x0000fffffffdf000-0x0001000000000000) rw- [stack]
[0x0001000000000000-0xffffffffffffffff) ---

The output matches exactly what you'd get from
repeating the command. Including that it shows
unmapped areas between the mapped regions.

(this is why Process GetMemoryRegions is not
used, that skips unmapped areas)

Help text has been updated to show that you can have
an address or --all but not both.

[DavidSpickett]

Correct help text. Make a parameter const (uses the previous stacked change).

[DavidSpickett]

Figured that the usage was now a bit confusing. Though I'm not sure how to write "either" in symbols,
I see {A|B} used in some other tools, I've just stuck with the word "or" for now.

Doesn't matter too much, the usage shown isn't meant to be exhaustive and you have help to give you that.

[JDevlieghere]

Minor nit but LGTM

lldb/source/Commands/CommandObjectMemory.cpp

1707

Can we move this down, closer to where it's being used?

[DavidSpickett]

Move variables closer to usage in DumpRegion.

[DavidSpickett]

Add a release note before I inevitably forget.

[DavidSpickett]

Fixup the tagged memory region test (it doesn't care about the
content of the usage just that the command eventually fails) and rebase.

[DavidSpickett]

Fixup the loop on systems that have more non-address bits than just top byte ignore.

With only top byte ignore:

• We ask the remote for the last region
• It says ok it's unmapped and it ends at 0xF....F
• Then we mask that address using just the top byte ignore but because the PAC sign bit is set we actually or in the mask which keeps all the bits set.
• We ask for a region at LLDB_INVALID_ADDRESS which fails and we break.

With top byte ignore plus pointer auth:

• We ask for the last region
• It returns a range with end address 0x0001000000000000
• Then we ask for the next region
• Which masks that address with 0xFFFF0...0 and gets 0
• Which gives you the first region and we loop forever

This is the same thing I did to fix the manual command repetition but
because we got away without it on a TBI only system I didn't notice it
until now.

I figured it needed the same fix but took a bit to figure out why.

[DavidSpickett]

Relanded in https://reviews.llvm.org/rG13e1cf806567bc4987817e14a774198c3e3f2709.

lldb/source/Commands/CommandObjectMemory.cpp

 //===-- CommandObjectMemory.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
bool DoExecute(Args &command, CommandReturnObject &result) override {
 
     return true;
   }
 };
 
 // CommandObjectMemoryRegion
 #pragma mark CommandObjectMemoryRegion
 
+#define LLDB_OPTIONS_memory_region
+#include "CommandOptions.inc"
+
 class CommandObjectMemoryRegion : public CommandObjectParsed {
 public:
+  class OptionGroupMemoryRegion : public OptionGroup {
+  public:
+    OptionGroupMemoryRegion() : OptionGroup(), m_all(false, false) {}
+
+    ~OptionGroupMemoryRegion() override = default;
+
+    llvm::ArrayRef<OptionDefinition> GetDefinitions() override {
+      return llvm::makeArrayRef(g_memory_region_options);
+    }
+
+    Status SetOptionValue(uint32_t option_idx, llvm::StringRef option_value,
+                          ExecutionContext *execution_context) override {
+      Status status;
+      const int short_option = g_memory_region_options[option_idx].short_option;
+
+      switch (short_option) {
+      case 'a':
+        m_all.SetCurrentValue(true);
+        m_all.SetOptionWasSet();
+        break;
+      default:
+        llvm_unreachable("Unimplemented option");
+      }
+
+      return status;
+    }
+
+    void OptionParsingStarting(ExecutionContext *execution_context) override {
+      m_all.Clear();
+    }
+
+    OptionValueBoolean m_all;
+  };
+
   CommandObjectMemoryRegion(CommandInterpreter &interpreter)
       : CommandObjectParsed(interpreter, "memory region",
                             "Get information on the memory region containing "
                             "an address in the current target process.",
-                            "memory region ADDR",
+                            "memory region <address-expression> (or --all)",
                             eCommandRequiresProcess | eCommandTryTargetAPILock |
-                                eCommandProcessMustBeLaunched) {}
+                                eCommandProcessMustBeLaunched) {
+    // Address in option set 1.
+    m_arguments.push_back(CommandArgumentEntry{CommandArgumentData(
+        eArgTypeAddressOrExpression, eArgRepeatPlain, LLDB_OPT_SET_1)});
+    // "--all" will go in option set 2.
+    m_option_group.Append(&m_memory_region_options);
+    m_option_group.Finalize();
+  }
 
   ~CommandObjectMemoryRegion() override = default;
 
+  Options *GetOptions() override { return &m_option_group; }
+
 protected:
+  void DumpRegion(CommandReturnObject &result, Target &target,
+                  const MemoryRegionInfo &range_info, lldb::addr_t load_addr) {
+    lldb::addr_t resolve_addr = load_addr;
+    if (m_memory_region_options.m_all)
+      resolve_addr = range_info.GetRange().GetRangeBase();
+
+    lldb_private::Address addr;
+    ConstString section_name;
+    if (target.ResolveLoadAddress(resolve_addr, addr)) {
+      SectionSP section_sp(addr.GetSection());
+      if (section_sp) {
+        // Got the top most section, not the deepest section
+        while (section_sp->GetParent())
+          section_sp = section_sp->GetParent();
+        section_name = section_sp->GetName();
+      }
+    }
+
+    ConstString name = range_info.GetName();
+    result.AppendMessageWithFormatv(
+        "[{0:x16}-{1:x16}) {2:r}{3:w}{4:x}{5}{6}{7}{8}",
+        range_info.GetRange().GetRangeBase(),
+        range_info.GetRange().GetRangeEnd(), range_info.GetReadable(),
+        range_info.GetWritable(), range_info.GetExecutable(), name ? " " : "",
+        name, section_name ? " " : "", section_name);
+    MemoryRegionInfo::OptionalBool memory_tagged = range_info.GetMemoryTagged();
+    if (memory_tagged == MemoryRegionInfo::OptionalBool::eYes)
+      result.AppendMessage("memory tagging: enabled");
+
+    const llvm::Optional<std::vector<addr_t>> &dirty_page_list =
+        range_info.GetDirtyPageList();
+    if (dirty_page_list.hasValue()) {
+      const size_t page_count = dirty_page_list.getValue().size();
+      result.AppendMessageWithFormat(
+          "Modified memory (dirty) page list provided, %zu entries.\n",
+          page_count);
+      if (page_count > 0) {
+        bool print_comma = false;
+        result.AppendMessageWithFormat("Dirty pages: ");
+        for (size_t i = 0; i < page_count; i++) {
+          if (print_comma)
+            result.AppendMessageWithFormat(", ");
+          else
+            print_comma = true;
+          result.AppendMessageWithFormat("0x%" PRIx64,
+                                         dirty_page_list.getValue()[i]);
+        }
+        result.AppendMessageWithFormat(".\n");
+      }
+    }
+  }
+
   bool DoExecute(Args &command, CommandReturnObject &result) override {
     ProcessSP process_sp = m_exe_ctx.GetProcessSP();
     if (!process_sp) {
       m_prev_end_addr = LLDB_INVALID_ADDRESS;
       result.AppendError("invalid process");
       return false;
     }
 
     Status error;
     lldb::addr_t load_addr = m_prev_end_addr;
     m_prev_end_addr = LLDB_INVALID_ADDRESS;
 
     const size_t argc = command.GetArgumentCount();
     const lldb::ABISP &abi = process_sp->GetABI();
 
     if (argc == 1) {
+      if (m_memory_region_options.m_all) {
+        result.AppendError(
+            "The \"--all\" option cannot be used when an address "
+            "argument is given");
+        return false;
+      }
+
       auto load_addr_str = command[0].ref();
       // Non-address bits in this will be handled later by GetMemoryRegion
       load_addr = OptionArgParser::ToAddress(&m_exe_ctx, load_addr_str,
                                              LLDB_INVALID_ADDRESS, &error);
       if (error.Fail() || load_addr == LLDB_INVALID_ADDRESS) {
         result.AppendErrorWithFormat("invalid address argument \"%s\": %s\n",
                                      command[0].c_str(), error.AsCString());
         return false;
       }
     } else if (argc > 1 ||
                // When we're repeating the command, the previous end address is
                // used for load_addr. If that was 0xF...F then we must have
                // reached the end of memory.
-               (argc == 0 && load_addr == LLDB_INVALID_ADDRESS) ||
+               (argc == 0 && !m_memory_region_options.m_all &&
+                load_addr == LLDB_INVALID_ADDRESS) ||
                // If the target has non-address bits (tags, limited virtual
                // address size, etc.), the end of mappable memory will be lower
                // than that. So if we find any non-address bit set, we must be
                // at the end of the mappable range.
                (abi && (abi->FixAnyAddress(load_addr) != load_addr))) {
-      result.AppendErrorWithFormat("'%s' takes one argument:\nUsage: %s\n",
-                                   m_cmd_name.c_str(), m_cmd_syntax.c_str());
+      result.AppendErrorWithFormat(
+          "'%s' takes one argument or \"--all\" option:\nUsage: %s\n",
       return false;
     }
 
-    lldb_private::MemoryRegionInfo range_info;
-    error = process_sp->GetMemoryRegionInfo(load_addr, range_info);
+    lldb_private::MemoryRegionInfos region_list;
+    if (m_memory_region_options.m_all) {
+      // We don't use GetMemoryRegions here because it doesn't include unmapped
+      // areas like repeating the command would. So instead, emulate doing that.
+      lldb::addr_t addr = 0;
+      while (error.Success()) {
+        lldb_private::MemoryRegionInfo region_info;
+        error = process_sp->GetMemoryRegionInfo(addr, region_info);
+
         if (error.Success()) {
-      lldb_private::Address addr;
-      ConstString name = range_info.GetName();
-      ConstString section_name;
-      if (process_sp->GetTarget().ResolveLoadAddress(load_addr, addr)) {
-        SectionSP section_sp(addr.GetSection());
-        if (section_sp) {
-          // Got the top most section, not the deepest section
-          while (section_sp->GetParent())
-            section_sp = section_sp->GetParent();
-          section_name = section_sp->GetName();
+          region_list.push_back(region_info);
+          addr = region_info.GetRange().GetRangeEnd();
         }
       }
 
-      result.AppendMessageWithFormatv(
-          "[{0:x16}-{1:x16}) {2:r}{3:w}{4:x}{5}{6}{7}{8}",
-          range_info.GetRange().GetRangeBase(),
-          range_info.GetRange().GetRangeEnd(), range_info.GetReadable(),
-          range_info.GetWritable(), range_info.GetExecutable(), name ? " " : "",
-          name, section_name ? " " : "", section_name);
-      MemoryRegionInfo::OptionalBool memory_tagged =
-          range_info.GetMemoryTagged();
-      if (memory_tagged == MemoryRegionInfo::OptionalBool::eYes)
-        result.AppendMessage("memory tagging: enabled");
-
-      const llvm::Optional<std::vector<addr_t>> &dirty_page_list =
-          range_info.GetDirtyPageList();
-      if (dirty_page_list.hasValue()) {
-        const size_t page_count = dirty_page_list.getValue().size();
-        result.AppendMessageWithFormat(
-            "Modified memory (dirty) page list provided, %zu entries.\n",
-            page_count);
-        if (page_count > 0) {
-          bool print_comma = false;
-          result.AppendMessageWithFormat("Dirty pages: ");
-          for (size_t i = 0; i < page_count; i++) {
-            if (print_comma)
-              result.AppendMessageWithFormat(", ");
-            else
-              print_comma = true;
-            result.AppendMessageWithFormat("0x%" PRIx64,
-                                           dirty_page_list.getValue()[i]);
-          }
-          result.AppendMessageWithFormat(".\n");
-        }
-      }
+      // Even if we read nothing, don't error for --all
+      error.Clear();
+    } else {
+      lldb_private::MemoryRegionInfo region_info;
+      error = process_sp->GetMemoryRegionInfo(load_addr, region_info);
+      if (error.Success())
+        region_list.push_back(region_info);
 
+    if (error.Success()) {
+      for (MemoryRegionInfo &range_info : region_list) {
+        DumpRegion(result, process_sp->GetTarget(), range_info, load_addr);
         m_prev_end_addr = range_info.GetRange().GetRangeEnd();
+      }
+
       result.SetStatus(eReturnStatusSuccessFinishResult);
       return true;
     }
 
     result.AppendErrorWithFormat("%s\n", error.AsCString());
     return false;
   }
 
   llvm::Optional<std::string> GetRepeatCommand(Args &current_command_args,
                                                uint32_t index) override {
     // If we repeat this command, repeat it without any arguments so we can
     // show the next memory range
     return m_cmd_name;
   }
 
   lldb::addr_t m_prev_end_addr = LLDB_INVALID_ADDRESS;
+
+  OptionGroupOptions m_option_group;
+  OptionGroupMemoryRegion m_memory_region_options;
 };
 
 // CommandObjectMemory
 
 CommandObjectMemory::CommandObjectMemory(CommandInterpreter &interpreter)
     : CommandObjectMultiword(
           interpreter, "memory",
           "Commands for operating on memory in the current target process.",

lldb/source/Commands/Options.td

 
 let Command = "memory write" in {
   def memory_write_infile : Option<"infile", "i">, Group<1>, Arg<"Filename">,
     Required, Desc<"Write memory using the contents of a file.">;
   def memory_write_offset : Option<"offset", "o">, Group<1>, Arg<"Offset">,
     Desc<"Start writing bytes from an offset within the input file.">;
 }
 
+let Command = "memory region" in {
+  def memory_region_all : Option<"all", "a">, Group<2>, Required,
+    Desc<"Show all memory regions. This is equivalent to starting from address "
+         "0 and repeating the command. Unmapped areas are included.">;
+}
+
 let Command = "memory tag write" in {
   def memory_write_end_addr : Option<"end-addr", "e">, Group<1>,
   Arg<"AddressOrExpression">, Desc<
     "Set tags for start address to end-addr, repeating tags as needed"
     " to cover the range. (instead of calculating the range from the"
     " number of tags given)">;
 }
 

lldb/test/API/functionalities/memory-region/TestMemoryRegion.py

class MemoryCommandRegion(TestBase):
 
     def setUp(self):
         TestBase.setUp(self)
         # Find the line number to break for main.c.
         self.line = line_number(
             'main.cpp',
             '// Run here before printing memory regions')
 
+    def test_help(self):
+        """ Test that help shows you must have one of address or --all, not both."""
+        self.expect("help memory region",
+            substrs=["memory region <address-expression>",
+                     "memory region -a"])
+
     def test(self):
         self.build()
 
         # Set breakpoint in main and run
         self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
         lldbutil.run_break_set_by_file_and_line(
             self, "main.cpp", self.line, num_expected_locations=-1, loc_exact=True)
 
         self.runCmd("run", RUN_SUCCEEDED)
 
         interp = self.dbg.GetCommandInterpreter()
         result = lldb.SBCommandReturnObject()
 
         # Test that the first 'memory region' command prints the usage.
         interp.HandleCommand("memory region", result)
         self.assertFalse(result.Succeeded())
-        self.assertRegexpMatches(result.GetError(), "Usage: memory region ADDR")
+        self.assertEqual(result.GetError(),
+                    "error: 'memory region' takes one argument or \"--all\" option:\n"
+                    "Usage: memory region <address-expression> (or --all)\n")
+
+        # We allow --all or an address argument, not both
+        interp.HandleCommand("memory region --all 0", result)
+        self.assertFalse(result.Succeeded())
+        self.assertRegexpMatches(result.GetError(),
+                "The \"--all\" option cannot be used when an address argument is given")
 
         # Test that when the address fails to parse, we show an error and do not continue
         interp.HandleCommand("memory region not_an_address", result)
         self.assertFalse(result.Succeeded())
         self.assertEqual(result.GetError(),
                 "error: invalid address argument \"not_an_address\": address expression \"not_an_address\" evaluation failed\n")
 
+        # Accumulate the results to compare with the --all output
+        all_regions = ""
+
         # Now let's print the memory region starting at 0 which should always work.
         interp.HandleCommand("memory region 0x0", result)
         self.assertTrue(result.Succeeded())
         self.assertRegexpMatches(result.GetOutput(), "\\[0x0+-")
+        all_regions += result.GetOutput()
 
         # Keep printing memory regions until we printed all of them.
         while True:
             interp.HandleCommand("memory region", result)
             if not result.Succeeded():
                 break
+            all_regions += result.GetOutput()
 
         # Now that we reached the end, 'memory region' should again print the usage.
         interp.HandleCommand("memory region", result)
         self.assertFalse(result.Succeeded())
-        self.assertRegexpMatches(result.GetError(), "Usage: memory region ADDR")
+        self.assertRegexpMatches(result.GetError(), "Usage: memory region <address\-expression> \(or \-\-all\)")
+
+        # --all should match what repeating the command gives you
+        interp.HandleCommand("memory region --all", result)
+        self.assertTrue(result.Succeeded())
+        self.assertEqual(result.GetOutput(), all_regions)

llvm/docs/ReleaseNotes.rst

 During this release ...
 
 Changes to the LLVM tools
 ---------------------------------
 
 Changes to LLDB
 ---------------------------------
 
+* The "memory region" command now has a "--all" option to list all
+  memory regions (including unmapped ranges). This is the equivalent
+  of using address 0 then repeating the command until all regions
+  have been listed.
+
 Changes to Sanitizers
 ---------------------
 
 
 Other Changes
 -------------
 * The code for the `LLVM Visual Studio integration
   <https://marketplace.visualstudio.com/items?itemName=LLVMExtensions.llvm-toolchain>`_