[lldb][AArch64] Add "memory tag read" command

[DavidSpickett]

Summary

This new command looks much like "memory read"
and mirrors its basic behaviour.

(lldb) memory tag read new_buf_ptr new_buf_ptr+32
Logical tag: 0x9
Allocation tags:
[0x900fffff7ffa000, 0x900fffff7ffa010): 0x9
[0x900fffff7ffa010, 0x900fffff7ffa020): 0x0

Important proprties:

• The end address is optional and defaults to reading 1 tag if ommitted
• It is an error to try to read tags if the architecture or process doesn't support it, or if the range asked for is not tagged.
• It is an error to read an inverted range (end < begin) (logical tags are removed for this check so you can pass tagged addresses here)
• The range will be expanded to fit the tagging granule, so you can get more tags than simply (end-begin)/granule size. Whatever you get back will always cover the original range.

[DavidSpickett]

This command is described in the RFC under the name "mtag showatag" (https://lists.llvm.org/pipermail/lldb-dev/2020-August/016402.html). The change of naming was suggested in feedback and I think it does fit better with the other commands so I went with it.

The output is very basic and could certainly be compacted but this is just the initial implementation.

We could also have this command highlight matches/mismatches. This would reduce (maybe remove) the need for a "memory tag check" command. I think that would be better done as a follow up if so.

[DavidSpickett]

lldb/test/API/linux/aarch64/mte_tag_read/main.c

47

https://developer.arm.com/documentation/101028/0012/10--Memory-tagging-intrinsics for descriptions of these functions.

I probably need to add some more comments here for the unfamiliar.

[DavidSpickett]

• Add some comments to explain the intrinsics used.

[DavidSpickett]

Update after changing previous patch.

[DavidSpickett]

• RemoveNonAddressBits over RemoveLogicalTag
• Set bits 63-60 of test file pointers to check we remove the top byte now.

[DavidSpickett]

• Update after changing earlier revisions
• Read page size only once in the test file

[DavidSpickett]

I still have one known (though rare) limitation to this command. That is, if you try to read across two neighbouring MTE regions it will fail because it thinks that the whole range isn't tagged. When it is, it's just split over two regions.

This would require you to manage to mmap them next to each other. So I'm looking for a way to do this so it can be tested.

[DavidSpickett]

Expand test to check that we can read tags for a range that spans
two mappings. This covers updated code in GetMemoryTagManager.

Refactor the test file a bit to account for the extra mmap.

I've tested the mmap sequence on my x86 box which has ASLR enabled
and on Qemu. I always get the ordering we want for the test.
I can't cite anything to say it should always work but I didn't
find any feature that goes out of its way to change the ordering
of mmap allocations.

[DavidSpickett]

• Rebase onto main

[DavidSpickett]

Rebase onto main.

[DavidSpickett]

Rebase

[DavidSpickett]

Rebase onto earlier patches.

[DavidSpickett]

Rebase

[DavidSpickett]

Rebase

[DavidSpickett]

Rebase. Remove result.SetStatus(eReturnStatusFailed) which
is now implicitly done when you add an error.

lldb/source/Commands/CMakeLists.txt

add_lldb_library(lldbCommands
   CommandObjectDisassemble.cpp
   CommandObjectExpression.cpp
   CommandObjectFrame.cpp
   CommandObjectGUI.cpp
   CommandObjectHelp.cpp
   CommandObjectLanguage.cpp
   CommandObjectLog.cpp
   CommandObjectMemory.cpp
+  CommandObjectMemoryTag.cpp
   CommandObjectMultiword.cpp
   CommandObjectPlatform.cpp
   CommandObjectPlugin.cpp
   CommandObjectProcess.cpp
   CommandObjectQuit.cpp
   CommandObjectRegexCommand.cpp
   CommandObjectRegister.cpp
   CommandObjectReproducer.cpp

lldb/source/Commands/CommandObjectMemory.cpp

 //===-- CommandObjectMemory.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "CommandObjectMemory.h"
+#include "CommandObjectMemoryTag.h"
 #include "lldb/Core/DumpDataExtractor.h"
 #include "lldb/Core/Section.h"
 #include "lldb/Core/ValueObjectMemory.h"
 #include "lldb/Expression/ExpressionVariable.h"
 #include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/CommandReturnObject.h"
 #include "lldb/Interpreter/OptionArgParser.h"
 #include "lldb/Interpreter/OptionGroupFormat.h"
CommandObjectMemory::CommandObjectMemory(CommandInterpreter &interpreter)
   LoadSubCommand("read",
                  CommandObjectSP(new CommandObjectMemoryRead(interpreter)));
   LoadSubCommand("write",
                  CommandObjectSP(new CommandObjectMemoryWrite(interpreter)));
   LoadSubCommand("history",
                  CommandObjectSP(new CommandObjectMemoryHistory(interpreter)));
   LoadSubCommand("region",
                  CommandObjectSP(new CommandObjectMemoryRegion(interpreter)));
+  LoadSubCommand("tag",
+                 CommandObjectSP(new CommandObjectMemoryTag(interpreter)));
 }
 
 CommandObjectMemory::~CommandObjectMemory() = default;

lldb/source/Commands/CommandObjectMemoryTag.h

+//===-- CommandObjectMemoryTag.h --------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_SOURCE_COMMANDS_COMMANDOBJECTMEMORYTAG_H
+#define LLDB_SOURCE_COMMANDS_COMMANDOBJECTMEMORYTAG_H
+
+#include "lldb/Interpreter/CommandObjectMultiword.h"
+
+namespace lldb_private {
+
+class CommandObjectMemoryTag : public CommandObjectMultiword {
+public:
+  CommandObjectMemoryTag(CommandInterpreter &interpreter);
+
+  ~CommandObjectMemoryTag() override;
+};
+
+} // namespace lldb_private
+
+#endif // LLDB_SOURCE_COMMANDS_COMMANDOBJECTMEMORYTAG_H

lldb/source/Commands/CommandObjectMemoryTag.cpp

+//===-- CommandObjectMemoryTag.cpp ----------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "CommandObjectMemoryTag.h"
+#include "lldb/Interpreter/CommandReturnObject.h"
+#include "lldb/Interpreter/OptionArgParser.h"
+#include "lldb/Target/Process.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+#define LLDB_OPTIONS_memory_tag_read
+#include "CommandOptions.inc"
+
+class CommandObjectMemoryTagRead : public CommandObjectParsed {
+public:
+  CommandObjectMemoryTagRead(CommandInterpreter &interpreter)
+      : CommandObjectParsed(interpreter, "tag",
+                            "Read memory tags for the given range of memory.",
+                            nullptr,
+                            eCommandRequiresTarget | eCommandRequiresProcess |
+                                eCommandProcessMustBePaused) {
+    // Address
+    m_arguments.push_back(
+        CommandArgumentEntry{CommandArgumentData(eArgTypeAddressOrExpression)});
+    // Optional end address
+    m_arguments.push_back(CommandArgumentEntry{
+        CommandArgumentData(eArgTypeAddressOrExpression, eArgRepeatOptional)});
+  }
+
+  ~CommandObjectMemoryTagRead() override = default;
+
+protected:
+  bool DoExecute(Args &command, CommandReturnObject &result) override {
+    if ((command.GetArgumentCount() < 1) || (command.GetArgumentCount() > 2)) {
+      result.AppendError(
+          "wrong number of arguments; expected at least <address-expression>, "
+          "at most <address-expression> <end-address-expression>");
+      return false;
+    }
+
+    Status error;
+    addr_t start_addr = OptionArgParser::ToAddress(
+        &m_exe_ctx, command[0].ref(), LLDB_INVALID_ADDRESS, &error);
+    if (start_addr == LLDB_INVALID_ADDRESS) {
+      result.AppendErrorWithFormatv("Invalid address expression, {0}",
+                                    error.AsCString());
+      return false;
+    }
+
+    // Default 1 byte beyond start, rounds up to at most 1 granule later
+    addr_t end_addr = start_addr + 1;
+
+    if (command.GetArgumentCount() > 1) {
+      end_addr = OptionArgParser::ToAddress(&m_exe_ctx, command[1].ref(),
+                                            LLDB_INVALID_ADDRESS, &error);
+      if (end_addr == LLDB_INVALID_ADDRESS) {
+        result.AppendErrorWithFormatv("Invalid end address expression, {0}",
+                                      error.AsCString());
+        return false;
+      }
+    }
+
+    Process *process = m_exe_ctx.GetProcessPtr();
+    llvm::Expected<const MemoryTagManager *> tag_manager_or_err =
+        process->GetMemoryTagManager(start_addr, end_addr);
+
+    if (!tag_manager_or_err) {
+      result.SetError(Status(tag_manager_or_err.takeError()));
+      return false;
+    }
+
+    const MemoryTagManager *tag_manager = *tag_manager_or_err;
+    ptrdiff_t len = tag_manager->AddressDiff(end_addr, start_addr);
+    llvm::Expected<std::vector<lldb::addr_t>> tags =
+        process->ReadMemoryTags(tag_manager, start_addr, len);
+
+    if (!tags) {
+      result.SetError(Status(tags.takeError()));
+      return false;
+    }
+
+    result.AppendMessageWithFormatv("Logical tag: {0:x}",
+                                    tag_manager->GetLogicalTag(start_addr));
+    result.AppendMessage("Allocation tags:");
+
+    MemoryTagManager::TagRange initial_range(start_addr, len);
+    addr_t addr = tag_manager->ExpandToGranule(initial_range).GetRangeBase();
+    for (auto tag : *tags) {
+      addr_t next_addr = addr + tag_manager->GetGranuleSize();
+      // Showing tagged adresses here until we have non address bit handling
+      result.AppendMessageWithFormatv("[{0:x}, {1:x}): {2:x}", addr, next_addr,
+                                      tag);
+      addr = next_addr;
+    }
+
+    result.SetStatus(eReturnStatusSuccessFinishResult);
+    return true;
+  }
+};
+
+CommandObjectMemoryTag::CommandObjectMemoryTag(CommandInterpreter &interpreter)
+    : CommandObjectMultiword(
+          interpreter, "tag", "Commands for manipulating memory tags",
+          "memory tag <sub-command> [<sub-command-options>]") {
+  CommandObjectSP read_command_object(
+      new CommandObjectMemoryTagRead(interpreter));
+  read_command_object->SetCommandName("memory tag read");
+  LoadSubCommand("read", read_command_object);
+}
+
+CommandObjectMemoryTag::~CommandObjectMemoryTag() = default;

lldb/test/API/functionalities/memory/tag/Makefile

+CXX_SOURCES := main.cpp
+
+include Makefile.rules

lldb/test/API/functionalities/memory/tag/TestMemoryTag.py

+"""
+Test errors from 'memory tag' commands on unsupported platforms.
+Tests for the only supported platform, AArch64 Linux, are in
+API/linux/aarch64/.
+"""
+
+import lldb
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test.decorators import *
+import lldbsuite.test.lldbutil as lldbutil
+
+
+class MemoryTagTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    NO_DEBUG_INFO_TESTCASE = True
+
+    def test_memory_tag_read_unsupported(self):
+        """Test that "memory tag read" errors on unsupported platforms"""
+        if self.isAArch64MTE():
+            self.skipTest("Requires a target without AArch64 MTE.")
+
+        self.build()
+        exe = self.getBuildArtifact("a.out")
+        self.runCmd("file " + exe, CURRENT_EXECUTABLE_SET)
+
+        lldbutil.run_break_set_by_file_and_line(self, "main.cpp",
+                            line_number('main.cpp', '// Breakpoint here'),
+                                        num_expected_locations=1)
+        self.runCmd("run", RUN_SUCCEEDED)
+
+        self.expect("memory tag read 0 1",
+                    substrs=["error: This architecture does not support memory tagging"],
+                    error=True)

lldb/test/API/functionalities/memory/tag/main.cpp

+int main(int argc, char const *argv[]) {
+  // Breakpoint here
+  return 0;
+}

lldb/test/API/linux/aarch64/mte_tag_read/Makefile

+C_SOURCES := main.c
+CFLAGS_EXTRAS := -march=armv8.5-a+memtag
+
+include Makefile.rules

lldb/test/API/linux/aarch64/mte_tag_read/TestAArch64LinuxMTEMemoryTagRead.py

+"""
+Test "memory tag read" command on AArch64 Linux with MTE.
+"""
+
+
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+
+class AArch64LinuxMTEMemoryTagReadTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    NO_DEBUG_INFO_TESTCASE = True
+
+    @skipUnlessArch("aarch64")
+    @skipUnlessPlatform(["linux"])
+    @skipUnlessAArch64MTELinuxCompiler
+    def test_mte_tag_read(self):
+        if not self.isAArch64MTE():
+            self.skipTest('Target must support MTE.')
+
+        self.build()
+        self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
+
+        lldbutil.run_break_set_by_file_and_line(self, "main.c",
+            line_number('main.c', '// Breakpoint here'),
+            num_expected_locations=1)
+
+        self.runCmd("run", RUN_SUCCEEDED)
+
+        if self.process().GetState() == lldb.eStateExited:
+            self.fail("Test program failed to run.")
+
+        self.expect("thread list", STOPPED_DUE_TO_BREAKPOINT,
+            substrs=['stopped',
+                     'stop reason = breakpoint'])
+
+        # Argument validation
+        self.expect("memory tag read",
+                substrs=["error: wrong number of arguments; expected at least <address-expression>, "
+                         "at most <address-expression> <end-address-expression>"],
+                error=True)
+        self.expect("memory tag read buf buf+16 32",
+                substrs=["error: wrong number of arguments; expected at least <address-expression>, "
+                         "at most <address-expression> <end-address-expression>"],
+                error=True)
+        self.expect("memory tag read not_a_symbol",
+                substrs=["error: Invalid address expression, address expression \"not_a_symbol\" "
+                         "evaluation failed"],
+                error=True)
+        self.expect("memory tag read buf not_a_symbol",
+                substrs=["error: Invalid end address expression, address expression \"not_a_symbol\" "
+                         "evaluation failed"],
+                error=True)
+        # Inverted range
+        self.expect("memory tag read buf buf-16",
+                patterns=["error: End address \(0x[A-Fa-f0-9]+\) must be "
+                          "greater than the start address \(0x[A-Fa-f0-9]+\)"],
+                error=True)
+        # Range of length 0
+        self.expect("memory tag read buf buf",
+                patterns=["error: End address \(0x[A-Fa-f0-9]+\) must be "
+                          "greater than the start address \(0x[A-Fa-f0-9]+\)"],
+                error=True)
+
+
+        # Can't read from a region without tagging
+        self.expect("memory tag read non_mte_buf",
+                patterns=["error: Address range 0x[0-9A-Fa-f]+00:0x[0-9A-Fa-f]+10 is not "
+                         "in a memory tagged region"],
+                error=True)
+
+        # If there's no end address we assume 1 granule
+        self.expect("memory tag read buf",
+                patterns=["Logical tag: 0x9\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+00, 0x[0-9A-Fa-f]+10\): 0x0$"])
+
+        # Range of <1 granule is rounded up to 1 granule
+        self.expect("memory tag read buf buf+8",
+                patterns=["Logical tag: 0x9\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+00, 0x[0-9A-Fa-f]+10\): 0x0$"])
+
+        # Start address is aligned down, end aligned up
+        self.expect("memory tag read buf+8 buf+24",
+                patterns=["Logical tag: 0x9\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+00, 0x[0-9A-Fa-f]+10\): 0x0\n"
+                          "\[0x[0-9A-Fa-f]+10, 0x[0-9A-Fa-f]+20\): 0x1$"])
+
+        # You may read up to the end of the tagged region
+        # Layout is buf (MTE), buf2 (MTE), <unmapped/non MTE>
+        # so we read from the end of buf2 here.
+        self.expect("memory tag read buf2+page_size-16 buf2+page_size",
+                patterns=["Logical tag: 0x0\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+, 0x[0-9A-Fa-f]+\): 0x0$"])
+
+        # Ranges with any part outside the region will error
+        self.expect("memory tag read buf2+page_size-16 buf2+page_size+32",
+                patterns=["error: Address range 0x[0-9A-fa-f]+f0:0x[0-9A-Fa-f]+20 "
+                          "is not in a memory tagged region"],
+                error=True)
+        self.expect("memory tag read buf2+page_size",
+                patterns=["error: Address range 0x[0-9A-fa-f]+00:0x[0-9A-Fa-f]+10 "
+                          "is not in a memory tagged region"],
+                error=True)
+
+        # You can read a range that spans more than one mapping
+        # This spills into buf2 which is also MTE
+        self.expect("memory tag read buf+page_size-16 buf+page_size+16",
+                patterns=["Logical tag: 0x9\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+f0, 0x[0-9A-Fa-f]+00\): 0xf\n"
+                          "\[0x[0-9A-Fa-f]+00, 0x[0-9A-Fa-f]+10\): 0x0$"])
+
+        # Tags in start/end are ignored when creating the range.
+        # So this is not an error despite start/end having different tags
+        self.expect("memory tag read buf buf_alt_tag+16 ",
+                patterns=["Logical tag: 0x9\n"
+                          "Allocation tags:\n"
+                          "\[0x[0-9A-Fa-f]+00, 0x[0-9A-Fa-f]+10\): 0x0$"])

lldb/test/API/linux/aarch64/mte_tag_read/main.c

+#include <arm_acle.h>
+#include <asm/hwcap.h>
+#include <asm/mman.h>
+#include <sys/auxv.h>
+#include <sys/mman.h>
+#include <sys/prctl.h>
+#include <unistd.h>
+
+int main(int argc, char const *argv[]) {
+  // We assume that the test runner has checked we're on an MTE system
+
+  if (prctl(PR_SET_TAGGED_ADDR_CTRL,
+            PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC |
+                // Allow all tags to be generated by the addg
+                // instruction __arm_mte_increment_tag produces.
+                (0xffff << PR_MTE_TAG_SHIFT),
+            0, 0, 0)) {
+    return 1;
+  }
+
+  size_t page_size = sysconf(_SC_PAGESIZE);
+
+  // Allocate memory with MTE
+  // We ask for two pages. One is read only so that we get
+  // 2 mappings in /proc/.../smaps so we can check reading
+  // a range across mappings.
+  // The first allocation will start at the highest address,
+  // so we allocate buf2 first to get:
+  // <low address> | buf | buf2 | <high address>
+  int prot = PROT_READ | PROT_MTE;
+  int flags = MAP_PRIVATE | MAP_ANONYMOUS;
+
+  char *buf2 = mmap(0, page_size, prot, flags, -1, 0);
+  if (buf2 == MAP_FAILED)
+    return 1;
+
+  // Writeable so we can set tags on it later
+  char *buf = mmap(0, page_size, prot | PROT_WRITE, flags, -1, 0);
+  if (buf == MAP_FAILED)
+    return 1;
+
+  // We expect the mappings to be next to each other
+  if (buf2 - buf != page_size)
+    return 1;
+
+  // And without MTE
+  char *non_mte_buf = mmap(0, page_size, PROT_READ | PROT_WRITE, flags, -1, 0);
+  if (non_mte_buf == MAP_FAILED)
+    return 1;
+
+  // Set incrementing tags until end of the first page
+  char *tagged_ptr = buf;
+  // This ignores tag bits when subtracting the addresses
+  while (__arm_mte_ptrdiff(tagged_ptr, buf) < page_size) {
+    // Set the allocation tag for this location
+    __arm_mte_set_tag(tagged_ptr);
+    // + 16 for 16 byte granules
+    // Earlier we allowed all tag values, so this will give us an
+    // incrementing pattern 0-0xF wrapping back to 0.
+    tagged_ptr = __arm_mte_increment_tag(tagged_ptr + 16, 1);
+  }
+
+  // Tag the original pointer with 9
+  buf = __arm_mte_create_random_tag(buf, ~(1 << 9));
+  // A different tag so that buf_alt_tag > buf if you don't handle the tag
+  char *buf_alt_tag = __arm_mte_create_random_tag(buf, ~(1 << 10));
+
+  // lldb should be removing the whole top byte, not just the tags.
+  // So fill 63-60 with something non zero so we'll fail if we only remove tags.
+#define SET_TOP_NIBBLE(ptr) (char *)((size_t)(ptr) | (0xA << 60))
+  buf = SET_TOP_NIBBLE(buf);
+  buf_alt_tag = SET_TOP_NIBBLE(buf_alt_tag);
+  buf2 = SET_TOP_NIBBLE(buf2);
+
+  // Breakpoint here
+  return 0;
+}