[lldb][AArch64] Add SME streaming vector length pseudo register

[DavidSpickett]

Summary

Reading the SVE registers of streaming mode from non-streaming mode,
and vice versa, returns invalid data. As their state is reset each
time you switch between them.

However, the vector length is distinct between the two modes.

The existing register "vg" will always report the vector length for
the current mode and this change adds "svg" which will always return
the streaming vector length.

non-streaming mode: vg  = non-streaming vector length
                    svg = streaming vector length
    streaming mode: vg  = streaming vector length
                    svg = streaming vector length

The content of svg is read from the NT_ARM_SSVE header, even if
we are in non-streaming mode. Which we are allowed to do
(the result is just missing the register data in this situation).

It is read only for the moment. It may be made writeable in future
patches. It has been added to the SME register set and I've converted
that into a struct for easier handling.

The SVE dynamic size test has been updated to check the expected
svg values as it is already setup to break just after a mode switch.

[jasonmolenda]

Just curious -- would it be better to have a single vg register shown to the user which is the vector length in non-streaming mode when the processor is in non-streaming mode, and the vector length in streaming mode when the processor is in streaming mode? I haven't worked with a target supporting SSVE mode, but if I'm following correctly we can show the SVE registers in non-streaming mode (and vg) and we can show the SSVE registers in streaming mode (and svg). As you say above, we can show svg when in non-streaming mode but we can't show vg when in streaming mode. Should we only show a single vg for the currently-available registers?

[DavidSpickett]

As you say above, we can show svg when in non-streaming mode but we can't show vg when in streaming mode. Should we only show a single vg for the currently-available registers?

I will admit I don't have practical experience to justify adding svg, I'm just guessing it would be nice to know what vector length you're about to switch into. That argument plays just as well for streaming -> non-streaming switches though.

The problem with that is we don't have an obvious name (or place) to show the non-streaming vg. For svg, we're adding the new register set to hold svcr anyway (which is essential because it's the only way to know the active mode). The architecture doesn't assign a new name to the non-streaming mode, so you could call it "svevg" and "ssvevg" but that's no help to the average person who hasn't (and shouldn't have to) read the manual.

Given that this isn't an essential feature, I'll put this on the backburner. At least until after 17, once I have more experience writing SME code.

[jasonmolenda]

In D155269#4505021, @DavidSpickett wrote:

As you say above, we can show svg when in non-streaming mode but we can't show vg when in streaming mode. Should we only show a single vg for the currently-available registers?

I will admit I don't have practical experience to justify adding svg, I'm just guessing it would be nice to know what vector length you're about to switch into. That argument plays just as well for streaming -> non-streaming switches though.

Oh yeah, don't take my question too seriously, my suggestion is formed entirely by looking at your patches and never having seen SVE or SME code myself.

[tschuett]

Could you offer higher abstractions? Show me the current SVME vector length? Show me the current SVME mode?

[DavidSpickett]

Could you offer higher abstractions? Show me the current SVME vector length? Show me the current SVME mode?

Adding it to process status is along those lines, we have stuff like the number of addressable bits there right now. Overall I prefer the registers route just for visibility but we'll see what the early users find.

[tschuett]

I would never question giving low-level access to the registers. As you mentioned less experienced users could accidentally switch between the modes with knowing.

lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_dynamic_resize/TestSVEThreadedDynamic.py

262

Newline please

[DavidSpickett]

I would never question giving low-level access to the registers.

Well in your defense, both svg and svcr will actually be pseudo registers. So the user isn't getting access to the "real" ones either way, we're emulating the behaviour with ptrace commands.

As you mentioned less experienced users could accidentally switch between the modes with knowing.

If we follow the kernel to the letter you can also mode switch by writing floating point registers while in streaming mode. That currently doesn't happen due to the way we model v registers as subsets of z but I might have to change that and if I do, that's another potential pitfall.

Ideally we would have as few routes to mode switch via the debugger as possible. Writing to the streaming vector control register is the single route I'd support given the choice.

[jasonmolenda]

In D155269#4509130, @DavidSpickett wrote:

Ideally we would have as few routes to mode switch via the debugger as possible. Writing to the streaming vector control register is the single route I'd support given the choice.

I wonder about function calls when in streaming mode, where someone might not even realize they're in it. Does lldb-server support QSaveRegisterState / QRestoreRegisterState around inferior function calls, or does lldb use the g/G packet (or write all the registers individually)? g/G are probably going to fetch / write all the floating point registers and reset the mode if you did a function call while in streaming mode?

[DavidSpickett]

I think in https://reviews.llvm.org/D154926, lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_static_config/TestSVERegisters.py addresses this. If what you mean is you are stopped in streaming mode, you evaluate an expression that may call a function which takes you into another mode.

If not, give me an example and I'll try to test it. This is the first I'm hearing of QSaveRegisterState / QRestoreRegisterState.

g/G are probably going to fetch / write all the floating point registers and reset the mode if you did a function call while in streaming mode?

We'd have to order them carefully I expect. Or say something like if we're restoring floating point and SVE registers, just ignore the floating point because we're about to supersede it.

I am not 100% sure that one cannot implement streaming SVE as a completely separate register state, I will be checking that today. If you can then it will complicate things in theory.

[jasonmolenda]

In D155269#4509364, @DavidSpickett wrote:

I think in https://reviews.llvm.org/D154926, lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_static_config/TestSVERegisters.py addresses this. If what you mean is you are stopped in streaming mode, you evaluate an expression that may call a function which takes you into another mode.

Oh it sounds like you've already been thinking about this, no worries. I was thinking about the mere act of restoring the register state. When lldb does a function call, it saves all of the register contents, does the function call, then restores the registers. This was traditionally done with the g/G (read/write the entire register context), if that is supported by the stub. Alternatively registers can all be read/written individually. As a simplification of all of this, and to avoid using g/G, we added QSaveRegisterState which tells the stub (debugserver etc) to save the current register context, and then after the inferior function call has completed, QRestoreRegisterState to restore them all.

In the process of restoring / writing the registers, I expect we will try to write the floating point register contents into the process which would drop it out of SSVE mode?

[DavidSpickett]

As a simplification of all of this, and to avoid using g/G, we added QSaveRegisterState which tells the stub (debugserver etc) to save the current register context, and then after the inferior function call has completed, QRestoreRegisterState to restore them all.

Looks like we do support that in lldb-server, I just hadn't come across it because I've been down at the native process level. I've updated ReadAllRegisterValues down there so it will restore to whatever the saved mode was.

In the process of restoring / writing the registers, I expect we will try to write the floating point register contents into the process which would drop it out of SSVE mode?

Right, yes it would if we follow this statement from the kernel docs:

Note that when SME is present and streaming SVE mode is in use the FPSIMD subset of registers will be read via NT_ARM_SVE and NT_ARM_SVE writes will exit streaming mode in the target.

I am talking to our kernel folks to understand the background to that. I suspect that it may be the case that for example, writing to the bottom 128 bits of streaming mode z0 may not be reflected in the SIMD unit's v0. Or at least, one could build a core that acted that way.

[DavidSpickett]

I suspect that it may be the case that for example, writing to the bottom 128 bits of streaming mode z0 may not be reflected in the SIMD unit's v0. Or at least, one could build a core that acted that way.

But the user would be very confused by this given that if you are stopped here in streaming mode:

mov v0.d[0] x0

That instruction would actually see the bottom 128 bits of streaming z0, even if elsewhere there is another, inactive v0 register in the core. If the user then does register write v0 {....} I doubt they would expect it to mode switch and write to a whole different v0, it should update z0.

So even if on a hardware level this configuration is possible, I don't think it's good to have the debugger act this way. Better that we think of this from the perspective of a running instruction, what would it see and therefore what would the user expect to happen.

[DavidSpickett]

I am talking to our kernel folks to understand the background to that.

The result is that yes cores an implement it as separate state but as mentioned here, taking that into account in lldb would be rather confusing in 99% of situations. If we simply want to read what instructions in the current context will see, using the bottom 128 bits of the Z registers is fine.

[DavidSpickett]

Turns out that for ZA support, we need to know the streaming vector length regardless of current mode. So SVG will be implemented as part of ZA support instead.

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.h

 //===-- NativeRegisterContextLinux_arm64.h ---------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
private:
   struct user_pac_mask {
     uint64_t data_mask;
     uint64_t insn_mask;
   };
 
   struct user_pac_mask m_pac_mask;
 
   uint64_t m_mte_ctrl_reg;
-  uint64_t m_sme_ctrl_reg;
+
+  struct sme_regs {
+    uint64_t ctrl_reg;
+    uint64_t svg_reg;
+  };
+
+  struct sme_regs m_sme_regs;
 
   struct tls_regs {
     uint64_t tpidr_reg;
     // Only valid when SME is present.
     uint64_t tpidr2_reg;
   };
 
   struct tls_regs m_tls_regs;
private:
 
   Status ReadTLS();
 
   Status WriteTLS();
 
   // SVCR is a pseudo register and we do not allow writes to it.
   Status ReadSMEControl();
 
+  Status ReadSMESVG();
+
   bool IsSVE(unsigned reg) const;
   bool IsPAuth(unsigned reg) const;
   bool IsMTE(unsigned reg) const;
   bool IsTLS(unsigned reg) const;
   bool IsSME(unsigned reg) const;
 
   uint64_t GetSVERegVG() { return CurrentSVEStateData().m_header.vl / 8; }
 
   void SetSVERegVG(uint64_t vg) { CurrentSVEStateData().m_header.vl = vg * 8; }
 
   void *GetPACMask() { return &m_pac_mask; }
 
   void *GetMTEControl() { return &m_mte_ctrl_reg; }
 
   void *GetTLS() { return &m_tls_regs; }
 
-  void *GetSMEControl() { return &m_sme_ctrl_reg; }
+  void *GetSME() { return &m_sme_regs; }
 
   void *GetSVEBuffer() { return CurrentSVEStateData().m_buffer.data(); }
 
   size_t GetSVEBufferSize() { return CurrentSVEStateData().m_buffer.size(); }
 
   void *GetSVEHeader() { return &(CurrentSVEStateData().m_header); }
 
   size_t GetSVEHeaderSize() { return sizeof(CurrentSVEStateData().m_header); }
 
   size_t GetPACMaskSize() { return sizeof(m_pac_mask); }
 
   size_t GetMTEControlSize() { return sizeof(m_mte_ctrl_reg); }
 
   size_t GetTLSSize() { return m_tls_size; }
 
-  size_t GetSMEControlSize() { return sizeof(m_sme_ctrl_reg); }
+  size_t GetSMESize() { return sizeof(m_sme_regs); }
 
   llvm::Error ReadHardwareDebugInfo() override;
 
   llvm::Error WriteHardwareDebugRegs(DREGType hwbType) override;
 
   uint32_t CalculateFprOffset(const RegisterInfo *reg_info) const;
 
   RegisterInfoPOSIX_arm64 &GetRegisterInfo() const;

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

 //===-- NativeRegisterContextLinux_arm64.cpp ------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
: NativeRegisterContextRegisterInfo(native_thread,
       NativeRegisterContextLinux(native_thread), m_sve_state_data(NT_ARM_SVE),
       m_ssve_state_data(NT_ARM_SSVE) {
   ::memset(&m_fpr, 0, sizeof(m_fpr));
   ::memset(&m_gpr_arm64, 0, sizeof(m_gpr_arm64));
   ::memset(&m_hwp_regs, 0, sizeof(m_hwp_regs));
   ::memset(&m_hbp_regs, 0, sizeof(m_hbp_regs));
   ::memset(&m_pac_mask, 0, sizeof(m_pac_mask));
   ::memset(&m_tls_regs, 0, sizeof(m_tls_regs));
+  ::memset(&m_sme_regs, 0, sizeof(m_sme_regs));
 
-  m_sme_ctrl_reg = 0;
   m_mte_ctrl_reg = 0;
 
   // 16 is just a maximum value, query hardware for actual watchpoint count
   m_max_hwp_supported = 16;
   m_max_hbp_supported = 16;
 
   m_refresh_hwdebug_info = true;
 
if (IsGPR(reg)) {
 
     offset = reg_info->byte_offset - GetRegisterInfo().GetMTEOffset();
     assert(offset < GetMTEControlSize());
     src = (uint8_t *)GetMTEControl() + offset;
   } else if (IsSME(reg)) {
     // This is a pseduo so it never fails.
     ReadSMEControl();
 
+    error = ReadSMESVG();
+    if (error.Fail())
+      return error;
+
     offset = reg_info->byte_offset - GetRegisterInfo().GetSMEOffset();
-    assert(offset < GetSMEControlSize());
-    src = (uint8_t *)GetSMEControl() + offset;
+    assert(offset < GetSMESize());
+    src = (uint8_t *)GetSME() + offset;
   } else
     return Status("failed - register wasn't recognized to be a GPR or an FPR, "
                   "write strategy unknown");
 
   reg_value.SetFromMemoryData(*reg_info, src, reg_info->byte_size,
                               eByteOrderLittle, error);
 
   return error;
if (IsGPR(reg)) {
 
     offset = reg_info->byte_offset - GetRegisterInfo().GetTLSOffset();
     assert(offset < GetTLSSize());
     dst = (uint8_t *)GetTLS() + offset;
     ::memcpy(dst, reg_value.GetBytes(), reg_info->byte_size);
 
     return WriteTLS();
   } else if (IsSME(reg)) {
-    return Status("Writing to SVCR is not supported.");
+    return Status("Writing to SVCR or SVG not supported.");
   }
 
   return Status("Failed to write register value");
 }
 
 Status NativeRegisterContextLinux_arm64::ReadAllRegisterValues(
     lldb::WritableDataBufferSP &data_sp) {
   // AArch64 register data must contain GPRs, either FPR or SVE registers

 }
 
 Status NativeRegisterContextLinux_arm64::ReadSMEControl() {
   // The real register is SVCR and is accessible from EL0. However we don't want
   // to have to JIT code into the target process so we'll just recreate it using
   // what we know from ptrace.
   // Bit 1 indicates whether streaming mode is active.
   // Bit 2 indicates whether the array storage is active (not yet implemented).
-  m_sme_ctrl_reg = m_sve_state == SVEState::Streaming;
+  m_sme_regs.ctrl_reg = m_sve_state == SVEState::Streaming;
   return {};
 }
 
+Status NativeRegisterContextLinux_arm64::ReadSMESVG() {
+  // This register is the streaming vector length, which needs to be read from
+  // the NT_ARM_SSVE set regardless of the active mode.
+  Status error;
+
+  SVEState previous_sve_state = m_sve_state;
+  m_sve_state = SVEState::Streaming;
+
+  error = ReadSVEHeader();
+  if (error.Success())
+    m_sme_regs.svg_reg = CurrentSVEStateData().m_header.vl / 8;
+
+  m_sve_state = previous_sve_state;
+  return error;
+}
+
 Status NativeRegisterContextLinux_arm64::ReadMTEControl() {
   Status error;
 
   if (m_mte_ctrl_is_valid)
     return error;
 
   struct iovec ioVec;
   ioVec.iov_base = GetMTEControl();

lldb/source/Plugins/Process/Utility/RegisterInfoPOSIX_arm64.cpp

 //===-- RegisterInfoPOSIX_arm64.cpp ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 
static lldb_private::RegisterInfo g_register_infos_mte[] = {
     DEFINE_EXTENSION_REG(mte_ctrl)};
 
 static lldb_private::RegisterInfo g_register_infos_tls[] = {
     DEFINE_EXTENSION_REG(tpidr),
     // Only present when SME is present
     DEFINE_EXTENSION_REG(tpidr2)};
 
 static lldb_private::RegisterInfo g_register_infos_sme[] = {
-    DEFINE_EXTENSION_REG(svcr)};
+    DEFINE_EXTENSION_REG(svcr), DEFINE_EXTENSION_REG(svg)};
 
 // Number of register sets provided by this context.
 enum {
   k_num_gpr_registers = gpr_w28 - gpr_x0 + 1,
   k_num_fpr_registers = fpu_fpcr - fpu_v0 + 1,
   k_num_sve_registers = sve_ffr - sve_vg + 1,
   k_num_mte_register = 1,
   // Number of TLS registers is dynamic so it is not listed here.
   k_num_pauth_register = 2,
-  k_num_sme_register = 1,
+  k_num_sme_register = 2,
   k_num_register_sets_default = 2,
   k_num_register_sets = 3
 };
 
 // ARM64 general purpose registers.
 static const uint32_t g_gpr_regnums_arm64[] = {
     gpr_x0,  gpr_x1,   gpr_x2,  gpr_x3,
     gpr_x4,  gpr_x5,   gpr_x6,  gpr_x7,
m_per_regset_regnum_range[m_register_set_count] =
       std::make_pair(tls_regnum, m_dynamic_reg_infos.size());
   m_dynamic_reg_sets.push_back(
       {"Thread Local Storage Registers", "tls", num_regs, nullptr});
   m_dynamic_reg_sets.back().registers = m_tls_regnum_collection.data();
 }
 
 void RegisterInfoPOSIX_arm64::AddRegSetSME() {
   uint32_t sme_regnum = m_dynamic_reg_infos.size();
-  m_sme_regnum_collection.push_back(sme_regnum);
-  m_dynamic_reg_infos.push_back(g_register_infos_sme[0]);
-  m_dynamic_reg_infos[sme_regnum].byte_offset =
-      m_dynamic_reg_infos[sme_regnum - 1].byte_offset +
-      m_dynamic_reg_infos[sme_regnum - 1].byte_size;
-  m_dynamic_reg_infos[sme_regnum].kinds[lldb::eRegisterKindLLDB] = sme_regnum;
+  for (uint32_t i = 0; i < k_num_sme_register; i++) {
+    m_sme_regnum_collection.push_back(sme_regnum + i);
+    m_dynamic_reg_infos.push_back(g_register_infos_sme[i]);
+    m_dynamic_reg_infos[sme_regnum + i].byte_offset =
+        m_dynamic_reg_infos[sme_regnum + i - 1].byte_offset +
+        m_dynamic_reg_infos[sme_regnum + i - 1].byte_size;
+    m_dynamic_reg_infos[sme_regnum + i].kinds[lldb::eRegisterKindLLDB] =
+        sme_regnum + i;
+  }
 
   m_per_regset_regnum_range[m_register_set_count] =
-      std::make_pair(sme_regnum, sme_regnum + 1);
+      std::make_pair(sme_regnum, m_dynamic_reg_infos.size());
   m_dynamic_reg_sets.push_back(g_reg_set_sme_arm64);
   m_dynamic_reg_sets.back().registers = m_sme_regnum_collection.data();
 }
 
 uint32_t RegisterInfoPOSIX_arm64::ConfigureVectorLength(uint32_t sve_vq) {
   // sve_vq contains SVE Quad vector length in context of AArch64 SVE.
   // SVE register infos if enabled cannot be disabled by selecting sve_vq = 0.
   // Also if an invalid or previously set vector length is passed to this

lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_dynamic_resize/TestSVEThreadedDynamic.py

def check_sve_registers(self, vg_test_value):
         for i in range(16):
             p_regs_value = (
                 "{" + " ".join(p_value_bytes[i % 5] for _ in range(p_reg_size)) + "}"
             )
             self.expect("register read " + "p%i" % (i), substrs=[p_regs_value])
 
         self.expect("register read ffr", substrs=[p_regs_value])
 
+    def build_for_mode(self, mode):
+        cflags = "-march=armv8-a+sve -lpthread"
+        if mode == Mode.SSVE:
+            cflags += " -DUSE_SSVE"
+        self.build(dictionary={"CFLAGS_EXTRAS": cflags})
+
     def run_sve_test(self, mode):
         if (mode == Mode.SVE) and not self.isAArch64SVE():
             self.skipTest("SVE registers must be supported.")
 
         if (mode == Mode.SSVE) and not self.isAArch64SME():
             self.skipTest("Streaming SVE registers must be supported.")
 
-        cflags = "-march=armv8-a+sve -lpthread"
-        if mode == Mode.SSVE:
-            cflags += " -DUSE_SSVE"
-        self.build(dictionary={"CFLAGS_EXTRAS": cflags})
+        self.build_for_mode(mode)
 
-        self.build()
         supported_vg = self.get_supported_vg()
 
         if not (2 in supported_vg and 4 in supported_vg):
             self.skipTest("Not all required SVE vector lengths are supported.")
 
         main_thread_stop_line = line_number("main.c", "// Break in main thread")
         lldbutil.run_break_set_by_file_and_line(self, "main.c", main_thread_stop_line)
 
def test_sve_registers_dynamic_config(self):
         self.run_sve_test(Mode.SVE)
 
     @no_debug_info_test
     @skipIf(archs=no_match(["aarch64"]))
     @skipIf(oslist=no_match(["linux"]))
     def test_ssve_registers_dynamic_config(self):
         """Test AArch64 SSVE registers multi-threaded dynamic resize."""
         self.run_sve_test(Mode.SSVE)
+
+    def setup_svg_test(self, mode):
+        # Even when running in SVE mode, we need access to SVG for these tests.
+        if not self.isAArch64SME():
+            self.skipTest("Streaming SVE registers must be supported.")
+
+        self.build_for_mode(mode)
+
+        supported_vg = self.get_supported_vg()
+
+        main_thread_stop_line = line_number("main.c", "// Break in main thread")
+        lldbutil.run_break_set_by_file_and_line(self, "main.c", main_thread_stop_line)
+
+        self.runCmd("run", RUN_SUCCEEDED)
+
+        process = self.dbg.GetSelectedTarget().GetProcess()
+
+        self.expect(
+            "thread info 1",
+            STOPPED_DUE_TO_BREAKPOINT,
+            substrs=["stop reason = breakpoint"],
+        )
+
+        target = self.dbg.GetSelectedTarget()
+        process = target.GetProcess()
+
+        return process, supported_vg
+
+    def read_vg(self, process):
+        registerSets = process.GetThreadAtIndex(0).GetFrameAtIndex(0).GetRegisters()
+        sve_registers = registerSets.GetFirstValueByName("Scalable Vector Extension Registers")
+        return sve_registers.GetChildMemberWithName("vg").GetValueAsUnsigned()
+
+    def read_svg(self, process):
+        registerSets = process.GetThreadAtIndex(0).GetFrameAtIndex(0).GetRegisters()
+        sve_registers = registerSets.GetFirstValueByName("Scalable Matrix Extension Registers")
+        return sve_registers.GetChildMemberWithName("svg").GetValueAsUnsigned()
+
+    def do_svg_test(self, process, vgs, expected_svgs):
+        for vg, svg in zip(vgs, expected_svgs):
+            self.runCmd("register write vg {}".format(vg))
+            self.assertEqual(svg, self.read_svg(process))
+
+    @no_debug_info_test
+    @skipIf(archs=no_match(["aarch64"]))
+    @skipIf(oslist=no_match(["linux"]))
+    def test_svg_sve_mode(self):
+        """ When in SVE mode, svg should remain constant as we change vg. """
+        process, supported_vg = self.setup_svg_test(Mode.SVE)
+        svg = self.read_svg(process)
+        self.do_svg_test(process, supported_vg, [svg]*len(supported_vg))
+
+    @no_debug_info_test
+    @skipIf(archs=no_match(["aarch64"]))
+    @skipIf(oslist=no_match(["linux"]))
+    def test_svg_ssve_mode(self):
+        """ When in SSVE mode, changing vg should change svg to the same value. """
+        process, supported_vg = self.setup_svg_test(Mode.SSVE)
+        self.do_svg_test(process, supported_vg, supported_vg)
 No newline at end of file