[lldb][AArch64] Add SME's streaming vector control register

[DavidSpickett]

Summary

Software can tell if it is in streaming SVE mode by checking
the Streaming Vector Control Register (SVCR).

"E3.1.9 SVCR, Streaming Vector Control Register" in
"Arm® Architecture Reference Manual Supplement, The Scalable Matrix
Extension (SME), for Armv9-A"

https://developer.arm.com/documentation/ddi0616/latest/

This is especially useful for debug because the names of the
SVE registers are the same betweeen non-streaming and streaming mode.

The Linux Kernel chose to not put this register behind ptrace,
and it can be read from EL0. However, this would mean running code
in process to read it. That can be done but we already know all
the information just from ptrace.

So this is a pseudo register that matches the architectural
content. The name is just "svcr", which aligns with GDB's proposed naming,
and it's added to the existing SME register set.

The SVCR register contains two bits:
0 : Whether streaming SVE mode is enabled (SM)
1 : Whether the array storage is enabled (ZA)

Array storage can be active when streaming mode is not, so this register
can have any permutation of those bits.

This register is currently read only. We can emulate the result of
writing to it, using ptrace. However at this point the utility of that
is not obvious.

Existing tests have been updated to check for appropriate SVCR values
at various points.

Given that this register is a read only pseudo, there is no need
to save and restore it around expressions.

[omjavaid]

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

333

typo: pseudo register?

1172

is there a need here to check if m_sve_state is valid?

[DavidSpickett]

Rebase, fix typo.

[DavidSpickett]

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

1172

The only "invalid" state we can check at the moment is SVEState::Unknown. Which would translate here to a value of 0 in the register, which isn't incorrect, it's just pesimistic.

By the time you're printing this register you'd likely have checked the state fully. Maybe it would come back as SVEState::Disabled, but in that case you wouldn't have SME anyway so this register isn't available to you.

[DavidSpickett]

Add missing newline.

lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_static_config/TestSVERegisters.py

32

This is the check promised in the previous patch. It ensures that when we return from evaluating the expression we do restore the previous operating mode.

[omjavaid]

I guess there is not much gain from telling the user between sve/ssve mode for the moment. Can we defer this patch till SME registers are implemented?

[DavidSpickett]

In some sense it already is, it'll go in after the streaming mode SVE registers. Should we wait until we can show the full content including the ZA bit? Yeah, why not, we're in no rush.

I have a patch locally for ZA support that I'm working on. I'll reorder this to after that and when I update this, it will have the full SVCR contents.

[DavidSpickett]

Rebase and reimplement this on top of ZA/SVG.

[DavidSpickett]

Correct regiter -> register in comment.

[DavidSpickett]

Rebase

[DavidSpickett]

ZA is now part of the SME set so SVCR is stuck in before SVG.

SVCR
SVG
ZA

Updated the various indexes to reflect that.

[DavidSpickett]

Format with black.

[DavidSpickett]

Rebase

[DavidSpickett]

Rebase, use new sme_pseudo_regs naming.

[DavidSpickett]

Rebase

[omjavaid]

Are

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

1181

Shouldnt we build a relevant error message here incase ReadSMEControl fails on ReadZAHeader

lldb/test/API/commands/register/register/aarch64_za_register/za_save_restore/TestZARegisterSaveRestore.py

168

should we also add a svcr write test here?

[DavidSpickett]

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

1181

I think this should be return error in fact, let me check what that does.

lldb/test/API/commands/register/register/aarch64_za_register/za_save_restore/TestZARegisterSaveRestore.py

168

It's read only. We could allow writes but it introduces a lot more ways to change state, most of which aren't obviously useful for the user. If someone really wants it, it can be added.

I will note this in a comment here.

[DavidSpickett]

• Return error in SVCR read.
• Note absence of write tests of SVCR because it's read only.

[DavidSpickett]

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

1181

I now return the underlying error so it won't fail silently.

lldb/test/API/commands/register/register/aarch64_za_register/za_save_restore/TestZARegisterSaveRestore.py

168

I've actually added comments to the other file, where it makes more sense that you might be writing to svcr.

Here the whole test is about having the program setup the registers, and modify them. So there's less expectation that you'd check writing here.

The fact that SVCR is read only (as is SVG) will be documented in fully https://github.com/llvm/llvm-project/pull/66767.

[omjavaid]

This is good too. Thanks!!!

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.h

 //===-- NativeRegisterContextLinux_arm64.h ---------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
struct user_pac_mask {
     uint64_t insn_mask;
   };
 
   struct user_pac_mask m_pac_mask;
 
   uint64_t m_mte_ctrl_reg;
 
   struct sme_regs {
+    uint64_t ctrl_reg;
     uint64_t svg_reg;
   };
 
   struct sme_regs m_sme_regs;
 
   struct tls_regs {
     uint64_t tpidr_reg;
     // Only valid when SME is present.
private:
   Status ReadZA();
 
   Status WriteZA();
 
   // No WriteZAHeader because writing only the header will disable ZA.
   // Instead use WriteZA and ensure you have the correct ZA buffer size set
   // beforehand if you wish to disable it.
 
+  // SVCR is a pseudo register and we do not allow writes to it.
+  Status ReadSMEControl();
+
   bool IsSVE(unsigned reg) const;
   bool IsSME(unsigned reg) const;
   bool IsPAuth(unsigned reg) const;
   bool IsMTE(unsigned reg) const;
   bool IsTLS(unsigned reg) const;
 
   uint64_t GetSVERegVG() { return m_sve_header.vl / 8; }
 

lldb/source/Plugins/Process/Linux/NativeRegisterContextLinux_arm64.cpp

 //===-- NativeRegisterContextLinux_arm64.cpp ------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
if (IsGPR(reg)) {
     error = ReadMTEControl();
     if (error.Fail())
       return error;
 
     offset = reg_info->byte_offset - GetRegisterInfo().GetMTEOffset();
     assert(offset < GetMTEControlSize());
     src = (uint8_t *)GetMTEControl() + offset;
   } else if (IsSME(reg)) {
     if (GetRegisterInfo().IsSMERegZA(reg)) {
       error = ReadZAHeader();
       if (error.Fail())
         return error;
 
       // If there is only a header and no registers, ZA is inactive. Read as 0
       // in this case.
       if (m_za_header.size == sizeof(m_za_header)) {
         // This will get reconfigured/reset later, so we are safe to use it.
if (GetRegisterInfo().IsSMERegZA(reg)) {
       // Therefore its effective byte offset is always 0 even if it isn't 0 within
       // the SME register set.
       src = (uint8_t *)GetZABuffer() + GetZAHeaderSize();
     } else {
       error = ReadSMESVG();
       if (error.Fail())
         return error;
 
+      // This is a psuedo so it never fails.
+      ReadSMEControl();
+
       offset = reg_info->byte_offset - GetRegisterInfo().GetSMEOffset();
       assert(offset < GetSMEBufferSize());
       src = (uint8_t *)GetSMEBuffer() + offset;
     }
   } else
     return Status("failed - register wasn't recognized to be a GPR or an FPR, "
                   "write strategy unknown");
 
if (IsGPR(reg)) {
     offset = reg_info->byte_offset - GetRegisterInfo().GetTLSOffset();
     assert(offset < GetTLSBufferSize());
     dst = (uint8_t *)GetTLSBuffer() + offset;
     ::memcpy(dst, reg_value.GetBytes(), reg_info->byte_size);
 
     return WriteTLS();
   } else if (IsSME(reg)) {
     if (!GetRegisterInfo().IsSMERegZA(reg))
-      return Status("Writing to SVG is not supported.");
+      return Status("Writing to SVG or SVCR is not supported.");
 
     error = ReadZA();
     if (error.Fail())
       return error;
 
     // ZA is part of the SME set but not stored with the other SME registers.
     // So its byte offset is effectively always 0.
     dst = (uint8_t *)GetZABuffer() + GetZAHeaderSize();

 enum RegisterSetType : uint32_t {
   GPR,
   SVE, // Used for SVE and SSVE.
   FPR, // When there is no SVE, or SVE in FPSIMD mode.
   // Pointer authentication registers are read only, so not included here.
   MTE,
   TLS,
   ZA,
-  // SME pseudo registers are read only.
+  // SME not here because SVG and SVCR are read only pseudo registers derived
+  // from the other registers.
 };
 
 static uint8_t *AddRegisterSetType(uint8_t *dst,
                                    RegisterSetType register_set_type) {
   *(reinterpret_cast<uint32_t *>(dst)) = register_set_type;
   return dst + sizeof(uint32_t);
 }
 
Status NativeRegisterContextLinux_arm64::WriteAllSVE() {
 
   m_sve_buffer_is_valid = false;
   m_sve_header_is_valid = false;
   m_fpu_is_valid = false;
 
   return WriteRegisterSet(&ioVec, GetSVEBufferSize(), GetSVERegSet());
 }
 
+Status NativeRegisterContextLinux_arm64::ReadSMEControl() {
+  // The real register is SVCR and is accessible from EL0. However we don't want
+  // to have to JIT code into the target process so we'll just recreate it using
+  // what we know from ptrace.
+
+  // Bit 0 indicates whether streaming mode is active.
+  m_sme_regs.ctrl_reg = m_sve_state == SVEState::Streaming;
+
+  // Bit 1 indicates whether the array storage is active.
+  // It is active if we can read the header and the size field tells us that
+  // there is register data following it.
+  Status error = ReadZAHeader();
+  if (error.Success() && (m_za_header.size > sizeof(m_za_header)))
+    m_sme_regs.ctrl_reg |= 2;
+
+  return {};
+}
+
 Status NativeRegisterContextLinux_arm64::ReadMTEControl() {
   Status error;
 
   if (m_mte_ctrl_is_valid)
     return error;
 
   struct iovec ioVec;
   ioVec.iov_base = GetMTEControl();

lldb/source/Plugins/Process/Utility/RegisterInfoPOSIX_arm64.cpp

 //===-- RegisterInfoPOSIX_arm64.cpp ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===---------------------------------------------------------------------===//
 
static lldb_private::RegisterInfo g_register_infos_mte[] = {
     DEFINE_EXTENSION_REG(mte_ctrl)};
 
 static lldb_private::RegisterInfo g_register_infos_tls[] = {
     DEFINE_EXTENSION_REG(tpidr),
     // Only present when SME is present
     DEFINE_EXTENSION_REG(tpidr2)};
 
 static lldb_private::RegisterInfo g_register_infos_sme[] = {
+    DEFINE_EXTENSION_REG(svcr),
     DEFINE_EXTENSION_REG(svg),
     // 16 is a default size we will change later.
     {"za", nullptr, 16, 0, lldb::eEncodingVector, lldb::eFormatVectorOfUInt8,
      KIND_ALL_INVALID, nullptr, nullptr, nullptr}};
 
 // Number of register sets provided by this context.
 enum {
   k_num_gpr_registers = gpr_w28 - gpr_x0 + 1,
   k_num_fpr_registers = fpu_fpcr - fpu_v0 + 1,
   k_num_sve_registers = sve_ffr - sve_vg + 1,
   k_num_mte_register = 1,
   // Number of TLS registers is dynamic so it is not listed here.
   k_num_pauth_register = 2,
-  k_num_sme_register = 2,
+  k_num_sme_register = 3,
   k_num_register_sets_default = 2,
   k_num_register_sets = 3
 };
 
 // ARM64 general purpose registers.
 static const uint32_t g_gpr_regnums_arm64[] = {
     gpr_x0,  gpr_x1,   gpr_x2,  gpr_x3,
     gpr_x4,  gpr_x5,   gpr_x6,  gpr_x7,
if (!VectorSizeIsValid(za_vq) || m_za_reg_vq == za_vq)
     return;
 
   m_za_reg_vq = za_vq;
 
   // For SVE changes, we replace m_register_info_p completely. ZA is in a
   // dynamic set and is just 1 register so we make an exception to const here.
   lldb_private::RegisterInfo *non_const_reginfo =
       const_cast<lldb_private::RegisterInfo *>(m_register_info_p);
-  non_const_reginfo[m_sme_regnum_collection[1]].byte_size =
+  non_const_reginfo[m_sme_regnum_collection[2]].byte_size =
       (za_vq * 16) * (za_vq * 16);
 }
 
 bool RegisterInfoPOSIX_arm64::IsSVEReg(unsigned reg) const {
   if (m_vector_reg_vq > eVectorQuadwordAArch64)
     return (sve_vg <= reg && reg <= sve_ffr);
   else
     return false;
 }
 
 bool RegisterInfoPOSIX_arm64::IsSVEZReg(unsigned reg) const {
   return (sve_z0 <= reg && reg <= sve_z31);
 }
 
 bool RegisterInfoPOSIX_arm64::IsSVEPReg(unsigned reg) const {
   return (sve_p0 <= reg && reg <= sve_p15);
 }
 
 bool RegisterInfoPOSIX_arm64::IsSVERegVG(unsigned reg) const {
   return sve_vg == reg;
 }
 
 bool RegisterInfoPOSIX_arm64::IsSMERegZA(unsigned reg) const {
-  return reg == m_sme_regnum_collection[1];
+  return reg == m_sme_regnum_collection[2];
 }
 
 bool RegisterInfoPOSIX_arm64::IsPAuthReg(unsigned reg) const {
   return llvm::is_contained(pauth_regnum_collection, reg);
 }
 
 bool RegisterInfoPOSIX_arm64::IsMTEReg(unsigned reg) const {
   return llvm::is_contained(m_mte_regnum_collection, reg);

 
 uint32_t RegisterInfoPOSIX_arm64::GetRegNumFPCR() const { return fpu_fpcr; }
 
 uint32_t RegisterInfoPOSIX_arm64::GetRegNumFPSR() const { return fpu_fpsr; }
 
 uint32_t RegisterInfoPOSIX_arm64::GetRegNumSVEVG() const { return sve_vg; }
 
 uint32_t RegisterInfoPOSIX_arm64::GetRegNumSMESVG() const {
-  return m_sme_regnum_collection[0];
+  return m_sme_regnum_collection[1];
 }
 
 uint32_t RegisterInfoPOSIX_arm64::GetPAuthOffset() const {
   return m_register_info_p[pauth_regnum_collection[0]].byte_offset;
 }
 
 uint32_t RegisterInfoPOSIX_arm64::GetMTEOffset() const {
   return m_register_info_p[m_mte_regnum_collection[0]].byte_offset;

lldb/test/API/commands/register/register/aarch64_dynamic_regset/TestArm64DynamicRegsets.py

def test_aarch64_dynamic_regset_config_sme(self):
         self.expect("register read za", substrs=[za_value])
 
         # SVG should match VG because we're in streaming mode.
 
         self.assertTrue(sme_registers.IsValid())
         svg = sme_registers.GetChildMemberWithName("svg").GetValueAsUnsigned()
         self.assertEqual(vg, svg)
 
+        # SVCR should be SVCR.SM | SVCR.ZA aka 3 because streaming mode is on
+        # and ZA is enabled.
+        svcr = sme_registers.GetChildMemberWithName("svcr").GetValueAsUnsigned()
+        self.assertEqual(3, svcr)
+
     @no_debug_info_test
     @skipIf(archs=no_match(["aarch64"]))
     @skipIf(oslist=no_match(["linux"]))
     def test_aarch64_dynamic_regset_config_sme_za_disabled(self):
         """Test that ZA shows as 0s when disabled and can be enabled by writing
            to it."""
         if not self.isAArch64SME():
             self.skipTest("SME must be present.")
 
         # No argument, so ZA will be disabled when we break.
         register_sets = self.setup_register_config_test()
 
         # vg is the non-streaming vg as we are in non-streaming mode, so we need
         # to use svg.
         sme_registers = register_sets.GetFirstValueByName(
             "Scalable Matrix Extension Registers")
         self.assertTrue(sme_registers.IsValid())
         svg = sme_registers.GetChildMemberWithName("svg").GetValueAsUnsigned()
 
+        # We are not in streaming mode, ZA is disabled, so this should be 0.
+        svcr = sme_registers.GetChildMemberWithName("svcr").GetValueAsUnsigned()
+        self.assertEqual(0, svcr)
+
         svl = svg * 8
         # A disabled ZA is shown as all 0s.
         self.expect("register read za", substrs=[self.make_za_value(svl, lambda r: 0)])
         za_value = self.make_za_value(svl, lambda r:r+1)
         # Writing to it enables ZA, so the value should be there when we read
         # it back.
         self.runCmd("register write za '{}'".format(za_value))
         self.expect("register read za", substrs=[za_value])
+
+        # Now SVCR.ZA should be set, which is bit 1.
+        self.expect("register read svcr", substrs=["0x0000000000000002"])

lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_static_config/TestSVERegisters.py

def check_sve_register_size(self, set, name, expected):
         reg_value = set.GetChildMemberWithName(name)
         self.assertTrue(
             reg_value.IsValid(), 'Verify we have a register named "%s"' % (name)
         )
         self.assertEqual(
             reg_value.GetByteSize(), expected, 'Verify "%s" == %i' % (name, expected)
         )
 
-    def check_sve_regs_read(self, z_reg_size):
+    def check_sve_regs_read(self, z_reg_size, expected_mode):
+        if self.isAArch64SME():
+            # This test uses SMSTART SM, which only enables streaming mode,
+            # leaving ZA disabled.
+            expected_value = "1" if expected_mode == Mode.SSVE else "0"
+            self.expect("register read svcr", substrs=[
+                "0x000000000000000" + expected_value])
+
         p_reg_size = int(z_reg_size / 8)
 
         for i in range(32):
             z_regs_value = (
                 "{"
                 + " ".join("0x{:02x}".format(i + 1) for _ in range(z_reg_size))
                 + "}"
             )
def sve_registers_read_write_impl(self, start_mode, eval_mode):
         registerSets = process.GetThreadAtIndex(0).GetFrameAtIndex(0).GetRegisters()
         sve_registers = registerSets.GetFirstValueByName(
             "Scalable Vector Extension Registers"
         )
         self.assertTrue(sve_registers)
 
         vg_reg_value = sve_registers.GetChildMemberWithName("vg").GetValueAsUnsigned()
         z_reg_size = vg_reg_value * 8
-        self.check_sve_regs_read(z_reg_size)
+        self.check_sve_regs_read(z_reg_size, start_mode)
 
         # Evaluate simple expression and print function expr_eval_func address.
         self.expect("expression expr_eval_func", substrs=["= 0x"])
 
         # Evaluate expression call function expr_eval_func.
         self.expect_expr(
             "expr_eval_func({})".format(
                 "true" if (eval_mode == Mode.SSVE) else "false"
             ),
             result_type="int",
             result_value="1",
         )
 
         # We called a jitted function above which must not have changed SVE
         # vector length or register values.
-        self.check_sve_regs_read(z_reg_size)
+        self.check_sve_regs_read(z_reg_size, start_mode)
 
         self.check_sve_regs_read_after_write(z_reg_size)
 
     # The following tests all setup some register values then evaluate an
     # expression. After the expression, the mode and register values should be
     # the same as before. Finally they read/write some values in the registers.
     # The only difference is the mode we start the program in, and the mode
     # the expression function uses.

lldb/test/API/commands/register/register/aarch64_sve_registers/rw_access_static_config/main.c

 #include <stdbool.h>
 #include <sys/prctl.h>
 
 #ifndef PR_SME_SET_VL
 #define PR_SME_SET_VL 63
 #endif
 
-#define SMSTART() asm volatile("msr  s0_3_c4_c7_3, xzr" /*smstart*/)
+#define SMSTART_SM() asm volatile("msr  s0_3_c4_c3_3, xzr" /*smstart sm*/)
 
 void write_sve_regs() {
   // We assume the smefa64 feature is present, which allows ffr access
   // in streaming mode.
   asm volatile("setffr\n\t");
   asm volatile("ptrue p0.b\n\t");
   asm volatile("ptrue p1.h\n\t");
   asm volatile("ptrue p2.s\n\t");

 // length and write SVE registers. Our program context should restore to
 // orignal vector length and register values after expression evaluation.
 int expr_eval_func(bool streaming) {
   int SET_VL_OPT = streaming ? PR_SME_SET_VL : PR_SVE_SET_VL;
   prctl(SET_VL_OPT, 8 * 2);
   // Note that doing a syscall brings you back to non-streaming mode, so we
   // don't need to SMSTOP here.
   if (streaming)
-    SMSTART();
+    SMSTART_SM();
   write_sve_regs_expr();
   prctl(SET_VL_OPT, 8 * 4);
   if (streaming)
-    SMSTART();
+    SMSTART_SM();
   write_sve_regs_expr();
   return 1;
 }
 
 int main() {
 #ifdef START_SSVE
-  SMSTART();
+  SMSTART_SM();
 #endif
   write_sve_regs();
 
   return 0; // Set a break point here.
 }

lldb/test/API/commands/register/register/aarch64_za_register/za_save_restore/TestZARegisterSaveRestore.py

def za_expr_test_impl(self, sve_mode, za_state, swap_start_vl):
         # and scaling ZA works properly. This is a brittle check, but failures
         # are likely to be catastrophic when they do happen anyway.
         sve_reg_names = "ffr {} {}".format(
             " ".join(["z{}".format(n) for n in range(32)]),
             " ".join(["p{}".format(n) for n in range(16)]))
         self.runCmd("register read " + sve_reg_names)
         sve_values = self.res.GetOutput()
 
+        svcr_value = 1 if sve_mode == Mode.SSVE else 0
+        if za_state == ZA.Enabled:
+            svcr_value += 2
+
         def check_regs():
             if za_state == ZA.Enabled:
                 self.check_za(start_vl)
             else:
                 self.check_za_disabled(start_vl)
 
             # svg and vg are in units of 8 bytes.
             self.assertEqual(start_vl, self.read_svg()*8)
             self.assertEqual(start_vg, self.read_vg())
 
             self.expect("register read " + sve_reg_names, substrs=[sve_values])
+            self.expect("register read svcr", substrs=["0x{:016x}".format(svcr_value)])
 
         for expr in exprs:
             expr_cmd = "expression {}()".format(expr)
 
             # We do this twice because there were issues in development where
             # using data stored by a previous WriteAllRegisterValues would crash
             # the second time around.
             self.runCmd(expr_cmd)
             check_regs()