[AArch64] Add new v9.4-A PM pstate system register

[pratlucas]

Summary

This adds support for the new PM pstate system register introduced by
the v9.4-A Exception-based Event Profiling extension (FEAT_EBEP).

The new PM pstate register takes a 1-bit immediate and requires
different values to be specified for the higher bits of the Crm field.
To enable that, this patch creates an explicit separation between the
pstate system registers that take 4-bit and 1-bit immediate operands,
allowing each entry to specify the value for the 3 high bits of Crm.

This also updates other pstate registers to correctly accept 4-bit
immediates, matching their decoding specification from the Arm ARM.
These include: PAN, UAO, DIT and SSBS.

More information about this extension and the new register can be found
at:

• https://developer.arm.com/documentation/ddi0601/2022-09/AArch64-Registers/PM--PMU-Exception-Mask

Contributors:

• Lucas Prates
• Sam Elliott

[lenary]

We likely have to release-note this change, as we are now are:

• not going to error for assembly where upper bits which do nothing are set in the immediate
• may codegen e.g. __arm_wsr64("uao", 2) to the assembly msr uao, 2 which is not accepted by binutils.

Both of these are intentional changes, though.

Also a nit about testing.

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1793

Do we have decoding tests for the case where you fallback from MSRpstateImm0_15 to MSRpstateImm0_1 to MSRI? This would presumably be a test where we use an unallocated MSR (imm) encoding?

[pratlucas]

Indeed. I'll add an entry to the release notes to capture that.

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1793

The fallback applies every time the decoder method fails due to the lack of the required feature for a particular pstate register.
This is currently covered by the negative tests for PAN, UAO, DIT, SSBS, and TCO.

[pratlucas]

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1793

On a deeper look, this existing behaviour doesn't look like the best approach to handle those situations. MSRI ends up decoding the instruction using the register form of the msr instruction, generating a somewhat meaningless result.
I'll have a look into an alternative for that, but given that's existing behaviour the change may go into a separate patch.

[lenary]

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1793

I'll have a look into an alternative for that, but given that's existing behaviour the change may go into a separate patch.

As long as we can parse whatever we produce, I think it should be ok. Maybe worth checking what binutils prints in these cases?

[pratlucas]

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1793

Yes, I've double checked the results we produce and they are assembled back into the correct instruction.
Binutils doesn't check for which features are enabled and just disassembles the feature-specific register name. For unallocated/reserved encodings, binutils behaves the same way as llvm, so I think we are ok indeed.

llvm/lib/Target/AArch64/AArch64ISelDAGToDAG.cpp

bool AArch64DAGToDAGISel::tryWriteRegister(SDNode *N) {
   bool WriteIs128Bit = N->getOpcode() == AArch64ISD::MSRR;
 
   if (!WriteIs128Bit) {
     // Check if the register was one of those allowed as the pstatefield value
     // in the MSR (immediate) instruction. To accept the values allowed in the
     // pstatefield for the MSR (immediate) instruction, we also require that an
     // immediate value has been provided as an argument, we know that this is
     // the case as it has been ensured by semantic checking.
-    auto PMapper = AArch64PState::lookupPStateByName(RegString->getString());
+    auto trySelectPState = [&](auto PMapper, unsigned State) {
       if (PMapper) {
         assert(isa<ConstantSDNode>(N->getOperand(2)) &&
                "Expected a constant integer expression.");
         unsigned Reg = PMapper->Encoding;
         uint64_t Immed = cast<ConstantSDNode>(N->getOperand(2))->getZExtValue();
-      unsigned State;
-      if (Reg == AArch64PState::PAN || Reg == AArch64PState::UAO ||
-          Reg == AArch64PState::SSBS) {
-        assert(Immed < 2 && "Bad imm");
-        State = AArch64::MSRpstateImm1;
-      } else {
-        assert(Immed < 16 && "Bad imm");
-        State = AArch64::MSRpstateImm4;
-      }
         CurDAG->SelectNodeTo(
             N, State, MVT::Other, CurDAG->getTargetConstant(Reg, DL, MVT::i32),
             CurDAG->getTargetConstant(Immed, DL, MVT::i16), N->getOperand(0));
         return true;
       }
+      return false;
+    };
+
+    if (trySelectPState(
+            AArch64PState::lookupPStateImm0_15ByName(RegString->getString()),
+            AArch64::MSRpstateImm4))
+      return true;
+    if (trySelectPState(
+            AArch64PState::lookupPStateImm0_1ByName(RegString->getString()),
+            AArch64::MSRpstateImm1))
+      return true;
   }
 
   int Imm = getIntOperandFromRegisterString(RegString->getString());
   if (Imm == -1) {
     // Use the sysreg mapper to attempt to map the remaining possible strings
     // to the value for the register to be used for the MSR (register)
     // instruction operand.
     auto TheReg = AArch64SysReg::lookupSysRegByName(RegString->getString());

llvm/lib/Target/AArch64/AArch64InstrFormats.td

 
 def SystemPStateFieldWithImm0_15Operand : AsmOperandClass {
   let Name = "SystemPStateFieldWithImm0_15";
   let ParserMethod = "tryParseSysReg";
 }
 def pstatefield4_op : Operand<i32> {
   let ParserMatchClass = SystemPStateFieldWithImm0_15Operand;
   let PrintMethod = "printSystemPStateField";
+  let MCOperandPredicate = [{
+    if (!MCOp.isImm())
+      return false;
+    return AArch64SVCR::lookupPStateImm0_15ByEncoding(MCOp.getImm()) != nullptr;
+  }];
 }
 
 // Instructions to modify PSTATE, no input reg
 let Defs = [NZCV] in
 class PstateWriteSimple<dag iops, string asm, string operands>
   : SimpleSystemI<0, iops, asm, operands> {
 
   let Inst{20-19} = 0b00;
   let Inst{15-12} = 0b0100;
 }
 
 class MSRpstateImm0_15
   : PstateWriteSimple<(ins pstatefield4_op:$pstatefield, imm0_15:$imm), "msr",
                   "\t$pstatefield, $imm">,
     Sched<[WriteSys]> {
 
   bits<6> pstatefield;
   bits<4> imm;
   let Inst{18-16} = pstatefield{5-3};
   let Inst{11-8} = imm;
   let Inst{7-5} = pstatefield{2-0};
 
-  let DecoderMethod = "DecodeSystemPStateInstruction";
+  let DecoderMethod = "DecodeSystemPStateImm0_15Instruction";
   // MSRpstateI aliases with MSRI. When the MSRpstateI decoder method returns
   // Fail the decoder should attempt to decode the instruction as MSRI.
-  let hasCompleteDecoder = 0;
+  let hasCompleteDecoder = false;
 }
 
 def SystemPStateFieldWithImm0_1Operand : AsmOperandClass {
   let Name = "SystemPStateFieldWithImm0_1";
   let ParserMethod = "tryParseSysReg";
 }
 def pstatefield1_op : Operand<i32> {
   let ParserMatchClass = SystemPStateFieldWithImm0_1Operand;
   let PrintMethod = "printSystemPStateField";
+  let MCOperandPredicate = [{
+    if (!MCOp.isImm())
+      return false;
+    return AArch64SVCR::lookupPStateImm0_1ByEncoding(MCOp.getImm()) != nullptr;
+  }];
 }
 
 class MSRpstateImm0_1
   : PstateWriteSimple<(ins pstatefield1_op:$pstatefield, imm0_1:$imm), "msr",
                  "\t$pstatefield, $imm">,
     Sched<[WriteSys]> {
 
-  bits<6> pstatefield;
+  bits<9> pstatefield;
   bit imm;
   let Inst{18-16} = pstatefield{5-3};
-  let Inst{11-9} = 0b000;
+  let Inst{11-9} = pstatefield{8-6};
   let Inst{8} = imm;
   let Inst{7-5} = pstatefield{2-0};
 
-  let DecoderMethod = "DecodeSystemPStateInstruction";
+  let DecoderMethod = "DecodeSystemPStateImm0_1Instruction";
   // MSRpstateI aliases with MSRI. When the MSRpstateI decoder method returns
   // Fail the decoder should attempt to decode the instruction as MSRI.
-  let hasCompleteDecoder = 0;
+  let hasCompleteDecoder = false;
+  let DecoderNamespace = "Fallback";
 }
 
 // SYS and SYSL generic system instructions.
 def SysCRAsmOperand : AsmOperandClass {
   let Name = "SysCR";
   let ParserMethod = "tryParseSysCROperand";
 }
 

llvm/lib/Target/AArch64/AArch64SystemOperands.td

 def : ExactFPImm<"half", "0.5", 0x1>;
 def : ExactFPImm<"one",  "1.0", 0x2>;
 def : ExactFPImm<"two",  "2.0", 0x3>;
 
 //===----------------------------------------------------------------------===//
 // PState instruction options.
 //===----------------------------------------------------------------------===//
 
-class PState<string name, bits<5> encoding> : SearchableTable {
+class PStateImm0_15<string name, bits<3> op1, bits<3> op2> : SearchableTable {
   let SearchableFields = ["Name", "Encoding"];
   let EnumValueField = "Encoding";
 
   string Name = name;
-  bits<5> Encoding;
-  let Encoding = encoding;
+  bits<6> Encoding;
+  let Encoding{5-3} = op1;
+  let Encoding{2-0} = op2;
+  code Requires = [{ {} }];
+}
+
+class PStateImm0_1<string name, bits<3> op1, bits<3> op2, bits<3> crm_high> : SearchableTable {
+  let SearchableFields = ["Name", "Encoding"];
+  let EnumValueField = "Encoding";
+
+  string Name = name;
+  bits<9> Encoding;
+  let Encoding{8-6} = crm_high;
+  let Encoding{5-3} = op1;
+  let Encoding{2-0} = op2;
   code Requires = [{ {} }];
 }
 
-def : PState<"SPSel",   0b00101>;
-def : PState<"DAIFSet", 0b11110>;
-def : PState<"DAIFClr", 0b11111>;
+//                   Name,     Op1,   Op2
+def : PStateImm0_15<"SPSel",   0b000, 0b101>;
+def : PStateImm0_15<"DAIFSet", 0b011, 0b110>;
+def : PStateImm0_15<"DAIFClr", 0b011, 0b111>;
 // v8.1a "Privileged Access Never" extension-specific PStates
 let Requires = [{ {AArch64::FeaturePAN} }] in
-def : PState<"PAN",     0b00100>;
+def : PStateImm0_15<"PAN",     0b000, 0b100>;
 
 // v8.2a "User Access Override" extension-specific PStates
 let Requires = [{ {AArch64::FeaturePsUAO} }] in
-def : PState<"UAO",     0b00011>;
+def : PStateImm0_15<"UAO",     0b000, 0b011>;
 // v8.4a timing insensitivity of data processing instructions
 let Requires = [{ {AArch64::FeatureDIT} }] in
-def : PState<"DIT",     0b11010>;
+def : PStateImm0_15<"DIT",     0b011, 0b010>;
 // v8.5a Spectre Mitigation
 let Requires = [{ {AArch64::FeatureSSBS} }] in
-def : PState<"SSBS",    0b11001>;
+def : PStateImm0_15<"SSBS",    0b011, 0b001>;
 // v8.5a Memory Tagging Extension
 let Requires = [{ {AArch64::FeatureMTE} }] in
-def : PState<"TCO",     0b11100>;
+def : PStateImm0_15<"TCO",     0b011, 0b100>;
+// v9.4a Exception-based event profiling
+//                  Name,      Op1,   Op2,   Crm_high
+def : PStateImm0_1<"PM",       0b001, 0b000, 0b001>;
 
 //===----------------------------------------------------------------------===//
 // SVCR instruction options.
 //===----------------------------------------------------------------------===//
 
 class SVCR<string name, bits<3> encoding> : SearchableTable {
   let SearchableFields = ["Name", "Encoding"];
   let EnumValueField = "Encoding";

 let Requires = [{ {AArch64::FeatureRASv2} }] in
 def : ROSysReg<"ERXGSR_EL1",        0b11, 0b000, 0b0101, 0b0011, 0b010>;
 
 // v8.9a/9.4a Physical Fault Address (FEAT_PFAR)
 //                                  Op0   Op1    CRn     CRm     Op2
 def : RWSysReg<"PFAR_EL1",          0b11, 0b000, 0b0110, 0b0000, 0b101>;
 def : RWSysReg<"PFAR_EL12",         0b11, 0b101, 0b0110, 0b0000, 0b101>;
 def : RWSysReg<"PFAR_EL2",          0b11, 0b100, 0b0110, 0b0000, 0b101>;
+
+// v9.4a Exception-based event profiling (FEAT_EBEP)
+//                                  Op0   Op1    CRn     CRm     Op2
+def : RWSysReg<"PM",                0b11, 0b000, 0b0100, 0b0011, 0b001>;

llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

public:
 
   bool isMSRSystemRegister() const {
     if (!isSysReg()) return false;
     return SysReg.MSRReg != -1U;
   }
 
   bool isSystemPStateFieldWithImm0_1() const {
     if (!isSysReg()) return false;
-    return (SysReg.PStateField == AArch64PState::PAN ||
-            SysReg.PStateField == AArch64PState::DIT ||
-            SysReg.PStateField == AArch64PState::UAO ||
-            SysReg.PStateField == AArch64PState::SSBS);
+    return AArch64PState::lookupPStateImm0_1ByEncoding(SysReg.PStateField);
   }
 
   bool isSystemPStateFieldWithImm0_15() const {
-    if (!isSysReg() || isSystemPStateFieldWithImm0_1()) return false;
-    return SysReg.PStateField != -1U;
+    if (!isSysReg())
+      return false;
+    return AArch64PState::lookupPStateImm0_15ByEncoding(SysReg.PStateField);
   }
 
   bool isSVCR() const {
     if (Kind != k_SVCR)
       return false;
     return SVCR.PStateField != -1U;
   }
 
AArch64AsmParser::tryParseSVCR(OperandVector &Operands) {
 
   if (Tok.isNot(AsmToken::Identifier)) {
     TokError("invalid operand for instruction");
     return MatchOperand_ParseFail;
   }
 
   unsigned PStateImm = -1;
   const auto *SVCR = AArch64SVCR::lookupSVCRByName(Tok.getString());
-  if (SVCR && SVCR->haveFeatures(getSTI().getFeatureBits()))
+  if (!SVCR)
+    return MatchOperand_NoMatch;
+  if (SVCR->haveFeatures(getSTI().getFeatureBits()))
     PStateImm = SVCR->Encoding;
 
   Operands.push_back(
       AArch64Operand::CreateSVCR(PStateImm, Tok.getString(), S, getContext()));
   Lex(); // Eat identifier token.
   return MatchOperand_Success;
 }
 
AArch64AsmParser::tryParseSysReg(OperandVector &Operands) {
   int MRSReg, MSRReg;
   auto SysReg = AArch64SysReg::lookupSysRegByName(Tok.getString());
   if (SysReg && SysReg->haveFeatures(getSTI().getFeatureBits())) {
     MRSReg = SysReg->Readable ? SysReg->Encoding : -1;
     MSRReg = SysReg->Writeable ? SysReg->Encoding : -1;
   } else
     MRSReg = MSRReg = AArch64SysReg::parseGenericRegister(Tok.getString());
 
-  auto PState = AArch64PState::lookupPStateByName(Tok.getString());
   unsigned PStateImm = -1;
-  if (PState && PState->haveFeatures(getSTI().getFeatureBits()))
-    PStateImm = PState->Encoding;
+  auto PState15 = AArch64PState::lookupPStateImm0_15ByName(Tok.getString());
+  if (PState15 && PState15->haveFeatures(getSTI().getFeatureBits()))
+    PStateImm = PState15->Encoding;
+  if (!PState15) {
+    auto PState1 = AArch64PState::lookupPStateImm0_1ByName(Tok.getString());
+    if (PState1 && PState1->haveFeatures(getSTI().getFeatureBits()))
+      PStateImm = PState1->Encoding;
+  }
 
   Operands.push_back(
       AArch64Operand::CreateSysReg(Tok.getString(), getLoc(), MRSReg, MSRReg,
                                    PStateImm, getContext()));
   Lex(); // Eat identifier
 
   return MatchOperand_Success;
 }

llvm/lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp

static DecodeStatus DecodeAdrInstruction(MCInst &Inst, uint32_t insn,
                                          const MCDisassembler *Decoder);
 static DecodeStatus DecodeAddSubImmShift(MCInst &Inst, uint32_t insn,
                                          uint64_t Address,
                                          const MCDisassembler *Decoder);
 static DecodeStatus DecodeUnconditionalBranch(MCInst &Inst, uint32_t insn,
                                               uint64_t Address,
                                               const MCDisassembler *Decoder);
 static DecodeStatus
-DecodeSystemPStateInstruction(MCInst &Inst, uint32_t insn, uint64_t Address,
+DecodeSystemPStateImm0_15Instruction(MCInst &Inst, uint32_t insn,
+                                     uint64_t Address,
+                                     const MCDisassembler *Decoder);
+static DecodeStatus
+DecodeSystemPStateImm0_1Instruction(MCInst &Inst, uint32_t insn,
+                                    uint64_t Address,
                                     const MCDisassembler *Decoder);
 static DecodeStatus DecodeTestAndBranch(MCInst &Inst, uint32_t insn,
                                         uint64_t Address,
                                         const MCDisassembler *Decoder);
 
 static DecodeStatus DecodeFMOVLaneInstruction(MCInst &Inst, unsigned Insn,
                                               uint64_t Address,
                                               const MCDisassembler *Decoder);
 static DecodeStatus DecodeVecShiftR64Imm(MCInst &Inst, unsigned Imm,
if (imm & (1 << (26 - 1)))
     imm |= ~((1LL << 26) - 1);
 
   if (!Decoder->tryAddingSymbolicOperand(Inst, imm * 4, Addr, true, 0, 0, 4))
     Inst.addOperand(MCOperand::createImm(imm));
 
   return Success;
 }
 
+static bool isInvalidPState(uint64_t Op1, uint64_t Op2) {
+  return Op1 == 0b000 && (Op2 == 0b000 || // CFINV
+                          Op2 == 0b001 || // XAFlag
+                          Op2 == 0b010);  // AXFlag
+}
+
 static DecodeStatus
-DecodeSystemPStateInstruction(MCInst &Inst, uint32_t insn, uint64_t Addr,
+DecodeSystemPStateImm0_15Instruction(MCInst &Inst, uint32_t insn, uint64_t Addr,
                                      const MCDisassembler *Decoder) {
   uint64_t op1 = fieldFromInstruction(insn, 16, 3);
   uint64_t op2 = fieldFromInstruction(insn, 5, 3);
-  uint64_t crm = fieldFromInstruction(insn, 8, 4);
+  uint64_t imm = fieldFromInstruction(insn, 8, 4);
   uint64_t pstate_field = (op1 << 3) | op2;
 
-  switch (pstate_field) {
-  case 0x01: // XAFlag
-  case 0x02: // AXFlag
-    return Fail;
-  }
+  if (isInvalidPState(op1, op2))
+    return Fail;
+
+  Inst.addOperand(MCOperand::createImm(pstate_field));
+  Inst.addOperand(MCOperand::createImm(imm));
+
+  auto PState = AArch64PState::lookupPStateImm0_15ByEncoding(pstate_field);
+  if (PState &&
+      PState->haveFeatures(Decoder->getSubtargetInfo().getFeatureBits()))
+    return Success;
 
-  if ((pstate_field == AArch64PState::PAN  ||
-       pstate_field == AArch64PState::UAO  ||
-       pstate_field == AArch64PState::SSBS) && crm > 1)
+static DecodeStatus
+DecodeSystemPStateImm0_1Instruction(MCInst &Inst, uint32_t insn, uint64_t Addr,
+                                    const MCDisassembler *Decoder) {
+  uint64_t op1 = fieldFromInstruction(insn, 16, 3);
+  uint64_t op2 = fieldFromInstruction(insn, 5, 3);
+  uint64_t crm_high = fieldFromInstruction(insn, 9, 3);
+  uint64_t imm = fieldFromInstruction(insn, 8, 1);
+  uint64_t pstate_field = (crm_high << 6) | (op1 << 3) | op2;
+
+  if (isInvalidPState(op1, op2))
     return Fail;
 
   Inst.addOperand(MCOperand::createImm(pstate_field));
-  Inst.addOperand(MCOperand::createImm(crm));
+  Inst.addOperand(MCOperand::createImm(imm));
 
-  auto PState = AArch64PState::lookupPStateByEncoding(pstate_field);
+  auto PState = AArch64PState::lookupPStateImm0_1ByEncoding(pstate_field);
   if (PState &&
       PState->haveFeatures(Decoder->getSubtargetInfo().getFeatureBits()))
     return Success;
   return Fail;
 }
 
 static DecodeStatus DecodeTestAndBranch(MCInst &Inst, uint32_t insn,
                                         uint64_t Addr,

llvm/lib/Target/AArch64/MCTargetDesc/AArch64InstPrinter.cpp

else
     O << AArch64SysReg::genericRegisterString(Val);
 }
 
 void AArch64InstPrinter::printSystemPStateField(const MCInst *MI, unsigned OpNo,
                                                 const MCSubtargetInfo &STI,
                                                 raw_ostream &O) {
   unsigned Val = MI->getOperand(OpNo).getImm();
 
-  auto PState = AArch64PState::lookupPStateByEncoding(Val);
-  if (PState && PState->haveFeatures(STI.getFeatureBits()))
-    O << PState->Name;
+  auto PStateImm15 = AArch64PState::lookupPStateImm0_15ByEncoding(Val);
+  auto PStateImm1 = AArch64PState::lookupPStateImm0_1ByEncoding(Val);
+  if (PStateImm15 && PStateImm15->haveFeatures(STI.getFeatureBits()))
+    O << PStateImm15->Name;
+  else if (PStateImm1 && PStateImm1->haveFeatures(STI.getFeatureBits()))
+    O << PStateImm1->Name;
   else
     O << "#" << formatImm(Val);
 }
 
 void AArch64InstPrinter::printSIMDType10Operand(const MCInst *MI, unsigned OpNo,
                                                 const MCSubtargetInfo &STI,
                                                 raw_ostream &O) {
   unsigned RawVal = MI->getOperand(OpNo).getImm();

llvm/lib/Target/AArch64/Utils/AArch64BaseInfo.h

struct ExactFPImm {
     int Enum;
     const char *Repr;
   };
 #define GET_EXACTFPIMM_DECL
 #include "AArch64GenSystemOperands.inc"
 }
 
 namespace AArch64PState {
-  struct PState : SysAlias{
+  struct PStateImm0_15 : SysAlias{
     using SysAlias::SysAlias;
   };
-  #define GET_PSTATE_DECL
+  #define GET_PSTATEIMM0_15_DECL
+  #include "AArch64GenSystemOperands.inc"
+
+  struct PStateImm0_1 : SysAlias{
+    using SysAlias::SysAlias;
+  };
+  #define GET_PSTATEIMM0_1_DECL
   #include "AArch64GenSystemOperands.inc"
 }
 
 namespace AArch64PSBHint {
   struct PSB : SysAlias {
     using SysAlias::SysAlias;
   };
   #define GET_PSB_DECL

llvm/lib/Target/AArch64/Utils/AArch64BaseInfo.cpp

namespace llvm {
   namespace AArch64ExactFPImm {
 #define GET_EXACTFPIMM_IMPL
 #include "AArch64GenSystemOperands.inc"
   }
 }
 
 namespace llvm {
   namespace AArch64PState {
-#define GET_PSTATE_IMPL
+#define GET_PSTATEIMM0_15_IMPL
+#include "AArch64GenSystemOperands.inc"
+#define GET_PSTATEIMM0_1_IMPL
 #include "AArch64GenSystemOperands.inc"
   }
 }
 
 namespace llvm {
   namespace AArch64PSBHint {
 #define GET_PSB_IMPL
 #include "AArch64GenSystemOperands.inc"

llvm/test/MC/AArch64/armv8.1a-pan.s

// CHECK:  msr PAN, #0           // encoding: [0x9f,0x40,0x00,0xd5]
   msr pan, #1
 // CHECK:  msr PAN, #1           // encoding: [0x9f,0x41,0x00,0xd5]
   msr pan, x5
 // CHECK:  msr PAN, x5           // encoding: [0x65,0x42,0x18,0xd5]
   mrs x13, pan
 // CHECK:  mrs x13, PAN          // encoding: [0x6d,0x42,0x38,0xd5]
 
   msr pan, #-1
-  msr pan, #2
+  msr pan, #16
   msr pan, w0
   mrs w0, pan
-// CHECK-ERROR: error: immediate must be an integer in range [0, 1].
+// CHECK-ERROR: error: immediate must be an integer in range [0, 15].
 // CHECK-ERROR:   msr pan, #-1
 // CHECK-ERROR:            ^
-// CHECK-ERROR: error: immediate must be an integer in range [0, 1].
-// CHECK-ERROR:   msr pan, #2
+// CHECK-ERROR: error: immediate must be an integer in range [0, 15].
+// CHECK-ERROR:   msr pan, #16
 // CHECK-ERROR:            ^
-// CHECK-ERROR: error: immediate must be an integer in range [0, 1].
+// CHECK-ERROR: error: immediate must be an integer in range [0, 15].
 // CHECK-ERROR:   msr pan, w0
 // CHECK-ERROR:            ^
 // CHECK-ERROR: error: invalid operand for instruction
 // CHECK-ERROR:   mrs w0, pan
 // CHECK-ERROR:       ^

llvm/test/MC/AArch64/armv8.2a-uao.s

 // RUN: not llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=+v8.2a < %s 2> %t | FileCheck %s
 // RUN: FileCheck --check-prefix=CHECK-ERROR %s < %t
 // RUN: not llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=+v8r -o - %s 2> %t | FileCheck %s
 // RUN: FileCheck --check-prefix=CHECK-ERROR %s < %t
 
   msr uao, #0
   msr uao, #1
 // CHECK: msr     UAO, #0                 // encoding: [0x7f,0x40,0x00,0xd5]
 // CHECK: msr     UAO, #1                 // encoding: [0x7f,0x41,0x00,0xd5]
 
-  msr uao, #2
-// CHECK-ERROR: error: immediate must be an integer in range [0, 1].
-// CHECK-ERROR:   msr uao, #2
+  msr uao, #16
+// CHECK-ERROR: error: immediate must be an integer in range [0, 15].
+// CHECK-ERROR:   msr uao, #16
 // CHECK-ERROR:            ^
 
   msr uao, x1
   mrs x2, uao
 // CHECK: msr     UAO, x1                 // encoding: [0x81,0x42,0x18,0xd5]
 // CHECK: mrs     x2, UAO                 // encoding: [0x82,0x42,0x38,0xd5]

llvm/test/MC/AArch64/armv8.4a-dit.s

 // RUN: not llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=+v8.4a < %s 2> %t  | FileCheck %s --check-prefix=CHECK
 // RUN: FileCheck --check-prefix=CHECK-ERROR < %t %s
 // RUN: not llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=-v8.4a < %s 2>&1 | FileCheck %s --check-prefix=CHECK-NO-V84
 
 //------------------------------------------------------------------------------
 // ARMV8.4-A Timing insensitivity of data processing instructions
 //------------------------------------------------------------------------------
 
 msr DIT, #1
 msr DIT, x0
 mrs x0, DIT
 
 //CHECK:      msr DIT, #1                 // encoding: [0x5f,0x41,0x03,0xd5]
 //CHECK-NEXT: msr DIT, x0                 // encoding: [0xa0,0x42,0x1b,0xd5]
 //CHECK-NEXT: mrs x0, DIT                 // encoding: [0xa0,0x42,0x3b,0xd5]
 
-msr DIT, #2
+msr DIT, #16
 msr DIT, #-1
 
-//CHECK-ERROR:      error: immediate must be an integer in range [0, 1].
-//CHECK-ERROR-NEXT: msr DIT, #2
+//CHECK-ERROR:      error: immediate must be an integer in range [0, 15].
+//CHECK-ERROR-NEXT: msr DIT, #16
 //CHECK-ERROR-NEXT:          ^
-//CHECK-ERROR-NEXT: error: immediate must be an integer in range [0, 1].
+//CHECK-ERROR-NEXT: error: immediate must be an integer in range [0, 15].
 //CHECK-ERROR-NEXT: msr DIT, #-1
 //CHECK-ERROR-NEXT:          ^
 
 //CHECK-NO-V84:      error: expected writable system register or pstate
 //CHECK-NO-V84-NEXT: msr DIT, #1
 //CHECK-NO-V84-NEXT:     ^
 //CHECK-NO-V84-NEXT: error: expected writable system register or pstate
 //CHECK-NO-V84-NEXT: msr DIT, x0
 //CHECK-NO-V84-NEXT:     ^
 //CHECK-NO-V84-NEXT: error: expected readable system register
 //CHECK-NO-V84-NEXT: mrs x0, DIT
 //CHECK-NO-V84-NEXT:         ^
 //CHECK-NO-V84-NEXT: error: expected writable system register or pstate
-//CHECK-NO-V84-NEXT: msr DIT, #2
+//CHECK-NO-V84-NEXT: msr DIT, #16
 //CHECK-NO-V84-NEXT:     ^
 //CHECK-NO-V84-NEXT: error: expected writable system register or pstate
 //CHECK-NO-V84-NEXT: msr DIT, #-1
 //CHECK-NO-V84-NEXT:     ^
 

llvm/test/MC/AArch64/armv8.5a-ssbs-error.s

 // RUN: not llvm-mc -triple aarch64 -show-encoding -mattr=+ssbs  < %s 2>&1 | FileCheck %s
 // RUN: not llvm-mc -triple aarch64 -show-encoding -mattr=+v8.5a < %s 2>&1 | FileCheck %s
 // RUN: not llvm-mc -triple aarch64 -show-encoding -mattr=-ssbs  < %s 2>&1 | FileCheck %s --check-prefix=NOSPECID
 
-msr SSBS, #2
+msr SSBS, #16
 
-// CHECK:         error: immediate must be an integer in range [0, 1].
-// CHECK-NEXT:    msr SSBS, #2
+// CHECK:         error: immediate must be an integer in range [0, 15].
+// CHECK-NEXT:    msr SSBS, #16
 // NOSPECID:      error: expected writable system register or pstate
-// NOSPECID-NEXT: msr {{ssbs|SSBS}}, #2
+// NOSPECID-NEXT: msr {{ssbs|SSBS}}, #16

llvm/test/MC/AArch64/armv9.4a-ebep-error.s

+// RUN: not llvm-mc -triple aarch64 -show-encoding < %s 2>&1 | FileCheck %s
+
+msr PM, #2
+// CHECK:         error: immediate must be an integer in range [0, 1].
+// CHECK-NEXT:    msr PM, #2
+// CHECK-NEXT:    ^

llvm/test/MC/AArch64/armv9.4a-ebep.s

+// RUN: llvm-mc -triple aarch64 -show-encoding < %s | FileCheck %s
+
+mrs x2, PM
+msr PM, x3
+msr PM, #1
+
+// CHECK:       mrs x2, {{pm|PM}} // encoding: [0x22,0x43,0x38,0xd5]
+// CHECK:       msr {{pm|PM}}, x3 // encoding: [0x23,0x43,0x18,0xd5]
+// CHECK:       msr {{pm|PM}}, #1 // encoding: [0x1f,0x43,0x01,0xd5]

llvm/test/MC/Disassembler/AArch64/armv8.1a-pan.txt

 # RUN: llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.1a --disassemble < %s | FileCheck %s
 
 0x9f,0x40,0x00,0xd5
 0x9f,0x41,0x00,0xd5
-0x9f,0x42,0x00,0xd5
+0x9f,0x4f,0x00,0xd5
 0x65,0x42,0x18,0xd5
 0x6d,0x42,0x38,0xd5
 # CHECK:  msr PAN, #0
 # CHECK:  msr PAN, #1
-# CHECK-NOT: msr PAN, #2
+# CHECK:  msr PAN, #15
 # CHECK:  msr PAN, x5
 # CHECK:  mrs x13, PAN

llvm/test/MC/Disassembler/AArch64/armv8.2a-uao.txt

 # RUN: llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.2a --disassemble < %s | FileCheck %s
 # RUN: llvm-mc -triple aarch64-none-linux-gnu --disassemble < %s 2>&1 | FileCheck --check-prefix=NO_V82A %s
 
 [0x7f,0x40,0x00,0xd5]
 [0x7f,0x41,0x00,0xd5]
-[0x7f,0x42,0x00,0xd5]
+[0x7f,0x4f,0x00,0xd5]
 # CHECK: msr UAO, #0
 # CHECK: msr UAO, #1
-# CHECK: msr S0_0_C4_C2_3, xzr
+# CHECK: msr UAO, #15
 # NO_V82A: msr S0_0_C4_C0_3, xzr
 # NO_V82A: msr S0_0_C4_C1_3, xzr
-# NO_V82A: msr S0_0_C4_C2_3, xzr
+# NO_V82A: msr S0_0_C4_C15_3, xzr
 
 [0x81,0x42,0x18,0xd5]
 [0x82,0x42,0x38,0xd5]
 # CHECK: msr UAO, x1
 # CHECK: mrs x2, UAO
 # NO_V82A: msr S3_0_C4_C2_4, x1
 # NO_V82A: mrs x2, S3_0_C4_C2_4

llvm/test/MC/Disassembler/AArch64/armv9.4a-ebep.txt

+# RUN: llvm-mc -triple=aarch64 -disassemble %s | FileCheck %s
+
+[0x23,0x43,0x38,0xd5]
+# CHECK:  mrs x3, PM
+
+[0x26,0x43,0x18,0xd5]
+# CHECK:  msr PM, x6
+
+[0x1f,0x42,0x01,0xd5]
+# CHECK:  msr PM, #0
+
+[0x1f,0x43,0x01,0xd5]
+# CHECK:  msr PM, #1