[AArch64] Add v8.1a atomic instructions

[t.p.northover]

I think I have more comments than are strictly justified by how I feel about this patch. Generally, it *is* good, I just think it could be streamlined to be even better.

Tim.

lib/Target/AArch64/AArch64InstrFormats.td

8633–8636

I don't think this is the best location for these changes. TokanAliases are very different from the bulk of the file, and are at the end of the file because of that.

8668–8669

These names are non-obvious. The AArch64 assembly syntax actually uses "L" for release. I'd suggest actually using "Acq" and "Rel" to make it more obvious for readers. We're not constrained for space here.

8685–8690

You should probably put BaseCAS and BaseCASP together. For a long while I thought this whole indirection was pointless (until I saw CASP).

8693

"BaseLD" is far too generic as a name for something that's only applicable to these atomicrmw operations.

8715–8716

I think the factoring of the multiclass hierarchy is inconsistent with the rest of the backend here. The general convention is that the capitalised prefix of the instruction mirrors the mnemonic, with suffixes (capitalised or not) showing the variant.

Obviously, that's not always possible, but here I think it is.

In this case, I'd expect in AArch64InstrInfo.td something like

defm LDADD : ...
defm LDADDA : ...
defm LDADDL : ...
defm LDADDAL : ...
...

with the second-level hierarchy appending a B/H/W/X as needed. See the usual loads & stores for example (though they're confounded slightly by both "ldrb w0, [x0]" and "ldr b0, [x0]" being byte loads).

8731–8732

It's also rather odd that the "BaseAliasLD" classes actually define mnemonics starting with "st".

lib/Target/AArch64/AArch64RegisterInfo.td

601–606

I think these should probably go with the existing SubRegIndex definitions.

608–609

We should probably mention GPRPair here, for readability in "-debug" if nothing else. (Elsewhere too, just because there are no FPRPairs now, doesn't mean there won't be in future).

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4311

We use lower-case "expected". Could also be more detailed here: "expected even general purpose register" for example.

4316

Space.

4354–4356

I tend to think you should either trust that the enums are consecutive here or not. Trusting that "x1 == x0 + 1" but then being paranoid over whether maybe "w1 == x0 + 1" seems daft.

4363

I think things might be a bit simpler in this function overall if you used functions like MCRegisterInfo::getMatchingSuperReg(FirstReg, AArch64::sube0, &PairClass. It could both diagnose an odd input and provide the correct Pair in one step.

More generally, you might want to consider a top-level "template<int Width> tryParsePair(...)" which passes the needed 32/64 info (like Pair32RegClass/Pair64Regclass) in as an argument to a non-templated "tryParsePair" function. It's a pattern I've found useful in assembly parsers that have to deal with multiple sizes in the past.

lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp

1551

Another possible case for a template deferring to the real implementation. These two functions are practically identical.

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

I could believe this change is needed, but why does this particular patch reveal the necessity?

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.cpp

1159

Local variables start with a capital letter (for now).

[vsukharev]

Hi Tim, thank you for your hints and comments. I'll address it soon.
Meanwhile shouldn't we perform careful review/re-do/re-base process from the first patch in the dependency chain( http://reviews.llvm.org/D8496 )?
That way I could save efforts rebasing the rest of chain after every correction..

[vsukharev]

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

Introduced usage of AArch64MCRegisterClasses causes the need of this explicit dependency.
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

[vsukharev]

Almost all points are corrected

[vsukharev]

lib/Target/AArch64/AArch64InstrFormats.td

8731–8732

I'd wish instructions would be better named as "LDST<op>" and "ST<op>"....
That way it semantics be clear, so ST just doesn't save loaded value in reg

8767

I'd wish instructions would be better named as "LDST<op>" and "ST<op>"....
That way it semantics be clear, so ST just doesn't save loaded value in reg

lib/Target/AArch64/AArch64RegisterInfo.td

608–609

I'm gonna rename it to WSeqPairs and XSeqPairs, similar to existing DSeqPairs and QSeqPairs

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4331

About top-level template "tryParseGPRSeqPair<>" and generic function:
hmmm, is there a way for AArch64AsmParser (methods MatchOperandParserImpl and tryCustomParseOperand in AArch64GenAsmMatcher.inc) to guess in advance which pair to expect - MCK_WSeqPair or MCK_XSeqPair?

I guess without that knowledge it is possible only to call this generic tryParseGPRSeqPair from tryCustomParseOperand, and it will find out the size of actual operands.
I've just tried to use a template and face only tryParseGPRSeqPair<32>() is being called for both test lines, and 64-bit variant is never called,

//CASP instruction
casp x0, x1, x2, x3, [x4]
casp w0, w1, w2, w3, [x4]

4361

No paranoia, second complicated check is needed against pairs like "w0, x1", I've added some tests for that, like

//CHECK-ERROR: error: expected consecutive registers of same size
//CHECK-ERROR: casp w0, x1, x2, x3, [x5]

4372

Introduced usage of AArch64MCRegisterClasses causes the need of explicit dependency AArch64Info -> AArch64Desc
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

Introduced usage of AArch64MCRegisterClasses causes the need of explicit dependency AArch64Info -> AArch64Desc
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

[vsukharev]

ping..

[vsukharev]

It might seem there are many comments on future actions, while they are all already addressed in last-uploaded diff http://reviews.llvm.org/differential/diff/22894/
So, it is ready to re-review

[t.p.northover]

Looks mostly reasonable. Just a few minor comments now.

Tim.

lib/Target/AArch64/AArch64RegisterInfo.td

608–609

These lines are longer than the column limit. Not sure about the surrounding ones.

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4322

Debugging code should be removed.

4332

Since this only ever called for an instruction that *must* have a pair, we can be more helpful in the error messages produced. Always say that we need an even/odd register pair, for example.

[t.p.northover]

Hi Vladimir,

One more comment...

lib/Target/AArch64/AArch64InstrFormats.td

8670

Arnold's pointed out to me that it probably should be Rs that gets written here. CASP seems to be right though.

[vsukharev]

Last minor comments are processed.
Thank you for comments and catches.

[richard.barton.arm]

Hi Tim

Are you happy with Vladimir's final patch?

Ta
Rich

[vsukharev]

Hi Arnold,
It seems Tim is unpingable for now...
As soon you have taken a look at this patch, would you be able to raise more comments, or approve the patch?
Ta, Vladimir

[t.p.northover]

Hi Vladimir,

Really sorry I didn't reply to the last two pings; I'd been neglecting the list recently. I think this looks fine now, thanks for updating it.

Tim.

lib/Target/AArch64/AArch64InstrFormats.td

                [(set (i32 FPR32:$Rd), (OpNode (i32 FPR32:$Rn)))]>;
 } // end of 'let Predicates = [HasCrypto]'
 
+//----------------------------------------------------------------------------
+// v8.1 atomic instructions extension:
+// * CAS
+// * CASP
+// * SWP
+// * LDOPregister<OP>, and aliases STOPregister<OP>
+
+// Instruction encodings:
+//
+//      31 30|29  24|23|22|21|20 16|15|14  10|9 5|4 0
+// CAS  SZ   |001000|1 |A |1 |Rs   |R |11111 |Rn |Rt
+// CASP  0|SZ|001000|0 |A |1 |Rs   |R |11111 |Rn |Rt
+// SWP  SZ   |111000|A |R |1 |Rs   |1 |OPC|00|Rn |Rt
+// LD   SZ   |111000|A |R |1 |Rs   |0 |OPC|00|Rn |Rt
+// ST   SZ   |111000|A |R |1 |Rs   |0 |OPC|00|Rn |11111
+
+// Instruction syntax:
+//
+// CAS{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// CAS{<order>} <Xs>, <Xt>, [<Xn|SP>]
+// CASP{<order>} <Ws>, <W(s+1)>, <Wt>, <W(t+1)>, [<Xn|SP>]
+// CASP{<order>} <Xs>, <X(s+1)>, <Xt>, <X(t+1)>, [<Xn|SP>]
+// SWP{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// SWP{<order>} <Xs>, <Xt>, [<Xn|SP>]
+// LD<OP>{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// LD<OP>{<order>} <Xs>, <Xt>, [<Xn|SP>]
+// ST<OP>{<order>}[<size>] <Ws>, [<Xn|SP>]
+// ST<OP>{<order>} <Xs>, [<Xn|SP>]
+
+let Predicates = [HasV8_1a], mayLoad = 1, mayStore = 1, hasSideEffects = 1 in
+class BaseCASEncoding<dag oops, dag iops, string asm, string operands,
+                      string cstr, list<dag> pattern>
+      : I<oops, iops, asm, operands, cstr, pattern> {
+  bits<2> Sz;
+  bit NP;
+  bit Acq;
+  bit Rel;
+  bits<5> Rs;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b001000;
+  let Inst{23} = NP;
+  let Inst{22} = Acq;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = Rel;
+  let Inst{14-10} = 0b11111;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+class BaseCAS<string order, string size, RegisterClass RC>
+      : BaseCASEncoding<(outs RC:$out),(ins RC:$Rs, RC:$Rt, GPR64sp:$Rn),
+                        "cas" # order # size, "\t$Rs, $Rt, [$Rn]",
+                        "$out = $Rt",[]> {
+  let NP = 1;
+}
+
+multiclass CompareAndSwap<bits<1> Acq, bits<1> Rel, string order> {
+  let Sz = 0b00, Acq = Acq, Rel = Rel in def b : BaseCAS<order, "b", GPR32>;
+  let Sz = 0b01, Acq = Acq, Rel = Rel in def h : BaseCAS<order, "h", GPR32>;
+  let Sz = 0b10, Acq = Acq, Rel = Rel in def s : BaseCAS<order, "", GPR32>;
+  let Sz = 0b11, Acq = Acq, Rel = Rel in def d : BaseCAS<order, "", GPR64>;
+}
+
+class BaseCASP<string order, string size, RegisterOperand RC>
+      : BaseCASEncoding<(outs RC:$out),(ins RC:$Rs, RC:$Rt, GPR64sp:$Rn),
+                        "casp" # order # size, "\t$Rs, $Rt, [$Rn]",
+                        "$out = $Rs",[]> {
+  let NP = 0;
+}
+
+multiclass CompareAndSwapPair<bits<1> Acq, bits<1> Rel, string order> {
+  let Sz = 0b00, Acq = Acq, Rel = Rel in 
+    def s : BaseCASP<order, "", WSeqPairClassOperand>;
+  let Sz = 0b01, Acq = Acq, Rel = Rel in 
+    def d : BaseCASP<order, "", XSeqPairClassOperand>;
+}
+
+let Predicates = [HasV8_1a] in
+class BaseSWP<string order, string size, RegisterClass RC>
+      : I<(outs RC:$Rt),(ins RC:$Rs, GPR64sp:$Rn), "swp" # order # size,
+          "\t$Rs, $Rt, [$Rn]","",[]> {
+  bits<2> Sz;
+  bit Acq;
+  bit Rel;
+  bits<5> Rs;
+  bits<3> opc = 0b000;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b111000;
+  let Inst{23} = Acq;
+  let Inst{22} = Rel;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = 0b1;
+  let Inst{14-12} = opc;
+  let Inst{11-10} = 0b00;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+multiclass Swap<bits<1> Acq, bits<1> Rel, string order> {
+  let Sz = 0b00, Acq = Acq, Rel = Rel in def b : BaseSWP<order, "b", GPR32>;
+  let Sz = 0b01, Acq = Acq, Rel = Rel in def h : BaseSWP<order, "h", GPR32>;
+  let Sz = 0b10, Acq = Acq, Rel = Rel in def s : BaseSWP<order, "", GPR32>;
+  let Sz = 0b11, Acq = Acq, Rel = Rel in def d : BaseSWP<order, "", GPR64>;
+}
+
+let Predicates = [HasV8_1a], mayLoad = 1, mayStore = 1, hasSideEffects = 1 in
+class BaseLDOPregister<string op, string order, string size, RegisterClass RC>
+      : I<(outs RC:$Rt),(ins RC:$Rs, GPR64sp:$Rn), "ld" # op # order # size,
+          "\t$Rs, $Rt, [$Rn]","",[]> {
+  bits<2> Sz;
+  bit Acq;
+  bit Rel;
+  bits<5> Rs;
+  bits<3> opc;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b111000;
+  let Inst{23} = Acq;
+  let Inst{22} = Rel;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = 0b0;
+  let Inst{14-12} = opc;
+  let Inst{11-10} = 0b00;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+multiclass LDOPregister<bits<3> opc, string op, bits<1> Acq, bits<1> Rel, 
+                        string order> {
+  let Sz = 0b00, Acq = Acq, Rel = Rel, opc = opc in 
+    def b : BaseLDOPregister<op, order, "b", GPR32>;
+  let Sz = 0b01, Acq = Acq, Rel = Rel, opc = opc in 
+    def h : BaseLDOPregister<op, order, "h", GPR32>;
+  let Sz = 0b10, Acq = Acq, Rel = Rel, opc = opc in 
+    def s : BaseLDOPregister<op, order, "", GPR32>;
+  let Sz = 0b11, Acq = Acq, Rel = Rel, opc = opc in 
+    def d : BaseLDOPregister<op, order, "", GPR64>;
+}
+
+let Predicates = [HasV8_1a] in
+class BaseSTOPregister<string asm, RegisterClass OP, Register Reg,
+                        Instruction inst> :
+      InstAlias<asm # "\t$Rs, [$Rn]", (inst Reg, OP:$Rs, GPR64sp:$Rn)>;
+
+multiclass STOPregister<string asm, string instr> {
+  def : BaseSTOPregister<asm # "lb", GPR32, WZR, 
+                    !cast<Instruction>(instr # "Lb")>;
+  def : BaseSTOPregister<asm # "lh", GPR32, WZR, 
+                    !cast<Instruction>(instr # "Lh")>;
+  def : BaseSTOPregister<asm # "l",  GPR32, WZR, 
+                    !cast<Instruction>(instr # "Ls")>;
+  def : BaseSTOPregister<asm # "l",  GPR64, XZR, 
+                    !cast<Instruction>(instr # "Ld")>;
+  def : BaseSTOPregister<asm # "b",  GPR32, WZR, 
+                    !cast<Instruction>(instr # "b")>;
+  def : BaseSTOPregister<asm # "h",  GPR32, WZR, 
+                    !cast<Instruction>(instr # "h")>;
+  def : BaseSTOPregister<asm,        GPR32, WZR, 
+                    !cast<Instruction>(instr # "s")>;
+  def : BaseSTOPregister<asm,        GPR64, XZR, 
+                    !cast<Instruction>(instr # "d")>;
+}
+
+//----------------------------------------------------------------------------
 // Allow the size specifier tokens to be upper case, not just lower.
 def : TokenAlias<".8B", ".8b">;
 def : TokenAlias<".4H", ".4h">;

lib/Target/AArch64/AArch64InstrInfo.td

 def CRC32CWrr : BaseCRC32<0, 0b10, 1, GPR32, int_aarch64_crc32cw, "crc32cw">;
 def CRC32CXrr : BaseCRC32<1, 0b11, 1, GPR64, int_aarch64_crc32cx, "crc32cx">;
 
+// v8.1 atomic CAS
+defm CAS   : CompareAndSwap<0, 0, "">;
+defm CASA  : CompareAndSwap<1, 0, "a">;
+defm CASL  : CompareAndSwap<0, 1, "l">;
+defm CASAL : CompareAndSwap<1, 1, "al">;
+
+// v8.1 atomic CASP
+defm CASP   : CompareAndSwapPair<0, 0, "">;
+defm CASPA  : CompareAndSwapPair<1, 0, "a">;
+defm CASPL  : CompareAndSwapPair<0, 1, "l">;
+defm CASPAL : CompareAndSwapPair<1, 1, "al">;
+
+// v8.1 atomic SWP
+defm SWP   : Swap<0, 0, "">;
+defm SWPA  : Swap<1, 0, "a">;
+defm SWPL  : Swap<0, 1, "l">;
+defm SWPAL : Swap<1, 1, "al">;
+
+// v8.1 atomic LD<OP>(register). Performs load and then ST<OP>(register)
+defm LDADD   : LDOPregister<0b000, "add", 0, 0, "">;
+defm LDADDA  : LDOPregister<0b000, "add", 1, 0, "a">;
+defm LDADDL  : LDOPregister<0b000, "add", 0, 1, "l">;
+defm LDADDAL : LDOPregister<0b000, "add", 1, 1, "al">;
+
+defm LDCLR   : LDOPregister<0b001, "clr", 0, 0, "">;
+defm LDCLRA  : LDOPregister<0b001, "clr", 1, 0, "a">;
+defm LDCLRL  : LDOPregister<0b001, "clr", 0, 1, "l">;
+defm LDCLRAL : LDOPregister<0b001, "clr", 1, 1, "al">;
+
+defm LDEOR   : LDOPregister<0b010, "eor", 0, 0, "">;
+defm LDEORA  : LDOPregister<0b010, "eor", 1, 0, "a">;
+defm LDEORL  : LDOPregister<0b010, "eor", 0, 1, "l">;
+defm LDEORAL : LDOPregister<0b010, "eor", 1, 1, "al">;
+
+defm LDSET   : LDOPregister<0b011, "set", 0, 0, "">;
+defm LDSETA  : LDOPregister<0b011, "set", 1, 0, "a">;
+defm LDSETL  : LDOPregister<0b011, "set", 0, 1, "l">;
+defm LDSETAL : LDOPregister<0b011, "set", 1, 1, "al">;
+
+defm LDSMAX   : LDOPregister<0b100, "smax", 0, 0, "">;
+defm LDSMAXA  : LDOPregister<0b100, "smax", 1, 0, "a">;
+defm LDSMAXL  : LDOPregister<0b100, "smax", 0, 1, "l">;
+defm LDSMAXAL : LDOPregister<0b100, "smax", 1, 1, "al">;
+
+defm LDSMIN   : LDOPregister<0b101, "smin`", 0, 0, "">;
+defm LDSMINA  : LDOPregister<0b101, "smin", 1, 0, "a">;
+defm LDSMINL  : LDOPregister<0b101, "smin", 0, 1, "l">;
+defm LDSMINAL : LDOPregister<0b101, "smin", 1, 1, "al">;
+
+defm LDUMAX   : LDOPregister<0b110, "umax", 0, 0, "">;
+defm LDUMAXA  : LDOPregister<0b110, "umax", 1, 0, "a">;
+defm LDUMAXL  : LDOPregister<0b110, "umax", 0, 1, "l">;
+defm LDUMAXAL : LDOPregister<0b110, "umax", 1, 1, "al">;
+
+defm LDUMIN   : LDOPregister<0b111, "umin", 0, 0, "">;
+defm LDUMINA  : LDOPregister<0b111, "umin", 1, 0, "a">;
+defm LDUMINL  : LDOPregister<0b111, "umin", 0, 1, "l">;
+defm LDUMINAL : LDOPregister<0b111, "umin", 1, 1, "al">;
+
+// v8.1 atomic ST<OP>(register) as aliases to "LD<OP>(register) when Rt=xZR"
+defm : STOPregister<"stadd","LDADD">; // STADDx
+defm : STOPregister<"stclr","LDCLR">; // STCLRx
+defm : STOPregister<"steor","LDEOR">; // STEORx
+defm : STOPregister<"stset","LDSET">; // STSETx
+defm : STOPregister<"stsmax","LDSMAX">;// STSMAXx
+defm : STOPregister<"stsmin","LDSMIN">;// STSMINx
+defm : STOPregister<"stumax","LDUMAX">;// STUMAXx
+defm : STOPregister<"stumin","LDUMIN">;// STUMINx
 
 //===----------------------------------------------------------------------===//
 // Logical instructions.

lib/Target/AArch64/AArch64RegisterInfo.td

   def hsub : SubRegIndex<16>;
   def ssub : SubRegIndex<32>;
   def dsub : SubRegIndex<32>;
+  def sube32 : SubRegIndex<32>;
+  def subo32 : SubRegIndex<32>;
   def qhisub : SubRegIndex<64>;
   def qsub : SubRegIndex<64>;
+  def sube64 : SubRegIndex<64>;
+  def subo64 : SubRegIndex<64>;
   // Note: Code depends on these having consecutive numbers
   def dsub0 : SubRegIndex<64>;
   def dsub1 : SubRegIndex<64>;

 def FPR32Op : RegisterOperand<FPR32, "printOperand">;
 def FPR64Op : RegisterOperand<FPR64, "printOperand">;
 def FPR128Op : RegisterOperand<FPR128, "printOperand">;
+
+
+//===----------------------------------------------------------------------===//
+// ARMv8.1a atomic CASP register operands
+
+
+def WSeqPairs : RegisterTuples<[sube32, subo32], [(rotl GPR32, 0), (rotl GPR32, 1)]>;
+def XSeqPairs : RegisterTuples<[sube64, subo64], [(rotl GPR64, 0), (rotl GPR64, 1)]>;
+
+def WSeqPairsClass   : RegisterClass<"AArch64", [untyped], 32, (add WSeqPairs)> {let Size = 64;}
+def XSeqPairsClass   : RegisterClass<"AArch64", [untyped], 64, (add XSeqPairs)> {let Size = 128;}
+
+
+let RenderMethod = "addRegOperands", ParserMethod="tryParseGPRSeqPair" in {
+  def WSeqPairsAsmOperandClass : AsmOperandClass { let Name = "WSeqPair"; }
+  def XSeqPairsAsmOperandClass : AsmOperandClass { let Name = "XSeqPair"; }
+}
+
+def WSeqPairClassOperand :
+    RegisterOperand<WSeqPairsClass, "printGPRSeqPairsClassOperand<32>"> {
+  let ParserMatchClass = WSeqPairsAsmOperandClass;
+}
+def XSeqPairClassOperand :
+    RegisterOperand<XSeqPairsClass, "printGPRSeqPairsClassOperand<64>"> {
+  let ParserMatchClass = XSeqPairsAsmOperandClass;
+}
+
+
+//===----- END: v8.1a atomic CASP register operands -----------------------===//

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

   OperandMatchResultTy tryParseAddSubImm(OperandVector &Operands);
   OperandMatchResultTy tryParseGPR64sp0Operand(OperandVector &Operands);
   bool tryParseVectorRegister(OperandVector &Operands);
+  OperandMatchResultTy tryParseGPRSeqPair(OperandVector &Operands);
 
 public:
   enum AArch64MatchResultTy {

     return Kind == k_Register && !Reg.isVector &&
       AArch64MCRegisterClasses[AArch64::GPR64RegClassID].contains(Reg.RegNum);
   }
+  bool isWSeqPair() const {
+    return Kind == k_Register && !Reg.isVector &&
+      AArch64MCRegisterClasses[AArch64::WSeqPairsClassRegClassID].contains(Reg.RegNum);
+  }
+  bool isXSeqPair() const {
+    return Kind == k_Register && !Reg.isVector &&
+      AArch64MCRegisterClasses[AArch64::XSeqPairsClassRegClassID].contains(Reg.RegNum);
+  }
 
   bool isGPR64sp0() const {
     return Kind == k_Register && !Reg.isVector &&

     return Match_Success;
   return Match_InvalidOperand;
 }
+
+
+AArch64AsmParser::OperandMatchResultTy
+AArch64AsmParser::tryParseGPRSeqPair(OperandVector &Operands) {
+
+  SMLoc S = getLoc();
+
+  if (getParser().getTok().isNot(AsmToken::Identifier)) {
+    Error(S, "expected register");
+    return MatchOperand_ParseFail;
+  }
+
+  int FirstReg = tryParseRegister();
+  if (FirstReg == -1) {
+    return MatchOperand_ParseFail;
+  }
+  printf ("FirstReg=%d\n",FirstReg);
+
+  const MCRegisterClass &WRegClass =
+      AArch64MCRegisterClasses[AArch64::GPR32RegClassID];
+  const MCRegisterClass &XRegClass =
+      AArch64MCRegisterClasses[AArch64::GPR64RegClassID];
+
+  bool isXReg = XRegClass.contains(FirstReg),
+       isWReg = WRegClass.contains(FirstReg);
+  if (!isXReg && !isWReg) {
+    Error(S, "expected register");
+    return MatchOperand_ParseFail;
+  }
+
+  const MCRegisterInfo *RI = getContext().getRegisterInfo();
+  unsigned FirstEncoding = RI->getEncodingValue(FirstReg);
+
+  if (FirstEncoding & 0x1) {
+    Error(S, "expected even register");
+    return MatchOperand_ParseFail;
+  }
+
+  SMLoc M = getLoc();
+  if (getParser().getTok().isNot(AsmToken::Comma)) {
+    Error(M, "expected comma");
+    return MatchOperand_ParseFail;
+  }
+  // Eat the comma
+  getParser().Lex();
+
+  SMLoc E = getLoc();
+  int SecondReg = tryParseRegister();
+  if (SecondReg ==-1) {
+    return MatchOperand_ParseFail;
+  }
+
+ if (RI->getEncodingValue(SecondReg) != FirstEncoding + 1 ||
+      (isXReg && !XRegClass.contains(SecondReg)) ||
+      (isWReg && !WRegClass.contains(SecondReg))) {
+    Error(E,"expected consecutive registers of same size");
+    return MatchOperand_ParseFail;
+  }
+  
+  unsigned Pair = 0;
+  if(isXReg) {
+    Pair = RI->getMatchingSuperReg(FirstReg, AArch64::sube64,
+           &AArch64MCRegisterClasses[AArch64::XSeqPairsClassRegClassID]);
+  } else {
+    Pair = RI->getMatchingSuperReg(FirstReg, AArch64::sube32,
+           &AArch64MCRegisterClasses[AArch64::WSeqPairsClassRegClassID]);
+  }
+
+  Operands.push_back(AArch64Operand::CreateReg(Pair, false, S, getLoc(),
+      getContext()));
+
+  return MatchOperand_Success;
+}

lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp

                                          uint64_t Addr, const void *Decoder);
 static DecodeStatus DecodeVecShiftL8Imm(llvm::MCInst &Inst, unsigned Imm,
                                         uint64_t Addr, const void *Decoder);
+static DecodeStatus DecodeWSeqPairsClassRegisterClass(MCInst &Inst,
+                                                      unsigned RegNo,
+                                                      uint64_t Addr,
+                                                      const void *Decoder);
+static DecodeStatus DecodeXSeqPairsClassRegisterClass(MCInst &Inst,
+                                                      unsigned RegNo,
+                                                      uint64_t Addr,
+                                                      const void *Decoder);
 
 static bool Check(DecodeStatus &Out, DecodeStatus In) {
   switch (In) {

 
   return Success;
 }
+
+static DecodeStatus DecodeGPRSeqPairsClassRegisterClass(MCInst &Inst,
+                                                        unsigned RegClassID,
+                                                        unsigned RegNo,
+                                                        uint64_t Addr,
+                                                        const void *Decoder) {
+  // Register number must be even (see CASP instruction)
+  if (RegNo & 0x1)
+    return Fail;
+
+  unsigned Register = AArch64MCRegisterClasses[RegClassID].getRegister(RegNo);
+  Inst.addOperand(MCOperand::CreateReg(Register));
+  return Success;
+}
+
+static DecodeStatus DecodeWSeqPairsClassRegisterClass(MCInst &Inst,
+                                                      unsigned RegNo,
+                                                      uint64_t Addr,
+                                                      const void *Decoder) {
+  return DecodeGPRSeqPairsClassRegisterClass(Inst, 
+                                             AArch64::WSeqPairsClassRegClassID,
+                                             RegNo, Addr, Decoder);
+}
+
+static DecodeStatus DecodeXSeqPairsClassRegisterClass(MCInst &Inst,
+                                                      unsigned RegNo,
+                                                      uint64_t Addr,
+                                                      const void *Decoder) {
+  return DecodeGPRSeqPairsClassRegisterClass(Inst, 
+                                             AArch64::XSeqPairsClassRegClassID,
+                                             RegNo, Addr, Decoder);
+}

lib/Target/AArch64/Disassembler/LLVMBuild.txt

 type = Library
 name = AArch64Disassembler
 parent = AArch64
-required_libraries = AArch64Info AArch64Utils MC MCDisassembler Support
+required_libraries = AArch64Desc AArch64Info AArch64Utils MC MCDisassembler Support
 add_to_library_groups = AArch64

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.h

   void printMRSSystemRegister(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printSystemPStateField(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printSIMDType10Operand(const MCInst *MI, unsigned OpNum, raw_ostream &O);
+  template<unsigned size>
+  void printGPRSeqPairsClassOperand(const MCInst *MI, unsigned OpNum, raw_ostream &O);
 };
 
 class AArch64AppleInstPrinter : public AArch64InstPrinter {

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.cpp

   return Reg;
 }
 
+template<unsigned size>
+void AArch64InstPrinter::printGPRSeqPairsClassOperand(const MCInst *MI, unsigned OpNum, raw_ostream &O) {
+  static_assert(size == 64 || size == 32,"Template parameter must be either 32 or 64");
+  unsigned Reg = MI->getOperand(OpNum).getReg();
+
+  unsigned Sube = (size == 32) ? AArch64::sube32 : AArch64::sube64;
+  unsigned Subo = (size == 32) ? AArch64::subo32 : AArch64::subo64;
+
+  unsigned Even = MRI.getSubReg(Reg,  Sube);
+  unsigned Odd = MRI.getSubReg(Reg,  Subo);
+  O << getRegisterName(Even) << ", " << getRegisterName(Odd);
+}
+
 void AArch64InstPrinter::printVectorList(const MCInst *MI, unsigned OpNum,
                                          raw_ostream &O,
                                          StringRef LayoutSuffix) {

test/MC/AArch64/armv8.1a-atomic.s

+// RUN: not llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.1a -show-encoding < %s 2> %t | FileCheck %s
+// RUN: FileCheck --check-prefix=CHECK-ERROR <%t %s
+  .text
+
+  //8 bits
+  casb   w0, w1, [x2]
+  casab  w0, w1, [x2]
+  caslb  w0, w1, [x2]
+  casalb   w0, w1, [x2]
+
+//CHECK:  casb   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x08]
+//CHECK:  casab  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x08]
+//CHECK:  caslb   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x08]
+//CHECK:  casalb   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x08]
+
+  casb w0, w1, [w2]
+  casalb x0, x1, [x2]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casb w0, w1, [w2]
+//CHECK-ERROR:                 ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casalb x0, x1, [x2]
+//CHECK-ERROR:          ^
+
+  //16 bits
+  cash   w0, w1, [x2]
+  casah  w0, w1, [x2]
+  caslh  w0, w1, [x2]
+  casalh   w0, w1, [x2]
+
+//CHECK:  cash   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x48]
+//CHECK:  casah  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x48]
+//CHECK:  caslh   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x48]
+//CHECK:  casalh   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x48]
+
+  //32 bits
+  cas   w0, w1, [x2]
+  casa  w0, w1, [x2]
+  casl  w0, w1, [x2]
+  casal   w0, w1, [x2]
+
+//CHECK:  cas   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x88]
+//CHECK:  casa  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x88]
+//CHECK:  casl   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x88]
+//CHECK:  casal   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x88]
+
+  cas   w0, w1, [w2]
+  casl  w0, x1, [x2]
+
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   cas   w0, w1, [w2]
+//CHECK-ERROR:                  ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casl  w0, x1, [x2]
+//CHECK-ERROR:             ^
+
+  //64 bits
+  cas   x0, x1, [x2]
+  casa  x0, x1, [x2]
+  casl   x0, x1, [x2]
+  casal   x0, x1, [x2]
+
+//CHECK:  cas   x0, x1, [x2]        //      encoding: [0x41,0x7c,0xa0,0xc8]
+//CHECK:  casa  x0, x1, [x2]        //      encoding: [0x41,0x7c,0xe0,0xc8]
+//CHECK:  casl   x0, x1, [x2]       //      encoding: [0x41,0xfc,0xa0,0xc8]
+//CHECK:  casal   x0, x1, [x2]      //      encoding: [0x41,0xfc,0xe0,0xc8]
+
+  casa   x0, x1, [w2]
+  casal  x0, w1, [x2]
+
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casa   x0, x1, [w2]
+//CHECK-ERROR:                   ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casal  x0, w1, [x2]
+//CHECK-ERROR:              ^
+
+  // LD<OP> intructions
+  ldadda x0, x1, [x2]
+  ldclrl x0, x1, [x2]
+  ldeoral x0, x1, [x2]
+  ldset x0, x1, [x2]
+  ldsmaxa w0, w1, [x2]
+  ldsminlb w0, w1, [x2]
+  ldumaxalh w0, w1, [x2]
+  ldumin w0, w1, [x2]
+//CHECK: ldadda     x0, x1, [x2]  // encoding: [0x41,0x00,0xa0,0xf8]
+//CHECK: ldclrl     x0, x1, [x2]  // encoding: [0x41,0x10,0x60,0xf8]
+//CHECK: ldeoral    x0, x1, [x2]  // encoding: [0x41,0x20,0xe0,0xf8]
+//CHECK: ldset      x0, x1, [x2]  // encoding: [0x41,0x30,0x20,0xf8]
+//CHECK: ldsmaxa    w0, w1, [x2]  // encoding: [0x41,0x40,0xa0,0xb8]
+//CHECK: ldsminlb   w0, w1, [x2]  // encoding: [0x41,0x50,0x60,0x38]
+//CHECK: ldumaxalh  w0, w1, [x2]  // encoding: [0x41,0x60,0xe0,0x78]
+//CHECK: ldumin     w0, w1, [x2]  // encoding: [0x41,0x70,0x20,0xb8]
+
+  // ST<OP> intructions: aliases to LD<OP>
+  stADDlb w0, [x2]
+  stclrlh w0, [x2]
+  steorl  w0, [x2]
+  stsetl  x0, [x2]
+  stsmaxb  w0, [x2]
+  stsminh  w0, [x2]
+  stumax   w0, [x2]
+  stumin   x0, [x2]
+//CHECK: staddlb    w0, [x2]  // encoding: [0x5f,0x00,0x60,0x38]
+//CHECK: stclrlh    w0, [x2]  // encoding: [0x5f,0x10,0x60,0x78]
+//CHECK: steorl     w0, [x2]  // encoding: [0x5f,0x20,0x60,0xb8]
+//CHECK: stsetl     x0, [x2]  // encoding: [0x5f,0x30,0x60,0xf8]
+//CHECK: stsmaxb     w0, [x2]  // encoding: [0x5f,0x40,0x20,0x38]
+//CHECK: stsminh     w0, [x2]  // encoding: [0x5f,0x50,0x20,0x78]
+//CHECK: stumax      w0, [x2]  // encoding: [0x5f,0x60,0x20,0xb8]
+//CHECK: stumin      x0, [x2]  // encoding: [0x5f,0x70,0x20,0xf8]
+
+  ldsmax x0, x1, [w2]
+  ldeorl w0, w1, [w2]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   ldsmax x0, x1, [w2]
+//CHECK-ERROR:                   ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   ldeorl w0, w1, [w2]
+//CHECK-ERROR:                   ^
+
+  //SWP instruction
+  swp   x0, x1, [x2]
+  swpb  w0, w1, [x2]
+  swplh w0, w1, [x2]
+  swpal x0, x1, [sp]
+//CHECK: swp   x0, x1, [x2]       // encoding: [0x41,0x80,0x20,0xf8]
+//CHECK: swpb  w0, w1, [x2]       // encoding: [0x41,0x80,0x20,0x38]
+//CHECK: swplh w0, w1, [x2]       // encoding: [0x41,0x80,0x60,0x78]
+//CHECK: swpal x0, x1, [sp]       // encoding: [0xe1,0x83,0xe0,0xf8]
+
+  swp   x0, x1, [w2]
+  swp   x0, x1, [xzr]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   swp   x0, x1, [w2]
+//CHECK-ERROR:                  ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   swp   x0, x1, [xzr]
+//CHECK-ERROR:                  ^
+
+  //CASP instruction
+  casp x0, x1, x2, x3, [x4]
+  casp w0, w1, w2, w3, [x4]
+//CHECK: casp x0, x1, x2, x3, [x4]      // encoding: [0x82,0x7c,0x20,0x48]
+//CHECK: casp w0, w1, w2, w3, [x4]      // encoding: [0x82,0x7c,0x20,0x08]
+
+  casp x1, x2, x4, x5, [x6]
+  casp x0, x1, x3, x4, [x5]
+  casp x0, x2, x4, x5, [x6]
+  casp x0, x1, x2, x4, [x5]
+  casp x0, w1, x2, x3, [x5]
+  casp w0, x1, x2, x3, [x5]
+  casp w0, x1, w2, w3, [x5]
+  casp x0, x1, w2, w3, [x5]
+//CHECK-ERROR: error: expected even register
+//CHECK-ERROR:  casp x1, x2, x4, x5, [x6]
+//CHECK-ERROR:       ^
+//CHECK-ERROR: error: expected even register
+//CHECK-ERROR:  casp x0, x1, x3, x4, [x5]
+//CHECK-ERROR:               ^
+//CHECK-ERROR: error: expected consecutive registers of same size
+//CHECK-ERROR:  casp x0, x2, x4, x5, [x6]
+//CHECK-ERROR:           ^
+//CHECK-ERROR: error: expected consecutive registers of same size
+//CHECK-ERROR:  casp x0, x1, x2, x4, [x5]
+//CHECK-ERROR:                   ^
+//CHECK-ERROR: error: expected consecutive registers of same size
+//CHECK-ERROR:  casp x0, w1, x2, x3, [x5]
+//CHECK-ERROR:           ^
+//CHECK-ERROR: error: expected consecutive registers of same size
+//CHECK-ERROR:  casp w0, x1, x2, x3, [x5]
+//CHECK-ERROR:           ^
+//CHECK-ERROR: error: expected consecutive registers of same size
+//CHECK-ERROR:  casp w0, x1, w2, w3, [x5]
+//CHECK-ERROR:           ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:  casp x0, x1, w2, w3, [x5]
+//CHECK-ERROR:               ^

test/MC/Disassembler/AArch64/armv8.1a-atomic.txt

+# RUN: llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.1a --disassemble < %s | FileCheck %s
+
+0x41,0x7c,0xa0,0x08
+0x41,0x7c,0xe0,0x08
+0x41,0xfc,0xa0,0x08
+0x41,0xfc,0xe0,0x08
+0x41,0x7c,0xa0,0x48
+0x41,0x7c,0xe0,0x48
+0x41,0xfc,0xa0,0x48
+0x41,0xfc,0xe0,0x48
+# CHECK:  casb   w0, w1, [x2]
+# CHECK:  casab  w0, w1, [x2]
+# CHECK:  caslb   w0, w1, [x2]
+# CHECK:  casalb   w0, w1, [x2]
+# CHECK:  cash   w0, w1, [x2]
+# CHECK:  casah  w0, w1, [x2]
+# CHECK:  caslh   w0, w1, [x2]
+# CHECK:  casalh   w0, w1, [x2]
+
+0x41,0x7c,0xa0,0x88
+0x41,0x7c,0xe0,0x88
+0x41,0xfc,0xa0,0x88
+0x41,0xfc,0xe0,0x88
+0x41,0x7c,0xa0,0xc8
+0x41,0x7c,0xe0,0xc8
+0x41,0xfc,0xa0,0xc8
+0x41,0xfc,0xe0,0xc8
+# CHECK:  cas   w0, w1, [x2]
+# CHECK:  casa  w0, w1, [x2]
+# CHECK:  casl   w0, w1, [x2]
+# CHECK:  casal   w0, w1, [x2]
+# CHECK:  cas   x0, x1, [x2]
+# CHECK:  casa  x0, x1, [x2]
+# CHECK:  casl   x0, x1, [x2]
+# CHECK:  casal   x0, x1, [x2]
+
+0x41,0x80,0x20,0xf8
+0x41,0x80,0x20,0x38
+0x41,0x80,0x60,0x78
+0xe1,0x83,0xe0,0xf8
+# CHECK: swp   x0, x1, [x2]
+# CHECK: swpb  w0, w1, [x2]
+# CHECK: swplh w0, w1, [x2]
+# CHECK: swpal x0, x1, [sp]
+
+0x41,0x00,0xa0,0xf8
+0x41,0x10,0x60,0xf8
+0x41,0x20,0xe0,0xf8
+0x41,0x30,0x20,0xf8
+0x41,0x40,0xa0,0xb8
+0x41,0x50,0x60,0x38
+0x41,0x60,0xe0,0x78
+0x41,0x70,0x20,0xb8
+# CHECK:  ldadda    x0, x1, [x2]
+# CHECK:  ldclrl    x0, x1, [x2]
+# CHECK:  ldeoral   x0, x1, [x2]
+# CHECK:  ldset     x0, x1, [x2]
+# CHECK:  ldsmaxa   w0, w1, [x2]
+# CHECK:  ldsminlb  w0, w1, [x2]
+# CHECK:  ldumaxalh w0, w1, [x2]
+# CHECK:  ldumin    w0, w1, [x2]
+
+0x5f,0x00,0x60,0x38
+0x5f,0x10,0x60,0x78
+0x5f,0x20,0x60,0xb8
+0x5f,0x30,0x60,0xf8
+0x5f,0x40,0x20,0x38
+0x5f,0x50,0x20,0x78
+0x5f,0x60,0x20,0xb8
+0x5f,0x70,0x20,0xf8
+# CHECK: staddlb    w0, [x2]
+# CHECK: stclrlh    w0, [x2]
+# CHECK: steorl     w0, [x2]
+# CHECK: stsetl     x0, [x2]
+# CHECK: stsmaxb     w0, [x2]
+# CHECK: stsminh     w0, [x2]
+# CHECK: stumax      w0, [x2]
+# CHECK: stumin      x0, [x2]
+
+0x82,0x7c,0x20,0x48
+0x82,0x7c,0x20,0x08
+# CHECK: casp x0, x1, x2, x3, [x4]
+# CHECK: casp w0, w1, w2, w3, [x4]