[AArch64] Add v8.1a atomic instructions

[t.p.northover]

I think I have more comments than are strictly justified by how I feel about this patch. Generally, it *is* good, I just think it could be streamlined to be even better.

Tim.

lib/Target/AArch64/AArch64InstrFormats.td

8633–8636

I don't think this is the best location for these changes. TokanAliases are very different from the bulk of the file, and are at the end of the file because of that.

8668–8669

These names are non-obvious. The AArch64 assembly syntax actually uses "L" for release. I'd suggest actually using "Acq" and "Rel" to make it more obvious for readers. We're not constrained for space here.

8685–8690

You should probably put BaseCAS and BaseCASP together. For a long while I thought this whole indirection was pointless (until I saw CASP).

8693

"BaseLD" is far too generic as a name for something that's only applicable to these atomicrmw operations.

8715–8716

I think the factoring of the multiclass hierarchy is inconsistent with the rest of the backend here. The general convention is that the capitalised prefix of the instruction mirrors the mnemonic, with suffixes (capitalised or not) showing the variant.

Obviously, that's not always possible, but here I think it is.

In this case, I'd expect in AArch64InstrInfo.td something like

defm LDADD : ...
defm LDADDA : ...
defm LDADDL : ...
defm LDADDAL : ...
...

with the second-level hierarchy appending a B/H/W/X as needed. See the usual loads & stores for example (though they're confounded slightly by both "ldrb w0, [x0]" and "ldr b0, [x0]" being byte loads).

8731–8732

It's also rather odd that the "BaseAliasLD" classes actually define mnemonics starting with "st".

lib/Target/AArch64/AArch64RegisterInfo.td

601–606

I think these should probably go with the existing SubRegIndex definitions.

608–609

We should probably mention GPRPair here, for readability in "-debug" if nothing else. (Elsewhere too, just because there are no FPRPairs now, doesn't mean there won't be in future).

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4311

We use lower-case "expected". Could also be more detailed here: "expected even general purpose register" for example.

4316

Space.

4354–4356

I tend to think you should either trust that the enums are consecutive here or not. Trusting that "x1 == x0 + 1" but then being paranoid over whether maybe "w1 == x0 + 1" seems daft.

4363

I think things might be a bit simpler in this function overall if you used functions like MCRegisterInfo::getMatchingSuperReg(FirstReg, AArch64::sube0, &PairClass. It could both diagnose an odd input and provide the correct Pair in one step.

More generally, you might want to consider a top-level "template<int Width> tryParsePair(...)" which passes the needed 32/64 info (like Pair32RegClass/Pair64Regclass) in as an argument to a non-templated "tryParsePair" function. It's a pattern I've found useful in assembly parsers that have to deal with multiple sizes in the past.

lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp

1551

Another possible case for a template deferring to the real implementation. These two functions are practically identical.

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

I could believe this change is needed, but why does this particular patch reveal the necessity?

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.cpp

1159

Local variables start with a capital letter (for now).

[vsukharev]

Hi Tim, thank you for your hints and comments. I'll address it soon.
Meanwhile shouldn't we perform careful review/re-do/re-base process from the first patch in the dependency chain( http://reviews.llvm.org/D8496 )?
That way I could save efforts rebasing the rest of chain after every correction..

[vsukharev]

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

Introduced usage of AArch64MCRegisterClasses causes the need of this explicit dependency.
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

[vsukharev]

Almost all points are corrected

[vsukharev]

lib/Target/AArch64/AArch64InstrFormats.td

8731–8732

I'd wish instructions would be better named as "LDST<op>" and "ST<op>"....
That way it semantics be clear, so ST just doesn't save loaded value in reg

8767

I'd wish instructions would be better named as "LDST<op>" and "ST<op>"....
That way it semantics be clear, so ST just doesn't save loaded value in reg

lib/Target/AArch64/AArch64RegisterInfo.td

608–609

I'm gonna rename it to WSeqPairs and XSeqPairs, similar to existing DSeqPairs and QSeqPairs

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4331

About top-level template "tryParseGPRSeqPair<>" and generic function:
hmmm, is there a way for AArch64AsmParser (methods MatchOperandParserImpl and tryCustomParseOperand in AArch64GenAsmMatcher.inc) to guess in advance which pair to expect - MCK_WSeqPair or MCK_XSeqPair?

I guess without that knowledge it is possible only to call this generic tryParseGPRSeqPair from tryCustomParseOperand, and it will find out the size of actual operands.
I've just tried to use a template and face only tryParseGPRSeqPair<32>() is being called for both test lines, and 64-bit variant is never called,

//CASP instruction
casp x0, x1, x2, x3, [x4]
casp w0, w1, w2, w3, [x4]

4361

No paranoia, second complicated check is needed against pairs like "w0, x1", I've added some tests for that, like

//CHECK-ERROR: error: expected consecutive registers of same size
//CHECK-ERROR: casp w0, x1, x2, x3, [x5]

4372

Introduced usage of AArch64MCRegisterClasses causes the need of explicit dependency AArch64Info -> AArch64Desc
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

lib/Target/AArch64/Disassembler/LLVMBuild.txt

22

Introduced usage of AArch64MCRegisterClasses causes the need of explicit dependency AArch64Info -> AArch64Desc
Without this change, linker issues an error in shared library mode:
llvm::AArch64MCRegisterClasses could not be found in AArch64Disassembler.cpp:1558 and 1571

[vsukharev]

ping..

[vsukharev]

It might seem there are many comments on future actions, while they are all already addressed in last-uploaded diff http://reviews.llvm.org/differential/diff/22894/
So, it is ready to re-review

[t.p.northover]

Looks mostly reasonable. Just a few minor comments now.

Tim.

lib/Target/AArch64/AArch64RegisterInfo.td

608–609

These lines are longer than the column limit. Not sure about the surrounding ones.

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

4322

Debugging code should be removed.

4332

Since this only ever called for an instruction that *must* have a pair, we can be more helpful in the error messages produced. Always say that we need an even/odd register pair, for example.

[t.p.northover]

Hi Vladimir,

One more comment...

lib/Target/AArch64/AArch64InstrFormats.td

8670

Arnold's pointed out to me that it probably should be Rs that gets written here. CASP seems to be right though.

[vsukharev]

Last minor comments are processed.
Thank you for comments and catches.

[richard.barton.arm]

Hi Tim

Are you happy with Vladimir's final patch?

Ta
Rich

[vsukharev]

Hi Arnold,
It seems Tim is unpingable for now...
As soon you have taken a look at this patch, would you be able to raise more comments, or approve the patch?
Ta, Vladimir

[t.p.northover]

Hi Vladimir,

Really sorry I didn't reply to the last two pings; I'd been neglecting the list recently. I think this looks fine now, thanks for updating it.

Tim.

lib/Target/AArch64/AArch64InstrFormats.td

 def : TokenAlias<".8H", ".8h">;
 def : TokenAlias<".4S", ".4s">;
 def : TokenAlias<".2D", ".2d">;
 def : TokenAlias<".1Q", ".1q">;
 def : TokenAlias<".B", ".b">;
 def : TokenAlias<".H", ".h">;
 def : TokenAlias<".S", ".s">;
 def : TokenAlias<".D", ".d">;
 def : TokenAlias<".Q", ".q">;
+
+//----------------------------------------------------------------------------
+// v8.1 atomic instructions extension:
+// * CAS
+// * CASP
+// * LD<OP>
+// * SWP
+
+// Instruction encodings:
+//
+//      31 30|29  24|23|22|21|20 16|15|14  10|9 5|4 0
+// CAS  SZ   |001000|1 |A |1 |Rs   |R |11111 |Rn |Rt
+// CASP  0|SZ|001000|0 |A |1 |Rs   |R |11111 |Rn |Rt
+// LD   SZ   |111000|A |R |1 |Rs   |0 |OPC|00|Rn |Rt
+// SWP  SZ   |111000|A |R |1 |Rs   |1 |OPC|00|Rn |Rt
+
+// Instruction syntax:
+//
+// CAS{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// CAS{<order>} <Xs>, <Xt>, [<Xn|SP>]
+// CASP{<order>} <Ws>, <W(s+1)>, <Wt>, <W(t+1)>, [<Xn|SP>]
+// CASP{<order>} <Xs>, <X(s+1)>, <Xt>, <X(t+1)>, [<Xn|SP>]
+// LD<OP>{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// LD<OP>{<order>} <Xs>, <Xt>, [<Xn|SP>]
+// SWP{<order>}[<size>] <Ws>, <Wt>, [<Xn|SP>]
+// SWP{<order>} <Xs>, <Xt>, [<Xn|SP>]
+
+
+let Predicates = [HasV8_1a], mayLoad = 1, mayStore = 1, hasSideEffects = 1 in
+class BaseCASEncoding<dag oops, dag iops, string asm, string operands,
+                      string cstr, list<dag> pattern>
+      : I<oops, iops, asm, operands, cstr, pattern> {
+  bits<2> Sz;
+  bit NP;
+  bit A;
+  bit R;
+  bits<5> Rs;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b001000;
+  let Inst{23} = NP;
+  let Inst{22} = A;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = R;
+  let Inst{14-10} = 0b11111;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+class BaseCAS<string order, string size, RegisterClass RC>
+      : BaseCASEncoding<(outs RC:$out),(ins RC:$Rs, RC:$Rt, GPR64sp:$Rn),
+                        "cas" # order # size, "\t$Rs, $Rt, [$Rn]",
+                        "$out = $Rt",[]> {
+  let NP = 1;
+}
+
+let Predicates = [HasV8_1a], mayLoad = 1, mayStore = 1, hasSideEffects = 1 in
+class BaseLD<string op, string order, string size, RegisterClass RC>
+      : I<(outs RC:$Rt),(ins RC:$Rs, GPR64sp:$Rn), "ld" # op # order # size, "\t$Rs, $Rt, [$Rn]","",[]> {
+  bits<2> Sz;
+  bit A;
+  bit R;
+  bits<5> Rs;
+  bits<3> opc;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b111000;
+  let Inst{23} = A;
+  let Inst{22} = R;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = 0b0;
+  let Inst{14-12} = opc;
+  let Inst{11-10} = 0b00;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+multiclass LDOp <string order, string size, RegisterClass RC> {
+  let opc = 0b000 in def _add  : BaseLD<"add" , order, size, RC>;
+  let opc = 0b001 in def _clr  : BaseLD<"clr" , order, size, RC>;
+  let opc = 0b010 in def _eor  : BaseLD<"eor" , order, size, RC>;
+  let opc = 0b011 in def _set  : BaseLD<"set" , order, size, RC>;
+  let opc = 0b100 in def _smax : BaseLD<"smax", order, size, RC>;
+  let opc = 0b101 in def _smin : BaseLD<"smin", order, size, RC>;
+  let opc = 0b110 in def _umax : BaseLD<"umax", order, size, RC>;
+  let opc = 0b111 in def _umin : BaseLD<"umin", order, size, RC>;
+}
+
+// Aliases for LD<OP>
+let Predicates = [HasV8_1a] in
+class BaseAliasLD<string asm, RegisterClass OP, Register Reg,Instruction inst> :
+      InstAlias<asm # "\t$Rs, [$Rn]", (inst Reg, OP:$Rs, GPR64sp:$Rn)>;
+
+multiclass BaseAliasLDOp<string asm> {
+  def : BaseAliasLD<"st" # asm # "lb", GPR32, WZR, !cast<Instruction>("LD" # _B_release_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "lh", GPR32, WZR, !cast<Instruction>("LD" # _H_release_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "l",  GPR32, WZR, !cast<Instruction>("LD" # _S_release_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "l",  GPR64, XZR, !cast<Instruction>("LD" # _D_release_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "b",  GPR32, WZR, !cast<Instruction>("LD" # _B_no_order_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "h",  GPR32, WZR, !cast<Instruction>("LD" # _H_no_order_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "",   GPR32, WZR, !cast<Instruction>("LD" # _S_no_order_ # asm)>;
+  def : BaseAliasLD<"st" # asm # "",   GPR64, XZR, !cast<Instruction>("LD" # _D_no_order_ # asm)>;
+}
+
+let Predicates = [HasV8_1a] in
+class BaseSWP<string order, string size, RegisterClass RC>
+      : I<(outs RC:$Rt),(ins RC:$Rs, GPR64sp:$Rn), "swp" # order # size,
+          "\t$Rs, $Rt, [$Rn]","",[]> {
+  bits<2> Sz;
+  bit A;
+  bit R;
+  bits<5> Rs;
+  bits<3> opc = 0b000;
+  bits<5> Rn;
+  bits<5> Rt;
+  let Inst{31-30} = Sz;
+  let Inst{29-24} = 0b111000;
+  let Inst{23} = A;
+  let Inst{22} = R;
+  let Inst{21} = 0b1;
+  let Inst{20-16} = Rs;
+  let Inst{15} = 0b1;
+  let Inst{14-12} = opc;
+  let Inst{11-10} = 0b00;
+  let Inst{9-5} = Rn;
+  let Inst{4-0} = Rt;
+}
+
+class BaseCASP<string order, string size, RegisterOperand RC>
+      : BaseCASEncoding<(outs RC:$out),(ins RC:$Rs, RC:$Rt, GPR64sp:$Rn),
+                        "casp" # order # size, "\t$Rs, $Rt, [$Rn]",
+                        "$out = $Rs",[]> {
+  let NP = 0;
+}

lib/Target/AArch64/AArch64InstrInfo.td

 def CRC32Wrr : BaseCRC32<0, 0b10, 0, GPR32, int_aarch64_crc32w, "crc32w">;
 def CRC32Xrr : BaseCRC32<1, 0b11, 0, GPR64, int_aarch64_crc32x, "crc32x">;
 
 def CRC32CBrr : BaseCRC32<0, 0b00, 1, GPR32, int_aarch64_crc32cb, "crc32cb">;
 def CRC32CHrr : BaseCRC32<0, 0b01, 1, GPR32, int_aarch64_crc32ch, "crc32ch">;
 def CRC32CWrr : BaseCRC32<0, 0b10, 1, GPR32, int_aarch64_crc32cw, "crc32cw">;
 def CRC32CXrr : BaseCRC32<1, 0b11, 1, GPR64, int_aarch64_crc32cx, "crc32cx">;
 
+// v8.1 atomic CAS
+multiclass CASOrder<string size, RegisterClass RC> {
+  let A = 0b1, R = 0b0 in def _acquire : BaseCAS<"a",size,RC>;
+  let A = 0b0, R = 0b1 in def _release : BaseCAS<"l",size,RC>;
+  let A = 0b1, R = 0b1 in def _acquire_and_release : BaseCAS<"al",size,RC>;
+  let A = 0b0, R = 0b0 in def _no_order : BaseCAS<"",size,RC>;
+}
+
+multiclass CASOrderSize {
+  let Sz = 0b00 in defm _B : CASOrder<"b",GPR32>;
+  let Sz = 0b01 in defm _H : CASOrder<"h",GPR32>;
+  let Sz = 0b10 in defm _S : CASOrder<"",GPR32>;
+  let Sz = 0b11 in defm _D : CASOrder<"",GPR64>;
+}
+
+defm CAS : CASOrderSize;
+
+// v8.1 atomic SWP
+multiclass SWPOrder<string size, RegisterClass RC> {
+  let A = 0b1, R = 0b0 in def _acquire : BaseSWP<"a",size,RC>;
+  let A = 0b0, R = 0b1 in def _release : BaseSWP<"l",size,RC>;
+  let A = 0b1, R = 0b1 in def _acquire_and_release : BaseSWP<"al",size,RC>;
+  let A = 0b0, R = 0b0 in def _no_order : BaseSWP<"",size,RC>;
+}
+
+multiclass SWPOrderSize {
+  let Sz = 0b00 in defm _B : SWPOrder<"b",GPR32>;
+  let Sz = 0b01 in defm _H : SWPOrder<"h",GPR32>;
+  let Sz = 0b10 in defm _S : SWPOrder<"",GPR32>;
+  let Sz = 0b11 in defm _D : SWPOrder<"",GPR64>;
+}
+
+defm SWP : SWPOrderSize;
+
+// v8.1 atomic LD<OP>
+multiclass LDOpOrder<string size, RegisterClass RC> {
+  let A = 0b1, R = 0b0 in defm _acquire : LDOp<"a", size, RC>;
+  let A = 0b0, R = 0b1 in defm _release : LDOp<"l", size, RC>;
+  let A = 0b1, R = 0b1 in defm _acquire_and_release : LDOp<"al", size, RC>;
+  let A = 0b0, R = 0b0 in defm _no_order : LDOp<"", size, RC>;
+}
+
+multiclass LDOpOrderSize {
+  let Sz = 0b00 in defm _B : LDOpOrder<"b", GPR32>;
+  let Sz = 0b01 in defm _H : LDOpOrder<"h", GPR32>;
+  let Sz = 0b10 in defm _S : LDOpOrder<"", GPR32>;
+  let Sz = 0b11 in defm _D : LDOpOrder<"", GPR64>;
+}
+
+defm LD : LDOpOrderSize;
+
+defm : BaseAliasLDOp<"add">;
+defm : BaseAliasLDOp<"clr">;
+defm : BaseAliasLDOp<"eor">;
+defm : BaseAliasLDOp<"set">;
+defm : BaseAliasLDOp<"smax">;
+defm : BaseAliasLDOp<"smin">;
+defm : BaseAliasLDOp<"umax">;
+defm : BaseAliasLDOp<"umin">;
+
+// v8.1 atomic CASP
+multiclass CASPOrder<string size, RegisterOperand RC> {
+  let A = 0b1, R = 0b0 in def _acquire    : BaseCASP<"a",size,RC>;
+  let A = 0b0, R = 0b1 in def _release    : BaseCASP<"l",size,RC>;
+  let A = 0b1, R = 0b1 in def _acquire_and_release   : BaseCASP<"al",size,RC>;
+  let A = 0b0, R = 0b0 in def _no_order : BaseCASP<"",size,RC>;
+}
+
+multiclass CASPOrderSize {
+  let Sz = 0b00 in defm _S : CASPOrder<"",ConsecutivePairClassOperand32>;
+  let Sz = 0b01 in defm _D : CASPOrder<"",ConsecutivePairClassOperand64>;
+}
+
+defm CASP : CASPOrderSize;
 
 //===----------------------------------------------------------------------===//
 // Logical instructions.
 //===----------------------------------------------------------------------===//
 
 // (immediate)
 defm ANDS : LogicalImmS<0b11, "ands", AArch64and_flag, "bics">;
 defm AND  : LogicalImm<0b00, "and", and, "bic">;

lib/Target/AArch64/AArch64RegisterInfo.td

 defm VecListFour  : VectorList<4, DDDD,  QQQQ>;
 
 
 // Register operand versions of the scalar FP registers.
 def FPR16Op : RegisterOperand<FPR16, "printOperand">;
 def FPR32Op : RegisterOperand<FPR32, "printOperand">;
 def FPR64Op : RegisterOperand<FPR64, "printOperand">;
 def FPR128Op : RegisterOperand<FPR128, "printOperand">;
+
+
+//===----------------------------------------------------------------------===//
+// ARMv8.1a atomic CASP register operands
+
+
+let Namespace = "AArch64" in {
+  def sube32 : SubRegIndex<32>;
+  def subo32 : SubRegIndex<32>;
+  def sube64 : SubRegIndex<64>;
+  def subo64 : SubRegIndex<64>;
+}
+
+def Pair32 : RegisterTuples<[sube32, subo32], [(rotl GPR32, 0), (rotl GPR32, 1)]>;
+def Pair64 : RegisterTuples<[sube64, subo64], [(rotl GPR64, 0), (rotl GPR64, 1)]>;
+
+def Pair32Class   : RegisterClass<"AArch64", [untyped], 32, (add Pair32)> {let Size = 64;}
+def Pair64Class   : RegisterClass<"AArch64", [untyped], 64, (add Pair64)> {let Size = 128;}
+
+
+let RenderMethod = "addRegOperands", ParserMethod="tryParsePair" in {
+  def Pair32AsmOperandClass : AsmOperandClass { let Name = "Pair32"; }
+  def Pair64AsmOperandClass : AsmOperandClass { let Name = "Pair64"; }
+}
+
+def ConsecutivePairClassOperand32 : RegisterOperand<Pair32Class,"printConsecutivePairClassOperand<32>"> {
+  let ParserMatchClass = Pair32AsmOperandClass;
+}
+def ConsecutivePairClassOperand64 : RegisterOperand<Pair64Class,"printConsecutivePairClassOperand<64>"> {
+  let ParserMatchClass = Pair64AsmOperandClass;
+}
+
+
+//===----- END: v8.1a atomic CASP register operands -----------------------===//

lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

#include "AArch64GenAsmMatcher.inc"
   OperandMatchResultTy tryParseSysCROperand(OperandVector &Operands);
   OperandMatchResultTy tryParsePrefetch(OperandVector &Operands);
   OperandMatchResultTy tryParseAdrpLabel(OperandVector &Operands);
   OperandMatchResultTy tryParseAdrLabel(OperandVector &Operands);
   OperandMatchResultTy tryParseFPImm(OperandVector &Operands);
   OperandMatchResultTy tryParseAddSubImm(OperandVector &Operands);
   OperandMatchResultTy tryParseGPR64sp0Operand(OperandVector &Operands);
   bool tryParseVectorRegister(OperandVector &Operands);
+  OperandMatchResultTy tryParsePair(OperandVector &Operands);
 
 public:
   enum AArch64MatchResultTy {
     Match_InvalidSuffix = FIRST_TARGET_MATCH_RESULT_TY,
 #define GET_OPERAND_DIAGNOSTIC_TYPES
 #include "AArch64GenAsmMatcher.inc"
   };
   AArch64AsmParser(MCSubtargetInfo &STI, MCAsmParser &Parser,
bool isVectorRegLo() const {
     return Kind == k_Register && Reg.isVector &&
            AArch64MCRegisterClasses[AArch64::FPR128_loRegClassID].contains(
                Reg.RegNum);
   }
   bool isGPR32as64() const {
     return Kind == k_Register && !Reg.isVector &&
       AArch64MCRegisterClasses[AArch64::GPR64RegClassID].contains(Reg.RegNum);
   }
-
+  bool isPair32() const {
+    return Kind == k_Register && !Reg.isVector &&
+      AArch64MCRegisterClasses[AArch64::Pair32ClassRegClassID].contains(Reg.RegNum);
+  }
+  bool isPair64() const {
+    return Kind == k_Register && !Reg.isVector &&
+      AArch64MCRegisterClasses[AArch64::Pair64ClassRegClassID].contains(Reg.RegNum);
+  }
   bool isGPR64sp0() const {
     return Kind == k_Register && !Reg.isVector &&
       AArch64MCRegisterClasses[AArch64::GPR64spRegClassID].contains(Reg.RegNum);
   }
 
   /// Is this a vector list with the type implicit (presumably attached to the
   /// instruction itself)?
   template <unsigned NumRegs> bool isImplicitlyTypedVectorList() const {
if (!Op.isImm())
     return Match_InvalidOperand;
   const MCConstantExpr *CE = dyn_cast<MCConstantExpr>(Op.getImm());
   if (!CE)
     return Match_InvalidOperand;
   if (CE->getValue() == ExpectedVal)
     return Match_Success;
   return Match_InvalidOperand;
 }
+
+
+AArch64AsmParser::OperandMatchResultTy
+AArch64AsmParser::tryParsePair(OperandVector &Operands) {
+
+  SMLoc S = getLoc();
+
+  if (getParser().getTok().isNot(AsmToken::Identifier)) {
+    Error(S, "Expected register");
+    return MatchOperand_ParseFail;
+  }
+
+  int FirstReg = tryParseRegister();
+  if (FirstReg ==-1) {
+    return MatchOperand_ParseFail;
+  }
+
+  const MCRegisterClass &WRegClass =
+      AArch64MCRegisterClasses[AArch64::GPR32RegClassID];
+  const MCRegisterClass &XRegClass =
+      AArch64MCRegisterClasses[AArch64::GPR64RegClassID];
+
+  bool isXReg = XRegClass.contains(FirstReg),
+       isWReg = WRegClass.contains(FirstReg);
+  if (!isXReg && !isWReg) {
+    Error(S, "Expected register");
+    return MatchOperand_ParseFail;
+  }
+
+  const MCRegisterInfo *RI = getContext().getRegisterInfo();
+  unsigned FirstEncoding = RI->getEncodingValue(FirstReg);
+
+  if (FirstEncoding & 0x1) {
+    Error(S, "Expected even register");
+    return MatchOperand_ParseFail;
+  }
+
+  SMLoc M = getLoc();
+  if (getParser().getTok().isNot(AsmToken::Comma)) {
+    Error(M, "Expected Comma");
+    return MatchOperand_ParseFail;
+  }
+  // Eat the comma
+  getParser().Lex();
+
+  SMLoc E = getLoc();
+  int SecondReg = tryParseRegister();
+  if (SecondReg ==-1) {
+    return MatchOperand_ParseFail;
+  }
+
+  if (RI->getEncodingValue(SecondReg) != FirstEncoding + 1 ||
+      (isXReg && !XRegClass.contains(SecondReg)) ||
+      (isWReg && !WRegClass.contains(SecondReg))) {
+    Error(E,"Expected consecutive registers");
+    return MatchOperand_ParseFail;
+  }
+
+  unsigned Pair = 0;
+  if (isXReg) {
+    Pair = AArch64MCRegisterClasses[AArch64::Pair64ClassRegClassID].getRegister(
+        FirstEncoding);
+  } else {
+    Pair = AArch64MCRegisterClasses[AArch64::Pair32ClassRegClassID].getRegister(
+        FirstEncoding);
+  }
+
+  Operands.push_back(AArch64Operand::CreateReg(Pair, false, S, getLoc(),
+      getContext()));
+
+  return MatchOperand_Success;
+}

lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp

 static DecodeStatus DecodeVecShiftL64Imm(llvm::MCInst &Inst, unsigned Imm,
                                          uint64_t Addr, const void *Decoder);
 static DecodeStatus DecodeVecShiftL32Imm(llvm::MCInst &Inst, unsigned Imm,
                                          uint64_t Addr, const void *Decoder);
 static DecodeStatus DecodeVecShiftL16Imm(llvm::MCInst &Inst, unsigned Imm,
                                          uint64_t Addr, const void *Decoder);
 static DecodeStatus DecodeVecShiftL8Imm(llvm::MCInst &Inst, unsigned Imm,
                                         uint64_t Addr, const void *Decoder);
+static DecodeStatus DecodePair32ClassRegisterClass(MCInst &Inst, unsigned RegNo,
+                                        uint64_t Addr, const void *Decoder);
+static DecodeStatus DecodePair64ClassRegisterClass(MCInst &Inst, unsigned RegNo,
+                                        uint64_t Addr, const void *Decoder);
 
 static bool Check(DecodeStatus &Out, DecodeStatus In) {
   switch (In) {
     case MCDisassembler::Success:
       // Out stays the same.
       return true;
     case MCDisassembler::SoftFail:
       Out = In;
static DecodeStatus DecodeTestAndBranch(llvm::MCInst &Inst, uint32_t insn,
   else
     DecodeGPR64RegisterClass(Inst, Rt, Addr, Decoder);
   Inst.addOperand(MCOperand::CreateImm(bit));
   if (!Dis->tryAddingSymbolicOperand(Inst, dst * 4, Addr, true, 0, 4))
     Inst.addOperand(MCOperand::CreateImm(dst));
 
   return Success;
 }
+
+static DecodeStatus DecodePair32ClassRegisterClass(MCInst &Inst, unsigned RegNo,
+                                             uint64_t Addr,
+                                             const void *Decoder) {
+  // Register number must be even (see CASP instruction)
+  if (RegNo & 0x1)
+    return Fail;
+
+  unsigned Register = AArch64MCRegisterClasses[AArch64::Pair32ClassRegClassID].
+      getRegister(RegNo);
+  Inst.addOperand(MCOperand::CreateReg(Register));
+  return Success;
+}
+
+static DecodeStatus DecodePair64ClassRegisterClass(MCInst &Inst, unsigned RegNo,
+                                             uint64_t Addr,
+                                             const void *Decoder) {
+  // Register number must be even (see CASP instruction)
+  if (RegNo & 0x1)
+    return Fail;
+
+  unsigned Register = AArch64MCRegisterClasses[AArch64::Pair64ClassRegClassID].
+      getRegister(RegNo);
+  Inst.addOperand(MCOperand::CreateReg(Register));
+  return Success;
+}

lib/Target/AArch64/Disassembler/LLVMBuild.txt

 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [component_0]
 type = Library
 name = AArch64Disassembler
 parent = AArch64
-required_libraries = AArch64Info AArch64Utils MC MCDisassembler Support
+required_libraries = AArch64Desc AArch64Info AArch64Utils MC MCDisassembler Support
 add_to_library_groups = AArch64

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.h

protected:
 
   void printVectorIndex(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printAdrpLabel(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printBarrierOption(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printMSRSystemRegister(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printMRSSystemRegister(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printSystemPStateField(const MCInst *MI, unsigned OpNum, raw_ostream &O);
   void printSIMDType10Operand(const MCInst *MI, unsigned OpNum, raw_ostream &O);
+  template<unsigned size>
+  void printConsecutivePairClassOperand(const MCInst *MI, unsigned OpNum, raw_ostream &O);
 };
 
 class AArch64AppleInstPrinter : public AArch64InstPrinter {
 public:
   AArch64AppleInstPrinter(const MCAsmInfo &MAI, const MCInstrInfo &MII,
                         const MCRegisterInfo &MRI, const MCSubtargetInfo &STI);
 
   void printInst(const MCInst *MI, raw_ostream &O, StringRef Annot) override;

lib/Target/AArch64/InstPrinter/AArch64InstPrinter.cpp

while (Stride--) {
     case AArch64::Q31:
       Reg = AArch64::Q0;
       break;
     }
   }
   return Reg;
 }
 
+template<unsigned size>
+void AArch64InstPrinter::printConsecutivePairClassOperand(const MCInst *MI, unsigned OpNum, raw_ostream &O) {
+  static_assert(size == 64 || size == 32,"Template parameter must be either 32 or 64");
+  unsigned Reg = MI->getOperand(OpNum).getReg();
+
+  unsigned sube = (size == 32) ? AArch64::sube32 : AArch64::sube64;
+  unsigned subo = (size == 32) ? AArch64::subo32 : AArch64::subo64;
+
+  unsigned even = MRI.getSubReg(Reg,  sube);
+  unsigned odd = MRI.getSubReg(Reg,  subo);
+  O << getRegisterName(even) << ", " << getRegisterName(odd);
+}
+
 void AArch64InstPrinter::printVectorList(const MCInst *MI, unsigned OpNum,
                                          raw_ostream &O,
                                          StringRef LayoutSuffix) {
   unsigned Reg = MI->getOperand(OpNum).getReg();
 
   O << "{ ";
 
   // Work out how many registers there are in the list (if there is an actual

test/MC/AArch64/armv8-extension-atomic.s

+// RUN: not llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.1a -show-encoding < %s 2> %t | FileCheck %s
+// RUN: FileCheck --check-prefix=CHECK-ERROR <%t %s
+  .text
+
+  //8 bits
+  casb   w0, w1, [x2]
+  casab  w0, w1, [x2]
+  caslb  w0, w1, [x2]
+  casalb   w0, w1, [x2]
+
+//CHECK:  casb   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x08]
+//CHECK:  casab  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x08]
+//CHECK:  caslb   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x08]
+//CHECK:  casalb   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x08]
+
+  casb w0, w1, [w2]
+  casalb x0, x1, [x2]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casb w0, w1, [w2]
+//CHECK-ERROR:                 ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casalb x0, x1, [x2]
+//CHECK-ERROR:          ^
+
+  //16 bits
+  cash   w0, w1, [x2]
+  casah  w0, w1, [x2]
+  caslh  w0, w1, [x2]
+  casalh   w0, w1, [x2]
+
+//CHECK:  cash   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x48]
+//CHECK:  casah  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x48]
+//CHECK:  caslh   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x48]
+//CHECK:  casalh   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x48]
+
+  //32 bits
+  cas   w0, w1, [x2]
+  casa  w0, w1, [x2]
+  casl  w0, w1, [x2]
+  casal   w0, w1, [x2]
+
+//CHECK:  cas   w0, w1, [x2]        //      encoding: [0x41,0x7c,0xa0,0x88]
+//CHECK:  casa  w0, w1, [x2]        //      encoding: [0x41,0x7c,0xe0,0x88]
+//CHECK:  casl   w0, w1, [x2]       //      encoding: [0x41,0xfc,0xa0,0x88]
+//CHECK:  casal   w0, w1, [x2]      //      encoding: [0x41,0xfc,0xe0,0x88]
+
+  cas   w0, w1, [w2]
+  casl  w0, x1, [x2]
+
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   cas   w0, w1, [w2]
+//CHECK-ERROR:                  ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casl  w0, x1, [x2]
+//CHECK-ERROR:             ^
+
+  //64 bits
+  cas   x0, x1, [x2]
+  casa  x0, x1, [x2]
+  casl   x0, x1, [x2]
+  casal   x0, x1, [x2]
+
+//CHECK:  cas   x0, x1, [x2]        //      encoding: [0x41,0x7c,0xa0,0xc8]
+//CHECK:  casa  x0, x1, [x2]        //      encoding: [0x41,0x7c,0xe0,0xc8]
+//CHECK:  casl   x0, x1, [x2]       //      encoding: [0x41,0xfc,0xa0,0xc8]
+//CHECK:  casal   x0, x1, [x2]      //      encoding: [0x41,0xfc,0xe0,0xc8]
+
+  casa   x0, x1, [w2]
+  casal  x0, w1, [x2]
+
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casa   x0, x1, [w2]
+//CHECK-ERROR:                   ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   casal  x0, w1, [x2]
+//CHECK-ERROR:              ^
+
+  // LD<OP> intructions
+  ldadda x0, x1, [x2]
+  ldclrl x0, x1, [x2]
+  ldeoral x0, x1, [x2]
+  ldset x0, x1, [x2]
+  ldsmaxa w0, w1, [x2]
+  ldsminlb w0, w1, [x2]
+  ldumaxalh w0, w1, [x2]
+  ldumin w0, w1, [x2]
+//CHECK: ldadda     x0, x1, [x2]  // encoding: [0x41,0x00,0xa0,0xf8]
+//CHECK: ldclrl     x0, x1, [x2]  // encoding: [0x41,0x10,0x60,0xf8]
+//CHECK: ldeoral    x0, x1, [x2]  // encoding: [0x41,0x20,0xe0,0xf8]
+//CHECK: ldset      x0, x1, [x2]  // encoding: [0x41,0x30,0x20,0xf8]
+//CHECK: ldsmaxa    w0, w1, [x2]  // encoding: [0x41,0x40,0xa0,0xb8]
+//CHECK: ldsminlb   w0, w1, [x2]  // encoding: [0x41,0x50,0x60,0x38]
+//CHECK: ldumaxalh  w0, w1, [x2]  // encoding: [0x41,0x60,0xe0,0x78]
+//CHECK: ldumin     w0, w1, [x2]  // encoding: [0x41,0x70,0x20,0xb8]
+
+  // ST<OP> intructions: aliases to LD<OP>
+  staddlb w0, [x2]
+  stclrlh w0, [x2]
+  steorl  w0, [x2]
+  stsetl  x0, [x2]
+  stsmaxb  w0, [x2]
+  stsminh  w0, [x2]
+  stumax   w0, [x2]
+  stumin   x0, [x2]
+//CHECK: staddlb    w0, [x2]  // encoding: [0x5f,0x00,0x60,0x38]
+//CHECK: stclrlh    w0, [x2]  // encoding: [0x5f,0x10,0x60,0x78]
+//CHECK: steorl     w0, [x2]  // encoding: [0x5f,0x20,0x60,0xb8]
+//CHECK: stsetl     x0, [x2]  // encoding: [0x5f,0x30,0x60,0xf8]
+//CHECK: stsmaxb     w0, [x2]  // encoding: [0x5f,0x40,0x20,0x38]
+//CHECK: stsminh     w0, [x2]  // encoding: [0x5f,0x50,0x20,0x78]
+//CHECK: stumax      w0, [x2]  // encoding: [0x5f,0x60,0x20,0xb8]
+//CHECK: stumin      x0, [x2]  // encoding: [0x5f,0x70,0x20,0xf8]
+
+  ldsmax x0, x1, [w2]
+  ldeorl w0, w1, [w2]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   ldsmax x0, x1, [w2]
+//CHECK-ERROR:                   ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   ldeorl w0, w1, [w2]
+//CHECK-ERROR:                   ^
+
+  //SWP instruction
+  swp   x0, x1, [x2]
+  swpb  w0, w1, [x2]
+  swplh w0, w1, [x2]
+  swpal x0, x1, [sp]
+//CHECK: swp   x0, x1, [x2]       // encoding: [0x41,0x80,0x20,0xf8]
+//CHECK: swpb  w0, w1, [x2]       // encoding: [0x41,0x80,0x20,0x38]
+//CHECK: swplh w0, w1, [x2]       // encoding: [0x41,0x80,0x60,0x78]
+//CHECK: swpal x0, x1, [sp]       // encoding: [0xe1,0x83,0xe0,0xf8]
+
+  swp   x0, x1, [w2]
+  swp   x0, x1, [xzr]
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   swp   x0, x1, [w2]
+//CHECK-ERROR:                  ^
+//CHECK-ERROR: error: invalid operand for instruction
+//CHECK-ERROR:   swp   x0, x1, [xzr]
+//CHECK-ERROR:                  ^
+
+  //CASP instruction
+  casp x0, x1, x2, x3, [x4]
+  casp w0, w1, w2, w3, [x4]
+//CHECK: casp x0, x1, x2, x3, [x4]      // encoding: [0x82,0x7c,0x20,0x48]
+//CHECK: casp w0, w1, w2, w3, [x4]      // encoding: [0x82,0x7c,0x20,0x08]
+
+  casp x1, x2, x4, x5, [x6]
+  casp x0, x1, x3, x4, [x5]
+  casp x0, x2, x4, x5, [x6]
+  casp x0, x1, x2, x4, [x5]
+//CHECK-ERROR: error: Expected even register
+//CHECK-ERROR:  casp x1, x2, x4, x5, [x6]
+//CHECK-ERROR:       ^
+//CHECK-ERROR: error: Expected even register
+//CHECK-ERROR:  casp x0, x1, x3, x4, [x5]
+//CHECK-ERROR:               ^
+//CHECK-ERROR: error: Expected consecutive registers
+//CHECK-ERROR:  casp x0, x2, x4, x5, [x6]
+//CHECK-ERROR:           ^
+//CHECK-ERROR: error: Expected consecutive registers
+//CHECK-ERROR:  casp x0, x1, x2, x4, [x5]
+//CHECK-ERROR:                   ^

test/MC/Disassembler/AArch64/armv8-extension-atomic.txt

+# RUN: llvm-mc -triple aarch64-none-linux-gnu -mattr=+v8.1a --disassemble < %s | FileCheck %s
+
+0x41,0x7c,0xa0,0x08
+0x41,0x7c,0xe0,0x08
+0x41,0xfc,0xa0,0x08
+0x41,0xfc,0xe0,0x08
+0x41,0x7c,0xa0,0x48
+0x41,0x7c,0xe0,0x48
+0x41,0xfc,0xa0,0x48
+0x41,0xfc,0xe0,0x48
+# CHECK:  casb   w0, w1, [x2]
+# CHECK:  casab  w0, w1, [x2]
+# CHECK:  caslb   w0, w1, [x2]
+# CHECK:  casalb   w0, w1, [x2]
+# CHECK:  cash   w0, w1, [x2]
+# CHECK:  casah  w0, w1, [x2]
+# CHECK:  caslh   w0, w1, [x2]
+# CHECK:  casalh   w0, w1, [x2]
+
+0x41,0x7c,0xa0,0x88
+0x41,0x7c,0xe0,0x88
+0x41,0xfc,0xa0,0x88
+0x41,0xfc,0xe0,0x88
+0x41,0x7c,0xa0,0xc8
+0x41,0x7c,0xe0,0xc8
+0x41,0xfc,0xa0,0xc8
+0x41,0xfc,0xe0,0xc8
+# CHECK:  cas   w0, w1, [x2]
+# CHECK:  casa  w0, w1, [x2]
+# CHECK:  casl   w0, w1, [x2]
+# CHECK:  casal   w0, w1, [x2]
+# CHECK:  cas   x0, x1, [x2]
+# CHECK:  casa  x0, x1, [x2]
+# CHECK:  casl   x0, x1, [x2]
+# CHECK:  casal   x0, x1, [x2]
+
+0x41,0x80,0x20,0xf8
+0x41,0x80,0x20,0x38
+0x41,0x80,0x60,0x78
+0xe1,0x83,0xe0,0xf8
+# CHECK: swp   x0, x1, [x2]
+# CHECK: swpb  w0, w1, [x2]
+# CHECK: swplh w0, w1, [x2]
+# CHECK: swpal x0, x1, [sp]
+
+0x41,0x00,0xa0,0xf8
+0x41,0x10,0x60,0xf8
+0x41,0x20,0xe0,0xf8
+0x41,0x30,0x20,0xf8
+0x41,0x40,0xa0,0xb8
+0x41,0x50,0x60,0x38
+0x41,0x60,0xe0,0x78
+0x41,0x70,0x20,0xb8
+# CHECK:  ldadda    x0, x1, [x2]
+# CHECK:  ldclrl    x0, x1, [x2]
+# CHECK:  ldeoral   x0, x1, [x2]
+# CHECK:  ldset     x0, x1, [x2]
+# CHECK:  ldsmaxa   w0, w1, [x2]
+# CHECK:  ldsminlb  w0, w1, [x2]
+# CHECK:  ldumaxalh w0, w1, [x2]
+# CHECK:  ldumin    w0, w1, [x2]
+
+0x5f,0x00,0x60,0x38
+0x5f,0x10,0x60,0x78
+0x5f,0x20,0x60,0xb8
+0x5f,0x30,0x60,0xf8
+0x5f,0x40,0x20,0x38
+0x5f,0x50,0x20,0x78
+0x5f,0x60,0x20,0xb8
+0x5f,0x70,0x20,0xf8
+# CHECK: staddlb    w0, [x2]
+# CHECK: stclrlh    w0, [x2]
+# CHECK: steorl     w0, [x2]
+# CHECK: stsetl     x0, [x2]
+# CHECK: stsmaxb     w0, [x2]
+# CHECK: stsminh     w0, [x2]
+# CHECK: stumax      w0, [x2]
+# CHECK: stumin      x0, [x2]
+
+0x82,0x7c,0x20,0x48
+0x82,0x7c,0x20,0x08
+# CHECK: casp x0, x1, x2, x3, [x4]
+# CHECK: casp w0, w1, w2, w3, [x4]