[SystemZ] Support Compare and Traps

[• zhanjunl]

Summary

Support and generate Compare and Traps like CRT, CIT, etc.

Support Trap as legal DAG opcodes and generate "j .+2" for them by default.
Add support for Conditional Traps and use the If Converter to convert them into the
corresponding compare and trap opcodes.

[• koriakin]

In addition to inline comments: there is no test for unconditional trap, or for non-fused conditional trap (you can get that one easily by a floating point comparison).

lib/Target/SystemZ/SystemZInstrFormats.td

223

nit: would be good to have them in order: InstRIEa, InstRIEb, InstRIEc.

lib/Target/SystemZ/SystemZInstrInfo.td

71

Alias<4 would be slightly more correct, though I don't think anything actually uses size information for aliases

74

This is not a barrier, and it does use custom inserter.

170

Not barriers.

293

Again, not barriers.

[• koriakin]

Oh, and remove the TODO item from lib/Target/SystemZ/README while you're at it :)

lib/Target/SystemZ/SystemZAsmPrinter.cpp

435

Nit: "jump to that label" would mean an infinite loop, say "use that label" or "jump to that label + 2" instead.

[• koriakin]

Oh, and there should be MC tests for the fused compares (an assembler test and a disassembler test).

It's also worth noting that there are also two more fused ops that could be supported here: CLT and CLGT (they don't have compare-and-branch variants), though they are a bit of a mess (they have different memory operands wrt their compare equivalents), so might be out of scope for this patch.

test/CodeGen/SystemZ/trap-01.ll

26

I'm not sure G stands for "grande", but I like it either way :) Got a good explanation for F?

[uweigand]

Thanks to koriakin for the review, I agree with the points raised there.

One additional question: why is the splitBlockAfter in emitTrap needed? I notice that other platforms don't seem to require similar code when emitting a trap. Shouldn't marking the trap as isTerminator in the .td file cause common code to terminate the basic block already?

Also, LLVM coding style limits line length to 80 characters, which your patch currently exceed in some places.

[• zhanjunl]

Thanks Marcin and Ulrich for the quick reviews. I've addressed most of them, and fixed the line width to 80 characters.

Ordered the InstRIEa instruction class properly now.
Set the alias size correctly.
Removed isBarrier from CondTrap and the Compare and Trap instruction sets.
Fixed the comment to "use that label instead".
Removed the TODO from the README file.
Added MC assembler and disassembler tests, and added tests for unconditional traps and invalid Compare and Trap, to test CondTrap.

In D21155#453347, @uweigand wrote:

One additional question: why is the splitBlockAfter in emitTrap needed? I notice that other platforms don't seem to require similar code when emitting a trap. Shouldn't marking the trap as isTerminator in the .td file cause common code to terminate the basic block already?

This part of the patch I was unsure of, because I'm not sure if I'm using the custom inserter properly. When compiling this LLVM IR:

define signext i32 @f0() {
entry:
  call void @llvm.trap()
  ret i32 0
}

Other platforms like PowerPC can compile that code fine, essentially generating "TRAP, LOAD 0, RETURN". SystemZ will throw an assertion, because there is a non-terminator between terminators (LOAD 0). I think the assertion makes sense, but I couldn't find any common code that terminated a basic block after seeing a terminator, which is why I had to use the custom inserter.

[• zhanjunl]

lib/Target/SystemZ/SystemZInstrInfo.td

74

I'm not sure why this should be marked as usesCustomInserter? The Trap instruction needed it because I needed to split the block, but CondTrap doesn't really need it since it should only have been inserted at the end of a basic block.

test/CodeGen/SystemZ/trap-01.ll

26

I have no explanation for F :(

[• koriakin]

lib/Target/SystemZ/SystemZInstrInfo.td

74

Whoops, that's right, my bad.

[uweigand]

In D21155#453635, @zhanjunl wrote:

In D21155#453347, @uweigand wrote:

One additional question: why is the splitBlockAfter in emitTrap needed? I notice that other platforms don't seem to require similar code when emitting a trap. Shouldn't marking the trap as isTerminator in the .td file cause common code to terminate the basic block already?

This part of the patch I was unsure of, because I'm not sure if I'm using the custom inserter properly. When compiling this LLVM IR:

define signext i32 @f0() {
entry:
  call void @llvm.trap()
  ret i32 0
}

Other platforms like PowerPC can compile that code fine, essentially generating "TRAP, LOAD 0, RETURN". SystemZ will throw an assertion, because there is a non-terminator between terminators (LOAD 0). I think the assertion makes sense, but I couldn't find any common code that terminated a basic block after seeing a terminator, which is why I had to use the custom inserter.

Hmm, it seems this in indeed incorrect on all platforms, it's just that the bug is silent by default except on SystemZ, where we run into an assertion in /SystemZLongBranch.cpp. However, you can see the bug on all platforms (I've tested powerpc64le and amd64) by using llc -verify-machineinstrs. This will report:

• Bad machine code: Non-terminator instruction after the first terminator ***

after the Instruction Selection pass.

Note that with -verify-machineinstrs, even in the presence of the emitTrap custom inserter on SystemZ, llc -verify-machineinstrs will *still* report the error, since the invalid machine code is still present initially after the instruction selection pass (even though it is cleaned up later by the custom inserter).

I think the correct place to fix this would be in common code, either during isel, or even at the IR stage. (Why is there any code after a call to a noreturn builtin in the first place?) It might be best to start a discussion on this on the mailing list.

In the meantime, maybe the preferable workaround would be to not mark the trap instruction as isTerminator?

[• zhanjunl]

I've unmarked isTerminator from the Trap instruction for now, and removed all custom insertion code. I'll bring it up in the mailing list and see if I can't find out where it should be fixed in the common code.

[uweigand]

LGTM, thanks!

lib/Target/SystemZ/README.txt

 
 --
 
 We might want to model all access registers and use them to spill
 32-bit values.
 
 --
 
-We might want to use 'j .+2' as a trap instruction, like gcc does.  It can
-also be made conditional like the return instruction, allowing us to utilize
-compare-and-trap and load-and-trap instructions.
-
---
-
 We might want to use the 'overflow' condition of eg. AR to support
 llvm.sadd.with.overflow.i32 and related instructions - the generated code
 for signed overflow check is currently quite bad.  This would improve
 the results of using -ftrapv.

lib/Target/SystemZ/SystemZAsmPrinter.cpp

else
         .addImm(15).addReg(SystemZ::R0D);
     break;
 
   // Emit nothing here but a comment if we can.
   case SystemZ::MemBarrier:
     OutStreamer->emitRawComment("MEMBARRIER");
     return;
 
+  // We want to emit "j .+2" for traps, jumping to the relative immediate field
+  // of the jump instruction, which is an illegal instruction. We cannot emit a
+  // "." symbol, so create and emit a temp label before the instruction and use
+  // that instead.
+  case SystemZ::Trap: {
+    MCSymbol *DotSym = OutContext.createTempSymbol();
+    OutStreamer->EmitLabel(DotSym);
+
+    const MCSymbolRefExpr *Expr = MCSymbolRefExpr::create(DotSym, OutContext);
+    const MCConstantExpr *ConstExpr = MCConstantExpr::create(2, OutContext);
+    LoweredMI = MCInstBuilder(SystemZ::J)
+      .addExpr(MCBinaryExpr::createAdd(Expr, ConstExpr, OutContext));
+    }
+    break;
+
+  // Conditional traps will create a branch on condition instruction that jumps
+  // to the relative immediate field of the jump instruction. (eg. "jo .+2")
+  case SystemZ::CondTrap: {
+    MCSymbol *DotSym = OutContext.createTempSymbol();
+    OutStreamer->EmitLabel(DotSym);
+
+    const MCSymbolRefExpr *Expr = MCSymbolRefExpr::create(DotSym, OutContext);
+    const MCConstantExpr *ConstExpr = MCConstantExpr::create(2, OutContext);
+    LoweredMI = MCInstBuilder(SystemZ::BRC)
+      .addImm(MI->getOperand(0).getImm())
+      .addImm(MI->getOperand(1).getImm())
+      .addExpr(MCBinaryExpr::createAdd(Expr, ConstExpr, OutContext));
+    }
+    break;
+
   default:
     Lower.lower(MI, LoweredMI);
     break;
   }
   EmitToStreamer(*OutStreamer, LoweredMI);
 }
 
 // Convert a SystemZ-specific constant pool modifier into the associated

lib/Target/SystemZ/SystemZElimCompare.cpp

private:
   Reference getRegReferences(MachineInstr *MI, unsigned Reg);
   bool convertToBRCT(MachineInstr *MI, MachineInstr *Compare,
                      SmallVectorImpl<MachineInstr *> &CCUsers);
   bool convertToLoadAndTest(MachineInstr *MI);
   bool adjustCCMasksForInstr(MachineInstr *MI, MachineInstr *Compare,
                              SmallVectorImpl<MachineInstr *> &CCUsers);
   bool optimizeCompareZero(MachineInstr *Compare,
                            SmallVectorImpl<MachineInstr *> &CCUsers);
-  bool fuseCompareAndBranch(MachineInstr *Compare,
+  bool fuseCompareOperations(MachineInstr *Compare,
                              SmallVectorImpl<MachineInstr *> &CCUsers);
 
   const SystemZInstrInfo *TII;
   const TargetRegisterInfo *TRI;
 };
 
 char SystemZElimCompare::ID = 0;
 } // end anonymous namespace
 
if (CCRefs.Use && CCRefs.Def)
       return false;
   }
   return false;
 }
 
 // Try to fuse comparison instruction Compare into a later branch.
 // Return true on success and if Compare is therefore redundant.
 bool SystemZElimCompare::
-fuseCompareAndBranch(MachineInstr *Compare,
+fuseCompareOperations(MachineInstr *Compare,
                       SmallVectorImpl<MachineInstr *> &CCUsers) {
   // See whether we have a single branch with which to fuse.
   if (CCUsers.size() != 1)
     return false;
   MachineInstr *Branch = CCUsers[0];
-  SystemZII::CompareAndBranchType Type;
+  SystemZII::FusedCompareType Type;
   switch (Branch->getOpcode()) {
   case SystemZ::BRC:
     Type = SystemZII::CompareAndBranch;
     break;
   case SystemZ::CondReturn:
     Type = SystemZII::CompareAndReturn;
     break;
   case SystemZ::CallBCR:
     Type = SystemZII::CompareAndSibcall;
     break;
+  case SystemZ::CondTrap:
+    Type = SystemZII::CompareAndTrap;
+    break;
   default:
     return false;
   }
 
   // See whether we have a comparison that can be fused.
-  unsigned FusedOpcode = TII->getCompareAndBranch(Compare->getOpcode(),
-                                                  Type, Compare);
+  unsigned FusedOpcode = TII->getFusedCompare(Compare->getOpcode(),
   if (!FusedOpcode)
     return false;
 
   // Make sure that the operands are available at the branch.
   unsigned SrcReg = Compare->getOperand(0).getReg();
   unsigned SrcReg2 = (Compare->getOperand(1).isReg() ?
                       Compare->getOperand(1).getReg() : 0);
   MachineBasicBlock::iterator MBBI = Compare, MBBE = Branch;
bool SystemZElimCompare::processBlock(MachineBasicBlock &MBB) {
   bool CompleteCCUsers = !isCCLiveOut(MBB);
   SmallVector<MachineInstr *, 4> CCUsers;
   MachineBasicBlock::iterator MBBI = MBB.end();
   while (MBBI != MBB.begin()) {
     MachineInstr *MI = --MBBI;
     if (CompleteCCUsers &&
         (MI->isCompare() || isLoadAndTestAsCmp(MI)) &&
         (optimizeCompareZero(MI, CCUsers) ||
-         fuseCompareAndBranch(MI, CCUsers))) {
+         fuseCompareOperations(MI, CCUsers))) {
       ++MBBI;
       MI->eraseFromParent();
       Changed = true;
       CCUsers.clear();
       continue;
     }
 
     if (MI->definesRegister(SystemZ::CC)) {

lib/Target/SystemZ/SystemZISelLowering.h

MachineBasicBlock *emitStringWrapper(MachineInstr *MI,
                                        unsigned Opcode) const;
   MachineBasicBlock *emitTransactionBegin(MachineInstr *MI,
                                           MachineBasicBlock *MBB,
                                           unsigned Opcode,
                                           bool NoFloat) const;
   MachineBasicBlock *emitLoadAndTestCmp0(MachineInstr *MI,
                                          MachineBasicBlock *MBB,
                                          unsigned Opcode) const;
+  MachineBasicBlock *emitTrap(MachineInstr *MI,
+                              MachineBasicBlock *MBB) const;
 
 };
 } // end namespace llvm
 
 #endif

lib/Target/SystemZ/SystemZISelLowering.cpp

SystemZTargetLowering::SystemZTargetLowering(const TargetMachine &TM,
   setOperationAction(ISD::ATOMIC_LOAD_MIN,  MVT::i32, Custom);
   setOperationAction(ISD::ATOMIC_LOAD_MAX,  MVT::i32, Custom);
   setOperationAction(ISD::ATOMIC_LOAD_UMIN, MVT::i32, Custom);
   setOperationAction(ISD::ATOMIC_LOAD_UMAX, MVT::i32, Custom);
   setOperationAction(ISD::ATOMIC_CMP_SWAP,  MVT::i32, Custom);
 
   setOperationAction(ISD::ATOMIC_FENCE, MVT::Other, Custom);
 
+  // Traps are legal, as we will convert them to "j .+2".
+  setOperationAction(ISD::TRAP, MVT::Other, Legal);
+
   // z10 has instructions for signed but not unsigned FP conversion.
   // Handle unsigned 32-bit types as signed 64-bit types.
   if (!Subtarget.hasFPExtension()) {
     setOperationAction(ISD::UINT_TO_FP, MVT::i32, Promote);
     setOperationAction(ISD::UINT_TO_FP, MVT::i64, Expand);
   }
 
   // We have native support for a 64-bit CTLZ, via FLOGR.
if (EndMBB) {
     MBB = EndMBB;
     MBB->addLiveIn(SystemZ::CC);
   }
 
   MI->eraseFromParent();
   return MBB;
 }
 
+MachineBasicBlock *
+SystemZTargetLowering::emitTrap(MachineInstr *MI,
+                                MachineBasicBlock *MBB) const {
+  // Trap instructions should be terminators for a basic block.
+  // Split the block in case the trap is not at the end of a basic block.
+  return splitBlockAfter(MI, MBB);
+}
+
 // Decompose string pseudo-instruction MI into a loop that continually performs
 // Opcode until CC != 3.
 MachineBasicBlock *
 SystemZTargetLowering::emitStringWrapper(MachineInstr *MI,
                                          MachineBasicBlock *MBB,
                                          unsigned Opcode) const {
   MachineFunction &MF = *MBB->getParent();
   const SystemZInstrInfo *TII =
case SystemZ::TBEGINC:
     return emitTransactionBegin(MI, MBB, SystemZ::TBEGINC, true);
   case SystemZ::LTEBRCompare_VecPseudo:
     return emitLoadAndTestCmp0(MI, MBB, SystemZ::LTEBR);
   case SystemZ::LTDBRCompare_VecPseudo:
     return emitLoadAndTestCmp0(MI, MBB, SystemZ::LTDBR);
   case SystemZ::LTXBRCompare_VecPseudo:
     return emitLoadAndTestCmp0(MI, MBB, SystemZ::LTXBR);
 
+  case SystemZ::Trap:
+    return emitTrap(MI, MBB);
+
   default:
     llvm_unreachable("Unexpected instr type to insert");
   }
 }

lib/Target/SystemZ/SystemZInstrFormats.td

class InstRI<bits<12> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   bits<16> I2;
 
   let Inst{31-24} = op{11-4};
   let Inst{23-20} = R1;
   let Inst{19-16} = op{3-0};
   let Inst{15-0}  = I2;
 }
 
+class InstRIEa<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
+  : InstSystemZ<6, outs, ins, asmstr, pattern> {
+  field bits<48> Inst;
+  field bits<48> SoftFail = 0;
+
+  bits<4> R1;
+  bits<16> I2;
+  bits<4> M3;
+
+  let Inst{47-40} = op{15-8};
+  let Inst{39-36} = R1;
+  let Inst{35-32} = 0;
+  let Inst{31-16} = I2;
+  let Inst{15-12} = M3;
+  let Inst{11-8}  = 0;
+  let Inst{7-0}   = op{7-0};
+}
+
 class InstRIEb<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   : InstSystemZ<6, outs, ins, asmstr, pattern> {
   field bits<48> Inst;
   field bits<48> SoftFail = 0;
 
   bits<4> R1;
   bits<4> R2;
   bits<4> M3;
   bits<16> RI4;
 
   let Inst{47-40} = op{15-8};
   let Inst{39-36} = R1;
   let Inst{35-32} = R2;
   let Inst{31-16} = RI4;
   let Inst{15-12} = M3;
   let Inst{11-8}  = 0;
   let Inst{7-0}   = op{7-0};
 }
 
 class InstRIEc<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   : InstSystemZ<6, outs, ins, asmstr, pattern> {
   field bits<48> Inst;
   field bits<48> SoftFail = 0;
 
   bits<4> R1;
   bits<8> I2;
   bits<4> M3;
   bits<16> RI4;
class InstRRF<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
 
   let Inst{31-16} = op;
   let Inst{15-12} = R3;
   let Inst{11-8}  = R4;
   let Inst{7-4}   = R1;
   let Inst{3-0}   = R2;
 }
 
+class InstRRFc<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
+  : InstSystemZ<4, outs, ins, asmstr, pattern> {
+  field bits<32> Inst;
+  field bits<32> SoftFail = 0;
+
+  bits<4> R1;
+  bits<4> R2;
+  bits<4> M3;
+
+  let Inst{31-16} = op;
+  let Inst{15-12} = M3;
+  let Inst{11-8}  = 0;
+  let Inst{7-4}   = R1;
+  let Inst{3-0}   = R2;
+}
+
 class InstRRS<bits<16> op, dag outs, dag ins, string asmstr, list<dag> pattern>
   : InstSystemZ<6, outs, ins, asmstr, pattern> {
   field bits<48> Inst;
   field bits<48> SoftFail = 0;
 
   bits<4> R1;
   bits<4> R2;
   bits<4> M3;

lib/Target/SystemZ/SystemZInstrInfo.h

struct Branch {
 
   // The target of the branch.
   const MachineOperand *Target;
 
   Branch(BranchType type, unsigned ccValid, unsigned ccMask,
          const MachineOperand *target)
     : Type(type), CCValid(ccValid), CCMask(ccMask), Target(target) {}
 };
-// Kinds of branch in compare-and-branch instructions.  Together with type
-// of the converted compare, this identifies the compare-and-branch
+// Kinds of fused compares in compare-and-* instructions.  Together with type
+// of the converted compare, this identifies the compare-and-*
 // instruction.
-enum CompareAndBranchType {
+enum FusedCompareType {
   // Relative branch - CRJ etc.
   CompareAndBranch,
 
   // Indirect branch, used for return - CRBReturn etc.
   CompareAndReturn,
 
   // Indirect branch, used for sibcall - CRBCall etc.
-  CompareAndSibcall
+  CompareAndSibcall,
+
+  // Trap
+  CompareAndTrap
 };
 } // end namespace SystemZII
 
 class SystemZSubtarget;
 class SystemZInstrInfo : public SystemZGenInstrInfo {
   const SystemZRegisterInfo RI;
   SystemZSubtarget &STI;
 
public:
   unsigned getLoadAndTest(unsigned Opcode) const;
 
   // Return true if ROTATE AND ... SELECTED BITS can be used to select bits
   // Mask of the R2 operand, given that only the low BitSize bits of Mask are
   // significant.  Set Start and End to the I3 and I4 operands if so.
   bool isRxSBGMask(uint64_t Mask, unsigned BitSize,
                    unsigned &Start, unsigned &End) const;
 
-  // If Opcode is a COMPARE opcode for which an associated COMPARE AND
-  // BRANCH exists, return the opcode for the latter, otherwise return 0.
+  // If Opcode is a COMPARE opcode for which an associated fused COMPARE AND *
+  // operation exists, return the opcode for the latter, otherwise return 0.
   // MI, if nonnull, is the compare instruction.
-  unsigned getCompareAndBranch(unsigned Opcode,
-                               SystemZII::CompareAndBranchType Type,
-                               const MachineInstr *MI = nullptr) const;
+  unsigned getFusedCompare(unsigned Opcode,
+                           SystemZII::FusedCompareType Type,
 
   // Emit code before MBBI in MI to move immediate value Value into
   // physical register Reg.
   void loadImmediate(MachineBasicBlock &MBB,
                      MachineBasicBlock::iterator MBBI,
                      unsigned Reg, uint64_t Value) const;
 };
 } // end namespace llvm
 
 #endif

lib/Target/SystemZ/SystemZInstrInfo.cpp

static unsigned getConditionalMove(unsigned Opcode) {
   }
 }
 
 bool SystemZInstrInfo::isPredicable(MachineInstr &MI) const {
   unsigned Opcode = MI.getOpcode();
   if (STI.hasLoadStoreOnCond() && getConditionalMove(Opcode))
     return true;
   if (Opcode == SystemZ::Return ||
+      Opcode == SystemZ::Trap ||
       Opcode == SystemZ::CallJG ||
       Opcode == SystemZ::CallBR)
     return true;
   return false;
 }
 
 bool SystemZInstrInfo::
 isProfitableToIfCvt(MachineBasicBlock &MBB,
                     unsigned NumCycles, unsigned ExtraPredCycles,
                     BranchProbability Probability) const {
   // Avoid using conditional returns at the end of a loop (since then
   // we'd need to emit an unconditional branch to the beginning anyway,
   // making the loop body longer).  This doesn't apply for low-probability
   // loops (eg. compare-and-swap retry), so just decide based on branch
   // probability instead of looping structure.
-  if (MBB.succ_empty() && Probability < BranchProbability(1, 8))
+  // However, since Compare and Trap instructions cost the same as a regular
+  // Compare instruction, we should allow the if conversion to convert this
+  // into a Conditional Compare regardless of the branch probability.
+  if (MBB.getFirstTerminator()->getOpcode() != SystemZ::Trap &&
+      MBB.succ_empty() && Probability < BranchProbability(1, 8))
     return false;
   // For now only convert single instructions.
   return NumCycles == 1;
 }
 
 bool SystemZInstrInfo::
 isProfitableToIfCvt(MachineBasicBlock &TMBB,
                     unsigned NumCyclesT, unsigned ExtraPredCyclesT,
if (unsigned CondOpcode = getConditionalMove(Opcode)) {
       MI.setDesc(get(CondOpcode));
       MachineInstrBuilder(*MI.getParent()->getParent(), MI)
           .addImm(CCValid)
           .addImm(CCMask)
           .addReg(SystemZ::CC, RegState::Implicit);
       return true;
     }
   }
+  if (Opcode == SystemZ::Trap) {
+    MI.setDesc(get(SystemZ::CondTrap));
+    MachineInstrBuilder(*MI.getParent()->getParent(), MI)
+      .addImm(CCValid).addImm(CCMask)
+      .addReg(SystemZ::CC, RegState::Implicit);
+    return true;
+  }
   if (Opcode == SystemZ::Return) {
     MI.setDesc(get(SystemZ::CondReturn));
     MachineInstrBuilder(*MI.getParent()->getParent(), MI)
       .addImm(CCValid).addImm(CCMask)
       .addReg(SystemZ::CC, RegState::Implicit);
     return true;
   }
   if (Opcode == SystemZ::CallJG) {
if (isStringOfOnes(Mask ^ allOnes(BitSize), LSB, Length)) {
     Start = 63 - (LSB - 1);
     End = 63 - (LSB + Length);
     return true;
   }
 
   return false;
 }
 
-unsigned SystemZInstrInfo::getCompareAndBranch(unsigned Opcode,
-                                               SystemZII::CompareAndBranchType Type,
-                                               const MachineInstr *MI) const {
+unsigned SystemZInstrInfo::getFusedCompare(unsigned Opcode,
+                                           SystemZII::FusedCompareType Type,
   switch (Opcode) {
   case SystemZ::CHI:
   case SystemZ::CGHI:
     if (!(MI && isInt<8>(MI->getOperand(1).getImm())))
       return 0;
     break;
   case SystemZ::CLFI:
   case SystemZ::CLGFI:
case SystemZ::CLGR:
       return SystemZ::CLGRBCall;
     case SystemZ::CLFI:
       return SystemZ::CLIBCall;
     case SystemZ::CLGFI:
       return SystemZ::CLGIBCall;
     default:
       return 0;
     }
+  case SystemZII::CompareAndTrap:
+    switch (Opcode) {
+    case SystemZ::CR:
+      return SystemZ::CRT;
+    case SystemZ::CGR:
+      return SystemZ::CGRT;
+    case SystemZ::CHI:
+      return SystemZ::CIT;
+    case SystemZ::CGHI:
+      return SystemZ::CGIT;
+    case SystemZ::CLR:
+      return SystemZ::CLRT;
+    case SystemZ::CLGR:
+      return SystemZ::CLGRT;
+    case SystemZ::CLFI:
+      return SystemZ::CLFIT;
+    case SystemZ::CLGFI:
+      return SystemZ::CLGIT;
+    default:
+      return 0;
+    }
   }
   return 0;
 }
 
 void SystemZInstrInfo::loadImmediate(MachineBasicBlock &MBB,
                                      MachineBasicBlock::iterator MBBI,
                                      unsigned Reg, uint64_t Value) const {
   DebugLoc DL = MBBI != MBB.end() ? MBBI->getDebugLoc() : DebugLoc();

lib/Target/SystemZ/SystemZInstrInfo.td

let isBranch = 1, isTerminator = 1, isBarrier = 1, R1 = 15 in {
   def J : InstRI<0xA74, (outs), (ins brtarget16:$I2), "j\t$I2",
                  [(br bb:$I2)]>;
 
   // An assembler extended mnemonic for BRCL.  (The extension is "G"
   // rather than "L" because "JL" is "Jump if Less".)
   def JG : InstRIL<0xC04, (outs), (ins brtarget32:$I2), "jg\t$I2", []>;
 }
 
+let isTerminator = 1, isBarrier = 1, hasCtrlDep = 1, usesCustomInserter = 1 in {
+  def Trap : Alias<4, (outs), (ins), [(trap)]>;
+}
+
+let isTerminator = 1, hasCtrlDep = 1, Uses = [CC] in {
+  def CondTrap : Alias<4, (outs), (ins cond4:$valid, cond4:$R1), []>;
+}
+
 // Conditional branches.  It's easier for LLVM to handle these branches
 // in their raw BRC/BRCL form, with the 4-bit condition-code mask being
 // the first operand.  It seems friendlier to use mnemonic forms like
 // JE and JLH when writing out the assembly though.
 let isBranch = 1, isTerminator = 1, Uses = [CC] in {
   let isCodeGenOnly = 1, CCMaskFirst = 1 in {
     def BRC : InstRI<0xA74, (outs), (ins cond4:$valid, cond4:$R1,
                                          brtarget16:$I2), "j$R1\t$I2",

 // These instructions do not use or clobber the condition codes.
 // We nevertheless pretend that they clobber CC, so that we can lower
 // them to separate comparisons and BRCLs if the branch ends up being
 // out of range.
 multiclass CompareBranches<Operand ccmask, string pos1, string pos2> {
   let isBranch = 1, isTerminator = 1, Defs = [CC] in {
     def RJ  : InstRIEb<0xEC76, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3,
                                             brtarget16:$RI4),
-                       "crj"##pos1##"\t$R1, $R2, "##pos2##"$RI4", []>;
+                       "crj"##pos1##"\t$R1, $R2"##pos2##", $RI4", []>;
     def GRJ : InstRIEb<0xEC64, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3,
                                             brtarget16:$RI4),
-                       "cgrj"##pos1##"\t$R1, $R2, "##pos2##"$RI4", []>;
+                       "cgrj"##pos1##"\t$R1, $R2"##pos2##", $RI4", []>;
     def IJ  : InstRIEc<0xEC7E, (outs), (ins GR32:$R1, imm32sx8:$I2, ccmask:$M3,
                                             brtarget16:$RI4),
-                       "cij"##pos1##"\t$R1, $I2, "##pos2##"$RI4", []>;
+                       "cij"##pos1##"\t$R1, $I2"##pos2##", $RI4", []>;
     def GIJ : InstRIEc<0xEC7C, (outs), (ins GR64:$R1, imm64sx8:$I2, ccmask:$M3,
                                             brtarget16:$RI4),
-                       "cgij"##pos1##"\t$R1, $I2, "##pos2##"$RI4", []>;
+                       "cgij"##pos1##"\t$R1, $I2"##pos2##", $RI4", []>;
     def LRJ  : InstRIEb<0xEC77, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3,
                                              brtarget16:$RI4),
-                        "clrj"##pos1##"\t$R1, $R2, "##pos2##"$RI4", []>;
+                        "clrj"##pos1##"\t$R1, $R2"##pos2##", $RI4", []>;
     def LGRJ : InstRIEb<0xEC65, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3,
                                              brtarget16:$RI4),
-                        "clgrj"##pos1##"\t$R1, $R2, "##pos2##"$RI4", []>;
+                        "clgrj"##pos1##"\t$R1, $R2"##pos2##", $RI4", []>;
     def LIJ  : InstRIEc<0xEC7F, (outs), (ins GR32:$R1, imm32zx8:$I2, ccmask:$M3,
                                              brtarget16:$RI4),
-                        "clij"##pos1##"\t$R1, $I2, "##pos2##"$RI4", []>;
+                        "clij"##pos1##"\t$R1, $I2"##pos2##", $RI4", []>;
     def LGIJ : InstRIEc<0xEC7D, (outs), (ins GR64:$R1, imm64zx8:$I2, ccmask:$M3,
                                              brtarget16:$RI4),
-                        "clgij"##pos1##"\t$R1, $I2, "##pos2##"$RI4", []>;
+                        "clgij"##pos1##"\t$R1, $I2"##pos2##", $RI4", []>;
     let isIndirectBranch = 1 in {
       def RB  : InstRRS<0xECF6, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3,
                                              bdaddr12only:$BD4),
-                        "crb"##pos1##"\t$R1, $R2, "##pos2##"$BD4", []>;
+                        "crb"##pos1##"\t$R1, $R2"##pos2##", $BD4", []>;
       def GRB : InstRRS<0xECE4, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3,
                                              bdaddr12only:$BD4),
-                        "cgrb"##pos1##"\t$R1, $R2, "##pos2##"$BD4", []>;
+                        "cgrb"##pos1##"\t$R1, $R2"##pos2##", $BD4", []>;
       def IB  : InstRIS<0xECFE, (outs), (ins GR32:$R1, imm32sx8:$I2, ccmask:$M3,
                                              bdaddr12only:$BD4),
-                        "cib"##pos1##"\t$R1, $I2, "##pos2##"$BD4", []>;
+                        "cib"##pos1##"\t$R1, $I2"##pos2##", $BD4", []>;
       def GIB : InstRIS<0xECFC, (outs), (ins GR64:$R1, imm64sx8:$I2, ccmask:$M3,
                                              bdaddr12only:$BD4),
-                        "cgib"##pos1##"\t$R1, $I2, "##pos2##"$BD4", []>;
+                        "cgib"##pos1##"\t$R1, $I2"##pos2##", $BD4", []>;
       def LRB  : InstRRS<0xECF7, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3,
                                               bdaddr12only:$BD4),
-                         "clrb"##pos1##"\t$R1, $R2, "##pos2##"$BD4", []>;
+                         "clrb"##pos1##"\t$R1, $R2"##pos2##", $BD4", []>;
       def LGRB : InstRRS<0xECE5, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3,
                                               bdaddr12only:$BD4),
-                         "clgrb"##pos1##"\t$R1, $R2, "##pos2##"$BD4", []>;
+                         "clgrb"##pos1##"\t$R1, $R2"##pos2##", $BD4", []>;
       def LIB  : InstRIS<0xECFF, (outs), (ins GR32:$R1, imm32zx8:$I2, ccmask:$M3,
                                               bdaddr12only:$BD4),
-                         "clib"##pos1##"\t$R1, $I2, "##pos2##"$BD4", []>;
+                         "clib"##pos1##"\t$R1, $I2"##pos2##", $BD4", []>;
       def LGIB : InstRIS<0xECFD, (outs), (ins GR64:$R1, imm64zx8:$I2, ccmask:$M3,
                                               bdaddr12only:$BD4),
-                         "clgib"##pos1##"\t$R1, $I2, "##pos2##"$BD4", []>;
+                         "clgib"##pos1##"\t$R1, $I2"##pos2##", $BD4", []>;
     }
   }
+
+  let isTerminator = 1, hasCtrlDep = 1 in {
+    def RT   : InstRRFc<0xB972, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3),
+                        "crt"##pos1##"\t$R1, $R2"##pos2, []>;
+    def GRT  : InstRRFc<0xB960, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3),
+                        "cgrt"##pos1##"\t$R1, $R2"##pos2, []>;
+    def LRT  : InstRRFc<0xB973, (outs), (ins GR32:$R1, GR32:$R2, ccmask:$M3),
+                        "clrt"##pos1##"\t$R1, $R2"##pos2, []>;
+    def LGRT : InstRRFc<0xB961, (outs), (ins GR64:$R1, GR64:$R2, ccmask:$M3),
+                        "clgrt"##pos1##"\t$R1, $R2"##pos2, []>;
+    def IT   : InstRIEa<0xEC72, (outs), (ins GR32:$R1, imm32sx16:$I2, ccmask:$M3),
+                         "cit"##pos1##"\t$R1, $I2"##pos2, []>;
+    def GIT  : InstRIEa<0xEC70, (outs), (ins GR64:$R1, imm32sx16:$I2, ccmask:$M3),
+                         "cgit"##pos1##"\t$R1, $I2"##pos2, []>;
+    def LFIT : InstRIEa<0xEC73, (outs), (ins GR32:$R1, imm32zx16:$I2, ccmask:$M3),
+                         "clfit"##pos1##"\t$R1, $I2"##pos2, []>;
+    def LGIT : InstRIEa<0xEC71, (outs), (ins GR64:$R1, imm32zx16:$I2, ccmask:$M3),
+                         "clgit"##pos1##"\t$R1, $I2"##pos2, []>;
+  }
 }
 let isCodeGenOnly = 1 in
   defm C : CompareBranches<cond4, "$M3", "">;
-defm AsmC : CompareBranches<imm32zx4, "", "$M3, ">;
+defm AsmC : CompareBranches<imm32zx4, "", ", $M3">;
 
 // Define AsmParser mnemonics for each general condition-code mask
 // (integer or floating-point)
 multiclass CondExtendedMnemonicA<bits<4> ccmask, string name> {
   let isBranch = 1, isTerminator = 1, R1 = ccmask in {
     def J : InstRI<0xA74, (outs), (ins brtarget16:$I2),
                    "j"##name##"\t$I2", []>;
     def JG : InstRIL<0xC04, (outs), (ins brtarget32:$I2),
let isIndirectBranch = 1 in {
       def CLIB  : InstRIS<0xECFF, (outs), (ins GR32:$R1, imm32zx8:$I2,
                                                bdaddr12only:$BD4),
                           "clib"##name##"\t$R1, $I2, $BD4", []>;
       def CLGIB : InstRIS<0xECFD, (outs), (ins GR64:$R1, imm64zx8:$I2,
                                                bdaddr12only:$BD4),
                           "clgib"##name##"\t$R1, $I2, $BD4", []>;
     }
   }
+
+  let hasCtrlDep = 1, isTerminator = 1, M3 = ccmask in {
+      def CRT   : InstRRFc<0xB972, (outs), (ins GR32:$R1, GR32:$R2),
+                          "crt"##name##"\t$R1, $R2", []>;
+      def CGRT  : InstRRFc<0xB960, (outs), (ins GR64:$R1, GR64:$R2),
+                          "cgrt"##name##"\t$R1, $R2", []>;
+      def CLRT  : InstRRFc<0xB973, (outs), (ins GR32:$R1, GR32:$R2),
+                          "clrt"##name##"\t$R1, $R2", []>;
+      def CLGRT : InstRRFc<0xB961, (outs), (ins GR64:$R1, GR64:$R2),
+                          "clgrt"##name##"\t$R1, $R2", []>;
+      def CIT   : InstRIEa<0xEC72, (outs), (ins GR32:$R1, imm32sx16:$I2),
+                           "cit"##name##"\t$R1, $I2", []>;
+      def CGIT  : InstRIEa<0xEC70, (outs), (ins GR64:$R1, imm32sx16:$I2),
+                           "cgit"##name##"\t$R1, $I2", []>;
+      def CLFIT : InstRIEa<0xEC73, (outs), (ins GR32:$R1, imm32zx16:$I2),
+                           "clfit"##name##"\t$R1, $I2", []>;
+      def CLGIT : InstRIEa<0xEC71, (outs), (ins GR64:$R1, imm32zx16:$I2),
+                           "clgit"##name##"\t$R1, $I2", []>;
+  }
 }
 multiclass IntCondExtendedMnemonic<bits<4> ccmask, string name1, string name2>
   : IntCondExtendedMnemonicA<ccmask, name1> {
   let isAsmParserOnly = 1 in
     defm Alt : IntCondExtendedMnemonicA<ccmask, name2>;
 }
 defm AsmJH   : IntCondExtendedMnemonic<2,  "h",  "nle">;
 defm AsmJL   : IntCondExtendedMnemonic<4,  "l",  "nhe">;

test/CodeGen/SystemZ/trap-01.ll

+; Test traps and conditional traps
+;
+; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s
+
+declare void @llvm.trap()
+
+; Check unconditional traps
+define i32 @f0() {
+; CHECK-LABEL: f0:
+; CHECK-LABEL: .Ltmp0
+; CHECK: j .Ltmp0+2
+entry:
+  tail call void @llvm.trap()
+  ret i32 0
+}
+
+; Check conditional compare immediate and trap
+define i32 @f1(i32 signext %a) {
+; CHECK-LABEL: f1:
+; CHECK: cithe %r2, 15
+; CHECK: lhi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp sgt i32 %a, 14
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i32 0
+}
+
+; Check conditional compare grande immediate and trap
+define i64 @f2(i64 signext %a) {
+; CHECK-LABEL: f2:
+; CHECK: cgitle %r2, 14
+; CHECK: lghi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp slt i64 %a, 15
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i64 0
+}
+
+; Check conditional compare logical immediate and trap
+define i32 @f3(i32 zeroext %a) {
+; CHECK-LABEL: f3:
+; CHECK: clfithe %r2, 15
+; CHECK: lhi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp ugt i32 %a, 14
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i32 0
+}
+
+; Check conditional compare grande logical immediate and trap
+define i64 @f4(i64 zeroext %a) {
+; CHECK-LABEL: f4:
+; CHECK: clgitle %r2, 14
+; CHECK: lghi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp ult i64 %a, 15
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i64 0
+}
+
+; Check conditional compare and trap
+define i32 @f5(i32 signext %a, i32 signext %b) {
+; CHECK-LABEL: f5:
+; CHECK: crte %r2, %r3
+; CHECK: lhi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp eq i32 %a, %b
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i32 0
+}
+
+; Check conditional compare grande and trap
+define i64 @f6(i64 signext %a, i64 signext %b) {
+; CHECK-LABEL: f6:
+; CHECK: cgrtl %r2, %r3
+; CHECK: lghi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp slt i64 %a, %b
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i64 0
+}
+
+; Check conditional compare logical and trap
+define i32 @f7(i32 zeroext %a, i32 zeroext %b) {
+; CHECK-LABEL: f7:
+; CHECK: clrth %r2, %r3
+; CHECK: lhi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp ugt i32 %a, %b
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i32 0
+}
+
+; Check conditional compare logical grande and trap
+define i64 @f8(i64 zeroext %a, i64 zeroext %b) {
+; CHECK-LABEL: f8:
+; CHECK: clgrtl %r2, %r3
+; CHECK: lghi %r2, 0
+; CHECK: br %r14
+entry:
+  %cmp = icmp ult i64 %a, %b
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret i64 0
+}
+
+; Check conditional traps that don't have a valid Compare and Trap
+define double @f9(double %a, double %b) {
+; CHECK-LABEL: f9:
+; CHECK: cdbr %f0, %f2
+; CHECK-LABEL: .Ltmp1
+; CHECK: je .Ltmp1+2
+; CHECK: lzdr %f0
+; CHECK: br %r14
+entry:
+  %cmp = fcmp oeq double %a, %b
+  br i1 %cmp, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  tail call void @llvm.trap()
+  unreachable
+
+if.end:                                           ; preds = %entry
+  ret double 0.000000e+00
+}

test/MC/Disassembler/SystemZ/insns.txt

 0xec 0x00 0x00 0x00 0xd0 0xe4
 
 # CHECK: cgrb %r0, %r0, 14, 0
 0xec 0x00 0x00 0x00 0xe0 0xe4
 
 # CHECK: cgrb %r0, %r0, 15, 0
 0xec 0x00 0x00 0x00 0xf0 0xe4
 
+# CHECK: cgrth %r0, %r1
+0xb9 0x60 0x20 0x01
+
+# CHECK: cgrtl %r0, %r1
+0xb9 0x60 0x40 0x01
+
+# CHECK: cgrte %r0, %r1
+0xb9 0x60 0x80 0x01
+
+# CHECK: cgrtlh %r0, %r1
+0xb9 0x60 0x60 0x01
+
+# CHECK: cgrthe %r0, %r1
+0xb9 0x60 0xa0 0x01
+
+# CHECK: cgrtle %r0, %r1
+0xb9 0x60 0xc0 0x01
+
 # CHECK: cg %r0, -524288
 0xe3 0x00 0x00 0x00 0x80 0x20
 
 # CHECK: cg %r0, -1
 0xe3 0x00 0x0f 0xff 0xff 0x20
 
 # CHECK: cg %r0, 0
 0xe3 0x00 0x00 0x00 0x00 0x20

 0xec 0x0d 0x00 0x00 0x00 0xfc
 
 # CHECK: cgib %r0, 0, 14, 0
 0xec 0x0e 0x00 0x00 0x00 0xfc
 
 # CHECK: cgib %r0, 0, 15, 0
 0xec 0x0f 0x00 0x00 0x00 0xfc
 
+# CHECK: cgith %r0, 0
+0xec 0x00 0x00 0x00 0x20 0x70
+
+# CHECK: cgitl %r0, 0
+0xec 0x00 0x00 0x00 0x40 0x70
+
+# CHECK: cgite %r0, 0
+0xec 0x00 0x00 0x00 0x80 0x70
+
+# CHECK: cgitlh %r0, 0
+0xec 0x00 0x00 0x00 0x60 0x70
+
+# CHECK: cgithe %r0, 0
+0xec 0x00 0x00 0x00 0xa0 0x70
+
+# CHECK: cgitle %r0, 0
+0xec 0x00 0x00 0x00 0xc0 0x70
+
 # CHECK: cgxbr %r0, 0, %f0
 0xb3 0xaa 0x00 0x00
 
 # CHECK: cgxbr %r0, 0, %f13
 0xb3 0xaa 0x00 0x0d
 
 # CHECK: cgxbr %r0, 15, %f0
 0xb3 0xaa 0xf0 0x00

 0xcc 0x0d 0x00 0x00 0x00 0x01
 
 # CHECK: cih %r0, 2147483647
 0xcc 0x0d 0x7f 0xff 0xff 0xff
 
 # CHECK: cih %r15, 0
 0xcc 0xfd 0x00 0x00 0x00 0x00
 
+# CHECK: cith %r0, 0
+0xec 0x00 0x00 0x00 0x20 0x72
+
+# CHECK: citl %r0, 0
+0xec 0x00 0x00 0x00 0x40 0x72
+
+# CHECK: cite %r0, 0
+0xec 0x00 0x00 0x00 0x80 0x72
+
+# CHECK: citlh %r0, 0
+0xec 0x00 0x00 0x00 0x60 0x72
+
+# CHECK: cithe %r0, 0
+0xec 0x00 0x00 0x00 0xa0 0x72
+
+# CHECK: citle %r0, 0
+0xec 0x00 0x00 0x00 0xc0 0x72
+
 # CHECK: clc 0(1), 0
 0xd5 0x00 0x00 0x00 0x00 0x00
 
 # CHECK: clc 0(1), 0(%r1)
 0xd5 0x00 0x00 0x00 0x10 0x00
 
 # CHECK: clc 0(1), 0(%r15)
 0xd5 0x00 0x00 0x00 0xf0 0x00

 0xc2 0x0f 0x00 0x00 0x00 0x00
 
 # CHECK: clfi %r0, 4294967295
 0xc2 0x0f 0xff 0xff 0xff 0xff
 
 # CHECK: clfi %r15, 0
 0xc2 0xff 0x00 0x00 0x00 0x00
 
+# CHECK: clfith %r0, 0
+0xec 0x00 0x00 0x00 0x20 0x73
+
+# CHECK: clfitl %r0, 0
+0xec 0x00 0x00 0x00 0x40 0x73
+
+# CHECK: clfite %r0, 0
+0xec 0x00 0x00 0x00 0x80 0x73
+
+# CHECK: clfitlh %r0, 0
+0xec 0x00 0x00 0x00 0x60 0x73
+
+# CHECK: clfithe %r0, 0
+0xec 0x00 0x00 0x00 0xa0 0x73
+
+# CHECK: clfitle %r0, 0
+0xec 0x00 0x00 0x00 0xc0 0x73
+
+# CHECK: clgith %r0, 0
+0xec 0x00 0x00 0x00 0x20 0x71
+
+# CHECK: clgitl %r0, 0
+0xec 0x00 0x00 0x00 0x40 0x71
+
+# CHECK: clgite %r0, 0
+0xec 0x00 0x00 0x00 0x80 0x71
+
+# CHECK: clgitlh %r0, 0
+0xec 0x00 0x00 0x00 0x60 0x71
+
+# CHECK: clgithe %r0, 0
+0xec 0x00 0x00 0x00 0xa0 0x71
+
+# CHECK: clgitle %r0, 0
+0xec 0x00 0x00 0x00 0xc0 0x71
+
 # CHECK: clgfi %r0, 0
 0xc2 0x0e 0x00 0x00 0x00 0x00
 
 # CHECK: clgfi %r0, 4294967295
 0xc2 0x0e 0xff 0xff 0xff 0xff
 
 # CHECK: clgfi %r15, 0
 0xc2 0xfe 0x00 0x00 0x00 0x00

 0xec 0x00 0x00 0x00 0xd0 0xf7
 
 # CHECK: clrb %r0, %r0, 14, 0
 0xec 0x00 0x00 0x00 0xe0 0xf7
 
 # CHECK: clrb %r0, %r0, 15, 0
 0xec 0x00 0x00 0x00 0xf0 0xf7
 
+# CHECK: clgrth %r0, %r1
+0xb9 0x61 0x20 0x01
+
+# CHECK: clgrtl %r0, %r1
+0xb9 0x61 0x40 0x01
+
+# CHECK: clgrte %r0, %r1
+0xb9 0x61 0x80 0x01
+
+# CHECK: clgrtlh %r0, %r1
+0xb9 0x61 0x60 0x01
+
+# CHECK: clgrthe %r0, %r1
+0xb9 0x61 0xa0 0x01
+
+# CHECK: clgrtle %r0, %r1
+0xb9 0x61 0xc0 0x01
+
+# CHECK: clrth %r0, %r1
+0xb9 0x73 0x20 0x01
+
+# CHECK: clrtl %r0, %r1
+0xb9 0x73 0x40 0x01
+
+# CHECK: clrte %r0, %r1
+0xb9 0x73 0x80 0x01
+
+# CHECK: clrtlh %r0, %r1
+0xb9 0x73 0x60 0x01
+
+# CHECK: clrthe %r0, %r1
+0xb9 0x73 0xa0 0x01
+
+# CHECK: clrtle %r0, %r1
+0xb9 0x73 0xc0 0x01
+
 # CHECK: clst %r0, %r0
 0xb2 0x5d 0x00 0x00
 
 # CHECK: clst %r0, %r15
 0xb2 0x5d 0x00 0x0f
 
 # CHECK: clst %r15, %r0
 0xb2 0x5d 0x00 0xf0

 0xec 0x00 0x00 0x00 0xd0 0xf6
 
 # CHECK: crb %r0, %r0, 14, 0
 0xec 0x00 0x00 0x00 0xe0 0xf6
 
 # CHECK: crb %r0, %r0, 15, 0
 0xec 0x00 0x00 0x00 0xf0 0xf6
 
+# CHECK: crth %r0, %r1
+0xb9 0x72 0x20 0x01
+
+# CHECK: crtl %r0, %r1
+0xb9 0x72 0x40 0x01
+
+# CHECK: crte %r0, %r1
+0xb9 0x72 0x80 0x01
+
+# CHECK: crtlh %r0, %r1
+0xb9 0x72 0x60 0x01
+
+# CHECK: crthe %r0, %r1
+0xb9 0x72 0xa0 0x01
+
+# CHECK: crtle %r0, %r1
+0xb9 0x72 0xc0 0x01
+
 # CHECK: csg %r0, %r0, -524288
 0xeb 0x00 0x00 0x00 0x80 0x30
 
 # CHECK: csg %r0, %r0, -1
 0xeb 0x00 0x0f 0xff 0xff 0x30
 
 # CHECK: csg %r0, %r0, 0
 0xeb 0x00 0x00 0x00 0x00 0x30

test/MC/SystemZ/insn-bad.s

 #CHECK: error: invalid instruction
 #CHECK: cgijo %r0, 0, 0, 0
 #CHECK: error: invalid instruction
 #CHECK: cgijno  %r0, 0, 0, 0
 
   cgijo %r0, 0, 0, 0
   cgijno  %r0, 0, 0, 0
 
+#CHECK: error: invalid operand
+#CHECK: cgit     %r0, -32769
+#CHECK: error: invalid operand
+#CHECK: cgit     %r0, 32768
+#CHECK: error: invalid instruction
+#CHECK: cgito    %r0, 0
+#CHECK: error: invalid instruction
+#CHECK: cgitno   %r0, 0
+
+        cgit     %r0, -32769
+        cgit     %r0, 32768
+        cgito    %r0, 0
+        cgitno   %r0, 0
+
 #CHECK: error: offset out of range
 #CHECK: cgrj  %r0, %r0, 0, -0x100002
 #CHECK: error: offset out of range
 #CHECK: cgrj  %r0, %r0, 0, -1
 #CHECK: error: offset out of range
 #CHECK: cgrj  %r0, %r0, 0, 1
 #CHECK: error: offset out of range
 #CHECK: cgrj  %r0, %r0, 0, 0x10000

 #CHECK: error: offset out of range
 #CHECK: cgrl  %r0, 0x100000000
 
   cgrl  %r0, -0x1000000002
   cgrl  %r0, -1
   cgrl  %r0, 1
   cgrl  %r0, 0x100000000
 
+#CHECK: error: invalid instruction
+#CHECK: cgrto    %r0, %r0
+#CHECK: error: invalid instruction
+#CHECK: cgrtno   %r0, %r0
+
+        cgrto    %r0, %r0
+        cgrtno   %r0, %r0
+
 #CHECK: error: invalid operand
 #CHECK: cgxbr %r0, -1, %f0
 #CHECK: error: invalid operand
 #CHECK: cgxbr %r0, 16, %f0
 #CHECK: error: invalid register pair
 #CHECK: cgxbr %r0, 0, %f2
 
   cgxbr %r0, -1, %f0

 #CHECK: cijo  %r0, 0, 0, 0
 #CHECK: error: invalid instruction
 #CHECK: cijno %r0, 0, 0, 0
 
   cijo  %r0, 0, 0, 0
   cijno %r0, 0, 0, 0
 
 #CHECK: error: invalid operand
+#CHECK: cit     %r0, -32769
+#CHECK: error: invalid operand
+#CHECK: cit     %r0, 32768
+#CHECK: error: invalid instruction
+#CHECK: cito    %r0, 0
+#CHECK: error: invalid instruction
+#CHECK: citno   %r0, 0
+
+        cit     %r0, -32769
+        cit     %r0, 32768
+        cito    %r0, 0
+        citno   %r0, 0
+
+#CHECK: error: invalid operand
 #CHECK: cl  %r0, -1
 #CHECK: error: invalid operand
 #CHECK: cl  %r0, 4096
 
   cl  %r0, -1
   cl  %r0, 4096
 
 #CHECK: error: missing length in address

 #CHECK: error: invalid operand
 #CHECK: clfi  %r0, -1
 #CHECK: error: invalid operand
 #CHECK: clfi  %r0, (1 << 32)
 
   clfi  %r0, -1
   clfi  %r0, (1 << 32)
 
+#CHECK: error: invalid operand
+#CHECK: clfit   %r0, -1
+#CHECK: error: invalid operand
+#CHECK: clfit   %r0, 65536
+#CHECK: error: invalid instruction
+#CHECK: clfito  %r0, 0
+#CHECK: error: invalid instruction
+#CHECK: clfitno %r0, 0
+
+        clfit   %r0, -1
+        clfit   %r0, 65536
+        clfito  %r0, 0
+        clfitno %r0, 0
+
 #CHECK: error: {{(instruction requires: fp-extension)?}}
 #CHECK: clfxbr  %r0, 0, %f0, 0
 
   clfxbr  %r0, 0, %f0, 0
 
 #CHECK: error: invalid operand
 #CHECK: clg %r0, -524289
 #CHECK: error: invalid operand
 #CHECK: clg %r0, 524288
 
   clg %r0, -524289
   clg %r0, 524288
 
+#CHECK: error: invalid operand
+#CHECK: clgit   %r0, -1
+#CHECK: error: invalid operand
+#CHECK: clgit   %r0, 65536
+#CHECK: error: invalid instruction
+#CHECK: clgito  %r0, 0
+#CHECK: error: invalid instruction
+#CHECK: clgitno %r0, 0
+
+        clgit   %r0, -1
+        clgit   %r0, 65536
+        clgito  %r0, 0
+        clgitno %r0, 0
+
 #CHECK: error: {{(instruction requires: fp-extension)?}}
 #CHECK: clgdbr  %r0, 0, %f0, 0
 
   clgdbr  %r0, 0, %f0, 0
 
 #CHECK: error: {{(instruction requires: fp-extension)?}}
 #CHECK: clgebr  %r0, 0, %f0, 0
 

 #CHECK: error: offset out of range
 #CHECK: clgrl %r0, 0x100000000
 
   clgrl %r0, -0x1000000002
   clgrl %r0, -1
   clgrl %r0, 1
   clgrl %r0, 0x100000000
 
+#CHECK: error: invalid instruction
+#CHECK: clgrto    %r0, %r0
+#CHECK: error: invalid instruction
+#CHECK: clgrtno   %r0, %r0
+
+        clgrto    %r0, %r0
+        clgrtno   %r0, %r0
+
 #CHECK: error: {{(instruction requires: fp-extension)?}}
 #CHECK: clgxbr  %r0, 0, %f0, 0
 
   clgxbr  %r0, 0, %f0, 0
 
 #CHECK: error: invalid operand
 #CHECK: clhhsi  -1, 0
 #CHECK: error: invalid operand

 #CHECK: error: offset out of range
 #CHECK: clrl  %r0, 0x100000000
 
   clrl  %r0, -0x1000000002
   clrl  %r0, -1
   clrl  %r0, 1
   clrl  %r0, 0x100000000
 
+#CHECK: error: invalid instruction
+#CHECK: clrto    %r0, %r0
+#CHECK: error: invalid instruction
+#CHECK: clrtno   %r0, %r0
+
+        clrto    %r0, %r0
+        clrtno   %r0, %r0
+
 #CHECK: error: invalid operand
 #CHECK: cly %r0, -524289
 #CHECK: error: invalid operand
 #CHECK: cly %r0, 524288
 
   cly %r0, -524289
   cly %r0, 524288
 

 #CHECK: error: offset out of range
 #CHECK: crl %r0, 0x100000000
 
   crl %r0, -0x1000000002
   crl %r0, -1
   crl %r0, 1
   crl %r0, 0x100000000
 
+#CHECK: error: invalid instruction
+#CHECK: crto    %r0, %r0
+#CHECK: error: invalid instruction
+#CHECK: crtno   %r0, %r0
+
+        crto    %r0, %r0
+        crtno   %r0, %r0
+
 #CHECK: error: invalid operand
 #CHECK: cs  %r0, %r0, -1
 #CHECK: error: invalid operand
 #CHECK: cs  %r0, %r0, 4096
 #CHECK: error: invalid use of indexed addressing
 #CHECK: cs  %r0, %r0, 0(%r1,%r2)
 
   cs  %r0, %r0, -1

test/MC/SystemZ/insn-good.s

 #CHECK: cgijle  %r1, -66, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xbe,0x7c]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   cgijle  %r1, -66, bar@PLT
 
 #CHECK: cgijnh  %r1, -66, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xbe,0x7c]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   cgijnh  %r1, -66, bar@PLT
 
+#CHECK: cgit     %r0, 0, 12             # encoding: [0xec,0x00,0x00,0x00,0xc0,0x70]
+#CHECK: cgit     %r0, -1, 12            # encoding: [0xec,0x00,0xff,0xff,0xc0,0x70]
+#CHECK: cgit     %r0, -32768, 12        # encoding: [0xec,0x00,0x80,0x00,0xc0,0x70]
+#CHECK: cgit     %r0, 32767, 12         # encoding: [0xec,0x00,0x7f,0xff,0xc0,0x70]
+#CHECK: cgith    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x20,0x70]
+#CHECK: cgitl    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x40,0x70]
+#CHECK: cgite    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x80,0x70]
+#CHECK: cgitne   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x60,0x70]
+#CHECK: cgitnl   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xa0,0x70]
+#CHECK: cgitnh   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xc0,0x70]
+
+        cgit     %r0, 0, 12
+        cgit     %r0, -1, 12
+        cgit     %r0, -32768, 12
+        cgit     %r0, 32767, 12
+        cgith    %r15, 1
+        cgitl    %r15, 1
+        cgite    %r15, 1
+        cgitne   %r15, 1
+        cgitnl   %r15, 1
+        cgitnh   %r15, 1
+
 #CHECK: cgr %r0, %r0                # encoding: [0xb9,0x20,0x00,0x00]
 #CHECK: cgr %r0, %r15               # encoding: [0xb9,0x20,0x00,0x0f]
 #CHECK: cgr %r15, %r0               # encoding: [0xb9,0x20,0x00,0xf0]
 #CHECK: cgr %r7, %r8                # encoding: [0xb9,0x20,0x00,0x78]
 
   cgr %r0,%r0
   cgr %r0,%r15
   cgr %r15,%r0

 #CHECK: cgrl  %r7, frob@PLT           # encoding: [0xc6,0x78,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 #CHECK: cgrl  %r8, frob@PLT           # encoding: [0xc6,0x88,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 
   cgrl  %r7,frob@PLT
   cgrl  %r8,frob@PLT
 
+#CHECK: cgrt     %r0, %r1, 12           # encoding: [0xb9,0x60,0xc0,0x01]
+#CHECK: cgrt     %r0, %r1, 12           # encoding: [0xb9,0x60,0xc0,0x01]
+#CHECK: cgrt     %r0, %r1, 12           # encoding: [0xb9,0x60,0xc0,0x01]
+#CHECK: cgrt     %r0, %r1, 12           # encoding: [0xb9,0x60,0xc0,0x01]
+#CHECK: cgrth    %r0, %r15              # encoding: [0xb9,0x60,0x20,0x0f]
+#CHECK: cgrtl    %r0, %r15              # encoding: [0xb9,0x60,0x40,0x0f]
+#CHECK: cgrte    %r0, %r15              # encoding: [0xb9,0x60,0x80,0x0f]
+#CHECK: cgrtne   %r0, %r15              # encoding: [0xb9,0x60,0x60,0x0f]
+#CHECK: cgrtnl   %r0, %r15              # encoding: [0xb9,0x60,0xa0,0x0f]
+#CHECK: cgrtnh   %r0, %r15              # encoding: [0xb9,0x60,0xc0,0x0f]
+
+        cgrt     %r0, %r1, 12
+        cgrt     %r0, %r1, 12
+        cgrt     %r0, %r1, 12
+        cgrt     %r0, %r1, 12
+        cgrth    %r0, %r15
+        cgrtl    %r0, %r15
+        cgrte    %r0, %r15
+        cgrtne   %r0, %r15
+        cgrtnl   %r0, %r15
+        cgrtnh   %r0, %r15
+
 #CHECK: cgxbr %r0, 0, %f0             # encoding: [0xb3,0xaa,0x00,0x00]
 #CHECK: cgxbr %r0, 0, %f13            # encoding: [0xb3,0xaa,0x00,0x0d]
 #CHECK: cgxbr %r0, 15, %f0            # encoding: [0xb3,0xaa,0xf0,0x00]
 #CHECK: cgxbr %r4, 5, %f8             # encoding: [0xb3,0xaa,0x50,0x48]
 #CHECK: cgxbr %r15, 0, %f0            # encoding: [0xb3,0xaa,0x00,0xf0]
 
   cgxbr %r0, 0, %f0
   cgxbr %r0, 0, %f13

 #CHECK: cijle %r1, -66, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xbe,0x7e]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   cijle %r1, -66, bar@PLT
 
 #CHECK: cijnh %r1, -66, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xbe,0x7e]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   cijnh %r1, -66, bar@PLT
 
+#CHECK: cit     %r0, 0, 12              # encoding: [0xec,0x00,0x00,0x00,0xc0,0x72]
+#CHECK: cit     %r0, -1, 12             # encoding: [0xec,0x00,0xff,0xff,0xc0,0x72]
+#CHECK: cit     %r0, -32768, 12         # encoding: [0xec,0x00,0x80,0x00,0xc0,0x72]
+#CHECK: cit     %r0, 32767, 12          # encoding: [0xec,0x00,0x7f,0xff,0xc0,0x72]
+#CHECK: cith    %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0x20,0x72]
+#CHECK: citl    %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0x40,0x72]
+#CHECK: cite    %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0x80,0x72]
+#CHECK: citne   %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0x60,0x72]
+#CHECK: citnl   %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0xa0,0x72]
+#CHECK: citnh   %r15, 1                 # encoding: [0xec,0xf0,0x00,0x01,0xc0,0x72]
+
+        cit     %r0, 0, 12
+        cit     %r0, -1, 12
+        cit     %r0, -32768, 12
+        cit     %r0, 32767, 12
+        cith    %r15, 1
+        citl    %r15, 1
+        cite    %r15, 1
+        citne   %r15, 1
+        citnl   %r15, 1
+        citnh   %r15, 1
+
 #CHECK: cl  %r0, 0                  # encoding: [0x55,0x00,0x00,0x00]
 #CHECK: cl  %r0, 4095               # encoding: [0x55,0x00,0x0f,0xff]
 #CHECK: cl  %r0, 0(%r1)             # encoding: [0x55,0x00,0x10,0x00]
 #CHECK: cl  %r0, 0(%r15)            # encoding: [0x55,0x00,0xf0,0x00]
 #CHECK: cl  %r0, 4095(%r1,%r15)     # encoding: [0x55,0x01,0xff,0xff]
 #CHECK: cl  %r0, 4095(%r15,%r1)     # encoding: [0x55,0x0f,0x1f,0xff]
 #CHECK: cl  %r15, 0                 # encoding: [0x55,0xf0,0x00,0x00]
 

 #CHECK: clfi  %r0, 0                  # encoding: [0xc2,0x0f,0x00,0x00,0x00,0x00]
 #CHECK: clfi  %r0, 4294967295         # encoding: [0xc2,0x0f,0xff,0xff,0xff,0xff]
 #CHECK: clfi  %r15, 0                 # encoding: [0xc2,0xff,0x00,0x00,0x00,0x00]
 
   clfi  %r0, 0
   clfi  %r0, (1 << 32) - 1
   clfi  %r15, 0
 
+#CHECK: clfit     %r0, 0, 12             # encoding: [0xec,0x00,0x00,0x00,0xc0,0x73]
+#CHECK: clfit     %r0, 65535, 12         # encoding: [0xec,0x00,0xff,0xff,0xc0,0x73]
+#CHECK: clfit     %r0, 32768, 12         # encoding: [0xec,0x00,0x80,0x00,0xc0,0x73]
+#CHECK: clfith    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x20,0x73]
+#CHECK: clfitl    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x40,0x73]
+#CHECK: clfite    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x80,0x73]
+#CHECK: clfitne   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x60,0x73]
+#CHECK: clfitnl   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xa0,0x73]
+#CHECK: clfitnh   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xc0,0x73]
+
+        clfit     %r0, 0, 12
+        clfit     %r0, 65535, 12
+        clfit     %r0, 32768, 12
+        clfith    %r15, 1
+        clfitl    %r15, 1
+        clfite    %r15, 1
+        clfitne   %r15, 1
+        clfitnl   %r15, 1
+        clfitnh   %r15, 1
+
 #CHECK: clg %r0, -524288            # encoding: [0xe3,0x00,0x00,0x00,0x80,0x21]
 #CHECK: clg %r0, -1                 # encoding: [0xe3,0x00,0x0f,0xff,0xff,0x21]
 #CHECK: clg %r0, 0                  # encoding: [0xe3,0x00,0x00,0x00,0x00,0x21]
 #CHECK: clg %r0, 1                  # encoding: [0xe3,0x00,0x00,0x01,0x00,0x21]
 #CHECK: clg %r0, 524287             # encoding: [0xe3,0x00,0x0f,0xff,0x7f,0x21]
 #CHECK: clg %r0, 0(%r1)             # encoding: [0xe3,0x00,0x10,0x00,0x00,0x21]
 #CHECK: clg %r0, 0(%r15)            # encoding: [0xe3,0x00,0xf0,0x00,0x00,0x21]
 #CHECK: clg %r0, 524287(%r1,%r15)   # encoding: [0xe3,0x01,0xff,0xff,0x7f,0x21]

 #CHECK: clgijle %r1, 193, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xc1,0x7d]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   clgijle %r1, 193, bar@PLT
 
 #CHECK: clgijnh %r1, 193, bar@PLT                 # encoding: [0xec,0x1c,A,A,0xc1,0x7d]
 #CHECK:  fixup A - offset: 2, value: bar@PLT+2, kind: FK_390_PC16DBL
   clgijnh %r1, 193, bar@PLT
 
+#CHECK: clgit     %r0, 0, 12             # encoding: [0xec,0x00,0x00,0x00,0xc0,0x71]
+#CHECK: clgit     %r0, 65535, 12         # encoding: [0xec,0x00,0xff,0xff,0xc0,0x71]
+#CHECK: clgit     %r0, 32768, 12         # encoding: [0xec,0x00,0x80,0x00,0xc0,0x71]
+#CHECK: clgith    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x20,0x71]
+#CHECK: clgitl    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x40,0x71]
+#CHECK: clgite    %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x80,0x71]
+#CHECK: clgitne   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0x60,0x71]
+#CHECK: clgitnl   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xa0,0x71]
+#CHECK: clgitnh   %r15, 1                # encoding: [0xec,0xf0,0x00,0x01,0xc0,0x71]
+
+        clgit     %r0, 0, 12
+        clgit     %r0, 65535, 12
+        clgit     %r0, 32768, 12
+        clgith    %r15, 1
+        clgitl    %r15, 1
+        clgite    %r15, 1
+        clgitne   %r15, 1
+        clgitnl   %r15, 1
+        clgitnh   %r15, 1
+
 #CHECK: clgr  %r0, %r0                # encoding: [0xb9,0x21,0x00,0x00]
 #CHECK: clgr  %r0, %r15               # encoding: [0xb9,0x21,0x00,0x0f]
 #CHECK: clgr  %r15, %r0               # encoding: [0xb9,0x21,0x00,0xf0]
 #CHECK: clgr  %r7, %r8                # encoding: [0xb9,0x21,0x00,0x78]
 
   clgr  %r0,%r0
   clgr  %r0,%r15
   clgr  %r15,%r0

 
 #CHECK: clrb  %r1, %r2, 12, 3(%r4)    # encoding: [0xec,0x12,0x40,0x03,0xc0,0xf7]
 #CHECK: clrble  %r1, %r2, 3(%r4)        # encoding: [0xec,0x12,0x40,0x03,0xc0,0xf7]
 #CHECK: clrbnh  %r1, %r2, 3(%r4)        # encoding: [0xec,0x12,0x40,0x03,0xc0,0xf7]
   clrb  %r1, %r2, 12, 3(%r4)
   clrble  %r1, %r2, 3(%r4)
   clrbnh  %r1, %r2, 3(%r4)
 
+#CHECK: clgrt     %r0, %r1, 12          # encoding: [0xb9,0x61,0xc0,0x01]
+#CHECK: clgrt     %r0, %r1, 12          # encoding: [0xb9,0x61,0xc0,0x01]
+#CHECK: clgrt     %r0, %r1, 12          # encoding: [0xb9,0x61,0xc0,0x01]
+#CHECK: clgrt     %r0, %r1, 12          # encoding: [0xb9,0x61,0xc0,0x01]
+#CHECK: clgrth    %r0, %r15             # encoding: [0xb9,0x61,0x20,0x0f]
+#CHECK: clgrtl    %r0, %r15             # encoding: [0xb9,0x61,0x40,0x0f]
+#CHECK: clgrte    %r0, %r15             # encoding: [0xb9,0x61,0x80,0x0f]
+#CHECK: clgrtne   %r0, %r15             # encoding: [0xb9,0x61,0x60,0x0f]
+#CHECK: clgrtnl   %r0, %r15             # encoding: [0xb9,0x61,0xa0,0x0f]
+#CHECK: clgrtnh   %r0, %r15             # encoding: [0xb9,0x61,0xc0,0x0f]
+
+        clgrt     %r0, %r1, 12
+        clgrt     %r0, %r1, 12
+        clgrt     %r0, %r1, 12
+        clgrt     %r0, %r1, 12
+        clgrth    %r0, %r15
+        clgrtl    %r0, %r15
+        clgrte    %r0, %r15
+        clgrtne   %r0, %r15
+        clgrtnl   %r0, %r15
+        clgrtnh   %r0, %r15
+
 #CHECK: clrj  %r0, %r0, 0, .[[LAB:L.*]] # encoding: [0xec,0x00,A,A,0x00,0x77]
 #CHECK:  fixup A - offset: 2, value: .[[LAB]]+2, kind: FK_390_PC16DBL
 #CHECK: clrj  %r0, %r15, 0, .[[LAB:L.*]]  # encoding: [0xec,0x0f,A,A,0x00,0x77]
 #CHECK:  fixup A - offset: 2, value: .[[LAB]]+2, kind: FK_390_PC16DBL
 #CHECK: clrj  %r15, %r0, 0, .[[LAB:L.*]]  # encoding: [0xec,0xf0,A,A,0x00,0x77]
 #CHECK:  fixup A - offset: 2, value: .[[LAB]]+2, kind: FK_390_PC16DBL
 #CHECK: clrj  %r7, %r8, 0, .[[LAB:L.*]] # encoding: [0xec,0x78,A,A,0x00,0x77]
 #CHECK:  fixup A - offset: 2, value: .[[LAB]]+2, kind: FK_390_PC16DBL

 #CHECK: clrl  %r7, frob@PLT           # encoding: [0xc6,0x7f,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 #CHECK: clrl  %r8, frob@PLT           # encoding: [0xc6,0x8f,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 
   clrl  %r7,frob@PLT
   clrl  %r8,frob@PLT
 
+#CHECK: clrt     %r0, %r1, 12           # encoding: [0xb9,0x73,0xc0,0x01]
+#CHECK: clrt     %r0, %r1, 12           # encoding: [0xb9,0x73,0xc0,0x01]
+#CHECK: clrt     %r0, %r1, 12           # encoding: [0xb9,0x73,0xc0,0x01]
+#CHECK: clrt     %r0, %r1, 12           # encoding: [0xb9,0x73,0xc0,0x01]
+#CHECK: clrth    %r0, %r15              # encoding: [0xb9,0x73,0x20,0x0f]
+#CHECK: clrtl    %r0, %r15              # encoding: [0xb9,0x73,0x40,0x0f]
+#CHECK: clrte    %r0, %r15              # encoding: [0xb9,0x73,0x80,0x0f]
+#CHECK: clrtne   %r0, %r15              # encoding: [0xb9,0x73,0x60,0x0f]
+#CHECK: clrtnl   %r0, %r15              # encoding: [0xb9,0x73,0xa0,0x0f]
+#CHECK: clrtnh   %r0, %r15              # encoding: [0xb9,0x73,0xc0,0x0f]
+
+        clrt     %r0, %r1, 12
+        clrt     %r0, %r1, 12
+        clrt     %r0, %r1, 12
+        clrt     %r0, %r1, 12
+        clrth    %r0, %r15
+        clrtl    %r0, %r15
+        clrte    %r0, %r15
+        clrtne   %r0, %r15
+        clrtnl   %r0, %r15
+        clrtnh   %r0, %r15
+
 #CHECK: clst  %r0, %r0                # encoding: [0xb2,0x5d,0x00,0x00]
 #CHECK: clst  %r0, %r15               # encoding: [0xb2,0x5d,0x00,0x0f]
 #CHECK: clst  %r15, %r0               # encoding: [0xb2,0x5d,0x00,0xf0]
 #CHECK: clst  %r7, %r8                # encoding: [0xb2,0x5d,0x00,0x78]
 
   clst  %r0,%r0
   clst  %r0,%r15
   clst  %r15,%r0

 #CHECK: crl %r7, frob@PLT           # encoding: [0xc6,0x7d,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 #CHECK: crl %r8, frob@PLT           # encoding: [0xc6,0x8d,A,A,A,A]
 # fixup A - offset: 2, value: frob@PLT+2, kind: FK_390_PC32DBL
 
   crl %r7,frob@PLT
   crl %r8,frob@PLT
 
+#CHECK: crt     %r0, %r1, 12            # encoding: [0xb9,0x72,0xc0,0x01]
+#CHECK: crt     %r0, %r1, 12            # encoding: [0xb9,0x72,0xc0,0x01]
+#CHECK: crt     %r0, %r1, 12            # encoding: [0xb9,0x72,0xc0,0x01]
+#CHECK: crt     %r0, %r1, 12            # encoding: [0xb9,0x72,0xc0,0x01]
+#CHECK: crth    %r0, %r15               # encoding: [0xb9,0x72,0x20,0x0f]
+#CHECK: crtl    %r0, %r15               # encoding: [0xb9,0x72,0x40,0x0f]
+#CHECK: crte    %r0, %r15               # encoding: [0xb9,0x72,0x80,0x0f]
+#CHECK: crtne   %r0, %r15               # encoding: [0xb9,0x72,0x60,0x0f]
+#CHECK: crtnl   %r0, %r15               # encoding: [0xb9,0x72,0xa0,0x0f]
+#CHECK: crtnh   %r0, %r15               # encoding: [0xb9,0x72,0xc0,0x0f]
+
+        crt     %r0, %r1, 12
+        crt     %r0, %r1, 12
+        crt     %r0, %r1, 12
+        crt     %r0, %r1, 12
+        crth    %r0, %r15
+        crtl    %r0, %r15
+        crte    %r0, %r15
+        crtne   %r0, %r15
+        crtnl   %r0, %r15
+        crtnh   %r0, %r15
+
 #CHECK: cs  %r0, %r0, 0             # encoding: [0xba,0x00,0x00,0x00]
 #CHECK: cs  %r0, %r0, 4095          # encoding: [0xba,0x00,0x0f,0xff]
 #CHECK: cs  %r0, %r0, 0(%r1)        # encoding: [0xba,0x00,0x10,0x00]
 #CHECK: cs  %r0, %r0, 0(%r15)       # encoding: [0xba,0x00,0xf0,0x00]
 #CHECK: cs  %r0, %r0, 4095(%r1)     # encoding: [0xba,0x00,0x1f,0xff]
 #CHECK: cs  %r0, %r0, 4095(%r15)    # encoding: [0xba,0x00,0xff,0xff]
 #CHECK: cs  %r0, %r15, 0            # encoding: [0xba,0x0f,0x00,0x00]
 #CHECK: cs  %r15, %r0, 0            # encoding: [0xba,0xf0,0x00,0x00]