[SystemZ] set 'guessInstructionProperties = 0' and add flags as needed.

[jonpa]

Summary

After setting 'guessInstructionProperties = 0', mayLoad, mayStore and hasSideEffects have been set to 0 per default in InstSystemZ.

Then those flags have been set on individual instructions so that there is no diff on the tablegen output compared to trunk.

I suspect that null_fag may have set this flag earlier in places where we could now remove it.

Not sure why, but SRAK seems to have the SideEffects flag...

hasSideEffects = 0 could also be removed in a few places.

[uweigand]

Thanks for looking at and reviewing those flags!

It's a good first step to ensure that flags remain identical to what they currently are. However, this seems to have exposed a couple of bugs in the current flags. Let's see what the best way to get to a correct solution is, without introducing too many intermediate changes ... I'm now wondering whether we should first fix wrong flags and then switch to guessInstructionProperties = 0 or the other way around.

First of all, I'm not sure it makes much sense to introduce "generic" settings of hasSideEffects in instruction format patterns. You've added them to all SideEffect* patterns -- however, while all those instructions have "side effects" in the sense that they not only operate on their operands, the many (most?) cases those side effect are simply memory accesses (loads or stores), which should already be handled via mayLoad / mayStore. Those should *not* also have hasSideEffects set; that flag is only needed if there are any *other* "unmodeled" side effects. I think I added the correct mayLoad/mayStore/hasSideEffects setting to all (I hope!) SideEffect* pattersn, but I didn't realize I had to actively reset hasSideEffects to zero. So I guess it is OK to add the flag to the SideEffect* patterns for now to ensure we get NFC, but then later on remove them all again, since they're actually incorrect. (Possibly doing another review of all users first.)

A similar issue are the "generic" vector instructions (i.e. the *Generic patterns). Those are all inline-assembler-only patterns, and therefore have no DAG pattern (which I guess is why they were classified as hasSideEffects by the default logic). You seem to have added the flag to some but not all *Generic patterns, but for those patterns where you haven't added it you instead added it to the instructions that use it ... Similar to above, I think it is OK to add the flag for now, but then you should just add it to *all* Generic patterns, to avoid unnecessary changes to the vector instruction patterns.

Yet again similar are the *K patterns. There the hasSideEffects flags is pretty much always wrong, it's just added incorrectly because the patterns use null_frag. Again, I'm fine with temporarily adding and later removing it. However, the duplication of the BinaryRSAndK pattern seems weird. Is this because some users of BinaryRSAndK manually set hasSideEffects = 0? This was probably already an attempt to work around. They should just *all* use that ...

The final major issue seem to be the HFP (hex floating-point) and DFP (decimal floating-point) settings. I guess they all have hasSideEffects because they are never used for code generation and therefore all use null_frag. Now I thought that floating-point operations usually have side effects anyway since they may trap. But that doesn't seem to be even modeled for the normal (binary) floating-point instructions, where it would be most important since those are actually used for codegen ... Not sure if this hides a bug somewhere. On the other hand, other platforms don't appear to set this for FP instructions either, so I guess we should leave it for now. But then we should probably treat HFP and DFP the same.

I've gone through the changes in detail. Note that my comments really refer to the state we should have in the end, i.e. in most cases *different* from the current setting. Given that we'd have to introduce many changes to those files just to revert them again, maybe in would be better to do the changes in a single step after all ...

lib/Target/SystemZ/SystemZInstrDFP.td

20

Given the discussion above, the comment should probably be different.

lib/Target/SystemZ/SystemZInstrFP.td

136

This shouldn't have hasSideEffects = 1, but rather mayLoad = 1. This would fix a current bug.

Note that the whole block of load instructions really should have mayLoad = 1.

157

Again, probably best if the whole block is marked mayStore = 1.

172

These aren't really different from other FP instructions, so if the rest don't have hasSideEffects = 1, this shouldn't either ... The same comment actually applies to the rest of these changes throughout this file.

541

These are actually wrong: EFPC and SFPC do not touch memory at all, while STFPC is only a store and LFPC is only a load. I think this is wrongly inferred because the definition of the int_s390_efpc / int_s390_sfpc intrinsics in include/llvm/IR/IntrinsicsSystemZ.td is wrong: they should be marked as IntrNoMem. (This specific problem can just be fixed before, I think.)

lib/Target/SystemZ/SystemZInstrFormats.td

4592

This is certainly also wrong, those compares have no special side effects ...

lib/Target/SystemZ/SystemZInstrInfo.td

14

Since those are fully no-op on SystemZ, they have no side effects.

37

I believe as long as branches are correctly marked as isBranch, they don't need to *also* carry a hasSideEffects flag.

204

But traps certainly should, so this one is fine.

376

This has no side effects.

381

This is a no-op and has no side effects.

417

Probably best to mark the whole block as mayLoad = 1.

461

Same here, mark the whole block as mayStore = 1.

520

This doesn't have side effects, just a mayStore ...

525

These should actually match the mayLoad / mayStore settings of the underlying instructions.

593

Doesn't actually have any side effects.

698

Doesn't have side effects.

710

Likewise.

746

Should just be mayStore.

756

Should be mayLoad / mayStore as appropriate, has no other side effects.

865

Should just be mayLoad.

921

No side effects with any of the adds. The to-memory variants should be mayLoad and mayStore.

1054

No side effects.

1063

Neither here. Why are SLFI and SLGFI different in the first place, that's weird ...

1280

No side effects on any of the multiplies ...

1362

As discussed above, none of the shifts or rotates has side effects.

1434

No side effects.

1488

Likewise.

1569

Just mayLoad

1583

This is actually fine, I guess.

1608

Should just have mayLoad/mayStore, like the others. Maybe for the whole block?

1744

Hmm. Not sure if atomics should have the SideEffect marker ... but either all of them or none.

1793

Just mayLoad is enough.

1854

mayLoad should be enough for these. (They may trap, but so can any load ...)

1868

Just mayLoad or mayStore

1906

Just mayLoad

1918

No particular side effects for the access register routines (except mayLoad / mayStore as appropriate)

1954

As above, no side effects for branches.

1988

This really should only be a mayStore. Maybe another bug in the intrinsics definition.

1998

A TABORT is probably best genuinely marked as hasSideEffects = 1. It shouldn't be mayLoad or mayStore.

2002

Should be store only.

2029

No side effects.

2075

This is OK.

lib/Target/SystemZ/SystemZInstrSystem.td

63

This as well.

186

Not really. None of the remaining changes in this file look good.

lib/Target/SystemZ/SystemZInstrVector.td

16

Nothing in this file really should have hasSideEffects = 1.

225

No, this should just be mayStore. Another intrinsics issue?

260

Likewise.

[jonpa]

Still NFC.

Patch updated to minimize changes in SystemZInstrVector.td, by moving the flag settings to the *Generic classes instead.

[jonpa]

Patch updated per review, with functional changes.

Three SystemZ tests have been updated, where two of them now have an extra COPY (instructions may now be more freely rescheduled). Hopefully, this should go away with other ongoing patches. One SystemZ test is crashing (asm-18.ll), see llvm-dev for explanation of the TwoAddress problem.

Hopefully, it should be possible to observe the changes by diffing to the previous diff, which was NFC.

lib/Target/SystemZ/SystemZInstrDFP.td

20

I removed the hasSideEffects flag in both DFP/HFP to be consistent , and instead made a TODO comment about perhaps adding this flag to all relevant FP instructions.

lib/Target/SystemZ/SystemZInstrFP.td

541

Not sure exactly how to do this - there is only one intrinsic (for read/set), but two instructions, where only one of them loads/stores.

It does not compile if just setting mayLoad=0, mayStore=0 on the non-mem instructions.

Would it work to add duplicate intrinsics for this in InstrinsicsSystemZ.td, like

 def int_s390_sfpc : GCCBuiltin<"__builtin_s390_sfpc">,
                      Intrinsic<[], [llvm_i32_ty], [IntrNoMem]>;

 def int_s390_sfpc_mem : GCCBuiltin<"__builtin_s390_sfpc">,
                          Intrinsic<[], [llvm_i32_ty], []>;
...

Or is this so rare that it could be ignored and left alone?

lib/Target/SystemZ/SystemZInstrInfo.td

14

Tried to remove, but did not work due to the callseq_... nodes in the pattern.

1063

yes... I don't know really

1744

Since we are unsure, I put all inside a new block with this flag set, and added a TODO comment regarding this. This is now *all* instructions in the atomic section, not just the ATOMIC instructions. Is that what you meant?

1954

OK - I am assuming this should also work the same for isCall / isReturn

1988

Yes. I tried to experiment with the intrinsics declarations like adding WriteOnly<0> and so on, but to no other effect than very long rebuild times...

I am leaving these flags as they are now as they are forced by the intrinsics definitions...

lib/Target/SystemZ/SystemZInstrVector.td

16

I suppose the vector FP instructions still follow the same discussion as for the FP files?

225

most likely -- leaving as for the others above.

[jonpa]

flagsdiff_master.diff167 KBDownload

This file is the output of a simple script I made, to perhaps get a better overview of the functional changes for the instructions.

[uweigand]

Thanks for providing the diff. I noticed two problems, also annotated in the detailed comment:

• You added mayLoad to a number of load-conditional instructions that actually don't access memory (reg-reg or reg-immed instructions)
• The guarded-storage instructions should really have the side effects flag (I overlooked this in my previous review).

The rest of the changes looks OK to me.

As to your question:

I suppose the vector FP instructions still follow the same discussion as for the FP files?

Yes, this is true.

lib/Target/SystemZ/SystemZInstrInfo.td

93–95

This should better be in the FixedCondBranchRXY pattern -- this already hard-codes the "load" DAG node, so it can also hard-code mayLoad. B.t.w. the other CondBranchRXY pattern variants really also should have the mayLoad flag ...

525–538

Most of these are load immediate or load register instructions, they do not access memory.

533

Only this one should have mayLoad.

555–557

Again, the first two are load register instructions, only the second two should have mayLoad.

947

Do we need this? Should be correctly autodetected from the DAG patterns. The same applied to various patterns below.

951

This is already in BinarySIY. Same applies to other BinarySIY patterns.

1853–1854

Thinking about this, the whole set of FeatureGuardedStorage instructions should have the HasSideEffects flags after all, since they use and/or modify the guarded storage control registers, which we do not otherwise model, so the must not be reordered with respect to each other.

[jonpa]

Updated per review.

The bugfix in TwoAddress is under review separately. Only the test case here will be committed by this patch, as it exposes the bug.

With the bugfix I could build benchmarks and I saw just some minor differences in instruction counts:

tmll           :                15798                15987     +189
lr             :                25915                25769     -146
wfsdb          :                 6655                 6795     +140
sdbr           :                 3134                 2994     -140
j              :                98539                98678     +139
nr             :                 2845                 2709     -136
n              :                 8918                 9052     +134
tmhl           :                  933                  799     -134
risbhg         :                 1996                 1880     -116
ldr            :                22280                22172     -108
stg            :               124184               124093      -91
lg             :               317058               316968      -90
lt             :                 7152                 7072      -80
cgrjlh         :                11594                11673      +79
risblg         :                 8448                 8527      +79
l              :                65752                65830      +78
st             :                51556                51628      +72
mdbr           :                 6686                 6615      -71
risbgn         :                34903                34837      -66

                 spec-llvm_A_clang-master/            spec-llvm
Spill|Reload   :               164860               164772      -88

Reduced spilling should be a good indicator.

lib/Target/SystemZ/SystemZInstrInfo.td

93–95

I added 'mayLoad = 1' to the three *CondBranchRXY patterns. This added the mayLoad flag on BIAsm* and BIC[Asm] instructions.

533

removed the mayLoad setting - FixedCondUnaryRSY already hard codes it.

555–557

Oops. Again removed the mayLoad = 1 flag, as it is already hard coded. I assume this to be the policy, so I also removed mayStore from STOC* instructions.

951

OK - I am guessing then that if the instruction class sets the mayLoad / mayStore flag, it should preferrably not also be set on the instruction. I tried to clean away the cases I could find in this file. I suppose the other files could also be cleaned up?

[uweigand]

These changes (at the code level) look reasonable to me now. (There is one mayStore on a STOC instruction that is probably still superfluous now.)

I've looked again at the atomic instructions, and I now think they should not have the "unmodelled side effects" flags: they do have side effects, but those are modeled (e.g. via the ordering flags). Common code should (and as far as I can tell, does) look at those properties to decide whether and how an atomic instruction may be scheduled. This also matches the fact that most of the atomics didn't have hasSideEffects before your patch, and they don't on other platforms either. So I think they should simply not have the flag.

Otherwise, there's still two things that should be fixed before this can go in: the TwoAddress fix, and the intrinsics flags.

Once this is all ready, could you please re-create another diff for the attribute settings, just so we can verify that nothing surprising has crept in? Thanks!

lib/Target/SystemZ/SystemZInstrInfo.td

525–538

Shouldn't this be unnecessary as well?

[jonpa]

Path updated.

Functional change:

UnmodeledSideEffects REMOVED: ATOMIC_CMP_SWAPW, ATOMIC_LOADW_AFI, ATOMIC_LOADW_AR, ATOMIC_LOADW_MAX, ATOMIC_LOADW_MIN, ATOMIC_LOADW_NILH, ATOMIC_LOADW_NILHi, ATOMIC_LOADW_NR, ATOMIC_LOADW_NRi, ATOMIC_LOADW_OILH, ATOMIC_LOADW_OR, ATOMIC_LOADW_SR, ATOMIC_LOADW_UMAX, ATOMIC_LOADW_UMIN, ATOMIC_LOADW_XILF, ATOMIC_LOADW_XR, ATOMIC_LOAD_AFI, ATOMIC_LOAD_AGFI, ATOMIC_LOAD_AGHI, ATOMIC_LOAD_AGR, ATOMIC_LOAD_AHI, ATOMIC_LOAD_AR, ATOMIC_LOAD_MAX_32, ATOMIC_LOAD_MAX_64, ATOMIC_LOAD_MIN_32, ATOMIC_LOAD_MIN_64, ATOMIC_LOAD_NGR, ATOMIC_LOAD_NGRi, ATOMIC_LOAD_NIHF64, ATOMIC_LOAD_NIHF64i, ATOMIC_LOAD_NIHH64, ATOMIC_LOAD_NIHH64i, ATOMIC_LOAD_NIHL64, ATOMIC_LOAD_NIHL64i, ATOMIC_LOAD_NILF, ATOMIC_LOAD_NILF64, ATOMIC_LOAD_NILF64i, ATOMIC_LOAD_NILFi, ATOMIC_LOAD_NILH, ATOMIC_LOAD_NILH64, ATOMIC_LOAD_NILH64i, ATOMIC_LOAD_NILHi, ATOMIC_LOAD_NILL, ATOMIC_LOAD_NILL64, ATOMIC_LOAD_NILL64i, ATOMIC_LOAD_NILLi, ATOMIC_LOAD_NR, ATOMIC_LOAD_NRi, ATOMIC_LOAD_OGR, ATOMIC_LOAD_OIHF64, ATOMIC_LOAD_OIHH64, ATOMIC_LOAD_OIHL64, ATOMIC_LOAD_OILF, ATOMIC_LOAD_OILF64, ATOMIC_LOAD_OILH, ATOMIC_LOAD_OILH64, ATOMIC_LOAD_OILL, ATOMIC_LOAD_OILL64, ATOMIC_LOAD_OR, ATOMIC_LOAD_SGR, ATOMIC_LOAD_SR, ATOMIC_LOAD_UMAX_32, ATOMIC_LOAD_UMAX_64, ATOMIC_LOAD_UMIN_32, ATOMIC_LOAD_UMIN_64, ATOMIC_LOAD_XGR, ATOMIC_LOAD_XIHF64, ATOMIC_LOAD_XILF, ATOMIC_LOAD_XILF64, ATOMIC_LOAD_XR, ATOMIC_SWAPW, ATOMIC_SWAP_32, ATOMIC_SWAP_64, CDS, CDSG, CDSY, CS, CSG, CSST, CSY, LAA, LAAG, LAAL, LAALG, LAN, LANG, LAO, LAOG, LAX, LAXG, LPD, LPDG, LPQ, PLO, STPQ, TS.

Otherwise, there's still two things that should be fixed before this can go in: the TwoAddress fix, and the intrinsics flags.

The TwoAddress fix has been committed, but the intrinsics flags fixing seem to require some work so I am not sure if we really want to wait with these changes until that is done?

Once this is all ready, could you please re-create another diff for the attribute settings, just so we can verify that nothing surprising has crept in?

Sure, this should be an updated diff of the patch effects:

flagsdiff_master_dec4156 KBDownload

[jonpa]

lib/Target/SystemZ/SystemZInstrInfo.td

525–538

Yes - removed this one and a few more in SystemZInstrSystem.td with NFC.

[uweigand]

The TwoAddress fix has been committed, but the intrinsics flags fixing seem to require some work so I am not sure if we really want to wait with these changes until that is done?

The problems with the intrinsics flags were related to IntrNoMem, right? This only happens for a few of the changes, the majority were about IntrWriteMem IIRC. It should be possible to add at least those changes right away without problem. And that would already remove the need for many of the wrongly-added mayLoad = 1 settings in this patch ...

Otherwise, this all looks good to me now.

[jonpa]

The TwoAddress fix has been committed, but the intrinsics flags fixing seem to require some work so I am not sure if we really want to wait with these changes until that is done?

The problems with the intrinsics flags were related to IntrNoMem, right? This only happens for a few of the changes, the majority were about IntrWriteMem IIRC. It should be possible to add at least those changes right away without problem. And that would already remove the need for many of the wrongly-added mayLoad = 1 settings in this patch ...

Yes, the IntrHasSideEffects would be needed for IntrNoMem (tabort, sfpc, efpc).

Patch updated for the other cases by adding IntrWriteMem and removing mayLoad flags, with functional change:

MayLoad REMOVED: NTSTG, TBEGINC, VSTL, VSTRL, VSTRLR

[uweigand]

For consistency, the other variants of the tbegin intrinsic (int_s390_tbegin and int_s390_tbegin_nofloat) should likewise get the IntrWriteMem flag.

Otherwise, this LGTM now. Thanks!

[jonpa]

r319756

lib/Target/SystemZ/SystemZ.td

 include "SystemZInstrFormats.td"
 include "SystemZInstrInfo.td"
 include "SystemZInstrVector.td"
 include "SystemZInstrFP.td"
 include "SystemZInstrHFP.td"
 include "SystemZInstrDFP.td"
 include "SystemZInstrSystem.td"
 
-def SystemZInstrInfo : InstrInfo {}
+def SystemZInstrInfo : InstrInfo { let guessInstructionProperties = 0; }
 
 //===----------------------------------------------------------------------===//
 // Assembly parser
 //===----------------------------------------------------------------------===//
 
 def SystemZAsmParser : AsmParser {
   let ShouldEmitMatchRegisterName = 0;
 }

lib/Target/SystemZ/SystemZInstrDFP.td

 // The instructions in this file implement SystemZ decimal floating-point
 // arithmetic.  These instructions are inot currently used for code generation,
 // are provided for use with the assembler and disassembler only.  If LLVM
 // ever supports decimal floating-point types (_Decimal64 etc.), they can
 // also be used for code generation for those types.
 //
 //===----------------------------------------------------------------------===//
 
+// In general, most floating-point instructions (except for simple moves and
+// the like) should have the side-effects flag, since they can raise
+// exceptions.
+let hasSideEffects = 1 in {
+
 //===----------------------------------------------------------------------===//
 // Move instructions
 //===----------------------------------------------------------------------===//
 
 // Load and test.
 let Defs = [CC] in {
   def LTDTR : UnaryRRE<"ltdtr", 0xB3D6, null_frag, FP64,  FP64>;
   def LTXTR : UnaryRRE<"ltxtr", 0xB3DE, null_frag, FP128, FP128>;

 
 // Test Data Group.
 let Defs = [CC] in {
   def TDGET : TestRXE<"tdget", 0xED51, null_frag, FP32>;
   def TDGDT : TestRXE<"tdgdt", 0xED55, null_frag, FP64>;
   def TDGXT : TestRXE<"tdgxt", 0xED59, null_frag, FP128>;
 }
 
+}  // hasSideEffects = 1

lib/Target/SystemZ/SystemZInstrFP.td

 defm CondStoreF64 : CondStores<FP64, nonvolatile_store,
                                nonvolatile_load, bdxaddr20only>;
 
 //===----------------------------------------------------------------------===//
 // Move instructions
 //===----------------------------------------------------------------------===//
 
 // Load zero.
-let hasSideEffects = 0, isAsCheapAsAMove = 1, isMoveImm = 1 in {
+let isAsCheapAsAMove = 1, isMoveImm = 1 in {
   def LZER : InherentRRE<"lzer", 0xB374, FP32,  fpimm0>;
   def LZDR : InherentRRE<"lzdr", 0xB375, FP64,  fpimm0>;
   def LZXR : InherentRRE<"lzxr", 0xB376, FP128, fpimm0>;
 }
 
 // Moves between two floating-point registers.
-let hasSideEffects = 0 in {
-  def LER : UnaryRR <"ler", 0x38,   null_frag, FP32,  FP32>;
-  def LDR : UnaryRR <"ldr", 0x28,   null_frag, FP64,  FP64>;
-  def LXR : UnaryRRE<"lxr", 0xB365, null_frag, FP128, FP128>;
 
-  // For z13 we prefer LDR over LER to avoid partial register dependencies.
-  let isCodeGenOnly = 1 in
-    def LDR32 : UnaryRR<"ldr", 0x28, null_frag, FP32, FP32>;
-}
 
 // Moves between two floating-point registers that also set the condition
 // codes.
 let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0xF in {
   defm LTEBR : LoadAndTestRRE<"ltebr", 0xB302, FP32>;
   defm LTDBR : LoadAndTestRRE<"ltdbr", 0xB312, FP64>;
   defm LTXBR : LoadAndTestRRE<"ltxbr", 0xB342, FP128>;
 }

 // Load instructions
 //===----------------------------------------------------------------------===//
 
 let canFoldAsLoad = 1, SimpleBDXLoad = 1 in {
   defm LE : UnaryRXPair<"le", 0x78, 0xED64, load, FP32, 4>;
   defm LD : UnaryRXPair<"ld", 0x68, 0xED65, load, FP64, 8>;
 
   // For z13 we prefer LDE over LE to avoid partial register dependencies.
-  let isCodeGenOnly = 1 in
+  let isCodeGenOnly = 1, hasSideEffects = 1 in
     def LDE32 : UnaryRXE<"lde", 0xED24, null_frag, FP32, 4>;
 
   // These instructions are split after register allocation, so we don't
   // want a custom inserter.
-  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1 in {
+  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1, mayLoad = 1 in {
     def LX : Pseudo<(outs FP128:$dst), (ins bdxaddr20only128:$src),
                      [(set FP128:$dst, (load bdxaddr20only128:$src))]>;
   }
 }
 
 //===----------------------------------------------------------------------===//
 // Store instructions
 //===----------------------------------------------------------------------===//
 
 let SimpleBDXStore = 1 in {
   defm STE : StoreRXPair<"ste", 0x70, 0xED66, store, FP32, 4>;
   defm STD : StoreRXPair<"std", 0x60, 0xED67, store, FP64, 8>;
 
   // These instructions are split after register allocation, so we don't
   // want a custom inserter.
-  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1 in {
+  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1, mayStore = 1 in {
     def STX : Pseudo<(outs), (ins FP128:$src, bdxaddr20only128:$dst),
                      [(store FP128:$src, bdxaddr20only128:$dst)]>;
   }
 }
 
 //===----------------------------------------------------------------------===//
 // Conversion instructions
 //===----------------------------------------------------------------------===//
 
 // Convert floating-point values to narrower representations, rounding
 // according to the current mode.  The destination of LEXBR and LDXBR
 // is a 128-bit value, but only the first register of the pair is used.
 def LEDBR : UnaryRRE<"ledbr", 0xB344, fpround,    FP32,  FP64>;
+
+let hasSideEffects = 1 in {
   def LEXBR : UnaryRRE<"lexbr", 0xB346, null_frag, FP128, FP128>;
   def LDXBR : UnaryRRE<"ldxbr", 0xB345, null_frag, FP128, FP128>;
 
   def LEDBRA : TernaryRRFe<"ledbra", 0xB344, FP32,  FP64>,
                Requires<[FeatureFPExtension]>;
   def LEXBRA : TernaryRRFe<"lexbra", 0xB346, FP128, FP128>,
                Requires<[FeatureFPExtension]>;
   def LDXBRA : TernaryRRFe<"ldxbra", 0xB345, FP128, FP128>,
                Requires<[FeatureFPExtension]>;
+}
 
 let Predicates = [FeatureNoVectorEnhancements1] in {
   def : Pat<(f32 (fpround FP128:$src)),
             (EXTRACT_SUBREG (LEXBR FP128:$src), subreg_hr32)>;
   def : Pat<(f64 (fpround FP128:$src)),
             (EXTRACT_SUBREG (LDXBR FP128:$src), subreg_h64)>;
 }
 

 def CXFBR : UnaryRRE<"cxfbr", 0xB396, sint_to_fp, FP128, GR32>;
 
 def CEGBR : UnaryRRE<"cegbr", 0xB3A4, sint_to_fp, FP32,  GR64>;
 def CDGBR : UnaryRRE<"cdgbr", 0xB3A5, sint_to_fp, FP64,  GR64>;
 def CXGBR : UnaryRRE<"cxgbr", 0xB3A6, sint_to_fp, FP128, GR64>;
 
 // The FP extension feature provides versions of the above that allow
 // specifying rounding mode and inexact-exception suppression flags.
-let Predicates = [FeatureFPExtension] in {
+let Predicates = [FeatureFPExtension], hasSideEffects = 1 in {
   def CEFBRA : TernaryRRFe<"cefbra", 0xB394, FP32,  GR32>;
   def CDFBRA : TernaryRRFe<"cdfbra", 0xB395, FP64,  GR32>;
   def CXFBRA : TernaryRRFe<"cxfbra", 0xB396, FP128, GR32>;
 
   def CEGBRA : TernaryRRFe<"cegbra", 0xB3A4, FP32,  GR64>;
   def CDGBRA : TernaryRRFe<"cdgbra", 0xB3A5, FP64,  GR64>;
   def CXGBRA : TernaryRRFe<"cxgbra", 0xB3A6, FP128, GR64>;
 }

 def : Pat<(i32 (fp_to_sint FP128:$src)), (CFXBR 5, FP128:$src)>;
 
 def : Pat<(i64 (fp_to_sint FP32:$src)),  (CGEBR 5, FP32:$src)>;
 def : Pat<(i64 (fp_to_sint FP64:$src)),  (CGDBR 5, FP64:$src)>;
 def : Pat<(i64 (fp_to_sint FP128:$src)), (CGXBR 5, FP128:$src)>;
 
 // The FP extension feature provides versions of the above that allow
 // also specifying the inexact-exception suppression flag.
-let Predicates = [FeatureFPExtension], Defs = [CC] in {
+let Predicates = [FeatureFPExtension], Defs = [CC], hasSideEffects = 1 in {
   def CFEBRA : TernaryRRFe<"cfebra", 0xB398, GR32, FP32>;
   def CFDBRA : TernaryRRFe<"cfdbra", 0xB399, GR32, FP64>;
   def CFXBRA : TernaryRRFe<"cfxbra", 0xB39A, GR32, FP128>;
 
   def CGEBRA : TernaryRRFe<"cgebra", 0xB3A8, GR64, FP32>;
   def CGDBRA : TernaryRRFe<"cgdbra", 0xB3A9, GR64, FP64>;
   def CGXBRA : TernaryRRFe<"cgxbra", 0xB3AA, GR64, FP128>;
 }

 // the CC is actually useful, the SystemZElimCompare pass will try to
 // convert generic instructions into opcodes that also set CC. Note
 // that lcdf / lpdf / lndf only affect the sign bit, and can therefore
 // be used with fp32 as well. This could be done for fp128, in which
 // case the operands would have to be tied.
 
 // Negation (Load Complement).
 let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0xF in {
+  let hasSideEffects = 1 in {
     def LCEBR : UnaryRRE<"lcebr", 0xB303, null_frag, FP32,  FP32>;
     def LCDBR : UnaryRRE<"lcdbr", 0xB313, null_frag, FP64,  FP64>;
+  }
   def LCXBR : UnaryRRE<"lcxbr", 0xB343, fneg, FP128, FP128>;
 }
 // Generic form, which does not set CC.
 def LCDFR : UnaryRRE<"lcdfr", 0xB373, fneg, FP64,  FP64>;
 let isCodeGenOnly = 1 in
   def LCDFR_32 : UnaryRRE<"lcdfr", 0xB373, fneg, FP32,  FP32>;
 
 // Absolute value (Load Positive).
 let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0xF in {
+  let hasSideEffects = 1 in {
     def LPEBR : UnaryRRE<"lpebr", 0xB300, null_frag, FP32,  FP32>;
     def LPDBR : UnaryRRE<"lpdbr", 0xB310, null_frag, FP64,  FP64>;
+  }
   def LPXBR : UnaryRRE<"lpxbr", 0xB340, fabs, FP128, FP128>;
 }
 // Generic form, which does not set CC.
 def LPDFR : UnaryRRE<"lpdfr", 0xB370, fabs, FP64,  FP64>;
 let isCodeGenOnly = 1 in
   def LPDFR_32 : UnaryRRE<"lpdfr", 0xB370, fabs, FP32,  FP32>;
 
 // Negative absolute value (Load Negative).
 let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0xF in {
+  let hasSideEffects = 1 in {
     def LNEBR : UnaryRRE<"lnebr", 0xB301, null_frag, FP32,  FP32>;
     def LNDBR : UnaryRRE<"lndbr", 0xB311, null_frag, FP64,  FP64>;
+  }
   def LNXBR : UnaryRRE<"lnxbr", 0xB341, fnabs, FP128, FP128>;
 }
 // Generic form, which does not set CC.
 def LNDFR : UnaryRRE<"lndfr", 0xB371, fnabs, FP64,  FP64>;
 let isCodeGenOnly = 1 in
   def LNDFR_32 : UnaryRRE<"lndfr", 0xB371, fnabs, FP32,  FP32>;
 
 // Square root.
let isCommutable = 1 in {
   def MEEBR : BinaryRRE<"meebr", 0xB317, fmul, FP32,  FP32>;
   def MDBR  : BinaryRRE<"mdbr",  0xB31C, fmul, FP64,  FP64>;
   def MXBR  : BinaryRRE<"mxbr",  0xB34C, fmul, FP128, FP128>;
 }
 def MEEB : BinaryRXE<"meeb", 0xED17, fmul, FP32, load, 4>;
 def MDB  : BinaryRXE<"mdb",  0xED1C, fmul, FP64, load, 8>;
 
 // f64 multiplication of two FP32 registers.
+let hasSideEffects = 1 in
   def MDEBR : BinaryRRE<"mdebr", 0xB30C, null_frag, FP64, FP32>;
 def : Pat<(fmul (f64 (fpextend FP32:$src1)), (f64 (fpextend FP32:$src2))),
           (MDEBR (INSERT_SUBREG (f64 (IMPLICIT_DEF)),
                                 FP32:$src1, subreg_r32), FP32:$src2)>;
 
 // f64 multiplication of an FP32 register and an f32 memory.
+let hasSideEffects = 1 in
   def MDEB : BinaryRXE<"mdeb", 0xED0C, null_frag, FP64, load, 4>;
 def : Pat<(fmul (f64 (fpextend FP32:$src1)),
                 (f64 (extloadf32 bdxaddr12only:$addr))),
           (MDEB (INSERT_SUBREG (f64 (IMPLICIT_DEF)), FP32:$src1, subreg_r32),
                 bdxaddr12only:$addr)>;
 
 // f128 multiplication of two FP64 registers.
+let hasSideEffects = 1 in
   def MXDBR : BinaryRRE<"mxdbr", 0xB307, null_frag, FP128, FP64>;
 let Predicates = [FeatureNoVectorEnhancements1] in
   def : Pat<(fmul (f128 (fpextend FP64:$src1)), (f128 (fpextend FP64:$src2))),
             (MXDBR (INSERT_SUBREG (f128 (IMPLICIT_DEF)),
                                   FP64:$src1, subreg_h64), FP64:$src2)>;
 
 // f128 multiplication of an FP64 register and an f64 memory.
+let hasSideEffects = 1 in
   def MXDB : BinaryRXE<"mxdb", 0xED07, null_frag, FP128, load, 8>;
 let Predicates = [FeatureNoVectorEnhancements1] in
   def : Pat<(fmul (f128 (fpextend FP64:$src1)),
                   (f128 (extloadf64 bdxaddr12only:$addr))),
             (MXDB (INSERT_SUBREG (f128 (IMPLICIT_DEF)), FP64:$src1, subreg_h64),
                   bdxaddr12only:$addr)>;
 
 // Fused multiply-add.
 def MAEBR : TernaryRRD<"maebr", 0xB30E, z_fma, FP32, FP32>;

 def DEBR : BinaryRRE<"debr", 0xB30D, fdiv, FP32,  FP32>;
 def DDBR : BinaryRRE<"ddbr", 0xB31D, fdiv, FP64,  FP64>;
 def DXBR : BinaryRRE<"dxbr", 0xB34D, fdiv, FP128, FP128>;
 
 def DEB : BinaryRXE<"deb", 0xED0D, fdiv, FP32, load, 4>;
 def DDB : BinaryRXE<"ddb", 0xED1D, fdiv, FP64, load, 8>;
 
 // Divide to integer.
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   def DIEBR : TernaryRRFb<"diebr", 0xB353, FP32, FP32, FP32>;
   def DIDBR : TernaryRRFb<"didbr", 0xB35B, FP64, FP64, FP64>;
 }
 
 //===----------------------------------------------------------------------===//
 // Comparisons
 //===----------------------------------------------------------------------===//
 
 let Defs = [CC], CCValues = 0xF in {
   def CEBR : CompareRRE<"cebr", 0xB309, z_fcmp, FP32,  FP32>;
   def CDBR : CompareRRE<"cdbr", 0xB319, z_fcmp, FP64,  FP64>;
   def CXBR : CompareRRE<"cxbr", 0xB349, z_fcmp, FP128, FP128>;
 
   def CEB : CompareRXE<"ceb", 0xED09, z_fcmp, FP32, load, 4>;
   def CDB : CompareRXE<"cdb", 0xED19, z_fcmp, FP64, load, 8>;
 
+  let hasSideEffects = 1 in {
     def KEBR : CompareRRE<"kebr", 0xB308, null_frag, FP32,  FP32>;
     def KDBR : CompareRRE<"kdbr", 0xB318, null_frag, FP64,  FP64>;
     def KXBR : CompareRRE<"kxbr", 0xB348, null_frag, FP128, FP128>;
 
     def KEB : CompareRXE<"keb", 0xED08, null_frag, FP32, load, 4>;
     def KDB : CompareRXE<"kdb", 0xED18, null_frag, FP64, load, 8>;
   }
+}
 
 // Test Data Class.
 let Defs = [CC], CCValues = 0xC in {
   def TCEB : TestRXE<"tceb", 0xED10, z_tdc, FP32>;
   def TCDB : TestRXE<"tcdb", 0xED11, z_tdc, FP64>;
   def TCXB : TestRXE<"tcxb", 0xED12, z_tdc, FP128>;
 }
 
 //===----------------------------------------------------------------------===//
 // Floating-point control register instructions
 //===----------------------------------------------------------------------===//
 
 let hasSideEffects = 1 in {
+  let mayLoad = 1, mayStore = 1 in {
     def EFPC  : InherentRRE<"efpc", 0xB38C, GR32, int_s390_efpc>;
     def STFPC : StoreInherentS<"stfpc", 0xB29C, storei<int_s390_efpc>, 4>;
 
     def SFPC : SideEffectUnaryRRE<"sfpc", 0xB384, GR32, int_s390_sfpc>;
     def LFPC : SideEffectUnaryS<"lfpc", 0xB29D, loadu<int_s390_sfpc>, 4>;
+  }
 
   def SFASR : SideEffectUnaryRRE<"sfasr", 0xB385, GR32, null_frag>;
   def LFAS  : SideEffectUnaryS<"lfas", 0xB2BD, null_frag, 4>;
 
   def SRNMB : SideEffectAddressS<"srnmb", 0xB2B8, null_frag, shift12only>,
               Requires<[FeatureFPExtension]>;
   def SRNM  : SideEffectAddressS<"srnm", 0xB299, null_frag, shift12only>;
   def SRNMT : SideEffectAddressS<"srnmt", 0xB2B9, null_frag, shift12only>;

lib/Target/SystemZ/SystemZInstrFormats.td

class InstSystemZ<int size, dag outs, dag ins, string asmstr,
   let Namespace = "SystemZ";
 
   dag OutOperandList = outs;
   dag InOperandList = ins;
   let Size = size;
   let Pattern = pattern;
   let AsmString = asmstr;
 
+  let hasSideEffects = 0;
+  let mayLoad = 0;
+  let mayStore = 0;
+
   // Some instructions come in pairs, one having a 12-bit displacement
   // and the other having a 20-bit displacement.  Both instructions in
   // the pair have the same DispKey and their DispSizes are "12" and "20"
   // respectively.
   string DispKey = "";
   string DispSize = "none";
 
   // Many register-based <INSN>R instructions have a memory-based <INSN>
class StoreInherentS<string mnemonic, bits<16> opcode,
                      SDPatternOperator operator, bits<5> bytes>
   : InstS<opcode, (outs), (ins bdaddr12only:$BD2),
           mnemonic#"\t$BD2", [(operator bdaddr12only:$BD2)]> {
   let mayStore = 1;
   let AccessBytes = bytes;
 }
 
 class SideEffectInherentE<string mnemonic, bits<16>opcode>
-  : InstE<opcode, (outs), (ins), mnemonic, []>;
+  : InstE<opcode, (outs), (ins), mnemonic, []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectInherentS<string mnemonic, bits<16> opcode,
                           SDPatternOperator operator>
   : InstS<opcode, (outs), (ins), mnemonic, [(operator)]> {
   let BD2 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectInherentRRE<string mnemonic, bits<16> opcode>
   : InstRRE<opcode, (outs), (ins), mnemonic, []> {
   let R1 = 0;
   let R2 = 0;
+  let hasSideEffects = 1;
 }
 
 // Allow an optional TLS marker symbol to generate TLS call relocations.
 class CallRI<string mnemonic, bits<12> opcode>
   : InstRIb<opcode, (outs), (ins GR64:$R1, brtarget16tls:$RI2),
             mnemonic#"\t$R1, $RI2", []>;
 
 // Allow an optional TLS marker symbol to generate TLS call relocations.
multiclass CondStoreRSYPair<string mnemonic, bits<16> opcode,
                             AddressingMode mode = bdaddr20only> {
   let isCodeGenOnly = 1 in
     def "" : CondStoreRSY<mnemonic, opcode, cls, bytes, mode>;
   def Asm : AsmCondStoreRSY<mnemonic, opcode, cls, bytes, mode>;
 }
 
 class SideEffectUnaryI<string mnemonic, bits<8> opcode, Immediate imm>
   : InstI<opcode, (outs), (ins imm:$I1),
-          mnemonic#"\t$I1", []>;
+          mnemonic#"\t$I1", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectUnaryRR<string mnemonic, bits<8>opcode, RegisterOperand cls>
   : InstRR<opcode, (outs), (ins cls:$R1),
            mnemonic#"\t$R1", []> {
   let R2 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectUnaryRRE<string mnemonic, bits<16> opcode, RegisterOperand cls,
                          SDPatternOperator operator>
   : InstRRE<opcode, (outs), (ins cls:$R1),
             mnemonic#"\t$R1", [(operator cls:$R1)]> {
   let R2 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectUnaryS<string mnemonic, bits<16> opcode,
                        SDPatternOperator operator, bits<5> bytes,
                        AddressingMode mode = bdaddr12only>
   : InstS<opcode, (outs), (ins mode:$BD2),
           mnemonic#"\t$BD2", [(operator mode:$BD2)]> {
   let mayLoad = 1;
   let AccessBytes = bytes;
+  let hasSideEffects = 1;
 }
 
 class SideEffectAddressS<string mnemonic, bits<16> opcode,
                         SDPatternOperator operator,
                         AddressingMode mode = bdaddr12only>
   : InstS<opcode, (outs), (ins mode:$BD2),
-          mnemonic#"\t$BD2", [(operator mode:$BD2)]>;
+          mnemonic#"\t$BD2", [(operator mode:$BD2)]> {
+  let hasSideEffects = 1;
+}
 
 class LoadAddressRX<string mnemonic, bits<8> opcode,
                     SDPatternOperator operator, AddressingMode mode>
   : InstRXa<opcode, (outs GR64:$R1), (ins mode:$XBD2),
             mnemonic#"\t$R1, $XBD2",
             [(set GR64:$R1, (operator mode:$XBD2))]>;
 
 class LoadAddressRXY<string mnemonic, bits<16> opcode,

 }
 
 multiclass CondUnaryRSYPair<string mnemonic, bits<16> opcode,
                             SDPatternOperator operator,
                             RegisterOperand cls, bits<5> bytes,
                             AddressingMode mode = bdaddr20only> {
   let isCodeGenOnly = 1 in
     def "" : CondUnaryRSY<mnemonic, opcode, operator, cls, bytes, mode>;
+  let hasSideEffects = 1 in
     def Asm : AsmCondUnaryRSY<mnemonic, opcode, cls, bytes, mode>;
 }
 
 
 class UnaryRX<string mnemonic, bits<8> opcode, SDPatternOperator operator,
               RegisterOperand cls, bits<5> bytes,
               AddressingMode mode = bdxaddr12only>
   : InstRXa<opcode, (outs cls:$R1), (ins mode:$XBD2),
             mnemonic#"\t$R1, $XBD2",
multiclass UnaryExtraVRRaSPair<string mnemonic, bits<16> opcode,
   def : InstAlias<mnemonic#"\t$V1, $V2",
                   (!cast<Instruction>(NAME) tr1.op:$V1, tr2.op:$V2, 0)>;
   let Defs = [CC] in
     def S : UnaryVRRa<mnemonic##"s", opcode, operator_cc, tr1, tr2,
                       type, 0, 1>;
 }
 
 multiclass UnaryExtraVRRaSPairGeneric<string mnemonic, bits<16> opcode> {
-  let M4 = 0 in
+  let M4 = 0, hasSideEffects = 1 in
     def "" : InstVRRa<opcode, (outs VR128:$V1),
                      (ins VR128:$V2, imm32zx4:$M3, imm32zx4:$M5),
                      mnemonic#"\t$V1, $V2, $M3, $M5", []>;
   def : InstAlias<mnemonic#"\t$V1, $V2, $M3",
                   (!cast<Instruction>(NAME) VR128:$V1, VR128:$V2,
                                             imm32zx4:$M3, 0)>;
 }
 
class UnaryVRXGeneric<string mnemonic, bits<16> opcode>
   : InstVRX<opcode, (outs VR128:$V1), (ins bdxaddr12only:$XBD2, imm32zx4:$M3),
             mnemonic#"\t$V1, $XBD2, $M3", []> {
   let mayLoad = 1;
 }
 
 class SideEffectBinaryRX<string mnemonic, bits<8> opcode,
                          RegisterOperand cls>
   : InstRXa<opcode, (outs), (ins cls:$R1, bdxaddr12only:$XBD2),
-            mnemonic##"\t$R1, $XBD2", []>;
+            mnemonic##"\t$R1, $XBD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinaryRXY<string mnemonic, bits<16> opcode,
                           RegisterOperand cls>
   : InstRXYa<opcode, (outs), (ins cls:$R1, bdxaddr20only:$XBD2),
-             mnemonic##"\t$R1, $XBD2", []>;
+             mnemonic##"\t$R1, $XBD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinaryRILPC<string mnemonic, bits<12> opcode,
                             RegisterOperand cls>
   : InstRILb<opcode, (outs), (ins cls:$R1, pcrel32:$RI2),
              mnemonic##"\t$R1, $RI2", []> {
   // We want PC-relative addresses to be tried ahead of BD and BDX addresses.
   // However, BDXs have two extra operands and are therefore 6 units more
   // complex.
   let AddedComplexity = 7;
+  let hasSideEffects = 1;
 }
 
 class SideEffectBinaryRRE<string mnemonic, bits<16> opcode,
                           RegisterOperand cls1, RegisterOperand cls2>
   : InstRRE<opcode, (outs), (ins cls1:$R1, cls2:$R2),
-            mnemonic#"\t$R1, $R2", []>;
+            mnemonic#"\t$R1, $R2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinaryRRFa<string mnemonic, bits<16> opcode,
                            RegisterOperand cls1, RegisterOperand cls2>
   : InstRRFa<opcode, (outs), (ins cls1:$R1, cls2:$R2),
              mnemonic#"\t$R1, $R2", []> {
   let R3 = 0;
   let M4 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectBinaryRRFc<string mnemonic, bits<16> opcode,
                            RegisterOperand cls1, RegisterOperand cls2>
   : InstRRFc<opcode, (outs), (ins cls1:$R1, cls2:$R2),
              mnemonic#"\t$R1, $R2", []> {
   let M3 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectBinaryIE<string mnemonic, bits<16> opcode,
                          Immediate imm1, Immediate imm2>
   : InstIE<opcode, (outs), (ins imm1:$I1, imm2:$I2),
-           mnemonic#"\t$I1, $I2", []>;
+           mnemonic#"\t$I1, $I2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySI<string mnemonic, bits<8> opcode, Operand imm>
   : InstSI<opcode, (outs), (ins bdaddr12only:$BD1, imm:$I2),
-           mnemonic#"\t$BD1, $I2", []>;
+           mnemonic#"\t$BD1, $I2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySIL<string mnemonic, bits<16> opcode,
                           SDPatternOperator operator, Immediate imm>
   : InstSIL<opcode, (outs), (ins bdaddr12only:$BD1, imm:$I2),
-            mnemonic#"\t$BD1, $I2", [(operator bdaddr12only:$BD1, imm:$I2)]>;
+            mnemonic#"\t$BD1, $I2", [(operator bdaddr12only:$BD1, imm:$I2)]> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySSa<string mnemonic, bits<8> opcode>
   : InstSSa<opcode, (outs), (ins bdladdr12onlylen8:$BDL1, bdaddr12only:$BD2),
-            mnemonic##"\t$BDL1, $BD2", []>;
+            mnemonic##"\t$BDL1, $BD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySSb<string mnemonic, bits<8> opcode>
   : InstSSb<opcode,
             (outs), (ins bdladdr12onlylen4:$BDL1, bdladdr12onlylen4:$BDL2),
-            mnemonic##"\t$BDL1, $BDL2", []>;
+            mnemonic##"\t$BDL1, $BDL2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySSf<string mnemonic, bits<8> opcode>
   : InstSSf<opcode, (outs), (ins bdaddr12only:$BD1, bdladdr12onlylen8:$BDL2),
-            mnemonic##"\t$BD1, $BDL2", []>;
+            mnemonic##"\t$BD1, $BDL2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinarySSE<string mnemonic, bits<16> opcode>
   : InstSSE<opcode, (outs), (ins bdaddr12only:$BD1, bdaddr12only:$BD2),
-            mnemonic#"\t$BD1, $BD2", []>;
+            mnemonic#"\t$BD1, $BD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectBinaryMemMemRR<string mnemonic, bits<8> opcode,
                                RegisterOperand cls1, RegisterOperand cls2>
   : InstRR<opcode, (outs cls1:$R1, cls2:$R2), (ins cls1:$R1src, cls2:$R2src),
            mnemonic#"\t$R1, $R2", []> {
     let Constraints = "$R1 = $R1src, $R2 = $R2src";
     let DisableEncoding = "$R1src, $R2src";
+    let hasSideEffects = 1;
 }
 
 class SideEffectBinaryMemRRE<string mnemonic, bits<16> opcode,
                              RegisterOperand cls1, RegisterOperand cls2>
   : InstRRE<opcode, (outs cls2:$R2), (ins cls1:$R1, cls2:$R2src),
             mnemonic#"\t$R1, $R2", []> {
   let Constraints = "$R2 = $R2src";
   let DisableEncoding = "$R2src";
+  let hasSideEffects = 1;
 }
 
 class SideEffectBinaryMemMemRRE<string mnemonic, bits<16> opcode,
                                 RegisterOperand cls1, RegisterOperand cls2>
   : InstRRE<opcode, (outs cls1:$R1, cls2:$R2), (ins cls1:$R1src, cls2:$R2src),
             mnemonic#"\t$R1, $R2", []> {
     let Constraints = "$R1 = $R1src, $R2 = $R2src";
     let DisableEncoding = "$R1src, $R2src";
+    let hasSideEffects = 1;
 }
 
 class SideEffectBinaryMemMemRRFc<string mnemonic, bits<16> opcode,
                                  RegisterOperand cls1, RegisterOperand cls2>
   : InstRRFc<opcode, (outs cls1:$R1, cls2:$R2), (ins cls1:$R1src, cls2:$R2src),
              mnemonic#"\t$R1, $R2", []> {
   let Constraints = "$R1 = $R1src, $R2 = $R2src";
   let DisableEncoding = "$R1src, $R2src";
   let M3 = 0;
+    let hasSideEffects = 1;
 }
 
 class BinaryRR<string mnemonic, bits<8> opcode, SDPatternOperator operator,
                RegisterOperand cls1, RegisterOperand cls2>
   : InstRR<opcode, (outs cls1:$R1), (ins cls1:$R1src, cls2:$R2),
            mnemonic#"\t$R1, $R2",
            [(set cls1:$R1, (operator cls1:$R1src, cls2:$R2))]> {
   let OpKey = mnemonic#cls1;
: InstRRFa<opcode, (outs cls1:$R1), (ins cls2:$R2, cls3:$R3),
              [(set cls1:$R1, (operator cls2:$R2, cls3:$R3))]> {
   let M4 = 0;
 }
 
 multiclass BinaryRRAndK<string mnemonic, bits<8> opcode1, bits<16> opcode2,
                         SDPatternOperator operator, RegisterOperand cls1,
                         RegisterOperand cls2> {
   let NumOpsKey = mnemonic in {
-    let NumOpsValue = "3" in
+    let NumOpsValue = "3", hasSideEffects = 1 in
       def K : BinaryRRFa<mnemonic#"k", opcode2, null_frag, cls1, cls1, cls2>,
               Requires<[FeatureDistinctOps]>;
     let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
       def "" : BinaryRR<mnemonic, opcode1, operator, cls1, cls2>;
   }
 }
 
 multiclass BinaryRREAndK<string mnemonic, bits<16> opcode1, bits<16> opcode2,
                          SDPatternOperator operator, RegisterOperand cls1,
                          RegisterOperand cls2> {
   let NumOpsKey = mnemonic in {
-    let NumOpsValue = "3" in
+    let NumOpsValue = "3", hasSideEffects = 1 in
       def K : BinaryRRFa<mnemonic#"k", opcode2, null_frag, cls1, cls1, cls2>,
               Requires<[FeatureDistinctOps]>;
     let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
       def "" : BinaryRRE<mnemonic, opcode1, operator, cls1, cls2>;
   }
 }
 
 class BinaryRRFb<string mnemonic, bits<16> opcode, SDPatternOperator operator,
class BinaryRIE<string mnemonic, bits<16> opcode, SDPatternOperator operator,
   : InstRIEd<opcode, (outs cls:$R1), (ins cls:$R3, imm:$I2),
              mnemonic#"\t$R1, $R3, $I2",
              [(set cls:$R1, (operator cls:$R3, imm:$I2))]>;
 
 multiclass BinaryRIAndK<string mnemonic, bits<12> opcode1, bits<16> opcode2,
                         SDPatternOperator operator, RegisterOperand cls,
                         Immediate imm> {
   let NumOpsKey = mnemonic in {
-    let NumOpsValue = "3" in
+    let NumOpsValue = "3", hasSideEffects = 1 in
       def K : BinaryRIE<mnemonic##"k", opcode2, null_frag, cls, imm>,
               Requires<[FeatureDistinctOps]>;
     let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
       def "" : BinaryRI<mnemonic, opcode1, operator, cls, imm>;
   }
 }
 
 class CondBinaryRIE<string mnemonic, bits<16> opcode, RegisterOperand cls,
let NumOpsKey = mnemonic in {
     let NumOpsValue = "3" in
       def K  : BinaryRSY<mnemonic##"k", opcode2, null_frag, cls>,
                Requires<[FeatureDistinctOps]>;
     let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
       def "" : BinaryRS<mnemonic, opcode1, operator, cls>;
   }
 }
 
+multiclass BinaryRSAndK_KSideEff<string mnemonic, bits<8> opcode1, bits<16> opcode2,
+                                 SDPatternOperator operator, RegisterOperand cls> {
+  let NumOpsKey = mnemonic in {
+    let NumOpsValue = "3", hasSideEffects = 1 in
+      def K  : BinaryRSY<mnemonic##"k", opcode2, null_frag, cls>,
+               Requires<[FeatureDistinctOps]>;
+    let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
+      def "" : BinaryRS<mnemonic, opcode1, operator, cls>;
+  }
+}
+
 class BinaryRSL<string mnemonic, bits<16> opcode, RegisterOperand cls>
   : InstRSLb<opcode, (outs cls:$R1),
              (ins bdladdr12onlylen8:$BDL2, imm32zx4:$M3),
              mnemonic#"\t$R1, $BDL2, $M3", []> {
   let mayLoad = 1;
 }
 
 class BinaryRX<string mnemonic, bits<8> opcode, SDPatternOperator operator,
multiclass BinaryVRRbSPair<string mnemonic, bits<16> opcode,
   let Defs = [CC] in
     def S : BinaryVRRb<mnemonic##"s", opcode, operator_cc, tr1, tr2, type,
                        !add (!and (modifier, 14), 1)>;
 }
 
 class BinaryVRRbSPairGeneric<string mnemonic, bits<16> opcode>
   : InstVRRb<opcode, (outs VR128:$V1),
              (ins VR128:$V2, VR128:$V3, imm32zx4:$M4, imm32zx4:$M5),
-             mnemonic#"\t$V1, $V2, $V3, $M4, $M5", []>;
+             mnemonic#"\t$V1, $V2, $V3, $M4, $M5", []> {
+  let hasSideEffects = 1;
+}
 
 // Declare a pair of instructions, one which sets CC and one which doesn't.
 // The CC-setting form ends with "S" and sets the low bit of M5.
 // The form that does not set CC has an extra operand to optionally allow
 // specifying arbitrary M5 values in assembler.
 multiclass BinaryExtraVRRbSPair<string mnemonic, bits<16> opcode,
                                 SDPatternOperator operator,
                                 SDPatternOperator operator_cc,
                                 TypedReg tr1, TypedReg tr2, bits<4> type> {
   let M4 = type in
     def "" : InstVRRb<opcode, (outs tr1.op:$V1),
                       (ins tr2.op:$V2, tr2.op:$V3, imm32zx4:$M5),
                       mnemonic#"\t$V1, $V2, $V3, $M5", []>;
   def : Pat<(tr1.vt (operator (tr2.vt tr2.op:$V2), (tr2.vt tr2.op:$V3))),
             (!cast<Instruction>(NAME) tr2.op:$V2, tr2.op:$V3, 0)>;
   def : InstAlias<mnemonic#"\t$V1, $V2, $V3",
                   (!cast<Instruction>(NAME) tr1.op:$V1, tr2.op:$V2,
                                             tr2.op:$V3, 0)>;
   let Defs = [CC] in
     def S : BinaryVRRb<mnemonic##"s", opcode, operator_cc, tr1, tr2, type, 1>;
 }
 
 multiclass BinaryExtraVRRbSPairGeneric<string mnemonic, bits<16> opcode> {
+  let hasSideEffects = 1 in
     def "" : InstVRRb<opcode, (outs VR128:$V1),
                      (ins VR128:$V2, VR128:$V3, imm32zx4:$M4, imm32zx4:$M5),
                      mnemonic#"\t$V1, $V2, $V3, $M4, $M5", []>;
   def : InstAlias<mnemonic#"\t$V1, $V2, $V3, $M4",
                   (!cast<Instruction>(NAME) VR128:$V1, VR128:$V2, VR128:$V3,
                                             imm32zx4:$M4, 0)>;
 }
 
 class BinaryVRRc<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                  TypedReg tr1, TypedReg tr2, bits<4> type = 0, bits<4> m5 = 0,
                  bits<4> m6 = 0>
   : InstVRRc<opcode, (outs tr1.op:$V1), (ins tr2.op:$V2, tr2.op:$V3),
              mnemonic#"\t$V1, $V2, $V3",
              [(set tr1.op:$V1, (tr1.vt (operator (tr2.vt tr2.op:$V2),
                                                  (tr2.vt tr2.op:$V3))))]> {
   let M4 = type;
   let M5 = m5;
   let M6 = m6;
 }
 
 class BinaryVRRcGeneric<string mnemonic, bits<16> opcode, bits<4> m5 = 0,
                         bits<4> m6 = 0>
   : InstVRRc<opcode, (outs VR128:$V1),
              (ins VR128:$V2, VR128:$V3, imm32zx4:$M4),
              mnemonic#"\t$V1, $V2, $V3, $M4", []> {
   let M5 = m5;
   let M6 = m6;
+  let hasSideEffects = 1;
 }
 
 class BinaryVRRcFloatGeneric<string mnemonic, bits<16> opcode, bits<4> m6 = 0>
   : InstVRRc<opcode, (outs VR128:$V1),
              (ins VR128:$V2, VR128:$V3, imm32zx4:$M4, imm32zx4:$M5),
              mnemonic#"\t$V1, $V2, $V3, $M4, $M5", []> {
   let M6 = m6;
+  let hasSideEffects = 1;
 }
 
 // Declare a pair of instructions, one which sets CC and one which doesn't.
 // The CC-setting form ends with "S" and sets the low bit of M5.
 multiclass BinaryVRRcSPair<string mnemonic, bits<16> opcode,
                            SDPatternOperator operator,
                            SDPatternOperator operator_cc, TypedReg tr1,
                            TypedReg tr2, bits<4> type, bits<4> m5,
                            bits<4> modifier = 0> {
   def "" : BinaryVRRc<mnemonic, opcode, operator, tr1, tr2, type,
                       m5, !and (modifier, 14)>;
   let Defs = [CC] in
     def S : BinaryVRRc<mnemonic##"s", opcode, operator_cc, tr1, tr2, type,
                        m5, !add (!and (modifier, 14), 1)>;
 }
 
 class BinaryVRRcSPairFloatGeneric<string mnemonic, bits<16> opcode>
   : InstVRRc<opcode, (outs VR128:$V1),
              (ins VR128:$V2, VR128:$V3, imm32zx4:$M4, imm32zx4:$M5,
                   imm32zx4:$M6),
-             mnemonic#"\t$V1, $V2, $V3, $M4, $M5, $M6", []>;
+             mnemonic#"\t$V1, $V2, $V3, $M4, $M5, $M6", []> {
+  let hasSideEffects = 1;
+}
 
 class BinaryVRRf<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                  TypedReg tr>
   : InstVRRf<opcode, (outs tr.op:$V1), (ins GR64:$R2, GR64:$R3),
              mnemonic#"\t$V1, $R2, $R3",
              [(set tr.op:$V1, (tr.vt (operator GR64:$R2, GR64:$R3)))]>;
 
 class BinaryVRRi<string mnemonic, bits<16> opcode, RegisterOperand cls>
   : InstVRRi<opcode, (outs cls:$R1), (ins VR128:$V2, imm32zx4:$M3),
              mnemonic#"\t$R1, $V2, $M3", []>;
 
 class BinaryVRSa<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                  TypedReg tr1, TypedReg tr2, bits<4> type>
   : InstVRSa<opcode, (outs tr1.op:$V1), (ins tr2.op:$V3, shift12only:$BD2),
              mnemonic#"\t$V1, $V3, $BD2",
              [(set tr1.op:$V1, (tr1.vt (operator (tr2.vt tr2.op:$V3),
                                                  shift12only:$BD2)))]> {
   let M4 = type;
 }
 
 class BinaryVRSaGeneric<string mnemonic, bits<16> opcode>
   : InstVRSa<opcode, (outs VR128:$V1),
              (ins VR128:$V3, shift12only:$BD2, imm32zx4:$M4),
-             mnemonic#"\t$V1, $V3, $BD2, $M4", []>;
+             mnemonic#"\t$V1, $V3, $BD2, $M4", []> {
+  let hasSideEffects = 1;
+}
 
 class BinaryVRSb<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                  bits<5> bytes>
   : InstVRSb<opcode, (outs VR128:$V1), (ins GR32:$R3, bdaddr12only:$BD2),
              mnemonic#"\t$V1, $R3, $BD2",
              [(set VR128:$V1, (operator GR32:$R3, bdaddr12only:$BD2))]> {
   let M4 = 0;
   let mayLoad = 1;

 
 class TestVRRg<string mnemonic, bits<16> opcode>
   : InstVRRg<opcode, (outs), (ins VR128:$V1),
              mnemonic#"\t$V1", []>;
 
 class SideEffectTernarySSc<string mnemonic, bits<8> opcode>
   : InstSSc<opcode, (outs), (ins bdladdr12onlylen4:$BDL1,
                                  shift12only:$BD2, imm32zx4:$I3),
-            mnemonic##"\t$BDL1, $BD2, $I3", []>;
+            mnemonic##"\t$BDL1, $BD2, $I3", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectTernaryRRFa<string mnemonic, bits<16> opcode,
                             RegisterOperand cls1, RegisterOperand cls2,
                             RegisterOperand cls3>
   : InstRRFa<opcode, (outs), (ins cls1:$R1, cls2:$R2, cls3:$R3),
              mnemonic#"\t$R1, $R2, $R3", []> {
   let M4 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectTernaryRRFb<string mnemonic, bits<16> opcode,
                             RegisterOperand cls1, RegisterOperand cls2,
                             RegisterOperand cls3>
   : InstRRFb<opcode, (outs), (ins cls1:$R1, cls2:$R2, cls3:$R3),
              mnemonic#"\t$R1, $R3, $R2", []> {
   let M4 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectTernaryMemMemMemRRFb<string mnemonic, bits<16> opcode,
                                      RegisterOperand cls1,
                                      RegisterOperand cls2,
                                      RegisterOperand cls3>
   : InstRRFb<opcode, (outs cls1:$R1, cls2:$R2, cls3:$R3),
              (ins cls1:$R1src, cls2:$R2src, cls3:$R3src),
              mnemonic#"\t$R1, $R3, $R2", []> {
   let Constraints = "$R1 = $R1src, $R2 = $R2src, $R3 = $R3src";
   let DisableEncoding = "$R1src, $R2src, $R3src";
   let M4 = 0;
+  let hasSideEffects = 1;
 }
 
 class SideEffectTernaryRRFc<string mnemonic, bits<16> opcode,
                             RegisterOperand cls1, RegisterOperand cls2,
                             Immediate imm>
   : InstRRFc<opcode, (outs), (ins cls1:$R1, cls2:$R2, imm:$M3),
-             mnemonic#"\t$R1, $R2, $M3", []>;
+             mnemonic#"\t$R1, $R2, $M3", []> {
+  let hasSideEffects = 1;
+}
 
 multiclass SideEffectTernaryRRFcOpt<string mnemonic, bits<16> opcode,
                                     RegisterOperand cls1,
                                     RegisterOperand cls2> {
   def "" : SideEffectTernaryRRFc<mnemonic, opcode, cls1, cls2, imm32zx4>;
   def Opt : SideEffectBinaryRRFc<mnemonic, opcode, cls1, cls2>;
 }
 
 class SideEffectTernaryMemMemRRFc<string mnemonic, bits<16> opcode,
                                   RegisterOperand cls1, RegisterOperand cls2,
                                   Immediate imm>
   : InstRRFc<opcode, (outs cls1:$R1, cls2:$R2),
              (ins cls1:$R1src, cls2:$R2src, imm:$M3),
              mnemonic#"\t$R1, $R2, $M3", []> {
   let Constraints = "$R1 = $R1src, $R2 = $R2src";
   let DisableEncoding = "$R1src, $R2src";
+  let hasSideEffects = 1;
 }
 
 multiclass SideEffectTernaryMemMemRRFcOpt<string mnemonic, bits<16> opcode,
                                           RegisterOperand cls1,
                                           RegisterOperand cls2> {
   def "" : SideEffectTernaryMemMemRRFc<mnemonic, opcode, cls1, cls2, imm32zx4>;
   def Opt : SideEffectBinaryMemMemRRFc<mnemonic, opcode, cls1, cls2>;
 }
 
 class SideEffectTernarySSF<string mnemonic, bits<12> opcode,
                            RegisterOperand cls>
   : InstSSF<opcode, (outs),
             (ins bdaddr12only:$BD1, bdaddr12only:$BD2, cls:$R3),
-            mnemonic#"\t$BD1, $BD2, $R3", []>;
+            mnemonic#"\t$BD1, $BD2, $R3", []> {
+  let hasSideEffects = 1;
+}
 
 class TernaryRRFa<string mnemonic, bits<16> opcode,
                  RegisterOperand cls1, RegisterOperand cls2,
                  RegisterOperand cls3>
   : InstRRFa<opcode, (outs cls1:$R1), (ins cls2:$R2, cls3:$R3, imm32zx4:$M4),
              mnemonic#"\t$R1, $R2, $R3, $M4", []>;
 
 class TernaryRRFb<string mnemonic, bits<16> opcode,
let DispSize = "20" in
       def Y  : TernaryRSY<mnemonic#"y", rsyOpcode, cls, bytes, bdaddr20pair>;
   }
 }
 
 class SideEffectTernaryRS<string mnemonic, bits<8> opcode,
                           RegisterOperand cls1, RegisterOperand cls2>
   : InstRSa<opcode, (outs),
             (ins cls1:$R1, cls2:$R3, bdaddr12only:$BD2),
-            mnemonic#"\t$R1, $R3, $BD2", []>;
+            mnemonic#"\t$R1, $R3, $BD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectTernaryRSY<string mnemonic, bits<16> opcode,
                            RegisterOperand cls1, RegisterOperand cls2>
   : InstRSYa<opcode, (outs),
              (ins cls1:$R1, cls2:$R3, bdaddr20only:$BD2),
-             mnemonic#"\t$R1, $R3, $BD2", []>;
+             mnemonic#"\t$R1, $R3, $BD2", []> {
+  let hasSideEffects = 1;
+}
 
 class SideEffectTernaryMemMemRS<string mnemonic, bits<8> opcode,
                                 RegisterOperand cls1, RegisterOperand cls2>
   : InstRSa<opcode, (outs cls1:$R1, cls2:$R3),
             (ins cls1:$R1src, cls2:$R3src, shift12only:$BD2),
             mnemonic#"\t$R1, $R3, $BD2", []> {
     let Constraints = "$R1 = $R1src, $R3 = $R3src";
     let DisableEncoding = "$R1src, $R3src";
+    let hasSideEffects = 1;
 }
 
 class SideEffectTernaryMemMemRSY<string mnemonic, bits<16> opcode,
                                  RegisterOperand cls1, RegisterOperand cls2>
   : InstRSYa<opcode, (outs cls1:$R1, cls2:$R3),
              (ins cls1:$R1src, cls2:$R3src, shift20only:$BD2),
              mnemonic#"\t$R1, $R3, $BD2", []> {
     let Constraints = "$R1 = $R1src, $R3 = $R3src";
     let DisableEncoding = "$R1src, $R3src";
+    let hasSideEffects = 1;
 }
 
 class TernaryRXF<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                  RegisterOperand cls1, RegisterOperand cls2,
                  SDPatternOperator load, bits<5> bytes>
   : InstRXF<opcode, (outs cls1:$R1),
             (ins cls2:$R1src, cls2:$R3, bdxaddr12only:$XBD2),
             mnemonic#"\t$R1, $R3, $XBD2",
: InstVRRa<opcode, (outs tr1.op:$V1),
                                                  imm32zx4:$M5)))],
              m4or> {
   let M3 = type;
 }
 
 class TernaryVRRaFloatGeneric<string mnemonic, bits<16> opcode>
   : InstVRRa<opcode, (outs VR128:$V1),
              (ins VR128:$V2, imm32zx4:$M3, imm32zx4:$M4, imm32zx4:$M5),
-             mnemonic#"\t$V1, $V2, $M3, $M4, $M5", []>;
+             mnemonic#"\t$V1, $V2, $M3, $M4, $M5", []> {
+  let hasSideEffects = 1;
+}
 
 class TernaryVRRb<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                   TypedReg tr1, TypedReg tr2, bits<4> type,
                   SDPatternOperator m5mask, bits<4> m5or>
   : InstVRRb<opcode, (outs tr1.op:$V1),
              (ins tr2.op:$V2, tr2.op:$V3, m5mask:$M5),
              mnemonic#"\t$V1, $V2, $V3, $M5",
              [(set tr1.op:$V1, (tr1.vt (operator (tr2.vt tr2.op:$V2),
let Defs = [CC] in
     def S : TernaryVRRb<mnemonic##"s", opcode, operator_cc, tr1, tr2, type,
                         imm32zx4even, !add(!and (modifier, 14), 1)>;
   def : InstAlias<mnemonic#"s\t$V1, $V2, $V3",
                   (!cast<Instruction>(NAME#"S") tr1.op:$V1, tr2.op:$V2,
                                                 tr2.op:$V3, 0)>;
 }
 
 multiclass TernaryOptVRRbSPairGeneric<string mnemonic, bits<16> opcode> {
+  let hasSideEffects = 1 in
     def "" : InstVRRb<opcode, (outs VR128:$V1),
                      (ins VR128:$V2, VR128:$V3, imm32zx4:$M4, imm32zx4:$M5),
                      mnemonic#"\t$V1, $V2, $V3, $M4, $M5", []>;
   def : InstAlias<mnemonic#"\t$V1, $V2, $V3, $M4",
                   (!cast<Instruction>(NAME) VR128:$V1, VR128:$V2, VR128:$V3,
                                             imm32zx4:$M4, 0)>;
 }
 
 class TernaryVRRc<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                   TypedReg tr1, TypedReg tr2>
   : InstVRRc<opcode, (outs tr1.op:$V1),
class TernaryVRRd<string mnemonic, bits<16> opcode, SDPatternOperator operator,
   let M6 = 0;
 }
 
 class TernaryVRRdGeneric<string mnemonic, bits<16> opcode>
   : InstVRRd<opcode, (outs VR128:$V1),
              (ins VR128:$V2, VR128:$V3, VR128:$V4, imm32zx4:$M5),
              mnemonic#"\t$V1, $V2, $V3, $V4, $M5", []> {
   let M6 = 0;
+  let hasSideEffects = 1;
 }
 
 class TernaryVRRe<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                   TypedReg tr1, TypedReg tr2, bits<4> m5 = 0, bits<4> type = 0>
   : InstVRRe<opcode, (outs tr1.op:$V1),
              (ins tr2.op:$V2, tr2.op:$V3, tr1.op:$V4),
              mnemonic#"\t$V1, $V2, $V3, $V4",
              [(set tr1.op:$V1, (tr1.vt (operator (tr2.vt tr2.op:$V2),
def S : QuaternaryVRRd<mnemonic##"s", opcode, operator_cc,
                            tr1, tr2, tr2, tr2, type,
                            imm32zx4even, !add (!and (modifier, 14), 1)>;
   def : InstAlias<mnemonic#"s\t$V1, $V2, $V3, $V4",
                   (!cast<Instruction>(NAME#"S") tr1.op:$V1, tr2.op:$V2,
                                                 tr2.op:$V3, tr2.op:$V4, 0)>;
 }
 
 multiclass QuaternaryOptVRRdSPairGeneric<string mnemonic, bits<16> opcode> {
+  let hasSideEffects = 1 in
     def "" : QuaternaryVRRdGeneric<mnemonic, opcode>;
   def : InstAlias<mnemonic#"\t$V1, $V2, $V3, $V4, $M5",
                   (!cast<Instruction>(NAME) VR128:$V1, VR128:$V2, VR128:$V3,
                                             VR128:$V4, imm32zx4:$M5, 0)>;
 }
 
 class SideEffectQuaternaryRRFa<string mnemonic, bits<16> opcode,
                                RegisterOperand cls1, RegisterOperand cls2,
                                RegisterOperand cls3>
   : InstRRFa<opcode, (outs), (ins cls1:$R1, cls2:$R2, cls3:$R3, imm32zx4:$M4),
-             mnemonic#"\t$R1, $R2, $R3, $M4", []>;
+             mnemonic#"\t$R1, $R2, $R3, $M4", []> {
+  let hasSideEffects = 1;
+}
 
 multiclass SideEffectQuaternaryRRFaOptOpt<string mnemonic, bits<16> opcode,
                                           RegisterOperand cls1,
                                           RegisterOperand cls2,
                                           RegisterOperand cls3> {
   def "" : SideEffectQuaternaryRRFa<mnemonic, opcode, cls1, cls2, cls3>;
   def Opt : SideEffectTernaryRRFa<mnemonic, opcode, cls1, cls2, cls3>;
   def OptOpt : SideEffectBinaryRRFa<mnemonic, opcode, cls1, cls2>;
 }
 
 class SideEffectQuaternaryRRFb<string mnemonic, bits<16> opcode,
                                RegisterOperand cls1, RegisterOperand cls2,
                                RegisterOperand cls3>
   : InstRRFb<opcode, (outs), (ins cls1:$R1, cls2:$R2, cls3:$R3, imm32zx4:$M4),
-             mnemonic#"\t$R1, $R3, $R2, $M4", []>;
+             mnemonic#"\t$R1, $R3, $R2, $M4", []> {
+  let hasSideEffects = 1;
+}
 
 multiclass SideEffectQuaternaryRRFbOpt<string mnemonic, bits<16> opcode,
                                        RegisterOperand cls1,
                                        RegisterOperand cls2,
                                        RegisterOperand cls3> {
   def "" : SideEffectQuaternaryRRFb<mnemonic, opcode, cls1, cls2, cls3>;
   def Opt : SideEffectTernaryRRFb<mnemonic, opcode, cls1, cls2, cls3>;
 }
 
 class SideEffectQuaternarySSe<string mnemonic, bits<8> opcode,
                               RegisterOperand cls>
   : InstSSe<opcode, (outs),
             (ins cls:$R1, bdaddr12only:$BD2, cls:$R3, bdaddr12only:$BD4),
-            mnemonic#"\t$R1, $BD2, $R3, $BD4", []>;
+            mnemonic#"\t$R1, $BD2, $R3, $BD4", []> {
+  let hasSideEffects = 1;
+}
 
 class LoadAndOpRSY<string mnemonic, bits<16> opcode, SDPatternOperator operator,
                   RegisterOperand cls, AddressingMode mode = bdaddr20only>
   : InstRSYa<opcode, (outs cls:$R1), (ins cls:$R3, mode:$BD2),
              mnemonic#"\t$R1, $R3, $BD2",
              [(set cls:$R1, (operator mode:$BD2, cls:$R3))]> {
   let mayLoad = 1;
   let mayStore = 1;

 
 // A floating-point load-and test operation.  Create both a normal unary
 // operation and one that acts as a comparison against zero.
 // Note that the comparison against zero operation is not available if we
 // have vector support, since load-and-test instructions will partially
 // clobber the target (vector) register.
 multiclass LoadAndTestRRE<string mnemonic, bits<16> opcode,
                           RegisterOperand cls> {
+  let hasSideEffects = 1 in
     def "" : UnaryRRE<mnemonic, opcode, null_frag, cls, cls>;
   let isCodeGenOnly = 1, Predicates = [FeatureNoVector] in
     def Compare : CompareRRE<mnemonic, opcode, null_frag, cls, cls>;
 }
 
 //===----------------------------------------------------------------------===//
 // Pseudo instructions
 //===----------------------------------------------------------------------===//
 //
 // Convenience instructions that get lowered to real instructions
 // by either SystemZTargetLowering::EmitInstrWithCustomInserter()
 // or SystemZInstrInfo::expandPostRAPseudo().
 //
 //===----------------------------------------------------------------------===//
 
 class Pseudo<dag outs, dag ins, list<dag> pattern>
   : InstSystemZ<0, outs, ins, "", pattern> {
   let isPseudo = 1;
   let isCodeGenOnly = 1;
 }
 
 // Like SideEffectBinarySIL, but expanded later.
 class SideEffectBinarySILPseudo<SDPatternOperator operator, Immediate imm>
   : Pseudo<(outs), (ins bdaddr12only:$BD1, imm:$I2),
-           [(operator bdaddr12only:$BD1, imm:$I2)]>;
+           [(operator bdaddr12only:$BD1, imm:$I2)]> {
+  let hasSideEffects = 1;
+}
 
 // Like UnaryRI, but expanded after RA depending on the choice of register.
 class UnaryRIPseudo<SDPatternOperator operator, RegisterOperand cls,
                     Immediate imm>
   : Pseudo<(outs cls:$R1), (ins imm:$I2),
            [(set cls:$R1, (operator imm:$I2))]>;
 
 // Like UnaryRXY, but expanded after RA depending on the choice of register.
class BinaryRIEPseudo<SDPatternOperator operator, RegisterOperand cls,
                       Immediate imm>
   : Pseudo<(outs cls:$R1), (ins cls:$R3, imm:$I2),
            [(set cls:$R1, (operator cls:$R3, imm:$I2))]>;
 
 // Like BinaryRIAndK, but expanded after RA depending on the choice of register.
 multiclass BinaryRIAndKPseudo<string key, SDPatternOperator operator,
                               RegisterOperand cls, Immediate imm> {
   let NumOpsKey = key in {
-    let NumOpsValue = "3" in
+    let NumOpsValue = "3", hasSideEffects = 1 in
       def K : BinaryRIEPseudo<null_frag, cls, imm>,
               Requires<[FeatureHighWord, FeatureDistinctOps]>;
     let NumOpsValue = "2", isConvertibleToThreeAddress = 1 in
       def "" : BinaryRIPseudo<operator, cls, imm>,
                Requires<[FeatureHighWord]>;
   }
 }
 
class SelectWrapper<ValueType vt, RegisterOperand cls>
   // across them.
   let Defs = [CC];
   let Uses = [CC];
 }
 
 // Stores $new to $addr if $cc is true ("" case) or false (Inv case).
 multiclass CondStores<RegisterOperand cls, SDPatternOperator store,
                       SDPatternOperator load, AddressingMode mode> {
-  let Defs = [CC], Uses = [CC], usesCustomInserter = 1 in {
+  let Defs = [CC], Uses = [CC], usesCustomInserter = 1,
+      mayLoad = 1, mayStore = 1 in {
     def "" : Pseudo<(outs),
                     (ins cls:$new, mode:$addr, imm32zx4:$valid, imm32zx4:$cc),
                     [(store (z_select_ccmask cls:$new, (load mode:$addr),
                                              imm32zx4:$valid, imm32zx4:$cc),
                             mode:$addr)]>;
     def Inv : Pseudo<(outs),
                      (ins cls:$new, mode:$addr, imm32zx4:$valid, imm32zx4:$cc),
                      [(store (z_select_ccmask (load mode:$addr), cls:$new,

lib/Target/SystemZ/SystemZInstrHFP.td

 //==- SystemZInstrHFP.td - Floating-point SystemZ instructions -*- tblgen-*-==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // The instructions in this file implement SystemZ hexadecimal floating-point
 // arithmetic.  Since this format is not mapped to any source-language data
 // type, these instructions are not used for code generation, but are provided
 // for use with the assembler and disassembler only.
 //
 //===----------------------------------------------------------------------===//
 
+// In general, most floating-point instructions (except for simple moves and
+// the like) should have the side-effects flag, since they can raise
+// exceptions.
+let hasSideEffects = 1 in {
+
 //===----------------------------------------------------------------------===//
 // Move instructions
 //===----------------------------------------------------------------------===//
 
 // Load and test.
 let Defs = [CC] in {
   def LTER : UnaryRR <"lter", 0x32,   null_frag, FP32,  FP32>;
   def LTDR : UnaryRR <"ltdr", 0x22,   null_frag, FP64,  FP64>;
let Defs = [CC] in {
   def CER : CompareRR <"cer", 0x39,   null_frag, FP32,  FP32>;
   def CDR : CompareRR <"cdr", 0x29,   null_frag, FP64,  FP64>;
   def CXR : CompareRRE<"cxr", 0xB369, null_frag, FP128, FP128>;
 
   def CE : CompareRX<"ce", 0x79, null_frag, FP32, load, 4>;
   def CD : CompareRX<"cd", 0x69, null_frag, FP64, load, 8>;
 }
 
+}  // hasSideEffects = 1

lib/Target/SystemZ/SystemZInstrInfo.td

 //===-- SystemZInstrInfo.td - General SystemZ instructions ----*- tblgen-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // Stack allocation
 //===----------------------------------------------------------------------===//
 
-let hasNoSchedulingInfo = 1 in {
+let hasNoSchedulingInfo = 1, hasSideEffects = 1 in {
   def ADJCALLSTACKDOWN : Pseudo<(outs), (ins i64imm:$amt1, i64imm:$amt2),
                                 [(callseq_start timm:$amt1, timm:$amt2)]>;
   def ADJCALLSTACKUP   : Pseudo<(outs), (ins i64imm:$amt1, i64imm:$amt2),
                                 [(callseq_end timm:$amt1, timm:$amt2)]>;
 }
 
-let hasSideEffects = 0 in {
-  // Takes as input the value of the stack pointer after a dynamic allocation
-  // has been made.  Sets the output to the address of the dynamically-
-  // allocated area itself, skipping the outgoing arguments.
-  //
-  // This expands to an LA or LAY instruction.  We restrict the offset
-  // to the range of LA and keep the LAY range in reserve for when
-  // the size of the outgoing arguments is added.
-  def ADJDYNALLOC : Pseudo<(outs GR64:$dst), (ins dynalloc12only:$src),
-                           [(set GR64:$dst, dynalloc12only:$src)]>;
-}
+
 
 //===----------------------------------------------------------------------===//
 // Branch instructions
 //===----------------------------------------------------------------------===//
 
 // Conditional branches.
-let isBranch = 1, isTerminator = 1, Uses = [CC] in {
+let isBranch = 1, isTerminator = 1, Uses = [CC], hasSideEffects = 1 in {
   // It's easier for LLVM to handle these branches in their raw BRC/BRCL form
   // with the condition-code mask being the first operand.  It seems friendlier
   // to use mnemonic forms like JE and JLH when writing out the assembly though.
   let isCodeGenOnly = 1 in {
     // An assembler extended mnemonic for BRC.
+    let hasSideEffects = 0 in
       def BRC  : CondBranchRI <"j#",  0xA74, z_br_ccmask>;
     // An assembler extended mnemonic for BRCL.  (The extension is "G"
     // rather than "L" because "JL" is "Jump if Less".)
     def BRCL : CondBranchRIL<"jg#", 0xC04>;
     let isIndirectBranch = 1 in {
       def BC  : CondBranchRX<"b#",  0x47>;
       def BCR : CondBranchRR<"b#r", 0x07>;
       def BIC : CondBranchRXY<"bi#", 0xe347>,
                 Requires<[FeatureMiscellaneousExtensions2]>;
let isBranch = 1, isTerminator = 1, Uses = [CC], hasSideEffects = 1 in {
   // (integer or floating-point)
   foreach V = [ "E", "NE", "H", "NH", "L", "NL", "HE", "NHE", "LE", "NLE",
                 "Z", "NZ", "P", "NP", "M", "NM", "LH", "NLH", "O", "NO" ] in {
     def JAsm#V  : FixedCondBranchRI <CV<V>, "j#",  0xA74>;
     def JGAsm#V : FixedCondBranchRIL<CV<V>, "jg#", 0xC04>;
     let isIndirectBranch = 1 in {
       def BAsm#V  : FixedCondBranchRX <CV<V>, "b#",  0x47>;
       def BRAsm#V : FixedCondBranchRR <CV<V>, "b#r", 0x07>;
+      let hasSideEffects = 1 in
         def BIAsm#V : FixedCondBranchRXY<CV<V>, "bi#", 0xe347>,
                       Requires<[FeatureMiscellaneousExtensions2]>;
     }
   }
 }
 
 // Unconditional branches.  These are in fact simply variants of the
 // conditional branches with the condition mask set to "always".
 let isBranch = 1, isTerminator = 1, isBarrier = 1 in {
   def J  : FixedCondBranchRI <CondAlways, "j",  0xA74, br>;
+  let hasSideEffects = 1 in
     def JG : FixedCondBranchRIL<CondAlways, "jg", 0xC04>;
   let isIndirectBranch = 1 in {
+    let hasSideEffects = 1 in
       def B  : FixedCondBranchRX<CondAlways, "b",  0x47>;
     def BR : FixedCondBranchRR<CondAlways, "br", 0x07, brind>;
+    let mayLoad = 1 in
       def BI : FixedCondBranchRXY<CondAlways, "bi", 0xe347, brind>,
                Requires<[FeatureMiscellaneousExtensions2]>;
   }
 }
 
 // NOPs.  These are again variants of the conditional branches,
 // with the condition mask set to "never".
 def NOP  : InstAlias<"nop\t$XBD", (BCAsm 0, bdxaddr12only:$XBD), 0>;
 def NOPR : InstAlias<"nopr\t$R", (BCRAsm 0, GR64:$R), 0>;
 
 // Fused compare-and-branch instructions.
 //
 // These instructions do not use or clobber the condition codes.
 // We nevertheless pretend that the relative compare-and-branch
 // instructions clobber CC, so that we can lower them to separate
 // comparisons and BRCLs if the branch ends up being out of range.
-let isBranch = 1, isTerminator = 1 in {
+let isBranch = 1, isTerminator = 1, hasSideEffects = 1 in {
   // As for normal branches, we handle these instructions internally in
   // their raw CRJ-like form, but use assembly macros like CRJE when writing
   // them out.  Using the *Pair multiclasses, we also create the raw forms.
   let Defs = [CC] in {
     defm CRJ   : CmpBranchRIEbPair<"crj",   0xEC76, GR32>;
     defm CGRJ  : CmpBranchRIEbPair<"cgrj",  0xEC64, GR64>;
     defm CIJ   : CmpBranchRIEcPair<"cij",   0xEC7E, GR32, imm32sx8>;
     defm CGIJ  : CmpBranchRIEcPair<"cgij",  0xEC7C, GR64, imm64sx8>;
let isIndirectBranch = 1 in {
       def CLGIBAsm#V : FixedCmpBranchRIS<ICV<V>, "clgib", 0xECFD, GR64,
                                          imm64zx8>;
     }
   }
 }
 
 // Decrement a register and branch if it is nonzero.  These don't clobber CC,
 // but we might need to split long relative branches into sequences that do.
-let isBranch = 1, isTerminator = 1 in {
+let isBranch = 1, isTerminator = 1, hasSideEffects = 1 in {
   let Defs = [CC] in {
     def BRCT  : BranchUnaryRI<"brct",  0xA76, GR32>;
     def BRCTG : BranchUnaryRI<"brctg", 0xA77, GR64>;
   }
   // This doesn't need to clobber CC since we never need to split it.
   def BRCTH : BranchUnaryRIL<"brcth", 0xCC6, GRH32>,
               Requires<[FeatureHighWord]>;
 
   def BCT   : BranchUnaryRX<"bct",  0x46,GR32>;
   def BCTR  : BranchUnaryRR<"bctr", 0x06, GR32>;
   def BCTG  : BranchUnaryRXY<"bctg",  0xE346, GR64>;
   def BCTGR : BranchUnaryRRE<"bctgr", 0xB946, GR64>;
 }
 
-let isBranch = 1, isTerminator = 1 in {
+let isBranch = 1, isTerminator = 1, hasSideEffects = 1 in {
   let Defs = [CC] in {
     def BRXH  : BranchBinaryRSI<"brxh",  0x84, GR32>;
     def BRXLE : BranchBinaryRSI<"brxle", 0x85, GR32>;
     def BRXHG : BranchBinaryRIEe<"brxhg", 0xEC44, GR64>;
     def BRXLG : BranchBinaryRIEe<"brxlg", 0xEC45, GR64>;
   }
   def BXH   : BranchBinaryRS<"bxh",  0x86, GR32>;
   def BXLE  : BranchBinaryRS<"bxle", 0x87, GR32>;
   def BXHG  : BranchBinaryRSY<"bxhg",  0xEB44, GR64>;
   def BXLEG : BranchBinaryRSY<"bxleg", 0xEB45, GR64>;
 }
 
 //===----------------------------------------------------------------------===//
 // Trap instructions
 //===----------------------------------------------------------------------===//
 
 // Unconditional trap.
-let hasCtrlDep = 1 in
+let hasCtrlDep = 1, hasSideEffects = 1 in
   def Trap : Alias<4, (outs), (ins), [(trap)]>;
 
 // Conditional trap.
-let hasCtrlDep = 1, Uses = [CC] in
+let hasCtrlDep = 1, Uses = [CC], hasSideEffects = 1 in
   def CondTrap : Alias<4, (outs), (ins cond4:$valid, cond4:$R1), []>;
 
 // Fused compare-and-trap instructions.
-let hasCtrlDep = 1 in {
+let hasCtrlDep = 1, hasSideEffects = 1 in {
   // These patterns work the same way as for compare-and-branch.
   defm CRT   : CmpBranchRRFcPair<"crt",   0xB972, GR32>;
   defm CGRT  : CmpBranchRRFcPair<"cgrt",  0xB960, GR64>;
   defm CLRT  : CmpBranchRRFcPair<"clrt",  0xB973, GR32>;
   defm CLGRT : CmpBranchRRFcPair<"clgrt", 0xB961, GR64>;
   defm CIT   : CmpBranchRIEaPair<"cit",   0xEC72, GR32, imm32sx16>;
   defm CGIT  : CmpBranchRIEaPair<"cgit",  0xEC70, GR64, imm64sx16>;
   defm CLFIT : CmpBranchRIEaPair<"clfit", 0xEC73, GR32, imm32zx16>;

 }
 
 //===----------------------------------------------------------------------===//
 // Call and return instructions
 //===----------------------------------------------------------------------===//
 
 // Define the general form of the call instructions for the asm parser.
 // These instructions don't hard-code %r14 as the return address register.
-let isCall = 1, Defs = [CC] in {
+let isCall = 1, Defs = [CC], hasSideEffects = 1 in {
   def BRAS  : CallRI <"bras", 0xA75>;
   def BRASL : CallRIL<"brasl", 0xC05>;
   def BAS   : CallRX <"bas", 0x4D>;
   def BASR  : CallRR <"basr", 0x0D>;
 }
 
 // Regular calls.
 let isCall = 1, Defs = [R14D, CC] in {

 let isCall = 1, isTerminator = 1, isReturn = 1, isBarrier = 1 in {
   def CallJG : Alias<6, (outs), (ins pcrel32:$I2),
                      [(z_sibcall pcrel32:$I2)]>;
   let Uses = [R1D] in
     def CallBR : Alias<2, (outs), (ins), [(z_sibcall R1D)]>;
 }
 
 // Conditional sibling calls.
-let CCMaskFirst = 1, isCall = 1, isTerminator = 1, isReturn = 1 in {
+let CCMaskFirst = 1, isCall = 1, isTerminator = 1, isReturn = 1,
+    hasSideEffects = 1 in {
   def CallBRCL : Alias<6, (outs), (ins cond4:$valid, cond4:$R1,
                                    pcrel32:$I2), []>;
   let Uses = [R1D] in
     def CallBCR : Alias<2, (outs), (ins cond4:$valid, cond4:$R1), []>;
 }
 
 // Fused compare and conditional sibling calls.
-let isCall = 1, isTerminator = 1, isReturn = 1, Uses = [R1D] in {
+let isCall = 1, isTerminator = 1, isReturn = 1, Uses = [R1D],
+    hasSideEffects = 1 in {
   def CRBCall : Alias<6, (outs), (ins GR32:$R1, GR32:$R2, cond4:$M3), []>;
   def CGRBCall : Alias<6, (outs), (ins GR64:$R1, GR64:$R2, cond4:$M3), []>;
   def CIBCall : Alias<6, (outs), (ins GR32:$R1, imm32sx8:$I2, cond4:$M3), []>;
   def CGIBCall : Alias<6, (outs), (ins GR64:$R1, imm64sx8:$I2, cond4:$M3), []>;
   def CLRBCall : Alias<6, (outs), (ins GR32:$R1, GR32:$R2, cond4:$M3), []>;
   def CLGRBCall : Alias<6, (outs), (ins GR64:$R1, GR64:$R2, cond4:$M3), []>;
   def CLIBCall : Alias<6, (outs), (ins GR32:$R1, imm32zx8:$I2, cond4:$M3), []>;
   def CLGIBCall : Alias<6, (outs), (ins GR64:$R1, imm64zx8:$I2, cond4:$M3), []>;
 }
 
 // A return instruction (br %r14).
 let isReturn = 1, isTerminator = 1, isBarrier = 1, hasCtrlDep = 1 in
   def Return : Alias<2, (outs), (ins), [(z_retflag)]>;
 
 // A conditional return instruction (bcr <cond>, %r14).
-let isReturn = 1, isTerminator = 1, hasCtrlDep = 1, CCMaskFirst = 1, Uses = [CC] in
+let isReturn = 1, isTerminator = 1, hasCtrlDep = 1, CCMaskFirst = 1, Uses = [CC],
+    hasSideEffects = 1 in
   def CondReturn : Alias<2, (outs), (ins cond4:$valid, cond4:$R1), []>;
 
 // Fused compare and conditional returns.
-let isReturn = 1, isTerminator = 1, hasCtrlDep = 1 in {
+let isReturn = 1, isTerminator = 1, hasCtrlDep = 1, hasSideEffects = 1 in {
   def CRBReturn : Alias<6, (outs), (ins GR32:$R1, GR32:$R2, cond4:$M3), []>;
   def CGRBReturn : Alias<6, (outs), (ins GR64:$R1, GR64:$R2, cond4:$M3), []>;
   def CIBReturn : Alias<6, (outs), (ins GR32:$R1, imm32sx8:$I2, cond4:$M3), []>;
   def CGIBReturn : Alias<6, (outs), (ins GR64:$R1, imm64sx8:$I2, cond4:$M3), []>;
   def CLRBReturn : Alias<6, (outs), (ins GR32:$R1, GR32:$R2, cond4:$M3), []>;
   def CLGRBReturn : Alias<6, (outs), (ins GR64:$R1, GR64:$R2, cond4:$M3), []>;
   def CLIBReturn : Alias<6, (outs), (ins GR32:$R1, imm32zx8:$I2, cond4:$M3), []>;
   def CLGIBReturn : Alias<6, (outs), (ins GR64:$R1, imm64zx8:$I2, cond4:$M3), []>;

 defm CondStore64 : CondStores<GR64, nonvolatile_store,
                               nonvolatile_load, bdxaddr20only>;
 
 //===----------------------------------------------------------------------===//
 // Move instructions
 //===----------------------------------------------------------------------===//
 
 // Register moves.
-let hasSideEffects = 0 in {
-  // Expands to LR, RISBHG or RISBLG, depending on the choice of registers.
-  def LRMux : UnaryRRPseudo<"lr", null_frag, GRX32, GRX32>,
-              Requires<[FeatureHighWord]>;
-  def LR  : UnaryRR <"lr",  0x18,   null_frag, GR32, GR32>;
-  def LGR : UnaryRRE<"lgr", 0xB904, null_frag, GR64, GR64>;
-}
-let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE in {
+
+let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE, hasSideEffects = 1 in {
   def LTR  : UnaryRR <"ltr",  0x12,   null_frag, GR32, GR32>;
   def LTGR : UnaryRRE<"ltgr", 0xB902, null_frag, GR64, GR64>;
 }
 
-let usesCustomInserter = 1, hasNoSchedulingInfo = 1 in
+let usesCustomInserter = 1, hasNoSchedulingInfo = 1, hasSideEffects = 1 in
   def PAIR128 : Pseudo<(outs GR128:$dst), (ins GR64:$hi, GR64:$lo), []>;
 
 // Immediate moves.
-let hasSideEffects = 0, isAsCheapAsAMove = 1, isMoveImm = 1,
-    isReMaterializable = 1 in {
+let isAsCheapAsAMove = 1, isMoveImm = 1, isReMaterializable = 1 in {
   // 16-bit sign-extended immediates.  LHIMux expands to LHI or IIHF,
   // deopending on the choice of register.
   def LHIMux : UnaryRIPseudo<bitconvert, GRX32, imm32sx16>,
                Requires<[FeatureHighWord]>;
   def LHI  : UnaryRI<"lhi",  0xA78, bitconvert, GR32, imm32sx16>;
   def LGHI : UnaryRI<"lghi", 0xA79, bitconvert, GR64, imm64sx16>;
 
   // Other 16-bit immediates.
def LMux : UnaryRXYPseudo<"l", load, GRX32, 4>,
              Requires<[FeatureHighWord]>;
   defm L : UnaryRXPair<"l", 0x58, 0xE358, load, GR32, 4>;
   def LFH : UnaryRXY<"lfh", 0xE3CA, load, GRH32, 4>,
             Requires<[FeatureHighWord]>;
   def LG : UnaryRXY<"lg", 0xE304, load, GR64, 8>;
 
   // These instructions are split after register allocation, so we don't
   // want a custom inserter.
-  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1 in {
+  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1, mayLoad = 1 in {
     def L128 : Pseudo<(outs GR128:$dst), (ins bdxaddr20only128:$src),
                       [(set GR128:$dst, (load bdxaddr20only128:$src))]>;
   }
 }
 let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE in {
   def LT  : UnaryRXY<"lt",  0xE312, load, GR32, 4>;
   def LTG : UnaryRXY<"ltg", 0xE302, load, GR64, 8>;
 }
let Predicates = [FeatureLoadAndZeroRightmostByte] in {
   def LZRG : UnaryRXY<"lzrg", 0xE32A, null_frag, GR64, 8>;
   def : Pat<(and (i32 (load bdxaddr20only:$src)), 0xffffff00),
             (LZRF bdxaddr20only:$src)>;
   def : Pat<(and (i64 (load bdxaddr20only:$src)), 0xffffffffffffff00),
             (LZRG bdxaddr20only:$src)>;
 }
 
 // Load and trap.
-let Predicates = [FeatureLoadAndTrap] in {
+let Predicates = [FeatureLoadAndTrap], hasSideEffects = 1 in {
   def LAT   : UnaryRXY<"lat",   0xE39F, null_frag, GR32, 4>;
   def LFHAT : UnaryRXY<"lfhat", 0xE3C8, null_frag, GRH32, 4>;
   def LGAT  : UnaryRXY<"lgat",  0xE385, null_frag, GR64, 8>;
 }
 
 // Register stores.
 let SimpleBDXStore = 1 in {
   // Expands to ST, STY or STFH, depending on the choice of register.
   def STMux : StoreRXYPseudo<store, GRX32, 4>,
               Requires<[FeatureHighWord]>;
   defm ST : StoreRXPair<"st", 0x50, 0xE350, store, GR32, 4>;
   def STFH : StoreRXY<"stfh", 0xE3CB, store, GRH32, 4>,
              Requires<[FeatureHighWord]>;
   def STG : StoreRXY<"stg", 0xE324, store, GR64, 8>;
 
   // These instructions are split after register allocation, so we don't
   // want a custom inserter.
-  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1 in {
+  let Has20BitOffset = 1, HasIndex = 1, Is128Bit = 1, mayStore = 1 in {
     def ST128 : Pseudo<(outs), (ins GR128:$src, bdxaddr20only128:$dst),
                        [(store GR128:$src, bdxaddr20only128:$dst)]>;
   }
 }
 def STRL  : StoreRILPC<"strl", 0xC4F, aligned_store, GR32>;
 def STGRL : StoreRILPC<"stgrl", 0xC4B, aligned_store, GR64>;
 
 // 8-bit immediate stores to 8-bit fields.

 
 //===----------------------------------------------------------------------===//
 // Conditional move instructions
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureLoadStoreOnCond2], Uses = [CC] in {
   // Load immediate on condition.  Matched via DAG pattern and created
   // by the PeepholeOptimizer via FoldImmediate.
-  let hasSideEffects = 0 in {
-    // Expands to LOCHI or LOCHHI, depending on the choice of register.
-    def LOCHIMux : CondBinaryRIEPseudo<GRX32, imm32sx16>;
-    defm LOCHHI  : CondBinaryRIEPair<"lochhi", 0xEC4E, GRH32, imm32sx16>;
-    defm LOCHI   : CondBinaryRIEPair<"lochi",  0xEC42, GR32, imm32sx16>;
-    defm LOCGHI  : CondBinaryRIEPair<"locghi", 0xEC46, GR64, imm64sx16>;
-  }
+
 
   // Move register on condition.  Expanded from Select* pseudos and
   // created by early if-conversion.
-  let hasSideEffects = 0, isCommutable = 1 in {
+  let isCommutable = 1 in {
     // Expands to LOCR or LOCFHR or a branch-and-move sequence,
     // depending on the choice of registers.
     def LOCRMux : CondBinaryRRFPseudo<GRX32, GRX32>;
     defm LOCFHR : CondBinaryRRFPair<"locfhr", 0xB9E0, GRH32, GRH32>;
   }
 
   // Load on condition.  Matched via DAG pattern.
   // Expands to LOC or LOCFH, depending on the choice of register.
   def LOCMux : CondUnaryRSYPseudo<nonvolatile_load, GRX32, 4>;
   defm LOCFH : CondUnaryRSYPair<"locfh", 0xEBE0, nonvolatile_load, GRH32, 4>;
 
   // Store on condition.  Expanded from CondStore* pseudos.
   // Expands to STOC or STOCFH, depending on the choice of register.
+  let hasSideEffects = 1 in {
     def STOCMux : CondStoreRSYPseudo<GRX32, 4>;
     defm STOCFH : CondStoreRSYPair<"stocfh", 0xEBE1, GRH32, 4>;
+  }
 
+  let hasSideEffects = 1 in {
     // Define AsmParser extended mnemonics for each general condition-code mask.
     foreach V = [ "E", "NE", "H", "NH", "L", "NL", "HE", "NHE", "LE", "NLE",
                   "Z", "NZ", "P", "NP", "M", "NM", "LH", "NLH", "O", "NO" ] in {
       def LOCHIAsm#V  : FixedCondBinaryRIE<CV<V>, "lochi",  0xEC42, GR32,
                                            imm32sx16>;
       def LOCGHIAsm#V : FixedCondBinaryRIE<CV<V>, "locghi", 0xEC46, GR64,
                                            imm64sx16>;
       def LOCHHIAsm#V : FixedCondBinaryRIE<CV<V>, "lochhi", 0xEC4E, GRH32,
                                            imm32sx16>;
       def LOCFHRAsm#V : FixedCondBinaryRRF<CV<V>, "locfhr", 0xB9E0, GRH32, GRH32>;
       def LOCFHAsm#V  : FixedCondUnaryRSY<CV<V>, "locfh",  0xEBE0, GRH32, 4>;
       def STOCFHAsm#V : FixedCondStoreRSY<CV<V>, "stocfh", 0xEBE1, GRH32, 4>;
     }
   }
+}
 
 let Predicates = [FeatureLoadStoreOnCond], Uses = [CC] in {
   // Move register on condition.  Expanded from Select* pseudos and
   // created by early if-conversion.
-  let hasSideEffects = 0, isCommutable = 1 in {
+  let isCommutable = 1 in {
     defm LOCR  : CondBinaryRRFPair<"locr",  0xB9F2, GR32, GR32>;
     defm LOCGR : CondBinaryRRFPair<"locgr", 0xB9E2, GR64, GR64>;
   }
 
   // Load on condition.  Matched via DAG pattern.
   defm LOC  : CondUnaryRSYPair<"loc",  0xEBF2, nonvolatile_load, GR32, 4>;
   defm LOCG : CondUnaryRSYPair<"locg", 0xEBE2, nonvolatile_load, GR64, 8>;
 
   // Store on condition.  Expanded from CondStore* pseudos.
+  let hasSideEffects = 1 in {
     defm STOC  : CondStoreRSYPair<"stoc",  0xEBF3, GR32, 4>;
     defm STOCG : CondStoreRSYPair<"stocg", 0xEBE3, GR64, 8>;
+  }
 
+  let hasSideEffects = 1 in {
     // Define AsmParser extended mnemonics for each general condition-code mask.
     foreach V = [ "E", "NE", "H", "NH", "L", "NL", "HE", "NHE", "LE", "NLE",
                   "Z", "NZ", "P", "NP", "M", "NM", "LH", "NLH", "O", "NO" ] in {
       def LOCRAsm#V   : FixedCondBinaryRRF<CV<V>, "locr",  0xB9F2, GR32, GR32>;
       def LOCGRAsm#V  : FixedCondBinaryRRF<CV<V>, "locgr", 0xB9E2, GR64, GR64>;
       def LOCAsm#V    : FixedCondUnaryRSY<CV<V>, "loc",   0xEBF2, GR32, 4>;
       def LOCGAsm#V   : FixedCondUnaryRSY<CV<V>, "locg",  0xEBE2, GR64, 8>;
       def STOCAsm#V   : FixedCondStoreRSY<CV<V>, "stoc",  0xEBF3, GR32, 4>;
       def STOCGAsm#V  : FixedCondStoreRSY<CV<V>, "stocg", 0xEBE3, GR64, 8>;
     }
-}
+  }    
+}
 //===----------------------------------------------------------------------===//
 // Sign extensions
 //===----------------------------------------------------------------------===//
 //
 // Note that putting these before zero extensions mean that we will prefer
 // them for anyextload*.  There's not really much to choose between the two
 // either way, but signed-extending loads have a short LH and a long LHY,
 // while zero-extending loads have only the long LLH.
 //
 //===----------------------------------------------------------------------===//
 
 // 32-bit extensions from registers.
-let hasSideEffects = 0 in {
-  def LBR : UnaryRRE<"lbr", 0xB926, sext8,  GR32, GR32>;
-  def LHR : UnaryRRE<"lhr", 0xB927, sext16, GR32, GR32>;
-}
 
 // 64-bit extensions from registers.
-let hasSideEffects = 0 in {
-  def LGBR : UnaryRRE<"lgbr", 0xB906, sext8,  GR64, GR64>;
-  def LGHR : UnaryRRE<"lghr", 0xB907, sext16, GR64, GR64>;
-  def LGFR : UnaryRRE<"lgfr", 0xB914, sext32, GR64, GR32>;
-}
-let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE in
+
+let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE, hasSideEffects = 1 in
   def LTGFR : UnaryRRE<"ltgfr", 0xB912, null_frag, GR64, GR32>;
 
 // Match 32-to-64-bit sign extensions in which the source is already
 // in a 64-bit register.
 def : Pat<(sext_inreg GR64:$src, i32),
           (LGFR (EXTRACT_SUBREG GR64:$src, subreg_l32))>;
 
 // 32-bit extensions from 8-bit memory.  LBMux expands to LB or LBH,

 let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE in
   def LTGF : UnaryRXY<"ltgf", 0xE332, asextloadi32, GR64, 4>;
 
 //===----------------------------------------------------------------------===//
 // Zero extensions
 //===----------------------------------------------------------------------===//
 
 // 32-bit extensions from registers.
-let hasSideEffects = 0 in {
-  // Expands to LLCR or RISB[LH]G, depending on the choice of registers.
-  def LLCRMux : UnaryRRPseudo<"llcr", zext8, GRX32, GRX32>,
-                Requires<[FeatureHighWord]>;
-  def LLCR    : UnaryRRE<"llcr", 0xB994, zext8,  GR32, GR32>;
-  // Expands to LLHR or RISB[LH]G, depending on the choice of registers.
-  def LLHRMux : UnaryRRPseudo<"llhr", zext16, GRX32, GRX32>,
-                Requires<[FeatureHighWord]>;
-  def LLHR    : UnaryRRE<"llhr", 0xB995, zext16, GR32, GR32>;
-}
+
 
 // 64-bit extensions from registers.
-let hasSideEffects = 0 in {
-  def LLGCR : UnaryRRE<"llgcr", 0xB984, zext8,  GR64, GR64>;
-  def LLGHR : UnaryRRE<"llghr", 0xB985, zext16, GR64, GR64>;
-  def LLGFR : UnaryRRE<"llgfr", 0xB916, zext32, GR64, GR32>;
-}
 
 // Match 32-to-64-bit zero extensions in which the source is already
 // in a 64-bit register.
 def : Pat<(and GR64:$src, 0xffffffff),
           (LLGFR (EXTRACT_SUBREG GR64:$src, subreg_l32))>;
 
 // 32-bit extensions from 8-bit memory.  LLCMux expands to LLC or LLCH,
 // depending on the choice of register.

 // Load and zero rightmost byte.
 let Predicates = [FeatureLoadAndZeroRightmostByte] in {
   def LLZRGF : UnaryRXY<"llzrgf", 0xE33A, null_frag, GR64, 4>;
   def : Pat<(and (i64 (azextloadi32 bdxaddr20only:$src)), 0xffffff00),
             (LLZRGF bdxaddr20only:$src)>;
 }
 
 // Load and trap.
-let Predicates = [FeatureLoadAndTrap] in {
+let Predicates = [FeatureLoadAndTrap], hasSideEffects = 1 in {
   def LLGFAT : UnaryRXY<"llgfat", 0xE39D, null_frag, GR64, 4>;
   def LLGTAT : UnaryRXY<"llgtat", 0xE39C, null_frag, GR64, 4>;
 }
 
 // Extend GR64s to GR128s.
-let usesCustomInserter = 1 in
+let usesCustomInserter = 1, hasSideEffects = 1 in
   def ZEXT128 : Pseudo<(outs GR128:$dst), (ins GR64:$src), []>;
 
 //===----------------------------------------------------------------------===//
 // "Any" extensions
 //===----------------------------------------------------------------------===//
 
 // Use subregs to populate the "don't care" bits in a 32-bit to 64-bit anyext.
 def : Pat<(i64 (anyext GR32:$src)),
           (INSERT_SUBREG (i64 (IMPLICIT_DEF)), GR32:$src, subreg_l32)>;
 
 // Extend GR64s to GR128s.
-let usesCustomInserter = 1 in
+let usesCustomInserter = 1, hasSideEffects = 1 in
   def AEXT128 : Pseudo<(outs GR128:$dst), (ins GR64:$src), []>;
 
 //===----------------------------------------------------------------------===//
 // Truncations
 //===----------------------------------------------------------------------===//
 
 // Truncations of 64-bit registers to 32-bit registers.
 def : Pat<(i32 (trunc GR64:$src)),

 // Truncations of 64-bit registers to memory.
 defm : StoreGR64Pair<STC, STCY, truncstorei8>;
 defm : StoreGR64Pair<STH, STHY, truncstorei16>;
 def  : StoreGR64PC<STHRL, aligned_truncstorei16>;
 defm : StoreGR64Pair<ST, STY, truncstorei32>;
 def  : StoreGR64PC<STRL, aligned_truncstorei32>;
 
 // Store characters under mask -- not (yet) used for codegen.
+let hasSideEffects = 1 in {
   defm STCM : StoreBinaryRSPair<"stcm", 0xBE, 0xEB2D, GR32, 0>;
   def STCMH : StoreBinaryRSY<"stcmh", 0xEB2C, GRH32, 0>;
+}
 
 //===----------------------------------------------------------------------===//
 // Multi-register moves
 //===----------------------------------------------------------------------===//
 
 // Multi-register loads.
+let hasSideEffects = 1 in {
   defm LM : LoadMultipleRSPair<"lm", 0x98, 0xEB98, GR32>;
   def LMG : LoadMultipleRSY<"lmg", 0xEB04, GR64>;
   def LMH : LoadMultipleRSY<"lmh", 0xEB96, GRH32>;
   def LMD : LoadMultipleSSe<"lmd", 0xEF, GR64>;
 
   // Multi-register stores.
   defm STM : StoreMultipleRSPair<"stm", 0x90, 0xEB90, GR32>;
   def STMG : StoreMultipleRSY<"stmg", 0xEB24, GR64>;
   def STMH : StoreMultipleRSY<"stmh", 0xEB26, GRH32>;
+}
 
 //===----------------------------------------------------------------------===//
 // Byte swaps
 //===----------------------------------------------------------------------===//
 
 // Byte-swapping register moves.
-let hasSideEffects = 0 in {
-  def LRVR  : UnaryRRE<"lrvr",  0xB91F, bswap, GR32, GR32>;
-  def LRVGR : UnaryRRE<"lrvgr", 0xB90F, bswap, GR64, GR64>;
-}
 
 // Byte-swapping loads.  Unlike normal loads, these instructions are
 // allowed to access storage more than once.
 def LRVH : UnaryRXY<"lrvh", 0xE31F, z_lrvh, GR32, 2>;
 def LRV  : UnaryRXY<"lrv",  0xE31E, z_lrv,  GR32, 4>;
 def LRVG : UnaryRXY<"lrvg", 0xE30F, z_lrvg, GR64, 8>;
 
 // Likewise byte-swapping stores.
 def STRVH : StoreRXY<"strvh", 0xE33F, z_strvh, GR32, 2>;
 def STRV  : StoreRXY<"strv",  0xE33E, z_strv,  GR32, 4>;
 def STRVG : StoreRXY<"strvg", 0xE32F, z_strvg, GR64, 8>;
 
 // Byte-swapping memory-to-memory moves.
 let mayLoad = 1, mayStore = 1 in
   def MVCIN : SideEffectBinarySSa<"mvcin", 0xE8>;
 
 //===----------------------------------------------------------------------===//
 // Load address instructions
 //===----------------------------------------------------------------------===//
 
 // Load BDX-style addresses.
-let hasSideEffects = 0, isAsCheapAsAMove = 1, isReMaterializable = 1 in
+let isAsCheapAsAMove = 1, isReMaterializable = 1 in
   defm LA : LoadAddressRXPair<"la", 0x41, 0xE371, bitconvert>;
 
 // Load a PC-relative address.  There's no version of this instruction
 // with a 16-bit offset, so there's no relaxation.
-let hasSideEffects = 0, isAsCheapAsAMove = 1, isMoveImm = 1,
-    isReMaterializable = 1 in
+let isAsCheapAsAMove = 1, isMoveImm = 1, isReMaterializable = 1 in
   def LARL : LoadAddressRIL<"larl", 0xC00, bitconvert>;
 
 // Load the Global Offset Table address.  This will be lowered into a
 //     larl $R1, _GLOBAL_OFFSET_TABLE_
 // instruction.
 def GOT : Alias<6, (outs GR64:$R1), (ins),
                 [(set GR64:$R1, (global_offset_table))]>;
 

 
 defm : InsertMem<"inserti8", IC32,  GR32, azextloadi8, bdxaddr12pair>;
 defm : InsertMem<"inserti8", IC32Y, GR32, azextloadi8, bdxaddr20pair>;
 
 defm : InsertMem<"inserti8", IC,  GR64, azextloadi8, bdxaddr12pair>;
 defm : InsertMem<"inserti8", ICY, GR64, azextloadi8, bdxaddr20pair>;
 
 // Insert characters under mask -- not (yet) used for codegen.
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   defm ICM : TernaryRSPair<"icm", 0xBF, 0xEB81, GR32, 0>;
   def ICMH : TernaryRSY<"icmh", 0xEB80, GRH32, 0>;
 }
 
 // Insertions of a 16-bit immediate, leaving other bits unaffected.
 // We don't have or_as_insert equivalents of these operations because
 // OI is available instead.
 //
let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0x8 in {
   // Addition of a register.
   let isCommutable = 1 in {
     defm AR : BinaryRRAndK<"ar", 0x1A, 0xB9F8, add, GR32, GR32>;
     defm AGR : BinaryRREAndK<"agr", 0xB908, 0xB9E8, add, GR64, GR64>;
   }
   def AGFR : BinaryRRE<"agfr", 0xB918, null_frag, GR64, GR32>;
 
   // Addition to a high register.
+  let hasSideEffects = 1 in {
     def AHHHR : BinaryRRFa<"ahhhr", 0xB9C8, null_frag, GRH32, GRH32, GRH32>,
                 Requires<[FeatureHighWord]>;
     def AHHLR : BinaryRRFa<"ahhlr", 0xB9D8, null_frag, GRH32, GRH32, GR32>,
                 Requires<[FeatureHighWord]>;
+  }
 
   // Addition of signed 16-bit immediates.
   defm AHIMux : BinaryRIAndKPseudo<"ahimux", add, GRX32, imm32sx16>;
   defm AHI  : BinaryRIAndK<"ahi",  0xA7A, 0xECD8, add, GR32, imm32sx16>;
   defm AGHI : BinaryRIAndK<"aghi", 0xA7B, 0xECD9, add, GR64, imm64sx16>;
 
   // Addition of signed 32-bit immediates.
   def AFIMux : BinaryRIPseudo<add, GRX32, simm32>,
                Requires<[FeatureHighWord]>;
   def AFI  : BinaryRIL<"afi",  0xC29, add, GR32, simm32>;
   def AIH  : BinaryRIL<"aih",  0xCC8, add, GRH32, simm32>,
              Requires<[FeatureHighWord]>;
   def AGFI : BinaryRIL<"agfi", 0xC28, add, GR64, imm64sx32>;
 
   // Addition of memory.
   defm AH  : BinaryRXPair<"ah", 0x4A, 0xE37A, add, GR32, asextloadi16, 2>;
   defm A   : BinaryRXPair<"a",  0x5A, 0xE35A, add, GR32, load, 4>;
   def  AGH : BinaryRXY<"agh", 0xE338, add, GR64, asextloadi16, 2>,
              Requires<[FeatureMiscellaneousExtensions2]>;
   def  AGF : BinaryRXY<"agf", 0xE318, add, GR64, asextloadi32, 4>;
   def  AG  : BinaryRXY<"ag",  0xE308, add, GR64, load, 8>;
 
   // Addition to memory.
   def ASI  : BinarySIY<"asi",  0xEB6A, add, imm32sx8>;
   def AGSI : BinarySIY<"agsi", 0xEB7A, add, imm64sx8>;
 }
 defm : SXB<add, GR64, AGFR>;
 
 // Addition producing a carry.
 let Defs = [CC] in {
   // Addition of a register.
   let isCommutable = 1 in {
     defm ALR : BinaryRRAndK<"alr", 0x1E, 0xB9FA, addc, GR32, GR32>;
     defm ALGR : BinaryRREAndK<"algr", 0xB90A, 0xB9EA, addc, GR64, GR64>;
   }
   def ALGFR : BinaryRRE<"algfr", 0xB91A, null_frag, GR64, GR32>;
 
   // Addition to a high register.
+  let hasSideEffects = 1 in {
     def ALHHHR : BinaryRRFa<"alhhhr", 0xB9CA, null_frag, GRH32, GRH32, GRH32>,
                  Requires<[FeatureHighWord]>;
     def ALHHLR : BinaryRRFa<"alhhlr", 0xB9DA, null_frag, GRH32, GRH32, GR32>,
                  Requires<[FeatureHighWord]>;
+  }
 
   // Addition of signed 16-bit immediates.
   def ALHSIK  : BinaryRIE<"alhsik",  0xECDA, addc, GR32, imm32sx16>,
                 Requires<[FeatureDistinctOps]>;
   def ALGHSIK : BinaryRIE<"alghsik", 0xECDB, addc, GR64, imm64sx16>,
                 Requires<[FeatureDistinctOps]>;
 
   // Addition of unsigned 32-bit immediates.
   def ALFI  : BinaryRIL<"alfi",  0xC2B, addc, GR32, uimm32>;
   def ALGFI : BinaryRIL<"algfi", 0xC2A, addc, GR64, imm64zx32>;
 
   // Addition of signed 32-bit immediates.
+  let hasSideEffects = 1 in
     def ALSIH : BinaryRIL<"alsih", 0xCCA, null_frag, GRH32, simm32>,
                 Requires<[FeatureHighWord]>;
 
   // Addition of memory.
   defm AL   : BinaryRXPair<"al", 0x5E, 0xE35E, addc, GR32, load, 4>;
   def  ALGF : BinaryRXY<"algf", 0xE31A, addc, GR64, azextloadi32, 4>;
   def  ALG  : BinaryRXY<"alg",  0xE30A, addc, GR64, load, 8>;
 
   // Addition to memory.
+  let hasSideEffects = 1 in {
     def ALSI  : BinarySIY<"alsi",  0xEB6E, null_frag, imm32sx8>;
     def ALGSI : BinarySIY<"algsi", 0xEB7E, null_frag, imm64sx8>;
   }
+}
 defm : ZXB<addc, GR64, ALGFR>;
 
 // Addition producing and using a carry.
 let Defs = [CC], Uses = [CC] in {
   // Addition of a register.
   def ALCR  : BinaryRRE<"alcr",  0xB998, adde, GR32, GR32>;
   def ALCGR : BinaryRRE<"alcgr", 0xB988, adde, GR64, GR64>;
 
   // Addition of memory.
   def ALC  : BinaryRXY<"alc",  0xE398, adde, GR32, load, 4>;
   def ALCG : BinaryRXY<"alcg", 0xE388, adde, GR64, load, 8>;
 }
 
 // Addition that does not modify the condition code.
+let hasSideEffects = 1 in
   def ALSIHN : BinaryRIL<"alsihn", 0xCCB, null_frag, GRH32, simm32>,
                Requires<[FeatureHighWord]>;
 
 //===----------------------------------------------------------------------===//
 // Subtraction
 //===----------------------------------------------------------------------===//
 
 // Plain subtraction.  Although immediate forms exist, we use the
 // add-immediate instruction instead.
 let Defs = [CC], CCValues = 0xF, CompareZeroCCMask = 0x8 in {
   // Subtraction of a register.
   defm SR : BinaryRRAndK<"sr", 0x1B, 0xB9F9, sub, GR32, GR32>;
   def SGFR : BinaryRRE<"sgfr", 0xB919, null_frag, GR64, GR32>;
   defm SGR : BinaryRREAndK<"sgr", 0xB909, 0xB9E9, sub, GR64, GR64>;
 
   // Subtraction from a high register.
+  let hasSideEffects = 1 in {
     def SHHHR : BinaryRRFa<"shhhr", 0xB9C9, null_frag, GRH32, GRH32, GRH32>,
                 Requires<[FeatureHighWord]>;
     def SHHLR : BinaryRRFa<"shhlr", 0xB9D9, null_frag, GRH32, GRH32, GR32>,
                 Requires<[FeatureHighWord]>;
+  }
 
   // Subtraction of memory.
   defm SH  : BinaryRXPair<"sh", 0x4B, 0xE37B, sub, GR32, asextloadi16, 2>;
   defm S   : BinaryRXPair<"s", 0x5B, 0xE35B, sub, GR32, load, 4>;
   def  SGH : BinaryRXY<"sgh", 0xE339, sub, GR64, asextloadi16, 2>,
              Requires<[FeatureMiscellaneousExtensions2]>;
   def  SGF : BinaryRXY<"sgf", 0xE319, sub, GR64, asextloadi32, 4>;
   def  SG  : BinaryRXY<"sg",  0xE309, sub, GR64, load, 8>;
 }
 defm : SXB<sub, GR64, SGFR>;
 
 // Subtraction producing a carry.
 let Defs = [CC] in {
   // Subtraction of a register.
   defm SLR : BinaryRRAndK<"slr", 0x1F, 0xB9FB, subc, GR32, GR32>;
   def SLGFR : BinaryRRE<"slgfr", 0xB91B, null_frag, GR64, GR32>;
   defm SLGR : BinaryRREAndK<"slgr", 0xB90B, 0xB9EB, subc, GR64, GR64>;
 
   // Subtraction from a high register.
+  let hasSideEffects = 1 in {
     def SLHHHR : BinaryRRFa<"slhhhr", 0xB9CB, null_frag, GRH32, GRH32, GRH32>,
                  Requires<[FeatureHighWord]>;
     def SLHHLR : BinaryRRFa<"slhhlr", 0xB9DB, null_frag, GRH32, GRH32, GR32>,
                  Requires<[FeatureHighWord]>;
+  }
 
   // Subtraction of unsigned 32-bit immediates.  These don't match
   // subc because we prefer addc for constants.
+  let hasSideEffects = 1 in
     def SLFI  : BinaryRIL<"slfi",  0xC25, null_frag, GR32, uimm32>;
   def SLGFI : BinaryRIL<"slgfi", 0xC24, null_frag, GR64, imm64zx32>;
 
   // Subtraction of memory.
   defm SL   : BinaryRXPair<"sl", 0x5F, 0xE35F, subc, GR32, load, 4>;
   def  SLGF : BinaryRXY<"slgf", 0xE31B, subc, GR64, azextloadi32, 4>;
   def  SLG  : BinaryRXY<"slg",  0xE30B, subc, GR64, load, 8>;
 }
 defm : ZXB<subc, GR64, SLGFR>;

 defm MH   : BinaryRXPair<"mh", 0x4C, 0xE37C, mul, GR32, asextloadi16, 2>;
 defm MS   : BinaryRXPair<"ms", 0x71, 0xE351, mul, GR32, load, 4>;
 def  MGH  : BinaryRXY<"mgh", 0xE33C, mul, GR64, asextloadi16, 2>,
             Requires<[FeatureMiscellaneousExtensions2]>;
 def  MSGF : BinaryRXY<"msgf", 0xE31C, mul, GR64, asextloadi32, 4>;
 def  MSG  : BinaryRXY<"msg",  0xE30C, mul, GR64, load, 8>;
 
 // Multiplication of memory, setting the condition code.
-let Predicates = [FeatureMiscellaneousExtensions2], Defs = [CC] in {
+let Predicates = [FeatureMiscellaneousExtensions2], Defs = [CC],
+    hasSideEffects = 1 in {
   def MSC  : BinaryRXY<"msc",  0xE353, null_frag, GR32, load, 4>;
   def MSGC : BinaryRXY<"msgc", 0xE383, null_frag, GR64, load, 8>;
 }
 
 // Multiplication of a register, producing two results.
+let hasSideEffects = 1 in
   def MR   : BinaryRR <"mr",    0x1C,   null_frag, GR128, GR32>;
 def MGRK : BinaryRRFa<"mgrk", 0xB9EC, null_frag, GR128, GR64, GR64>,
            Requires<[FeatureMiscellaneousExtensions2]>;
+let hasSideEffects = 1 in {
   def MLR  : BinaryRRE<"mlr",  0xB996, null_frag, GR128, GR32>;
   def MLGR : BinaryRRE<"mlgr", 0xB986, null_frag, GR128, GR64>;
+}
 def : Pat<(z_smul_lohi GR64:$src1, GR64:$src2),
           (MGRK GR64:$src1, GR64:$src2)>;
 def : Pat<(z_umul_lohi GR64:$src1, GR64:$src2),
           (MLGR (AEXT128 GR64:$src1), GR64:$src2)>;
 
 // Multiplication of memory, producing two results.
+let hasSideEffects = 1 in {
   def M   : BinaryRX <"m",   0x5C,   null_frag, GR128, load, 4>;
   def MFY : BinaryRXY<"mfy", 0xE35C, null_frag, GR128, load, 4>;
   def MG  : BinaryRXY<"mg",  0xE384, null_frag, GR128, load, 8>,
             Requires<[FeatureMiscellaneousExtensions2]>;
   def ML  : BinaryRXY<"ml",  0xE396, null_frag, GR128, load, 4>;
   def MLG : BinaryRXY<"mlg", 0xE386, null_frag, GR128, load, 8>;
+}
 def : Pat<(z_smul_lohi GR64:$src1, (i64 (load bdxaddr20only:$src2))),
           (MG (AEXT128 GR64:$src1), bdxaddr20only:$src2)>;
 def : Pat<(z_umul_lohi GR64:$src1, (i64 (load bdxaddr20only:$src2))),
           (MLG (AEXT128 GR64:$src1), bdxaddr20only:$src2)>;
 
 //===----------------------------------------------------------------------===//
 // Division and remainder
 //===----------------------------------------------------------------------===//

 def : Pat<(z_udivrem GR64:$src1, (i64 (load bdxaddr20only:$src2))),
           (DLG (ZEXT128 GR64:$src1), bdxaddr20only:$src2)>;
 
 //===----------------------------------------------------------------------===//
 // Shifts
 //===----------------------------------------------------------------------===//
 
 // Logical shift left.
-let hasSideEffects = 0 in {
-  defm SLL : BinaryRSAndK<"sll", 0x89, 0xEBDF, shl, GR32>;
-  def SLLG : BinaryRSY<"sllg", 0xEB0D, shl, GR64>;
-  def SLDL : BinaryRS<"sldl", 0x8D, null_frag, GR128>;
-}
 
 // Arithmetic shift left.
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   defm SLA : BinaryRSAndK<"sla", 0x8B, 0xEBDD, null_frag, GR32>;
   def SLAG : BinaryRSY<"slag", 0xEB0B, null_frag, GR64>;
   def SLDA : BinaryRS<"slda", 0x8F, null_frag, GR128>;
 }
 
 // Logical shift right.
-let hasSideEffects = 0 in {
-  defm SRL : BinaryRSAndK<"srl", 0x88, 0xEBDE, srl, GR32>;
-  def SRLG : BinaryRSY<"srlg", 0xEB0C, srl, GR64>;
-  def SRDL : BinaryRS<"srdl", 0x8C, null_frag, GR128>;
-}
 
 // Arithmetic shift right.
 let Defs = [CC], CCValues = 0xE, CompareZeroCCMask = 0xE in {
-  defm SRA : BinaryRSAndK<"sra", 0x8A, 0xEBDC, sra, GR32>;
+  defm SRA : BinaryRSAndK_KSideEff<"sra", 0x8A, 0xEBDC, sra, GR32>;
   def SRAG : BinaryRSY<"srag", 0xEB0A, sra, GR64>;
+  let hasSideEffects = 1 in
     def SRDA : BinaryRS<"srda", 0x8E, null_frag, GR128>;
 }
 
 // Rotate left.
-let hasSideEffects = 0 in {
-  def RLL  : BinaryRSY<"rll",  0xEB1D, rotl, GR32>;
-  def RLLG : BinaryRSY<"rllg", 0xEB1C, rotl, GR64>;
-}
 
 // Rotate second operand left and inserted selected bits into first operand.
 // These can act like 32-bit operands provided that the constant start and
 // end bits (operands 2 and 3) are in the range [32, 64).
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   let isCodeGenOnly = 1 in
     def RISBG32 : RotateSelectRIEf<"risbg", 0xEC55, GR32, GR32>;
   let CCValues = 0xE, CompareZeroCCMask = 0xE in
     def RISBG : RotateSelectRIEf<"risbg", 0xEC55, GR64, GR64>;
 }
 
 // On zEC12 we have a variant of RISBG that does not set CC.
-let Predicates = [FeatureMiscellaneousExtensions] in
+let Predicates = [FeatureMiscellaneousExtensions], hasSideEffects = 1 in
   def RISBGN : RotateSelectRIEf<"risbgn", 0xEC59, GR64, GR64>;
 
 // Forms of RISBG that only affect one word of the destination register.
 // They do not set CC.
-let Predicates = [FeatureHighWord] in {
+let Predicates = [FeatureHighWord], hasSideEffects = 1 in {
   def RISBMux : RotateSelectRIEfPseudo<GRX32, GRX32>;
   def RISBLL  : RotateSelectAliasRIEf<GR32,  GR32>;
   def RISBLH  : RotateSelectAliasRIEf<GR32,  GRH32>;
   def RISBHL  : RotateSelectAliasRIEf<GRH32, GR32>;
   def RISBHH  : RotateSelectAliasRIEf<GRH32, GRH32>;
   def RISBLG  : RotateSelectRIEf<"risblg", 0xEC51, GR32, GR64>;
   def RISBHG  : RotateSelectRIEf<"risbhg", 0xEC5D, GRH32, GR64>;
 }
 
 // Rotate second operand left and perform a logical operation with selected
 // bits of the first operand.  The CC result only describes the selected bits,
 // so isn't useful for a full comparison against zero.
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   def RNSBG : RotateSelectRIEf<"rnsbg", 0xEC54, GR64, GR64>;
   def ROSBG : RotateSelectRIEf<"rosbg", 0xEC56, GR64, GR64>;
   def RXSBG : RotateSelectRIEf<"rxsbg", 0xEC57, GR64, GR64>;
 }
 
 //===----------------------------------------------------------------------===//
 // Comparison
 //===----------------------------------------------------------------------===//
 
 // Signed comparisons.  We put these before the unsigned comparisons because
 // some of the signed forms have COMPARE AND BRANCH equivalents whereas none
 // of the unsigned forms do.
 let Defs = [CC], CCValues = 0xE in {
   // Comparison with a register.
   def CR   : CompareRR <"cr",   0x19,   z_scmp,    GR32, GR32>;
   def CGFR : CompareRRE<"cgfr", 0xB930, null_frag, GR64, GR32>;
   def CGR  : CompareRRE<"cgr",  0xB920, z_scmp,    GR64, GR64>;
 
   // Comparison with a high register.
+  let hasSideEffects = 1 in {
     def CHHR : CompareRRE<"chhr", 0xB9CD, null_frag, GRH32, GRH32>,
                Requires<[FeatureHighWord]>;
     def CHLR : CompareRRE<"chlr", 0xB9DD, null_frag, GRH32, GR32>,
                Requires<[FeatureHighWord]>;
+  }
 
   // Comparison with a signed 16-bit immediate.  CHIMux expands to CHI or CIH,
   // depending on the choice of register.
   def CHIMux : CompareRIPseudo<z_scmp, GRX32, imm32sx16>,
                Requires<[FeatureHighWord]>;
   def CHI  : CompareRI<"chi",  0xA7E, z_scmp, GR32, imm32sx16>;
   def CGHI : CompareRI<"cghi", 0xA7F, z_scmp, GR64, imm64sx16>;
 

 // Unsigned comparisons.
 let Defs = [CC], CCValues = 0xE, IsLogical = 1 in {
   // Comparison with a register.
   def CLR   : CompareRR <"clr",   0x15,   z_ucmp,    GR32, GR32>;
   def CLGFR : CompareRRE<"clgfr", 0xB931, null_frag, GR64, GR32>;
   def CLGR  : CompareRRE<"clgr",  0xB921, z_ucmp,    GR64, GR64>;
 
   // Comparison with a high register.
+  let hasSideEffects = 1 in {
     def CLHHR : CompareRRE<"clhhr", 0xB9CF, null_frag, GRH32, GRH32>,
                 Requires<[FeatureHighWord]>;
     def CLHLR : CompareRRE<"clhlr", 0xB9DF, null_frag, GRH32, GR32>,
                 Requires<[FeatureHighWord]>;
+  }
 
   // Comparison with an unsigned 32-bit immediate.  CLFIMux expands to CLFI
   // or CLIH, depending on the choice of register.
   def CLFIMux : CompareRIPseudo<z_ucmp, GRX32, uimm32>,
                 Requires<[FeatureHighWord]>;
   def CLFI  : CompareRIL<"clfi",  0xC2F, z_ucmp, GR32, uimm32>;
   def CLIH  : CompareRIL<"clih",  0xCCF, z_ucmp, GRH32, uimm32>,
               Requires<[FeatureHighWord]>;
let Defs = [CC] in {
 
   defm TM : CompareSIPair<"tm", 0x91, 0xEB51, z_tm_mem, anyextloadi8, imm32zx8>;
 }
 
 def TML : InstAlias<"tml\t$R, $I", (TMLL GR32:$R, imm32ll16:$I), 0>;
 def TMH : InstAlias<"tmh\t$R, $I", (TMLH GR32:$R, imm32lh16:$I), 0>;
 
 // Compare logical characters under mask -- not (yet) used for codegen.
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   defm CLM : CompareRSPair<"clm", 0xBD, 0xEB21, GR32, 0>;
   def CLMH : CompareRSY<"clmh", 0xEB20, GRH32, 0>;
 }
 
 //===----------------------------------------------------------------------===//
 // Prefetch and execution hint
 //===----------------------------------------------------------------------===//
 
+let mayLoad = 1, mayStore = 1 in {
   def PFD : PrefetchRXY<"pfd", 0xE336, z_prefetch>;
   def PFDRL : PrefetchRILPC<"pfdrl", 0xC62, z_prefetch>;
+}
 
-let Predicates = [FeatureExecutionHint] in {
+let Predicates = [FeatureExecutionHint], hasSideEffects = 1 in {
   // Branch Prediction Preload
   def BPP : BranchPreloadSMI<"bpp", 0xC7>;
   def BPRP : BranchPreloadMII<"bprp", 0xC5>;
 
   // Next Instruction Access Intent
   def NIAI : SideEffectBinaryIE<"niai", 0xB2FA, imm32zx4, imm32zx4>;
 }
 
 //===----------------------------------------------------------------------===//
 // Atomic operations
 //===----------------------------------------------------------------------===//
 
 // A serialization instruction that acts as a barrier for all memory
 // accesses, which expands to "bcr 14, 0".
 let hasSideEffects = 1 in
 def Serialize : Alias<2, (outs), (ins), []>;
 
 // A pseudo instruction that serves as a compiler barrier.
 let hasSideEffects = 1, hasNoSchedulingInfo = 1 in
 def MemBarrier : Pseudo<(outs), (ins), [(z_membarrier)]>;
 
 let Predicates = [FeatureInterlockedAccess1], Defs = [CC] in {
   def LAA   : LoadAndOpRSY<"laa",   0xEBF8, atomic_load_add_32, GR32>;
   def LAAG  : LoadAndOpRSY<"laag",  0xEBE8, atomic_load_add_64, GR64>;
+  let hasSideEffects = 1 in {
     def LAAL  : LoadAndOpRSY<"laal",  0xEBFA, null_frag, GR32>;
     def LAALG : LoadAndOpRSY<"laalg", 0xEBEA, null_frag, GR64>;
+  }
   def LAN   : LoadAndOpRSY<"lan",   0xEBF4, atomic_load_and_32, GR32>;
   def LANG  : LoadAndOpRSY<"lang",  0xEBE4, atomic_load_and_64, GR64>;
   def LAO   : LoadAndOpRSY<"lao",   0xEBF6, atomic_load_or_32, GR32>;
   def LAOG  : LoadAndOpRSY<"laog",  0xEBE6, atomic_load_or_64, GR64>;
   def LAX   : LoadAndOpRSY<"lax",   0xEBF7, atomic_load_xor_32, GR32>;
   def LAXG  : LoadAndOpRSY<"laxg",  0xEBE7, atomic_load_xor_64, GR64>;
 }
 
def ATOMIC_CMP_SWAPW
   let Defs = [CC];
   let mayLoad = 1;
   let mayStore = 1;
   let usesCustomInserter = 1;
   let hasNoSchedulingInfo = 1;
 }
 
 // Test and set.
-let mayLoad = 1, Defs = [CC] in
+let mayLoad = 1, Defs = [CC], hasSideEffects = 1 in
   def TS : StoreInherentS<"ts", 0x9300, null_frag, 1>;
 
 // Compare and swap.
 let Defs = [CC] in {
   defm CS  : CmpSwapRSPair<"cs", 0xBA, 0xEB14, z_atomic_cmp_swap, GR32>;
   def  CSG : CmpSwapRSY<"csg", 0xEB30, z_atomic_cmp_swap, GR64>;
 }
 
 // Compare double and swap.
 let Defs = [CC] in {
+  let hasSideEffects = 1 in
     defm CDS  : CmpSwapRSPair<"cds", 0xBB, 0xEB31, null_frag, GR128>;
   def  CDSG : CmpSwapRSY<"cdsg", 0xEB3E, z_atomic_cmp_swap_128, GR128>;
 }
 
 // Compare and swap and store.
 let Uses = [R0L, R1D], Defs = [CC], mayStore = 1, mayLoad = 1 in
   def CSST : SideEffectTernarySSF<"csst", 0xC82, GR64>;
 
 // Perform locked operation.
 let Uses = [R0L, R1D], Defs = [CC], mayStore = 1, mayLoad =1 in
   def PLO : SideEffectQuaternarySSe<"plo", 0xEE, GR64>;
 
 // Load/store pair from/to quadword.
 def LPQ  : UnaryRXY<"lpq", 0xE38F, z_atomic_load_128, GR128, 16>;
 def STPQ : StoreRXY<"stpq", 0xE38E, z_atomic_store_128, GR128, 16>;
 
 // Load pair disjoint.
-let Predicates = [FeatureInterlockedAccess1], Defs = [CC] in {
+let Predicates = [FeatureInterlockedAccess1], Defs = [CC], hasSideEffects = 1 in {
   def LPD  : BinarySSF<"lpd", 0xC84, GR128>;
   def LPDG : BinarySSF<"lpdg", 0xC85, GR128>;
 }
 
 //===----------------------------------------------------------------------===//
 // Translate and convert
 //===----------------------------------------------------------------------===//
 
 let mayLoad = 1, mayStore = 1 in
   def TR : SideEffectBinarySSa<"tr", 0xDC>;
 
 let mayLoad = 1, Defs = [CC, R0L, R1D] in {
   def TRT  : SideEffectBinarySSa<"trt", 0xDD>;
   def TRTR : SideEffectBinarySSa<"trtr", 0xD0>;
 }
 
 let mayLoad = 1, mayStore = 1, Uses = [R0L] in
   def TRE : SideEffectBinaryMemMemRRE<"tre", 0xB2A5, GR128, GR64>;
 
-let mayLoad = 1, Uses = [R1D], Defs = [CC] in {
+let mayLoad = 1, Uses = [R1D], Defs = [CC], hasSideEffects = 1 in {
   defm TRTE  : BinaryMemRRFcOpt<"trte",  0xB9BF, GR128, GR64>;
   defm TRTRE : BinaryMemRRFcOpt<"trtre", 0xB9BD, GR128, GR64>;
 }
 
 let mayLoad = 1, mayStore = 1, Uses = [R0L, R1D], Defs = [CC] in {
   defm TROO : SideEffectTernaryMemMemRRFcOpt<"troo", 0xB993, GR128, GR64>;
   defm TROT : SideEffectTernaryMemMemRRFcOpt<"trot", 0xB992, GR128, GR64>;
   defm TRTO : SideEffectTernaryMemMemRRFcOpt<"trto", 0xB991, GR128, GR64>;
let Predicates = [FeatureMessageSecurityAssist8] in
     def KMA : SideEffectTernaryMemMemMemRRFb<"kma", 0xB929,
                                               GR128, GR128, GR128>;
 }
 
 //===----------------------------------------------------------------------===//
 // Guarded storage
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureGuardedStorage] in {
+  let hasSideEffects = 1 in {
     def LGG : UnaryRXY<"lgg", 0xE34C, null_frag, GR64, 8>;
     def LLGFSG : UnaryRXY<"llgfsg", 0xE348, null_frag, GR64, 4>;
-
+  }
   let mayLoad = 1 in
     def LGSC : SideEffectBinaryRXY<"lgsc", 0xE34D, GR64>;
   let mayStore = 1 in
     def STGSC : SideEffectBinaryRXY<"stgsc", 0xE349, GR64>;
 }
 
 //===----------------------------------------------------------------------===//
 // Decimal arithmetic
 //===----------------------------------------------------------------------===//
 
+let hasSideEffects = 1 in {
   defm CVB  : BinaryRXPair<"cvb",0x4F, 0xE306, null_frag, GR32, load, 4>;
   def  CVBG : BinaryRXY<"cvbg", 0xE30E, null_frag, GR64, load, 8>;
 
   defm CVD  : StoreRXPair<"cvd", 0x4E, 0xE326, null_frag, GR32, 4>;
   def  CVDG : StoreRXY<"cvdg", 0xE32E, null_frag, GR64, 8>;
+}
 
 let mayLoad = 1, mayStore = 1 in {
   def MVN : SideEffectBinarySSa<"mvn", 0xD1>;
   def MVZ : SideEffectBinarySSa<"mvz", 0xD3>;
   def MVO : SideEffectBinarySSb<"mvo", 0xF1>;
 
   def PACK : SideEffectBinarySSb<"pack", 0xF2>;
   def PKA  : SideEffectBinarySSf<"pka", 0xE9>;
let mayLoad = 1, mayStore = 1 in {
   def MP : SideEffectBinarySSb<"mp", 0xFC>;
   def DP : SideEffectBinarySSb<"dp", 0xFD>;
   let Defs = [CC] in {
     def ED : SideEffectBinarySSa<"ed", 0xDE>;
     def EDMK : SideEffectBinarySSa<"edmk", 0xDF>;
   }
 }
 
-let Defs = [CC] in {
+let Defs = [CC], hasSideEffects = 1 in {
   def CP : CompareSSb<"cp", 0xF9>;
   def TP : TestRSL<"tp", 0xEBC0>;
 }
 
 //===----------------------------------------------------------------------===//
 // Access registers
 //===----------------------------------------------------------------------===//
 
 // Read a 32-bit access register into a GR32.  As with all GR32 operations,
 // the upper 32 bits of the enclosing GR64 remain unchanged, which is useful
 // when a 64-bit address is stored in a pair of access registers.
+let hasSideEffects = 1 in
   def EAR : UnaryRRE<"ear", 0xB24F, null_frag, GR32, AR32>;
 
 // Set access register.
+let hasSideEffects = 1 in
   def SAR : UnaryRRE<"sar", 0xB24E, null_frag, AR32, GR32>;
 
 // Copy access register.
+let hasSideEffects = 1 in
   def CPYA : UnaryRRE<"cpya", 0xB24D, null_frag, AR32, AR32>;
 
 // Load address extended.
+let hasSideEffects = 1 in
   defm LAE : LoadAddressRXPair<"lae", 0x51, 0xE375, null_frag>;
 
 // Load access multiple.
+let hasSideEffects = 1 in
   defm LAM : LoadMultipleRSPair<"lam", 0x9A, 0xEB9A, AR32>;
 
 // Load access multiple.
+let hasSideEffects = 1 in
   defm STAM : StoreMultipleRSPair<"stam", 0x9B, 0xEB9B, AR32>;
 
 //===----------------------------------------------------------------------===//
 // Program mask and addressing mode
 //===----------------------------------------------------------------------===//
 
 // Extract CC and program mask into a register.  CC ends up in bits 29 and 28.
 let Uses = [CC] in
   def IPM : InherentRRE<"ipm", 0xB222, GR32, z_ipm>;
 
 // Set CC and program mask from a register.
 let hasSideEffects = 1, Defs = [CC] in
   def SPM : SideEffectUnaryRR<"spm", 0x04, GR32>;
 
 // Branch and link - like BAS, but also extracts CC and program mask.
-let isCall = 1, Uses = [CC], Defs = [CC] in {
+let isCall = 1, Uses = [CC], Defs = [CC], hasSideEffects = 1 in {
   def BAL  : CallRX<"bal", 0x45>;
   def BALR : CallRR<"balr", 0x05>;
 }
 
 // Test addressing mode.
 let Defs = [CC] in
   def TAM : SideEffectInherentE<"tam", 0x010B>;
 
 // Set addressing mode.
 let hasSideEffects = 1 in {
   def SAM24 : SideEffectInherentE<"sam24", 0x010C>;
   def SAM31 : SideEffectInherentE<"sam31", 0x010D>;
   def SAM64 : SideEffectInherentE<"sam64", 0x010E>;
 }
 
 // Branch and set mode.  Not really a call, but also sets an output register.
-let isBranch = 1, isTerminator = 1, isBarrier = 1 in
+let isBranch = 1, isTerminator = 1, isBarrier = 1, hasSideEffects = 1 in
   def BSM : CallRR<"bsm", 0x0B>;
 
 // Branch and save and set mode.
-let isCall = 1, Defs = [CC] in
+let isCall = 1, Defs = [CC], hasSideEffects = 1 in
   def BASSM : CallRR<"bassm", 0x0C>;
 
 //===----------------------------------------------------------------------===//
 // Transactional execution
 //===----------------------------------------------------------------------===//
 
 let hasSideEffects = 1, Predicates = [FeatureTransactionalExecution] in {
   // Transaction Begin
   let mayStore = 1, usesCustomInserter = 1, Defs = [CC] in {
     def TBEGIN : SideEffectBinarySIL<"tbegin", 0xE560, z_tbegin, imm32zx16>;
     def TBEGIN_nofloat : SideEffectBinarySILPseudo<z_tbegin_nofloat, imm32zx16>;
 
+    let mayLoad = 1 in
       def TBEGINC : SideEffectBinarySIL<"tbeginc", 0xE561,
                                         int_s390_tbeginc, imm32zx16>;
   }
 
   // Transaction End
   let Defs = [CC] in
     def TEND : SideEffectInherentS<"tend", 0xB2F8, z_tend>;
 
   // Transaction Abort
-  let isTerminator = 1, isBarrier = 1 in
+  let isTerminator = 1, isBarrier = 1, mayLoad = 1, mayStore = 1 in
     def TABORT : SideEffectAddressS<"tabort", 0xB2FC, int_s390_tabort>;
 
   // Nontransactional Store
+  let mayLoad = 1, mayStore = 1 in
     def NTSTG : StoreRXY<"ntstg", 0xE325, int_s390_ntstg, GR64, 8>;
 
   // Extract Transaction Nesting Depth
   def ETND : InherentRRE<"etnd", 0xB2EC, GR32, int_s390_etnd>;
 }
 
 //===----------------------------------------------------------------------===//
 // Processor assist
 //===----------------------------------------------------------------------===//

 //===----------------------------------------------------------------------===//
 // Miscellaneous Instructions.
 //===----------------------------------------------------------------------===//
 
 // Find leftmost one, AKA count leading zeros.  The instruction actually
 // returns a pair of GR64s, the first giving the number of leading zeros
 // and the second giving a copy of the source with the leftmost one bit
 // cleared.  We only use the first result here.
-let Defs = [CC] in
+let Defs = [CC], hasSideEffects = 1 in
   def FLOGR : UnaryRRE<"flogr", 0xB983, null_frag, GR128, GR64>;
 def : Pat<(ctlz GR64:$src),
           (EXTRACT_SUBREG (FLOGR GR64:$src), subreg_h64)>;
 
 // Population count.  Counts bits set per byte.
 let Predicates = [FeaturePopulationCount], Defs = [CC] in
   def POPCNT : UnaryRRE<"popcnt", 0xB9E1, z_popcnt, GR64, GR64>;
 
let hasSideEffects = 1 in {
   def EX   : SideEffectBinaryRX<"ex", 0x44, GR64>;
   def EXRL : SideEffectBinaryRILPC<"exrl", 0xC60, GR64>;
 }
 
 //===----------------------------------------------------------------------===//
 // .insn directive instructions
 //===----------------------------------------------------------------------===//
 
-let isCodeGenOnly = 1 in {
+let isCodeGenOnly = 1, hasSideEffects = 1 in {
   def InsnE   : DirectiveInsnE<(outs), (ins imm64zx16:$enc), ".insn e,$enc", []>;
   def InsnRI  : DirectiveInsnRI<(outs), (ins imm64zx32:$enc, AnyReg:$R1,
                                              imm32sx16:$I2),
                                 ".insn ri,$enc,$R1,$I2", []>;
   def InsnRIE : DirectiveInsnRIE<(outs), (ins imm64zx48:$enc, AnyReg:$R1,
                                               AnyReg:$R3, brtarget16:$I2),
                                  ".insn rie,$enc,$R1,$R3,$I2", []>;
   def InsnRIL : DirectiveInsnRIL<(outs), (ins imm64zx48:$enc, AnyReg:$R1,

lib/Target/SystemZ/SystemZInstrSystem.td

let hasSideEffects = 1 in {
   def SAC : SideEffectAddressS<"sac", 0xB219, null_frag>;
   def SACF : SideEffectAddressS<"sacf", 0xB279, null_frag>;
 }
 
 //===----------------------------------------------------------------------===//
 // Control Register Instructions.
 //===----------------------------------------------------------------------===//
 
+let hasSideEffects = 1 in {
   // Load control.
   def LCTL : LoadMultipleRS<"lctl", 0xB7, CR64>;
   def LCTLG : LoadMultipleRSY<"lctlg", 0xEB2F, CR64>;
 
   // Store control.
   def STCTL : StoreMultipleRS<"stctl", 0xB6, CR64>;
   def STCTG : StoreMultipleRSY<"stctg", 0xEB25, CR64>;
+}
 
 // Extract primary ASN (and instance).
 let hasSideEffects = 1 in {
   def EPAR : InherentRRE<"epar", 0xB226, GR32, null_frag>;
   def EPAIR : InherentRRE<"epair", 0xB99A, GR64, null_frag>;
 }
 
 // Extract secondary ASN (and instance).

 
 // Load real address.
 let hasSideEffects = 1, Defs = [CC] in {
   defm LRA : LoadAddressRXPair<"lra", 0xB1, 0xE313, null_frag>;
   def LRAG : LoadAddressRXY<"lrag", 0xE303, null_frag, laaddr20pair>;
 }
 
 // Store real address.
+let hasSideEffects = 1 in
   def STRAG : StoreSSE<"strag", 0xE502>;
 
 // Load using real address.
-let mayLoad = 1 in {
+let mayLoad = 1, hasSideEffects = 1 in {
  def LURA : UnaryRRE<"lura", 0xB24B, null_frag, GR32, GR64>;
  def LURAG : UnaryRRE<"lurag", 0xB905, null_frag, GR64, GR64>;
 }
 
 // Store using real address.
 let mayStore = 1 in {
  def STURA : SideEffectBinaryRRE<"stura", 0xB246, GR32, GR64>;
  def STURG : SideEffectBinaryRRE<"sturg", 0xB925, GR64, GR64>;
 }
 
 // Test protection.
 let hasSideEffects = 1, Defs = [CC] in
   def TPROT : SideEffectBinarySSE<"tprot", 0xE501>;
 
 //===----------------------------------------------------------------------===//
 // Memory-move Instructions.
 //===----------------------------------------------------------------------===//
 
 // Move with key.
-let mayLoad = 1, mayStore = 1, Defs = [CC] in
+let mayLoad = 1, mayStore = 1, Defs = [CC], hasSideEffects = 1 in
   def MVCK : MemoryBinarySSd<"mvck", 0xD9, GR64>;
 
 // Move to primary / secondary.
-let mayLoad = 1, mayStore = 1, Defs = [CC] in {
+let mayLoad = 1, mayStore = 1, Defs = [CC], hasSideEffects = 1 in {
   def MVCP : MemoryBinarySSd<"mvcp", 0xDA, GR64>;
   def MVCS : MemoryBinarySSd<"mvcs", 0xDB, GR64>;
 }
 
 // Move with source / destination key.
 let mayLoad = 1, mayStore = 1, Uses = [R0L, R1L] in {
   def MVCSK : SideEffectBinarySSE<"mvcsk", 0xE50E>;
   def MVCDK : SideEffectBinarySSE<"mvcdk", 0xE50F>;

lib/Target/SystemZ/SystemZInstrVector.td

 //==- SystemZInstrVector.td - SystemZ Vector instructions ------*- tblgen-*-==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // Move instructions
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureVector] in {
   // Register move.
+  let hasSideEffects = 1 in {
     def VLR : UnaryVRRa<"vlr", 0xE756, null_frag, v128any, v128any>;
     def VLR32 : UnaryAliasVRR<null_frag, v32sb, v32sb>;
     def VLR64 : UnaryAliasVRR<null_frag, v64db, v64db>;
+  }
   
   // Load GR from VR element.
+  let hasSideEffects = 1 in {
     def VLGV  : BinaryVRScGeneric<"vlgv", 0xE721>;
     def VLGVB : BinaryVRSc<"vlgvb", 0xE721, null_frag, v128b, 0>;
     def VLGVH : BinaryVRSc<"vlgvh", 0xE721, null_frag, v128h, 1>;
     def VLGVF : BinaryVRSc<"vlgvf", 0xE721, null_frag, v128f, 2>;
+  }
   def VLGVG : BinaryVRSc<"vlgvg", 0xE721, z_vector_extract, v128g, 3>;
   
   // Load VR element from GR.
+  let hasSideEffects = 1 in
     def VLVG  : TernaryVRSbGeneric<"vlvg", 0xE722>;
   def VLVGB : TernaryVRSb<"vlvgb", 0xE722, z_vector_insert,
                           v128b, v128b, GR32, 0>;
   def VLVGH : TernaryVRSb<"vlvgh", 0xE722, z_vector_insert,
                           v128h, v128h, GR32, 1>;
   def VLVGF : TernaryVRSb<"vlvgf", 0xE722, z_vector_insert,
                           v128f, v128f, GR32, 2>;
   def VLVGG : TernaryVRSb<"vlvgg", 0xE722, z_vector_insert,
                           v128g, v128g, GR64, 3>;
 
   // Load VR from GRs disjoint.
   def VLVGP : BinaryVRRf<"vlvgp", 0xE762, z_join_dwords, v128g>;
+  let hasSideEffects = 1 in
     def VLVGP32 : BinaryAliasVRRf<GR32>;
 }
 
 // Extractions always assign to the full GR64, even if the element would
 // fit in the lower 32 bits.  Sub-i64 extracts therefore need to take a
 // subreg of the result.
 class VectorExtractSubreg<ValueType type, Instruction insn>
   : Pat<(i32 (z_vector_extract (type VR128:$vec), shift12only:$index)),
         (EXTRACT_SUBREG (insn VR128:$vec, shift12only:$index), subreg_l32)>;
 
 def : VectorExtractSubreg<v16i8, VLGVB>;
 def : VectorExtractSubreg<v8i16, VLGVH>;
 def : VectorExtractSubreg<v4i32, VLGVF>;
 
 //===----------------------------------------------------------------------===//
 // Immediate instructions
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureVector] in {
-  let hasSideEffects = 0, isAsCheapAsAMove = 1, isMoveImm = 1,
-      isReMaterializable = 1 in {
+  let isAsCheapAsAMove = 1, isMoveImm = 1, isReMaterializable = 1 in {
 
     // Generate byte mask.
     def VZERO : InherentVRIa<"vzero", 0xE744, 0>;
     def VONE  : InherentVRIa<"vone", 0xE744, 0xffff>;
     def VGBM  : UnaryVRIa<"vgbm", 0xE744, z_byte_mask, v128b, imm32zx16>;
 
     // Generate mask.
     def VGM  : BinaryVRIbGeneric<"vgm", 0xE746>;
def LCBB : InstRXE<0xE727, (outs GR32:$R1),
                        "lcbb\t$R1, $XBD2, $M3",
                        [(set GR32:$R1, (int_s390_lcbb bdxaddr12only:$XBD2,
                                                       imm32zx4:$M3))]>;
 
   // Load with length.  The number of loaded bytes is only known at run time.
   def VLL : BinaryVRSb<"vll", 0xE737, int_s390_vll, 0>;
 
   // Load multiple.
+  let hasSideEffects = 1 in
     def VLM : LoadMultipleVRSa<"vlm", 0xE736>;
 
   // Load and replicate
+  let hasSideEffects = 1 in
     def VLREP  : UnaryVRXGeneric<"vlrep", 0xE705>;
   def VLREPB : UnaryVRX<"vlrepb", 0xE705, z_replicate_loadi8,  v128b, 1, 0>;
   def VLREPH : UnaryVRX<"vlreph", 0xE705, z_replicate_loadi16, v128h, 2, 1>;
   def VLREPF : UnaryVRX<"vlrepf", 0xE705, z_replicate_loadi32, v128f, 4, 2>;
   def VLREPG : UnaryVRX<"vlrepg", 0xE705, z_replicate_loadi64, v128g, 8, 3>;
   def : Pat<(v4f32 (z_replicate_loadf32 bdxaddr12only:$addr)),
             (VLREPF bdxaddr12only:$addr)>;
   def : Pat<(v2f64 (z_replicate_loadf64 bdxaddr12only:$addr)),
             (VLREPG bdxaddr12only:$addr)>;
 
   // Use VLREP to load subvectors.  These patterns use "12pair" because
   // LEY and LDY offer full 20-bit displacement fields.  It's often better
   // to use those instructions rather than force a 20-bit displacement
   // into a GPR temporary.
+  let mayLoad = 1 in {
     def VL32 : UnaryAliasVRX<load, v32sb, bdxaddr12pair>;
     def VL64 : UnaryAliasVRX<load, v64db, bdxaddr12pair>;
+  }
 
   // Load logical element and zero.
+  let hasSideEffects = 1 in
     def VLLEZ  : UnaryVRXGeneric<"vllez", 0xE704>;
   def VLLEZB : UnaryVRX<"vllezb", 0xE704, z_vllezi8,  v128b, 1, 0>;
   def VLLEZH : UnaryVRX<"vllezh", 0xE704, z_vllezi16, v128h, 2, 1>;
   def VLLEZF : UnaryVRX<"vllezf", 0xE704, z_vllezi32, v128f, 4, 2>;
   def VLLEZG : UnaryVRX<"vllezg", 0xE704, z_vllezi64, v128g, 8, 3>;
   def : Pat<(v4f32 (z_vllezf32 bdxaddr12only:$addr)),
             (VLLEZF bdxaddr12only:$addr)>;
   def : Pat<(v2f64 (z_vllezf64 bdxaddr12only:$addr)),
             (VLLEZG bdxaddr12only:$addr)>;
let Predicates = [FeatureVector] in {
   def VLEF : TernaryVRX<"vlef", 0xE703, z_vlei32, v128f, v128f, 4, imm32zx2>;
   def VLEG : TernaryVRX<"vleg", 0xE702, z_vlei64, v128g, v128g, 8, imm32zx1>;
   def : Pat<(z_vlef32 (v4f32 VR128:$val), bdxaddr12only:$addr, imm32zx2:$index),
             (VLEF VR128:$val, bdxaddr12only:$addr, imm32zx2:$index)>;
   def : Pat<(z_vlef64 (v2f64 VR128:$val), bdxaddr12only:$addr, imm32zx1:$index),
             (VLEG VR128:$val, bdxaddr12only:$addr, imm32zx1:$index)>;
 
   // Gather element.
+  let hasSideEffects = 1 in {
     def VGEF : TernaryVRV<"vgef", 0xE713, 4, imm32zx2>;
     def VGEG : TernaryVRV<"vgeg", 0xE712, 8, imm32zx1>;
   }
+}
 
 let Predicates = [FeatureVectorPackedDecimal] in {
   // Load rightmost with length.  The number of loaded bytes is only known
   // at run time.
   def VLRL : BinaryVSI<"vlrl", 0xE635, int_s390_vlrl, 0>;
   def VLRLR : BinaryVRSd<"vlrlr", 0xE637, int_s390_vlrl, 0>;
 }
 

 // Stores
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureVector] in {
   // Store.
   def VST : StoreVRX<"vst", 0xE70E, null_frag, v128any, 16>;
 
   // Store with length.  The number of stored bytes is only known at run time.
+  let mayLoad = 1 in
     def VSTL : StoreLengthVRSb<"vstl", 0xE73F, int_s390_vstl, 0>;
 
   // Store multiple.
+  let hasSideEffects = 1 in
     def VSTM : StoreMultipleVRSa<"vstm", 0xE73E>;
 
   // Store element.
   def VSTEB : StoreBinaryVRX<"vsteb", 0xE708, z_vstei8,  v128b, 1, imm32zx4>;
   def VSTEH : StoreBinaryVRX<"vsteh", 0xE709, z_vstei16, v128h, 2, imm32zx3>;
   def VSTEF : StoreBinaryVRX<"vstef", 0xE70B, z_vstei32, v128f, 4, imm32zx2>;
   def VSTEG : StoreBinaryVRX<"vsteg", 0xE70A, z_vstei64, v128g, 8, imm32zx1>;
   def : Pat<(z_vstef32 (v4f32 VR128:$val), bdxaddr12only:$addr,
                        imm32zx2:$index),
             (VSTEF VR128:$val, bdxaddr12only:$addr, imm32zx2:$index)>;
   def : Pat<(z_vstef64 (v2f64 VR128:$val), bdxaddr12only:$addr,
                        imm32zx1:$index),
             (VSTEG VR128:$val, bdxaddr12only:$addr, imm32zx1:$index)>;
 
   // Use VSTE to store subvectors.  These patterns use "12pair" because
   // STEY and STDY offer full 20-bit displacement fields.  It's often better
   // to use those instructions rather than force a 20-bit displacement
   // into a GPR temporary.
+  let mayStore = 1 in {
     def VST32 : StoreAliasVRX<store, v32sb, bdxaddr12pair>;
     def VST64 : StoreAliasVRX<store, v64db, bdxaddr12pair>;
+  }
 
   // Scatter element.
+  let hasSideEffects = 1 in {
     def VSCEF : StoreBinaryVRV<"vscef", 0xE71B, 4, imm32zx2>;
     def VSCEG : StoreBinaryVRV<"vsceg", 0xE71A, 8, imm32zx1>;
   }
+}
 
-let Predicates = [FeatureVectorPackedDecimal] in {
+let Predicates = [FeatureVectorPackedDecimal], mayLoad = 1 in {
   // Store rightmost with length.  The number of stored bytes is only known
   // at run time.
   def VSTRL : StoreLengthVSI<"vstrl", 0xE63D, int_s390_vstrl, 0>;
   def VSTRLR : StoreLengthVRSd<"vstrlr", 0xE63F, int_s390_vstrl, 0>;
 }
 
 //===----------------------------------------------------------------------===//
 // Selects and permutes
let Predicates = [FeatureVector] in {
   // Permute doubleword immediate.
   def VPDI : TernaryVRRc<"vpdi", 0xE784, z_permute_dwords, v128g, v128g>;
 
   // Bit Permute.
   let Predicates = [FeatureVectorEnhancements1] in
     def VBPERM : BinaryVRRc<"vbperm", 0xE785, int_s390_vbperm, v128g, v128b>;
 
   // Replicate.
+  let hasSideEffects = 1 in
     def VREP:   BinaryVRIcGeneric<"vrep", 0xE74D>;
   def VREPB : BinaryVRIc<"vrepb", 0xE74D, z_splat, v128b, v128b, 0>;
   def VREPH : BinaryVRIc<"vreph", 0xE74D, z_splat, v128h, v128h, 1>;
   def VREPF : BinaryVRIc<"vrepf", 0xE74D, z_splat, v128f, v128f, 2>;
   def VREPG : BinaryVRIc<"vrepg", 0xE74D, z_splat, v128g, v128g, 3>;
   def : Pat<(v4f32 (z_splat VR128:$vec, imm32zx16:$index)),
             (VREPF VR128:$vec, imm32zx16:$index)>;
   def : Pat<(v2f64 (z_splat VR128:$vec, imm32zx16:$index)),
             (VREPG VR128:$vec, imm32zx16:$index)>;
let Predicates = [FeatureVector] in {
   defm VPKLSH : BinaryVRRbSPair<"vpklsh", 0xE795, int_s390_vpklsh, z_packls_cc,
                                 v128b, v128h, 1>;
   defm VPKLSF : BinaryVRRbSPair<"vpklsf", 0xE795, int_s390_vpklsf, z_packls_cc,
                                 v128h, v128f, 2>;
   defm VPKLSG : BinaryVRRbSPair<"vpklsg", 0xE795, int_s390_vpklsg, z_packls_cc,
                                 v128f, v128g, 3>;
 
   // Sign-extend to doubleword.
+  let hasSideEffects = 1 in
     def VSEG  : UnaryVRRaGeneric<"vseg", 0xE75F>;
   def VSEGB : UnaryVRRa<"vsegb", 0xE75F, z_vsei8,  v128g, v128g, 0>;
   def VSEGH : UnaryVRRa<"vsegh", 0xE75F, z_vsei16, v128g, v128g, 1>;
   def VSEGF : UnaryVRRa<"vsegf", 0xE75F, z_vsei32, v128g, v128g, 2>;
   def : Pat<(z_vsei8_by_parts  (v16i8 VR128:$src)), (VSEGB VR128:$src)>;
   def : Pat<(z_vsei16_by_parts (v8i16 VR128:$src)), (VSEGH VR128:$src)>;
   def : Pat<(z_vsei32_by_parts (v4i32 VR128:$src)), (VSEGF VR128:$src)>;
 
   // Unpack high.
+  let hasSideEffects = 1 in
     def VUPH  : UnaryVRRaGeneric<"vuph", 0xE7D7>;
   def VUPHB : UnaryVRRa<"vuphb", 0xE7D7, z_unpack_high, v128h, v128b, 0>;
   def VUPHH : UnaryVRRa<"vuphh", 0xE7D7, z_unpack_high, v128f, v128h, 1>;
   def VUPHF : UnaryVRRa<"vuphf", 0xE7D7, z_unpack_high, v128g, v128f, 2>;
 
   // Unpack logical high.
+  let hasSideEffects = 1 in
     def VUPLH  : UnaryVRRaGeneric<"vuplh", 0xE7D5>;
   def VUPLHB : UnaryVRRa<"vuplhb", 0xE7D5, z_unpackl_high, v128h, v128b, 0>;
   def VUPLHH : UnaryVRRa<"vuplhh", 0xE7D5, z_unpackl_high, v128f, v128h, 1>;
   def VUPLHF : UnaryVRRa<"vuplhf", 0xE7D5, z_unpackl_high, v128g, v128f, 2>;
 
   // Unpack low.
+  let hasSideEffects = 1 in
     def VUPL   : UnaryVRRaGeneric<"vupl", 0xE7D6>;
   def VUPLB  : UnaryVRRa<"vuplb",  0xE7D6, z_unpack_low, v128h, v128b, 0>;
   def VUPLHW : UnaryVRRa<"vuplhw", 0xE7D6, z_unpack_low, v128f, v128h, 1>;
   def VUPLF  : UnaryVRRa<"vuplf",  0xE7D6, z_unpack_low, v128g, v128f, 2>;
 
   // Unpack logical low.
+  let hasSideEffects = 1 in
     def VUPLL  : UnaryVRRaGeneric<"vupll", 0xE7D4>;
   def VUPLLB : UnaryVRRa<"vupllb", 0xE7D4, z_unpackl_low, v128h, v128b, 0>;
   def VUPLLH : UnaryVRRa<"vupllh", 0xE7D4, z_unpackl_low, v128f, v128h, 1>;
   def VUPLLF : UnaryVRRa<"vupllf", 0xE7D4, z_unpackl_low, v128g, v128f, 2>;
 }
 
 //===----------------------------------------------------------------------===//
 // Instantiating generic operations for specific types.
 //===----------------------------------------------------------------------===//
let Predicates = [FeatureVector] in {
   def VAVGLH : BinaryVRRc<"vavglh", 0xE7F0, int_s390_vavglh, v128h, v128h, 1>;
   def VAVGLF : BinaryVRRc<"vavglf", 0xE7F0, int_s390_vavglf, v128f, v128f, 2>;
   def VAVGLG : BinaryVRRc<"vavglg", 0xE7F0, int_s390_vavglg, v128g, v128g, 3>;
 
   // Checksum.
   def VCKSM : BinaryVRRc<"vcksm", 0xE766, int_s390_vcksm, v128f, v128f>;
 
   // Count leading zeros.
+  let hasSideEffects = 1 in
     def VCLZ  : UnaryVRRaGeneric<"vclz", 0xE753>;
   def VCLZB : UnaryVRRa<"vclzb", 0xE753, ctlz, v128b, v128b, 0>;
   def VCLZH : UnaryVRRa<"vclzh", 0xE753, ctlz, v128h, v128h, 1>;
   def VCLZF : UnaryVRRa<"vclzf", 0xE753, ctlz, v128f, v128f, 2>;
   def VCLZG : UnaryVRRa<"vclzg", 0xE753, ctlz, v128g, v128g, 3>;
 
   // Count trailing zeros.
+  let hasSideEffects = 1 in
     def VCTZ  : UnaryVRRaGeneric<"vctz", 0xE752>;
   def VCTZB : UnaryVRRa<"vctzb", 0xE752, cttz, v128b, v128b, 0>;
   def VCTZH : UnaryVRRa<"vctzh", 0xE752, cttz, v128h, v128h, 1>;
   def VCTZF : UnaryVRRa<"vctzf", 0xE752, cttz, v128f, v128f, 2>;
   def VCTZG : UnaryVRRa<"vctzg", 0xE752, cttz, v128g, v128g, 3>;
 
   // Not exclusive or.
   let Predicates = [FeatureVectorEnhancements1] in
     def VNX : BinaryVRRc<"vnx", 0xE76C, null_frag, v128any, v128any>;
let Predicates = [FeatureVector] in {
   // Galois field multiply sum and accumulate.
   def VGFMA  : TernaryVRRdGeneric<"vgfma", 0xE7BC>;
   def VGFMAB : TernaryVRRd<"vgfmab", 0xE7BC, int_s390_vgfmab, v128h, v128b, 0>;
   def VGFMAH : TernaryVRRd<"vgfmah", 0xE7BC, int_s390_vgfmah, v128f, v128h, 1>;
   def VGFMAF : TernaryVRRd<"vgfmaf", 0xE7BC, int_s390_vgfmaf, v128g, v128f, 2>;
   def VGFMAG : TernaryVRRd<"vgfmag", 0xE7BC, int_s390_vgfmag, v128q, v128g, 3>;
 
   // Load complement.
+  let hasSideEffects = 1 in
     def VLC  : UnaryVRRaGeneric<"vlc", 0xE7DE>;
   def VLCB : UnaryVRRa<"vlcb", 0xE7DE, z_vneg, v128b, v128b, 0>;
   def VLCH : UnaryVRRa<"vlch", 0xE7DE, z_vneg, v128h, v128h, 1>;
   def VLCF : UnaryVRRa<"vlcf", 0xE7DE, z_vneg, v128f, v128f, 2>;
   def VLCG : UnaryVRRa<"vlcg", 0xE7DE, z_vneg, v128g, v128g, 3>;
 
   // Load positive.
+  let hasSideEffects = 1 in
     def VLP  : UnaryVRRaGeneric<"vlp", 0xE7DF>;
   def VLPB : UnaryVRRa<"vlpb", 0xE7DF, z_viabs8,  v128b, v128b, 0>;
   def VLPH : UnaryVRRa<"vlph", 0xE7DF, z_viabs16, v128h, v128h, 1>;
   def VLPF : UnaryVRRa<"vlpf", 0xE7DF, z_viabs32, v128f, v128f, 2>;
   def VLPG : UnaryVRRa<"vlpg", 0xE7DF, z_viabs64, v128g, v128g, 3>;
 
   // Maximum.
   def VMX  : BinaryVRRcGeneric<"vmx", 0xE7FF>;
   def VMXB : BinaryVRRc<"vmxb", 0xE7FF, null_frag, v128b, v128b, 0>;
let Predicates = [FeatureVector] in {
   // Multiply logical odd.
   def VMLO  : BinaryVRRcGeneric<"vmlo", 0xE7A5>;
   def VMLOB : BinaryVRRc<"vmlob", 0xE7A5, int_s390_vmlob, v128h, v128b, 0>;
   def VMLOH : BinaryVRRc<"vmloh", 0xE7A5, int_s390_vmloh, v128f, v128h, 1>;
   def VMLOF : BinaryVRRc<"vmlof", 0xE7A5, int_s390_vmlof, v128g, v128f, 2>;
 
   // Multiply sum logical.
   let Predicates = [FeatureVectorEnhancements1] in {
+    let hasSideEffects = 1 in
       def VMSL  : QuaternaryVRRdGeneric<"vmsl", 0xE7B8>;
     def VMSLG : QuaternaryVRRd<"vmslg", 0xE7B8, int_s390_vmslg,
                                v128q, v128g, v128g, v128q, 3>;
   }
 
   // Nand.
   let Predicates = [FeatureVectorEnhancements1] in
     def VNN : BinaryVRRc<"vnn", 0xE76E, null_frag, v128any, v128any>;
 
let Predicates = [FeatureVector] in {
   let Predicates = [FeatureVectorEnhancements1] in {
     def VPOPCTB : UnaryVRRa<"vpopctb", 0xE750, ctpop, v128b, v128b, 0>;
     def VPOPCTH : UnaryVRRa<"vpopcth", 0xE750, ctpop, v128h, v128h, 1>;
     def VPOPCTF : UnaryVRRa<"vpopctf", 0xE750, ctpop, v128f, v128f, 2>;
     def VPOPCTG : UnaryVRRa<"vpopctg", 0xE750, ctpop, v128g, v128g, 3>;
   }
 
   // Element rotate left logical (with vector shift amount).
+  let hasSideEffects = 1 in
     def VERLLV  : BinaryVRRcGeneric<"verllv", 0xE773>;
   def VERLLVB : BinaryVRRc<"verllvb", 0xE773, int_s390_verllvb,
                            v128b, v128b, 0>;
   def VERLLVH : BinaryVRRc<"verllvh", 0xE773, int_s390_verllvh,
                            v128h, v128h, 1>;
   def VERLLVF : BinaryVRRc<"verllvf", 0xE773, int_s390_verllvf,
                            v128f, v128f, 2>;
   def VERLLVG : BinaryVRRc<"verllvg", 0xE773, int_s390_verllvg,
                            v128g, v128g, 3>;
 
   // Element rotate left logical (with scalar shift amount).
   def VERLL  : BinaryVRSaGeneric<"verll", 0xE733>;
   def VERLLB : BinaryVRSa<"verllb", 0xE733, int_s390_verllb, v128b, v128b, 0>;
   def VERLLH : BinaryVRSa<"verllh", 0xE733, int_s390_verllh, v128h, v128h, 1>;
   def VERLLF : BinaryVRSa<"verllf", 0xE733, int_s390_verllf, v128f, v128f, 2>;
   def VERLLG : BinaryVRSa<"verllg", 0xE733, int_s390_verllg, v128g, v128g, 3>;
 
   // Element rotate and insert under mask.
+  let hasSideEffects = 1 in
     def VERIM  : QuaternaryVRIdGeneric<"verim", 0xE772>;
   def VERIMB : QuaternaryVRId<"verimb", 0xE772, int_s390_verimb, v128b, v128b, 0>;
   def VERIMH : QuaternaryVRId<"verimh", 0xE772, int_s390_verimh, v128h, v128h, 1>;
   def VERIMF : QuaternaryVRId<"verimf", 0xE772, int_s390_verimf, v128f, v128f, 2>;
   def VERIMG : QuaternaryVRId<"verimg", 0xE772, int_s390_verimg, v128g, v128g, 3>;
 
   // Element shift left (with vector shift amount).
   def VESLV  : BinaryVRRcGeneric<"veslv", 0xE770>;
   def VESLVB : BinaryVRRc<"veslvb", 0xE770, z_vshl, v128b, v128b, 0>;

 defm : IntegerMinMaxVectorOps<v2i64, z_vicmphl, VMNLG, VMXLG>;
 
 //===----------------------------------------------------------------------===//
 // Integer comparison
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureVector] in {
   // Element compare.
-  let Defs = [CC] in {
+  let Defs = [CC], hasSideEffects = 1 in {
     def VEC  : CompareVRRaGeneric<"vec", 0xE7DB>;
     def VECB : CompareVRRa<"vecb", 0xE7DB, null_frag, v128b, 0>;
     def VECH : CompareVRRa<"vech", 0xE7DB, null_frag, v128h, 1>;
     def VECF : CompareVRRa<"vecf", 0xE7DB, null_frag, v128f, 2>;
     def VECG : CompareVRRa<"vecg", 0xE7DB, null_frag, v128g, 3>;
   }
 
   // Element compare logical.
-  let Defs = [CC] in {
+  let Defs = [CC], hasSideEffects = 1 in {
     def VECL  : CompareVRRaGeneric<"vecl", 0xE7D9>;
     def VECLB : CompareVRRa<"veclb", 0xE7D9, null_frag, v128b, 0>;
     def VECLH : CompareVRRa<"veclh", 0xE7D9, null_frag, v128h, 1>;
     def VECLF : CompareVRRa<"veclf", 0xE7D9, null_frag, v128f, 2>;
     def VECLG : CompareVRRa<"veclg", 0xE7D9, null_frag, v128g, 3>;
   }
 
   // Compare equal.
let Predicates = [FeatureVectorEnhancements1] in {
     def VFASB : BinaryVRRc<"vfasb", 0xE7E3, fadd, v128sb, v128sb, 2, 0>;
     def WFASB : BinaryVRRc<"wfasb", 0xE7E3, fadd, v32sb, v32sb, 2, 8>;
     def WFAXB : BinaryVRRc<"wfaxb", 0xE7E3, fadd, v128xb, v128xb, 4, 8>;
   }
 
   // Convert from fixed 64-bit.
   def VCDG  : TernaryVRRaFloatGeneric<"vcdg", 0xE7C3>;
   def VCDGB : TernaryVRRa<"vcdgb", 0xE7C3, null_frag, v128db, v128g, 3, 0>;
+  let hasSideEffects = 1 in
     def WCDGB : TernaryVRRa<"wcdgb", 0xE7C3, null_frag, v64db, v64g, 3, 8>;
   def : FPConversion<VCDGB, sint_to_fp, v128db, v128g, 0, 0>;
 
   // Convert from logical 64-bit.
   def VCDLG  : TernaryVRRaFloatGeneric<"vcdlg", 0xE7C1>;
   def VCDLGB : TernaryVRRa<"vcdlgb", 0xE7C1, null_frag, v128db, v128g, 3, 0>;
+  let hasSideEffects = 1 in
     def WCDLGB : TernaryVRRa<"wcdlgb", 0xE7C1, null_frag, v64db, v64g, 3, 8>;
   def : FPConversion<VCDLGB, uint_to_fp, v128db, v128g, 0, 0>;
 
   // Convert to fixed 64-bit.
   def VCGD  : TernaryVRRaFloatGeneric<"vcgd", 0xE7C2>;
   def VCGDB : TernaryVRRa<"vcgdb", 0xE7C2, null_frag, v128g, v128db, 3, 0>;
+  let hasSideEffects = 1 in
     def WCGDB : TernaryVRRa<"wcgdb", 0xE7C2, null_frag, v64g, v64db, 3, 8>;
   // Rounding mode should agree with SystemZInstrFP.td.
   def : FPConversion<VCGDB, fp_to_sint, v128g, v128db, 0, 5>;
 
   // Convert to logical 64-bit.
   def VCLGD  : TernaryVRRaFloatGeneric<"vclgd", 0xE7C0>;
   def VCLGDB : TernaryVRRa<"vclgdb", 0xE7C0, null_frag, v128g, v128db, 3, 0>;
+  let hasSideEffects = 1 in
     def WCLGDB : TernaryVRRa<"wclgdb", 0xE7C0, null_frag, v64g, v64db, 3, 8>;
   // Rounding mode should agree with SystemZInstrFP.td.
   def : FPConversion<VCLGDB, fp_to_uint, v128g, v128db, 0, 5>;
 
   // Divide.
   def VFD   : BinaryVRRcFloatGeneric<"vfd", 0xE7E5>;
   def VFDDB : BinaryVRRc<"vfddb", 0xE7E5, fdiv, v128db, v128db, 3, 0>;
   def WFDDB : BinaryVRRc<"wfddb", 0xE7E5, fdiv, v64db, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
let Predicates = [FeatureVectorEnhancements1] in {
     def WFISB : TernaryVRRa<"wfisb", 0xE7C7, null_frag, v32sb, v32sb, 2, 8>;
     def WFIXB : TernaryVRRa<"wfixb", 0xE7C7, null_frag, v128xb, v128xb, 4, 8>;
     defm : VectorRounding<VFISB, v128sb>;
     defm : VectorRounding<WFISB, v32sb>;
     defm : VectorRounding<WFIXB, v128xb>;
   }
 
   // Load lengthened.
+  let hasSideEffects = 1 in
     def VLDE  : UnaryVRRaFloatGeneric<"vlde", 0xE7C4>;
   def VLDEB : UnaryVRRa<"vldeb", 0xE7C4, z_vextend, v128db, v128sb, 2, 0>;
   def WLDEB : UnaryVRRa<"wldeb", 0xE7C4, fpextend, v64db, v32sb, 2, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     let isAsmParserOnly = 1 in {
+      let hasSideEffects = 1 in {
         def VFLL  : UnaryVRRaFloatGeneric<"vfll", 0xE7C4>;
         def VFLLS : UnaryVRRa<"vflls", 0xE7C4, null_frag, v128db, v128sb, 2, 0>;
         def WFLLS : UnaryVRRa<"wflls", 0xE7C4, null_frag, v64db, v32sb, 2, 8>;
       }
+    }
     def WFLLD : UnaryVRRa<"wflld", 0xE7C4, fpextend, v128xb, v64db, 3, 8>;
     def : Pat<(f128 (fpextend (f32 VR32:$src))),
               (WFLLD (WLDEB VR32:$src))>;
   }
 
   // Load rounded.
   def VLED  : TernaryVRRaFloatGeneric<"vled", 0xE7C5>;
   def VLEDB : TernaryVRRa<"vledb", 0xE7C5, null_frag, v128sb, v128db, 3, 0>;
   def WLEDB : TernaryVRRa<"wledb", 0xE7C5, null_frag, v32sb, v64db, 3, 8>;
   def : Pat<(v4f32 (z_vround (v2f64 VR128:$src))), (VLEDB VR128:$src, 0, 0)>;
   def : FPConversion<WLEDB, fpround, v32sb, v64db, 0, 0>;
   let Predicates = [FeatureVectorEnhancements1] in {
     let isAsmParserOnly = 1 in {
       def VFLR  : TernaryVRRaFloatGeneric<"vflr", 0xE7C5>;
+      let hasSideEffects = 1 in {
         def VFLRD : TernaryVRRa<"vflrd", 0xE7C5, null_frag, v128sb, v128db, 3, 0>;
         def WFLRD : TernaryVRRa<"wflrd", 0xE7C5, null_frag, v32sb, v64db, 3, 8>;
       }
+    }
     def WFLRX : TernaryVRRa<"wflrx", 0xE7C5, null_frag, v64db, v128xb, 4, 8>;
     def : FPConversion<WFLRX, fpround, v64db, v128xb, 0, 0>;
     def : Pat<(f32 (fpround (f128 VR128:$src))),
               (WLEDB (WFLRX VR128:$src, 0, 3), 0, 0)>;
   }
 
   // Maximum.
   multiclass VectorMax<Instruction insn, TypedReg tr> {
     def : FPMinMax<insn, fmaxnum, tr, 4>;
     def : FPMinMax<insn, fmaxnan, tr, 1>;
   }
   let Predicates = [FeatureVectorEnhancements1] in {
+    let hasSideEffects = 1 in
       def VFMAX   : TernaryVRRcFloatGeneric<"vfmax", 0xE7EF>;
     def VFMAXDB : TernaryVRRcFloat<"vfmaxdb", 0xE7EF, int_s390_vfmaxdb,
                                    v128db, v128db, 3, 0>;
     def WFMAXDB : TernaryVRRcFloat<"wfmaxdb", 0xE7EF, null_frag,
                                    v64db, v64db, 3, 8>;
     def VFMAXSB : TernaryVRRcFloat<"vfmaxsb", 0xE7EF, int_s390_vfmaxsb,
                                    v128sb, v128sb, 2, 0>;
     def WFMAXSB : TernaryVRRcFloat<"wfmaxsb", 0xE7EF, null_frag,
                                    v32sb, v32sb, 2, 8>;
     def WFMAXXB : TernaryVRRcFloat<"wfmaxxb", 0xE7EF, null_frag,
                                    v128xb, v128xb, 4, 8>;
     defm : VectorMax<VFMAXDB, v128db>;
     defm : VectorMax<WFMAXDB, v64db>;
     defm : VectorMax<VFMAXSB, v128sb>;
     defm : VectorMax<WFMAXSB, v32sb>;
     defm : VectorMax<WFMAXXB, v128xb>;
   }
 
   // Minimum.
   multiclass VectorMin<Instruction insn, TypedReg tr> {
     def : FPMinMax<insn, fminnum, tr, 4>;
     def : FPMinMax<insn, fminnan, tr, 1>;
   }
   let Predicates = [FeatureVectorEnhancements1] in {
+    let hasSideEffects = 1 in
       def VFMIN   : TernaryVRRcFloatGeneric<"vfmin", 0xE7EE>;
     def VFMINDB : TernaryVRRcFloat<"vfmindb", 0xE7EE, int_s390_vfmindb,
                                    v128db, v128db, 3, 0>;
     def WFMINDB : TernaryVRRcFloat<"wfmindb", 0xE7EE, null_frag,
                                    v64db, v64db, 3, 8>;
     def VFMINSB : TernaryVRRcFloat<"vfminsb", 0xE7EE, int_s390_vfminsb,
                                    v128sb, v128sb, 2, 0>;
     def WFMINSB : TernaryVRRcFloat<"wfminsb", 0xE7EE, null_frag,
                                    v32sb, v32sb, 2, 8>;
let Predicates = [FeatureVector] in {
   def WFMDB : BinaryVRRc<"wfmdb", 0xE7E7, fmul, v64db, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFMSB : BinaryVRRc<"vfmsb", 0xE7E7, fmul, v128sb, v128sb, 2, 0>;
     def WFMSB : BinaryVRRc<"wfmsb", 0xE7E7, fmul, v32sb, v32sb, 2, 8>;
     def WFMXB : BinaryVRRc<"wfmxb", 0xE7E7, fmul, v128xb, v128xb, 4, 8>;
   }
 
   // Multiply and add.
+  let hasSideEffects = 1 in
     def VFMA   : TernaryVRReFloatGeneric<"vfma", 0xE78F>;
   def VFMADB : TernaryVRRe<"vfmadb", 0xE78F, fma, v128db, v128db, 0, 3>;
   def WFMADB : TernaryVRRe<"wfmadb", 0xE78F, fma, v64db, v64db, 8, 3>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFMASB : TernaryVRRe<"vfmasb", 0xE78F, fma, v128sb, v128sb, 0, 2>;
     def WFMASB : TernaryVRRe<"wfmasb", 0xE78F, fma, v32sb, v32sb, 8, 2>;
     def WFMAXB : TernaryVRRe<"wfmaxb", 0xE78F, fma, v128xb, v128xb, 8, 4>;
   }
 
   // Multiply and subtract.
+  let hasSideEffects = 1 in
     def VFMS   : TernaryVRReFloatGeneric<"vfms", 0xE78E>;
   def VFMSDB : TernaryVRRe<"vfmsdb", 0xE78E, fms, v128db, v128db, 0, 3>;
   def WFMSDB : TernaryVRRe<"wfmsdb", 0xE78E, fms, v64db, v64db, 8, 3>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFMSSB : TernaryVRRe<"vfmssb", 0xE78E, fms, v128sb, v128sb, 0, 2>;
     def WFMSSB : TernaryVRRe<"wfmssb", 0xE78E, fms, v32sb, v32sb, 8, 2>;
     def WFMSXB : TernaryVRRe<"wfmsxb", 0xE78E, fms, v128xb, v128xb, 8, 4>;
   }
 
   // Negative multiply and add.
   let Predicates = [FeatureVectorEnhancements1] in {
+    let hasSideEffects = 1 in
       def VFNMA   : TernaryVRReFloatGeneric<"vfnma", 0xE79F>;
     def VFNMADB : TernaryVRRe<"vfnmadb", 0xE79F, fnma, v128db, v128db, 0, 3>;
     def WFNMADB : TernaryVRRe<"wfnmadb", 0xE79F, fnma, v64db, v64db, 8, 3>;
     def VFNMASB : TernaryVRRe<"vfnmasb", 0xE79F, fnma, v128sb, v128sb, 0, 2>;
     def WFNMASB : TernaryVRRe<"wfnmasb", 0xE79F, fnma, v32sb, v32sb, 8, 2>;
     def WFNMAXB : TernaryVRRe<"wfnmaxb", 0xE79F, fnma, v128xb, v128xb, 8, 4>;
   }
 
   // Negative multiply and subtract.
   let Predicates = [FeatureVectorEnhancements1] in {
+    let hasSideEffects = 1 in
       def VFNMS   : TernaryVRReFloatGeneric<"vfnms", 0xE79E>;
     def VFNMSDB : TernaryVRRe<"vfnmsdb", 0xE79E, fnms, v128db, v128db, 0, 3>;
     def WFNMSDB : TernaryVRRe<"wfnmsdb", 0xE79E, fnms, v64db, v64db, 8, 3>;
     def VFNMSSB : TernaryVRRe<"vfnmssb", 0xE79E, fnms, v128sb, v128sb, 0, 2>;
     def WFNMSSB : TernaryVRRe<"wfnmssb", 0xE79E, fnms, v32sb, v32sb, 8, 2>;
     def WFNMSXB : TernaryVRRe<"wfnmsxb", 0xE79E, fnms, v128xb, v128xb, 8, 4>;
   }
 
   // Perform sign operation.
+  let hasSideEffects = 1 in {
     def VFPSO   : BinaryVRRaFloatGeneric<"vfpso", 0xE7CC>;
     def VFPSODB : BinaryVRRa<"vfpsodb", 0xE7CC, null_frag, v128db, v128db, 3, 0>;
     def WFPSODB : BinaryVRRa<"wfpsodb", 0xE7CC, null_frag, v64db, v64db, 3, 8>;
     let Predicates = [FeatureVectorEnhancements1] in {
       def VFPSOSB : BinaryVRRa<"vfpsosb", 0xE7CC, null_frag, v128sb, v128sb, 2, 0>;
       def WFPSOSB : BinaryVRRa<"wfpsosb", 0xE7CC, null_frag, v32sb, v32sb, 2, 8>;
       def WFPSOXB : BinaryVRRa<"wfpsoxb", 0xE7CC, null_frag, v128xb, v128xb, 4, 8>;
     }
+  }
   
   // Load complement.
   def VFLCDB : UnaryVRRa<"vflcdb", 0xE7CC, fneg, v128db, v128db, 3, 0, 0>;
   def WFLCDB : UnaryVRRa<"wflcdb", 0xE7CC, fneg, v64db, v64db, 3, 8, 0>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFLCSB : UnaryVRRa<"vflcsb", 0xE7CC, fneg, v128sb, v128sb, 2, 0, 0>;
     def WFLCSB : UnaryVRRa<"wflcsb", 0xE7CC, fneg, v32sb, v32sb, 2, 8, 0>;
     def WFLCXB : UnaryVRRa<"wflcxb", 0xE7CC, fneg, v128xb, v128xb, 4, 8, 0>;
   }
let Predicates = [FeatureVector] in {
   def WFLPDB : UnaryVRRa<"wflpdb", 0xE7CC, fabs, v64db, v64db, 3, 8, 2>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFLPSB : UnaryVRRa<"vflpsb", 0xE7CC, fabs, v128sb, v128sb, 2, 0, 2>;
     def WFLPSB : UnaryVRRa<"wflpsb", 0xE7CC, fabs, v32sb, v32sb, 2, 8, 2>;
     def WFLPXB : UnaryVRRa<"wflpxb", 0xE7CC, fabs, v128xb, v128xb, 4, 8, 2>;
   }
 
   // Square root.
+  let hasSideEffects = 1 in
     def VFSQ   : UnaryVRRaFloatGeneric<"vfsq", 0xE7CE>;
   def VFSQDB : UnaryVRRa<"vfsqdb", 0xE7CE, fsqrt, v128db, v128db, 3, 0>;
   def WFSQDB : UnaryVRRa<"wfsqdb", 0xE7CE, fsqrt, v64db, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFSQSB : UnaryVRRa<"vfsqsb", 0xE7CE, fsqrt, v128sb, v128sb, 2, 0>;
     def WFSQSB : UnaryVRRa<"wfsqsb", 0xE7CE, fsqrt, v32sb, v32sb, 2, 8>;
     def WFSQXB : UnaryVRRa<"wfsqxb", 0xE7CE, fsqrt, v128xb, v128xb, 4, 8>;
   }
 
   // Subtract.
   def VFS   : BinaryVRRcFloatGeneric<"vfs", 0xE7E2>;
   def VFSDB : BinaryVRRc<"vfsdb", 0xE7E2, fsub, v128db, v128db, 3, 0>;
   def WFSDB : BinaryVRRc<"wfsdb", 0xE7E2, fsub, v64db, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     def VFSSB : BinaryVRRc<"vfssb", 0xE7E2, fsub, v128sb, v128sb, 2, 0>;
     def WFSSB : BinaryVRRc<"wfssb", 0xE7E2, fsub, v32sb, v32sb, 2, 8>;
     def WFSXB : BinaryVRRc<"wfsxb", 0xE7E2, fsub, v128xb, v128xb, 4, 8>;
   }
 
   // Test data class immediate.
   let Defs = [CC] in {
+    let hasSideEffects = 1 in
       def VFTCI   : BinaryVRIeFloatGeneric<"vftci", 0xE74A>;
     def VFTCIDB : BinaryVRIe<"vftcidb", 0xE74A, z_vftci, v128g, v128db, 3, 0>;
+    let hasSideEffects = 1 in
       def WFTCIDB : BinaryVRIe<"wftcidb", 0xE74A, null_frag, v64g, v64db, 3, 8>;
     let Predicates = [FeatureVectorEnhancements1] in {
       def VFTCISB : BinaryVRIe<"vftcisb", 0xE74A, z_vftci, v128f, v128sb, 2, 0>;
+      let hasSideEffects = 1 in {
         def WFTCISB : BinaryVRIe<"wftcisb", 0xE74A, null_frag, v32f, v32sb, 2, 8>;
         def WFTCIXB : BinaryVRIe<"wftcixb", 0xE74A, null_frag, v128q, v128xb, 4, 8>;
       }
     }
   }
+}
 
 //===----------------------------------------------------------------------===//
 // Floating-point comparison
 //===----------------------------------------------------------------------===//
 
 let Predicates = [FeatureVector] in {
   // Compare scalar.
   let Defs = [CC] in {
+    let hasSideEffects = 1 in
       def WFC   : CompareVRRaFloatGeneric<"wfc", 0xE7CB>;
     def WFCDB : CompareVRRa<"wfcdb", 0xE7CB, z_fcmp, v64db, 3>;
     let Predicates = [FeatureVectorEnhancements1] in {
       def WFCSB : CompareVRRa<"wfcsb", 0xE7CB, z_fcmp, v32sb, 2>;
       def WFCXB : CompareVRRa<"wfcxb", 0xE7CB, z_fcmp, v128xb, 4>;
     }
   }
 
   // Compare and signal scalar.
-  let Defs = [CC] in {
+  let Defs = [CC], hasSideEffects = 1 in {
     def WFK   : CompareVRRaFloatGeneric<"wfk", 0xE7CA>;
     def WFKDB : CompareVRRa<"wfkdb", 0xE7CA, null_frag, v64db, 3>;
     let Predicates = [FeatureVectorEnhancements1] in {
       def WFKSB : CompareVRRa<"wfksb", 0xE7CA, null_frag, v32sb, 2>;
       def WFKXB : CompareVRRa<"wfkxb", 0xE7CA, null_frag, v128xb, 4>;
     }
   }
 
   // Compare equal.
   def  VFCE   : BinaryVRRcSPairFloatGeneric<"vfce", 0xE7E8>;
   defm VFCEDB : BinaryVRRcSPair<"vfcedb", 0xE7E8, z_vfcmpe, z_vfcmpes,
                                 v128g, v128db, 3, 0>;
+  let hasSideEffects = 1 in
     defm WFCEDB : BinaryVRRcSPair<"wfcedb", 0xE7E8, null_frag, null_frag,
                                   v64g, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     defm VFCESB : BinaryVRRcSPair<"vfcesb", 0xE7E8, z_vfcmpe, z_vfcmpes,
                                   v128f, v128sb, 2, 0>;
+    let hasSideEffects = 1 in {
       defm WFCESB : BinaryVRRcSPair<"wfcesb", 0xE7E8, null_frag, null_frag,
                                     v32f, v32sb, 2, 8>;
       defm WFCEXB : BinaryVRRcSPair<"wfcexb", 0xE7E8, null_frag, null_frag,
                                     v128q, v128xb, 4, 8>;
     }
+  }
 
   // Compare and signal equal.
-  let Predicates = [FeatureVectorEnhancements1] in {
+  let Predicates = [FeatureVectorEnhancements1], hasSideEffects = 1 in {
     defm VFKEDB : BinaryVRRcSPair<"vfkedb", 0xE7E8, null_frag, null_frag,
                                   v128g, v128db, 3, 4>;
     defm WFKEDB : BinaryVRRcSPair<"wfkedb", 0xE7E8, null_frag, null_frag,
                                   v64g, v64db, 3, 12>;
     defm VFKESB : BinaryVRRcSPair<"vfkesb", 0xE7E8, null_frag, null_frag,
                                   v128f, v128sb, 2, 4>;
     defm WFKESB : BinaryVRRcSPair<"wfkesb", 0xE7E8, null_frag, null_frag,
                                   v32f, v32sb, 2, 12>;
     defm WFKEXB : BinaryVRRcSPair<"wfkexb", 0xE7E8, null_frag, null_frag,
                                   v128q, v128xb, 4, 12>;
   }
 
   // Compare high.
   def  VFCH   : BinaryVRRcSPairFloatGeneric<"vfch", 0xE7EB>;
   defm VFCHDB : BinaryVRRcSPair<"vfchdb", 0xE7EB, z_vfcmph, z_vfcmphs,
                                 v128g, v128db, 3, 0>;
+  let hasSideEffects = 1 in
     defm WFCHDB : BinaryVRRcSPair<"wfchdb", 0xE7EB, null_frag, null_frag,
                                   v64g, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     defm VFCHSB : BinaryVRRcSPair<"vfchsb", 0xE7EB, z_vfcmph, z_vfcmphs,
                                   v128f, v128sb, 2, 0>;
+    let hasSideEffects = 1 in {
       defm WFCHSB : BinaryVRRcSPair<"wfchsb", 0xE7EB, null_frag, null_frag,
                                     v32f, v32sb, 2, 8>;
       defm WFCHXB : BinaryVRRcSPair<"wfchxb", 0xE7EB, null_frag, null_frag,
                                     v128q, v128xb, 4, 8>;
     }
+  }
 
   // Compare and signal high.
-  let Predicates = [FeatureVectorEnhancements1] in {
+  let Predicates = [FeatureVectorEnhancements1], hasSideEffects = 1 in {
     defm VFKHDB : BinaryVRRcSPair<"vfkhdb", 0xE7EB, null_frag, null_frag,
                                   v128g, v128db, 3, 4>;
     defm WFKHDB : BinaryVRRcSPair<"wfkhdb", 0xE7EB, null_frag, null_frag,
                                   v64g, v64db, 3, 12>;
     defm VFKHSB : BinaryVRRcSPair<"vfkhsb", 0xE7EB, null_frag, null_frag,
                                   v128f, v128sb, 2, 4>;
     defm WFKHSB : BinaryVRRcSPair<"wfkhsb", 0xE7EB, null_frag, null_frag,
                                   v32f, v32sb, 2, 12>;
     defm WFKHXB : BinaryVRRcSPair<"wfkhxb", 0xE7EB, null_frag, null_frag,
                                   v128q, v128xb, 4, 12>;
   }
 
   // Compare high or equal.
   def  VFCHE   : BinaryVRRcSPairFloatGeneric<"vfche", 0xE7EA>;
   defm VFCHEDB : BinaryVRRcSPair<"vfchedb", 0xE7EA, z_vfcmphe, z_vfcmphes,
                                  v128g, v128db, 3, 0>;
+  let hasSideEffects = 1 in
     defm WFCHEDB : BinaryVRRcSPair<"wfchedb", 0xE7EA, null_frag, null_frag,
                                    v64g, v64db, 3, 8>;
   let Predicates = [FeatureVectorEnhancements1] in {
     defm VFCHESB : BinaryVRRcSPair<"vfchesb", 0xE7EA, z_vfcmphe, z_vfcmphes,
                                    v128f, v128sb, 2, 0>;
+    let hasSideEffects = 1 in {
       defm WFCHESB : BinaryVRRcSPair<"wfchesb", 0xE7EA, null_frag, null_frag,
                                      v32f, v32sb, 2, 8>;
       defm WFCHEXB : BinaryVRRcSPair<"wfchexb", 0xE7EA, null_frag, null_frag,
                                      v128q, v128xb, 4, 8>;
     }
+  }
 
   // Compare and signal high or equal.
-  let Predicates = [FeatureVectorEnhancements1] in {
+  let Predicates = [FeatureVectorEnhancements1], hasSideEffects = 1 in {
     defm VFKHEDB : BinaryVRRcSPair<"vfkhedb", 0xE7EA, null_frag, null_frag,
                                    v128g, v128db, 3, 4>;
     defm WFKHEDB : BinaryVRRcSPair<"wfkhedb", 0xE7EA, null_frag, null_frag,
                                    v64g, v64db, 3, 12>;
     defm VFKHESB : BinaryVRRcSPair<"vfkhesb", 0xE7EA, null_frag, null_frag,
                                    v128f, v128sb, 2, 4>;
     defm WFKHESB : BinaryVRRcSPair<"wfkhesb", 0xE7EA, null_frag, null_frag,
                                    v32f, v32sb, 2, 12>;

 //===----------------------------------------------------------------------===//
 // Floating-point insertion and extraction
 //===----------------------------------------------------------------------===//
 
 // Moving 32-bit values between GPRs and FPRs can be done using VLVGF
 // and VLGVF.
 let Predicates = [FeatureVector] in {
   def LEFR : UnaryAliasVRS<VR32, GR32>;
+  let hasSideEffects = 1 in
     def LFER : UnaryAliasVRS<GR64, VR32>;
   def : Pat<(f32 (bitconvert (i32 GR32:$src))), (LEFR GR32:$src)>;
   def : Pat<(i32 (bitconvert (f32 VR32:$src))),
             (EXTRACT_SUBREG (LFER VR32:$src), subreg_l32)>;
 }
 
 // Floating-point values are stored in element 0 of the corresponding
 // vector register.  Scalar to vector conversion is just a subreg and
 // scalar replication can just replicate element 0 of the vector register.
let Predicates = [FeatureVector] in {
   defm VSTRCZF : QuaternaryOptVRRdSPair<"vstrczf", 0xE78A, int_s390_vstrczf,
                                         z_vstrcz_cc, v128f, v128f, 2, 2>;
 }
 
 //===----------------------------------------------------------------------===//
 // Packed-decimal instructions
 //===----------------------------------------------------------------------===//
 
-let Predicates = [FeatureVectorPackedDecimal] in {
+let Predicates = [FeatureVectorPackedDecimal], hasSideEffects = 1 in {
   def VLIP : BinaryVRIh<"vlip", 0xE649>;
 
   def VPKZ : BinaryVSI<"vpkz", 0xE634, null_frag, 0>;
   def VUPKZ : StoreLengthVSI<"vupkz", 0xE63C, null_frag, 0>;
 
   let Defs = [CC] in {
     def VCVB : BinaryVRRi<"vcvb", 0xE650, GR32>;
     def VCVBG : BinaryVRRi<"vcvbg", 0xE652, GR64>;