[MRI] isConstantPhysReg should also check if the register is clobbered by a RegMask

[Carrot]

Summary

Current implementation of MachineRegisterInfo::isConstantPhysReg checks if the specified physical is modified in a function, but it only checks if it is used in a Def register operand, it may still be modified in a RegMask operand, like a function call. So it can give wrong result in this situation.

Other changes are used to prevent test regressions.

[arsenm]

llvm/lib/CodeGen/MachineRegisterInfo.cpp

591

This can just be a range loop. Also braces

llvm/lib/CodeGen/RegisterCoalescer.cpp

4116 ↗

(On Diff #449070)

Adding a specific collection step here feels wrong. Other contexts the mask is added as soon as it's created. Not sure why we have the virtregrewriter adding these now

llvm/lib/Target/AMDGPU/SIPreAllocateWWMRegs.cpp

104 ↗

(On Diff #449070)

Don't see why this would skip regmask tests

llvm/lib/Target/Mips/MipsRegisterInfo.cpp

322–323 ↗

(On Diff #449070)

This should be a separate mips change

[Carrot]

llvm/lib/CodeGen/RegisterCoalescer.cpp

4116 ↗

(On Diff #449070)

I agree it's ugly. Do you have any good idea where I can collect the RegMasks?

llvm/lib/Target/AMDGPU/SIPreAllocateWWMRegs.cpp

104 ↗

(On Diff #449070)

To be honest, the MRI change causes two gpu tests failed, because previously there was no RegMask information collected, the checking of RegMask in isPhysRegUsed was actually nop. Now with RegMask collected, isPhysRegUsed behaves differently. I am completely ignorant with AMD GPU, so I just kept its original behavior by skipping regmask test.

Do you think it's better to enable the regmask test and send out the modified tests for review?

llvm/lib/Target/Mips/MipsRegisterInfo.cpp

322–323 ↗

(On Diff #449070)

I sent out D130932 for it.

[arichardson]

It's been a while since I looked at constant registers but I made a few changes in CHERI LLVM to avoid unnecessary copies. I just had a quick look at some of the commits and https://github.com/CTSRD-CHERI/llvm-project/commit/8588d8b81458ed6d87b674893e7752e6a6915574 looks somewhat related.

[arichardson]

Uploaded the patch I committed to CHERI LLVM as https://reviews.llvm.org/D131958.

[arsenm]

Can this be closed with D131958?

[arichardson]

In D130919#3802733, @arsenm wrote:

Can this be closed with D131958?

I think so (about to commit that revision once I check tests still pass), but it's hard to say if it covers all cases this revision does since there are no tests.

[Carrot]

In D130919#3802764, @arichardson wrote:

In D130919#3802733, @arsenm wrote:

Can this be closed with D131958?

I think so (about to commit that revision once I check tests still pass), but it's hard to say if it covers all cases this revision does since there are no tests.

No, a phyiscal register is not a target constant register, but if it is not modified inside a specific function, it can still be treated as a constant register by MachineRegisterInfo::isConstantPhysReg inside the specific function. The modification should include both defined register operand and regmask.

I will update this patch. But it still depends on D132987 and D132561 to prevent regressions. These are real problems exposed by this patch.

[Carrot]

Update the patch to collect RegMask information whenever a new RegMask operand is created or its content is changed.

[arsenm]

llvm/lib/CodeGen/RegUsageInfoPropagate.cpp

147–149

Won't this be reflected in the regmask updates (BTW I think this pass needs to be changed to directly modify the regmasks in the instruction instead of the analysis)?

[Carrot]

llvm/lib/CodeGen/RegUsageInfoPropagate.cpp

147–149

Query register information through MRI is precise through the linked Register operands. But RegMask is different, all RegMask information is cumulated in MRI.UsedPhysRegMask. So usually we can only add new used physical registers into UsedPhysRegMask. This pass is used to reduce used physical registers, we need to reset UsedPhysRegMask and collect RegMask again.

[Carrot]

ping

[Carrot]

Rebase.

[Carrot]

Any other comments?

[foad]

Why are there so many changes in AMDGPU codegen, even in tests that do not contain function calls?

[Carrot]

In D130919#3891206, @foad wrote:

Why are there so many changes in AMDGPU codegen, even in tests that do not contain function calls?

In a previous comment I mentioned I'm completely ignorant with AMDGPU, the instructions looks very different when compared with traditional CPU. So it's better to have an AMDGPU expert to have a look at these changes.

I tried to debug the simplest one wwm-reserved.ll. The difference comes from pass si-pre-allocate-wwm-regs, the related code is

...
160B      dead $sgpr30_sgpr31 = SI_CALL %8:sreg_64, @called, <regmask $noreg $exec $exec_hi $exec_lo $flat_scr $flat_scr_hi $flat_scr_hi_ci $flat_scr_hi_vi $flat_scr_lo $flat_scr_lo_ci $flat_scr_lo_vi $flat_scr_ci $flat_scr_vi $fp_reg $lds_direct $mode $pc_reg $private_rsrc_reg $scc $sgpr_null $sgpr_null_hi $sp_reg $src_execz $src_pops_exiting_wave_id $src_private_base $src_private_limit $src_scc $src_shared_base $src_shared_limit $src_vccz $tba $tba_hi $tba_lo and 6831 more...>, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $vgpr0, implicit-def $vgpr0
168B      %10:vgpr_32 = COPY $vgpr0
...

When this pass processing the COPY instruction, function SIPreAllocateWWMRegs::processDef tries to find a physical register for virtual register %10. It tries every physical register in class vgpr_32 to see if it fits.

...
  for (MCRegister PhysReg : RegClassInfo.getOrder(MRI->getRegClass(Reg))) {
    if (!MRI->isPhysRegUsed(PhysReg) &&                                                                             // Different result
        Matrix->checkInterference(LI, PhysReg) == LiveRegMatrix::IK_Free) {
      Matrix->assign(LI, PhysReg);
      assert(PhysReg != 0); 
      RegsToRewrite.push_back(Reg);
      return true;
    }   
  }
...

When PhysReg is $vgpr1, MRI->isPhysRegUsed(PhysReg) computes different result because of this patch. Function MRI->isPhysRegUsed checks regmask before register operands. But regmask information was not collected until very late, so at this time MRI doesn't have any regmask information, and there is no explicit $vgpr1 in the function, so MRI->isPhysRegUsed returns true for $vgpr1, and it is assigned to %10.

With this patch, MRI regmask information is updated whenever there is a regmask change, and recollected in reg-usage-propagation pass. So when $vgpr1 is passed to MRI->isPhysRegUsed, false is returned because $vgpr1 is used in @called and propagated to MRI in current function, it means $vgpr1 is not appropriate for %10, and next physical register $vgpr2 is tried.

Although the original behavior is correct for this specific function, but it is very dangerous. If we have another function call to @called inside the live range of %10, wrong code can be generated.

[foad]

llvm/lib/CodeGen/MachineCSE.cpp

280 ↗

(On Diff #471353)

Can you please split this part out into a separate patch for review? It causes significant codegen changes for AMDGPU and we need to be sure that the changes are OK.

llvm/test/CodeGen/AMDGPU/GlobalISel/udiv.i64.ll

1 ↗

(On Diff #471353)

I don't understant why these diffs are included in the patch. Can you double check? I don't think your patch affects codegen for any of:

LLVM :: CodeGen/AMDGPU/GlobalISel/udiv.i64.ll
LLVM :: CodeGen/AMDGPU/GlobalISel/urem.i64.ll
LLVM :: CodeGen/AMDGPU/sgpr-control-flow.ll

[Carrot]

@foad, thanks a lot for reviewing AMDGPU changes!

llvm/lib/CodeGen/MachineCSE.cpp

280 ↗

(On Diff #471353)

It is split to D137222. I also did an analysis for it.

llvm/test/CodeGen/AMDGPU/GlobalISel/udiv.i64.ll

1 ↗

(On Diff #471353)

Will investigate it.

[Carrot]

llvm/test/CodeGen/AMDGPU/GlobalISel/udiv.i64.ll

1 ↗

(On Diff #471353)

They are also caused by the change in MachineCSE. They don't show up in D137222 because there are other changes in MachineCSE::isPRECandidate between the two versions of my two clients. They will disappear after rebaseing.

[Carrot]

Rebase this patch after committing the MachineCSE part.

[Carrot]

ping

[Carrot]

Any other comments?

[Carrot]

Add a test case to show that llvm can generate wrong instructions without this patch.

[Carrot]

Rebase the code.

The new failed test pr59305.ll is another case that wrong code was generated without this patch.

Several DIV instructions crossing function call to fesetround got CSEd, fesetround is used to change rounding mode, so these DIV instructions may generate different results, they can't be CSEd.

llvm/include/llvm/CodeGen/MachineInstrBuilder.h

public:
   const MachineInstrBuilder &addBlockAddress(const BlockAddress *BA,
                                              int64_t Offset = 0,
                                              unsigned TargetFlags = 0) const {
     MI->addOperand(*MF, MachineOperand::CreateBA(BA, Offset, TargetFlags));
     return *this;
   }
 
   const MachineInstrBuilder &addRegMask(const uint32_t *Mask) const {
-    MI->addOperand(*MF, MachineOperand::CreateRegMask(Mask));
+    MI->addOperand(*MF, MachineOperand::CreateRegMask(Mask, MF));
     return *this;
   }
 
   const MachineInstrBuilder &addMemOperand(MachineMemOperand *MMO) const {
     MI->addMemOperand(*MF, MMO);
     return *this;
   }
 

llvm/include/llvm/CodeGen/MachineOperand.h

 
 class LLT;
 class BlockAddress;
 class Constant;
 class ConstantFP;
 class ConstantInt;
 class GlobalValue;
 class MachineBasicBlock;
+class MachineFunction;
 class MachineInstr;
 class MachineRegisterInfo;
 class MCCFIInstruction;
 class MDNode;
 class ModuleSlotTracker;
 class TargetIntrinsicInfo;
 class TargetRegisterInfo;
 class hash_code;
void setMBB(MachineBasicBlock *MBB) {
     assert(isMBB() && "Wrong MachineOperand mutator");
     Contents.MBB = MBB;
   }
 
   /// Sets value of register mask operand referencing Mask.  The
   /// operand does not take ownership of the memory referenced by Mask, it must
   /// remain valid for the lifetime of the operand. See CreateRegMask().
   /// Any physreg with a 0 bit in the mask is clobbered by the instruction.
-  void setRegMask(const uint32_t *RegMaskPtr) {
-    assert(isRegMask() && "Wrong MachineOperand mutator");
-    Contents.RegMask = RegMaskPtr;
-  }
+  void setRegMask(const uint32_t *RegMaskPtr, MachineInstr *MI = nullptr);
 
   void setIntrinsicID(Intrinsic::ID IID) {
     assert(isIntrinsicID() && "Wrong MachineOperand mutator");
     Contents.IntrinsicID = IID;
   }
 
   void setPredicate(unsigned Predicate) {
     assert(isPredicate() && "Wrong MachineOperand mutator");
public:
   /// A RegMask operand represents a set of non-clobbered physical registers
   /// on an instruction that clobbers many registers, typically a call.  The
   /// bit mask has a bit set for each physreg that is preserved by this
   /// instruction, as described in the documentation for
   /// TargetRegisterInfo::getCallPreservedMask().
   ///
   /// Any physreg with a 0 bit in the mask is clobbered by the instruction.
   ///
-  static MachineOperand CreateRegMask(const uint32_t *Mask) {
-    assert(Mask && "Missing register mask");
-    MachineOperand Op(MachineOperand::MO_RegisterMask);
-    Op.Contents.RegMask = Mask;
-    return Op;
-  }
+  static MachineOperand CreateRegMask(const uint32_t *Mask,
+                                      MachineFunction *MF);
+
   static MachineOperand CreateRegLiveOut(const uint32_t *Mask) {
     assert(Mask && "Missing live-out register mask");
     MachineOperand Op(MachineOperand::MO_RegisterLiveOut);
     Op.Contents.RegMask = Mask;
     return Op;
   }
   static MachineOperand CreateMetadata(const MDNode *Meta) {
     MachineOperand Op(MachineOperand::MO_Metadata);

llvm/include/llvm/CodeGen/MachineRegisterInfo.h

public:
   bool isPhysRegUsed(MCRegister PhysReg, bool SkipRegMaskTest = false) const;
 
   /// addPhysRegsUsedFromRegMask - Mark any registers not in RegMask as used.
   /// This corresponds to the bit mask attached to register mask operands.
   void addPhysRegsUsedFromRegMask(const uint32_t *RegMask) {
     UsedPhysRegMask.setBitsNotInMask(RegMask);
   }
 
+  void recollectUsedPhysRegMask();
+
   const BitVector &getUsedPhysRegsMask() const { return UsedPhysRegMask; }
 
   //===--------------------------------------------------------------------===//
   // Reserved Register Info
   //===--------------------------------------------------------------------===//
   //
   // The set of reserved registers must be invariant during register
   // allocation.  For example, the target cannot suddenly decide it needs a

llvm/lib/CodeGen/MIRParser/MIParser.cpp

if (Token.isNot(MIToken::rparen)) {
       Mask[Reg / 32] |= 1U << (Reg % 32);
     }
 
     // TODO: Report an error if the same register is used more than once.
   } while (consumeIfPresent(MIToken::comma));
 
   if (expectAndConsume(MIToken::rparen))
     return true;
-  Dest = MachineOperand::CreateRegMask(Mask);
+  Dest = MachineOperand::CreateRegMask(Mask, &MF);
   return false;
 }
 
 bool MIParser::parseLiveoutRegisterMaskOperand(MachineOperand &Dest) {
   assert(Token.is(MIToken::kw_liveout));
   uint32_t *Mask = MF.allocateRegMask();
   lex();
   if (expectAndConsume(MIToken::lparen))
bool MIParser::parseMachineOperand(const unsigned OpCode, const unsigned OpIdx,
   case MIToken::kw_intpred:
     return parsePredicateOperand(Dest);
   case MIToken::kw_shufflemask:
     return parseShuffleMaskOperand(Dest);
   case MIToken::Error:
     return true;
   case MIToken::Identifier:
     if (const auto *RegMask = PFS.Target.getRegMask(Token.stringValue())) {
-      Dest = MachineOperand::CreateRegMask(RegMask);
+      Dest = MachineOperand::CreateRegMask(RegMask, &MF);
       lex();
       break;
     } else if (Token.stringValue() == "CustomRegMask") {
       return parseCustomRegisterMaskOperand(Dest);
     } else
       return parseTypedImmediateOperand(Dest);
   case MIToken::dot: {
     const auto *TII = MF.getSubtarget().getInstrInfo();

llvm/lib/CodeGen/MachineOperand.cpp

 void MachineOperand::removeRegFromUses() {
   if (!isReg() || !isOnRegUseList())
     return;
 
   if (MachineFunction *MF = getMFIfAvailable(*this))
     MF->getRegInfo().removeRegOperandFromUseList(this);
 }
 
+void MachineOperand::setRegMask(const uint32_t *RegMaskPtr, MachineInstr *MI) {
+  assert(isRegMask() && "Wrong MachineOperand mutator");
+  Contents.RegMask = RegMaskPtr;
+  if (MI == nullptr)
+    MI = ParentMI;
+  MI->getMF()->getRegInfo().addPhysRegsUsedFromRegMask(RegMaskPtr);
+}
+
+MachineOperand MachineOperand::CreateRegMask(const uint32_t *Mask,
+                                             MachineFunction *MF) {
+  assert(Mask && "Missing register mask");
+  MachineOperand Op(MachineOperand::MO_RegisterMask);
+  Op.Contents.RegMask = Mask;
+  MF->getRegInfo().addPhysRegsUsedFromRegMask(Mask);
+  return Op;
+}
+
 /// ChangeToImmediate - Replace this operand with a new immediate operand of
 /// the specified value.  If an operand is known to be an immediate already,
 /// the setImm method should be used.
 void MachineOperand::ChangeToImmediate(int64_t ImmVal, unsigned TargetFlags) {
   assert((!isReg() || !isTied()) && "Cannot change a tied operand into an imm");
 
   removeRegFromUses();
 

llvm/lib/CodeGen/MachineRegisterInfo.cpp

bool MachineRegisterInfo::isConstantPhysReg(MCRegister PhysReg) const {
   const TargetRegisterInfo *TRI = getTargetRegisterInfo();
   if (TRI->isConstantPhysReg(PhysReg))
     return true;
 
   // Check if any overlapping register is modified, or allocatable so it may be
   // used later.
   for (MCRegAliasIterator AI(PhysReg, TRI, true);
        AI.isValid(); ++AI)
-    if (!def_empty(*AI) || isAllocatable(*AI))
+    if (!def_empty(*AI) || isAllocatable(*AI) || UsedPhysRegMask.test(PhysReg))
       return false;
+
   return true;
 }
 
 /// markUsesInDebugValueAsUndef - Mark every DBG_VALUE referencing the
 /// specified register as undefined which causes the DBG_VALUE to be
 /// deleted during LiveDebugVariables analysis.
 void MachineRegisterInfo::markUsesInDebugValueAsUndef(Register Reg) const {
   // Mark any DBG_VALUE* that uses Reg as undef (but don't delete it.)
for (const MachineOperand &MO : make_range(def_begin(*AI), def_end())) {
       if (!SkipNoReturnDef && isNoReturnDef(MO))
         continue;
       return true;
     }
   }
   return false;
 }
 
+void MachineRegisterInfo::recollectUsedPhysRegMask() {
+  UsedPhysRegMask.reset();
+  for (MachineBasicBlock &MBB : *MF) {
+    for (MachineInstr &MI : MBB.instrs()) {
+      for (MachineOperand &MO : MI.operands()) {
+        if (MO.isRegMask())
+          addPhysRegsUsedFromRegMask(MO.getRegMask());
+      }
+    }
+  }
+}
+
 bool MachineRegisterInfo::isPhysRegUsed(MCRegister PhysReg,
                                         bool SkipRegMaskTest) const {
   if (!SkipRegMaskTest && UsedPhysRegMask.test(PhysReg))
     return true;
   const TargetRegisterInfo *TRI = getTargetRegisterInfo();
   for (MCRegAliasIterator AliasReg(PhysReg, TRI, true); AliasReg.isValid();
        ++AliasReg) {
     if (!reg_nodbg_empty(*AliasReg))

llvm/lib/CodeGen/RegUsageInfoPropagate.cpp

for (MachineInstr &MI : MBB) {
 
       LLVM_DEBUG(
           dbgs()
           << "Call Instruction After Register Usage Info Propagation : \n"
           << MI << '\n');
     }
   }
 
+  // MRI.UsedPhysRegMask should reflect the updated RegMask.
+  if (Changed)
+    MF.getRegInfo().recollectUsedPhysRegMask();
+
   LLVM_DEBUG(
       dbgs() << " +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
                 "++++++ \n");
   return Changed;
 }
 
 FunctionPass *llvm::createRegUsageInfoPropPass() {
   return new RegUsageInfoPropagation();
 }

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

for (auto Reg : CLI.OutRegs)
     Ops.push_back(MachineOperand::CreateReg(Reg, /*isDef=*/false));
 
   // Push live variables for the stack map.
   if (!addStackMapLiveVars(Ops, I, NumMetaOpers + NumArgs))
     return false;
 
   // Push the register mask info.
   Ops.push_back(MachineOperand::CreateRegMask(
-      TRI.getCallPreservedMask(*FuncInfo.MF, CC)));
+      TRI.getCallPreservedMask(*FuncInfo.MF, CC), FuncInfo.MF));
 
   // Add scratch registers as implicit def and early clobber.
   const MCPhysReg *ScratchRegs = TLI.getScratchRegisters(CC);
   for (unsigned i = 0; ScratchRegs[i]; ++i)
     Ops.push_back(MachineOperand::CreateReg(
         ScratchRegs[i], /*isDef=*/true, /*isImp=*/true, /*isKill=*/false,
         /*isDead=*/false, /*isUndef=*/false, /*isEarlyClobber=*/true));
 

llvm/lib/CodeGen/TargetInstrInfo.cpp

if (Reg == 0)
       continue;
 
     // Check for a well-behaved physical register.
     if (Register::isPhysicalRegister(Reg)) {
       if (MO.isUse()) {
         // If the physreg has no defs anywhere, it's just an ambient register
         // and we can freely move its uses. Alternatively, if it's allocatable,
         // it could get allocated to something with a def during allocation.
-        if (!MRI.isConstantPhysReg(Reg))
+        if (!MRI.isConstantPhysReg(Reg) && !isIgnorableUse(MO))
           return false;
       } else {
         // A physreg def. We can't remat it.
         return false;
       }
       continue;
     }
 

llvm/lib/Target/X86/X86InstrInfo.h

#include "X86GenInstrInfo.inc"
 
   static bool hasLockPrefix(const MachineInstr &MI) {
     return MI.getDesc().TSFlags & X86II::LOCK;
   }
 
   std::optional<ParamLoadedValue>
   describeLoadedValue(const MachineInstr &MI, Register Reg) const override;
 
+  /// Given \p MO is a PhysReg use return if it can be ignored for the purpose
+  /// of instruction rematerialization or sinking.
+  bool isIgnorableUse(const MachineOperand &MO) const override {
+    // An RIP relative address is a constant.
+    return MO.getReg() == X86::RIP;
+  }
+
 protected:
   /// Commutes the operands in the given instruction by changing the operands
   /// order and/or changing the instruction's opcode and/or the immediate value
   /// operand.
   ///
   /// The arguments 'CommuteOpIdx1' and 'CommuteOpIdx2' specify the operands
   /// to be commuted.
   ///

llvm/test/CodeGen/AArch64/regmask.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=aarch64-- --reserve-regs-for-regalloc=X8 | FileCheck %s
+
+declare i64 @bar()
+%struct.S = type { i64, i64, i64 }
+
+; X8 can be clobbered by function call. So we can't use X8 to represent
+; %agg.result after the call to @bar.
+
+define void @foo(ptr sret(%struct.S) %agg.result) {
+; CHECK-LABEL: foo:
+; CHECK:       // %bb.0: // %entry
+; CHECK-NEXT:    stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
+; CHECK-NEXT:    .cfi_offset w19, -8
+; CHECK-NEXT:    .cfi_offset w30, -16
+; CHECK-NEXT:    mov x19, x8
+; CHECK-NEXT:    mov w9, #4
+; CHECK-NEXT:    str x9, [x19, #8]
+; CHECK-NEXT:    bl bar
+; CHECK-NEXT:    str x0, [x19, #16]
+; CHECK-NEXT:    ldp x30, x19, [sp], #16 // 16-byte Folded Reload
+; CHECK-NEXT:    ret
+entry:
+  %f1 = getelementptr inbounds %struct.S, ptr %agg.result, i64 0, i32 1
+  store i64 4, ptr %f1, align 8
+  %call = call i64 @bar()
+  %f2 = getelementptr inbounds %struct.S, ptr %agg.result, i64 0, i32 2
+  store i64 %call, ptr %f2, align 8
+  ret void
+}
+

llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll

 define amdgpu_kernel void @v3i16_registers(i1 %cond) #0 {
 ; GCN-LABEL: v3i16_registers:
 ; GCN:       ; %bb.0: ; %entry
 ; GCN-NEXT:    s_add_u32 flat_scratch_lo, s12, s17
 ; GCN-NEXT:    s_load_dword s12, s[8:9], 0x0
 ; GCN-NEXT:    s_addc_u32 flat_scratch_hi, s13, 0
 ; GCN-NEXT:    s_add_u32 s0, s0, s17
 ; GCN-NEXT:    s_addc_u32 s1, s1, 0
-; GCN-NEXT:    s_mov_b32 s32, 0
+; GCN-NEXT:    s_mov_b32 s18, 0
 ; GCN-NEXT:    s_waitcnt lgkmcnt(0)
 ; GCN-NEXT:    s_bitcmp1_b32 s12, 0
+; GCN-NEXT:    v_mov_b32_e32 v3, v1
+; GCN-NEXT:    v_mov_b32_e32 v4, v0
 ; GCN-NEXT:    s_cselect_b64 s[12:13], -1, 0
+; GCN-NEXT:    s_mov_b32 s19, s18
+; GCN-NEXT:    v_mov_b32_e32 v0, s18
 ; GCN-NEXT:    s_and_b64 vcc, exec, s[12:13]
+; GCN-NEXT:    v_mov_b32_e32 v1, s19
+; GCN-NEXT:    s_mov_b32 s32, 0
 ; GCN-NEXT:    s_cbranch_vccnz .LBB4_2
 ; GCN-NEXT:  ; %bb.1: ; %if.else
 ; GCN-NEXT:    s_add_u32 s8, s8, 8
-; GCN-NEXT:    v_lshlrev_b32_e32 v2, 20, v2
-; GCN-NEXT:    v_lshlrev_b32_e32 v1, 10, v1
+; GCN-NEXT:    v_lshlrev_b32_e32 v0, 20, v2
+; GCN-NEXT:    v_lshlrev_b32_e32 v1, 10, v3
 ; GCN-NEXT:    s_addc_u32 s9, s9, 0
-; GCN-NEXT:    v_or3_b32 v31, v0, v1, v2
+; GCN-NEXT:    v_or3_b32 v31, v4, v1, v0
 ; GCN-NEXT:    s_mov_b32 s12, s14
 ; GCN-NEXT:    s_mov_b32 s13, s15
 ; GCN-NEXT:    s_mov_b32 s14, s16
 ; GCN-NEXT:    s_getpc_b64 s[18:19]
 ; GCN-NEXT:    s_add_u32 s18, s18, func_v3i16@rel32@lo+4
 ; GCN-NEXT:    s_addc_u32 s19, s19, func_v3i16@rel32@hi+12
 ; GCN-NEXT:    s_swappc_b64 s[30:31], s[18:19]
-; GCN-NEXT:    s_branch .LBB4_3
-; GCN-NEXT:  .LBB4_2:
-; GCN-NEXT:    s_mov_b32 s4, 0
-; GCN-NEXT:    s_mov_b32 s5, s4
-; GCN-NEXT:    v_mov_b32_e32 v0, s4
-; GCN-NEXT:    v_mov_b32_e32 v1, s5
-; GCN-NEXT:  .LBB4_3: ; %if.end
+; GCN-NEXT:  .LBB4_2: ; %if.end
 ; GCN-NEXT:    global_store_short v[0:1], v1, off
 ; GCN-NEXT:    global_store_dword v[0:1], v0, off
 ; GCN-NEXT:    s_endpgm
 entry:
   br i1 %cond, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
   br label %if.end

 define amdgpu_kernel void @v3f16_registers(i1 %cond) #0 {
 ; GCN-LABEL: v3f16_registers:
 ; GCN:       ; %bb.0: ; %entry
 ; GCN-NEXT:    s_add_u32 flat_scratch_lo, s12, s17
 ; GCN-NEXT:    s_load_dword s12, s[8:9], 0x0
 ; GCN-NEXT:    s_addc_u32 flat_scratch_hi, s13, 0
 ; GCN-NEXT:    s_add_u32 s0, s0, s17
 ; GCN-NEXT:    s_addc_u32 s1, s1, 0
-; GCN-NEXT:    s_mov_b32 s32, 0
+; GCN-NEXT:    s_mov_b32 s18, 0
 ; GCN-NEXT:    s_waitcnt lgkmcnt(0)
 ; GCN-NEXT:    s_bitcmp1_b32 s12, 0
+; GCN-NEXT:    v_mov_b32_e32 v3, v1
+; GCN-NEXT:    v_mov_b32_e32 v4, v0
 ; GCN-NEXT:    s_cselect_b64 s[12:13], -1, 0
+; GCN-NEXT:    s_mov_b32 s19, s18
+; GCN-NEXT:    v_mov_b32_e32 v0, s18
 ; GCN-NEXT:    s_and_b64 vcc, exec, s[12:13]
+; GCN-NEXT:    v_mov_b32_e32 v1, s19
+; GCN-NEXT:    s_mov_b32 s32, 0
 ; GCN-NEXT:    s_cbranch_vccnz .LBB5_2
 ; GCN-NEXT:  ; %bb.1: ; %if.else
 ; GCN-NEXT:    s_add_u32 s8, s8, 8
-; GCN-NEXT:    v_lshlrev_b32_e32 v2, 20, v2
-; GCN-NEXT:    v_lshlrev_b32_e32 v1, 10, v1
+; GCN-NEXT:    v_lshlrev_b32_e32 v0, 20, v2
+; GCN-NEXT:    v_lshlrev_b32_e32 v1, 10, v3
 ; GCN-NEXT:    s_addc_u32 s9, s9, 0
-; GCN-NEXT:    v_or3_b32 v31, v0, v1, v2
+; GCN-NEXT:    v_or3_b32 v31, v4, v1, v0
 ; GCN-NEXT:    s_mov_b32 s12, s14
 ; GCN-NEXT:    s_mov_b32 s13, s15
 ; GCN-NEXT:    s_mov_b32 s14, s16
 ; GCN-NEXT:    s_getpc_b64 s[18:19]
 ; GCN-NEXT:    s_add_u32 s18, s18, func_v3f16@rel32@lo+4
 ; GCN-NEXT:    s_addc_u32 s19, s19, func_v3f16@rel32@hi+12
 ; GCN-NEXT:    s_swappc_b64 s[30:31], s[18:19]
-; GCN-NEXT:    s_branch .LBB5_3
-; GCN-NEXT:  .LBB5_2:
-; GCN-NEXT:    s_mov_b32 s4, 0
-; GCN-NEXT:    s_mov_b32 s5, s4
-; GCN-NEXT:    v_mov_b32_e32 v0, s4
-; GCN-NEXT:    v_mov_b32_e32 v1, s5
-; GCN-NEXT:  .LBB5_3: ; %if.end
+; GCN-NEXT:  .LBB5_2: ; %if.end
 ; GCN-NEXT:    global_store_short v[0:1], v1, off
 ; GCN-NEXT:    global_store_dword v[0:1], v0, off
 ; GCN-NEXT:    s_endpgm
 entry:
   br i1 %cond, label %if.then, label %if.else
 
 if.then:                                          ; preds = %entry
   br label %if.end

llvm/test/CodeGen/AMDGPU/tuple-allocation-failure.ll

 }
 
 define amdgpu_kernel void @kernel(ptr addrspace(1) %arg1.global, i1 %tmp3.i.i, i32 %tmp5.i.i, i32 %tmp427.i, i1 %tmp438.i, double %tmp27.i, i1 %tmp48.i) {
 ; GLOBALNESS1-LABEL: kernel:
 ; GLOBALNESS1:       ; %bb.0: ; %bb
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[54:55], s[6:7]
 ; GLOBALNESS1-NEXT:    s_load_dwordx4 s[36:39], s[8:9], 0x0
 ; GLOBALNESS1-NEXT:    s_load_dword s6, s[8:9], 0x14
-; GLOBALNESS1-NEXT:    v_mov_b32_e32 v41, v0
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v42, 0
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[0:1], 0, 0
-; GLOBALNESS1-NEXT:    global_store_dword v[0:1], v42, off
+; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[44:45], 0, 0
+; GLOBALNESS1-NEXT:    global_store_dword v[44:45], v42, off
+; GLOBALNESS1-NEXT:    v_mov_b32_e32 v41, v0
 ; GLOBALNESS1-NEXT:    s_waitcnt lgkmcnt(0)
 ; GLOBALNESS1-NEXT:    global_load_dword v0, v42, s[36:37]
 ; GLOBALNESS1-NEXT:    s_add_u32 flat_scratch_lo, s12, s17
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[64:65], s[4:5]
 ; GLOBALNESS1-NEXT:    s_load_dwordx2 s[4:5], s[8:9], 0x18
 ; GLOBALNESS1-NEXT:    s_load_dword s7, s[8:9], 0x20
 ; GLOBALNESS1-NEXT:    s_addc_u32 flat_scratch_hi, s13, 0
 ; GLOBALNESS1-NEXT:    s_add_u32 s0, s0, s17

 ; GLOBALNESS1-NEXT:    v_accvgpr_write_b32 a34, v2
 ; GLOBALNESS1-NEXT:    v_accvgpr_write_b32 a33, v1
 ; GLOBALNESS1-NEXT:    v_accvgpr_write_b32 a32, v0
 ; GLOBALNESS1-NEXT:    s_cbranch_vccnz .LBB1_30
 ; GLOBALNESS1-NEXT:  .LBB1_4: ; %bb5
 ; GLOBALNESS1-NEXT:    ; =>This Loop Header: Depth=1
 ; GLOBALNESS1-NEXT:    ; Child Loop BB1_15 Depth 2
 ; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[0:1], s[92:93], s[92:93] op_sel:[0,1]
-; GLOBALNESS1-NEXT:    flat_load_dword v44, v[0:1]
+; GLOBALNESS1-NEXT:    flat_load_dword v46, v[0:1]
 ; GLOBALNESS1-NEXT:    s_add_u32 s8, s62, 40
 ; GLOBALNESS1-NEXT:    buffer_store_dword v42, off, s[0:3], 0
-; GLOBALNESS1-NEXT:    flat_load_dword v45, v[0:1]
+; GLOBALNESS1-NEXT:    flat_load_dword v47, v[0:1]
 ; GLOBALNESS1-NEXT:    s_addc_u32 s9, s63, 0
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[4:5], s[64:65]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS1-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS1-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS1-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v31, v41

 ; GLOBALNESS1-NEXT:    ; implicit-def: $sgpr4_sgpr5
 ; GLOBALNESS1-NEXT:    s_branch .LBB1_23
 ; GLOBALNESS1-NEXT:  .LBB1_8: ; %Flow16
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS1-NEXT:    s_and_b64 vcc, exec, s[6:7]
 ; GLOBALNESS1-NEXT:    s_cbranch_vccz .LBB1_24
 ; GLOBALNESS1-NEXT:  .LBB1_9: ; %baz.exit.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS1-NEXT:    flat_load_dword v0, v[32:33]
+; GLOBALNESS1-NEXT:    flat_load_dword v0, v[44:45]
 ; GLOBALNESS1-NEXT:    s_mov_b32 s68, s93
 ; GLOBALNESS1-NEXT:    s_mov_b32 s70, s93
 ; GLOBALNESS1-NEXT:    s_mov_b32 s71, s69
 ; GLOBALNESS1-NEXT:    s_mov_b32 s72, s93
 ; GLOBALNESS1-NEXT:    s_mov_b32 s73, s69
 ; GLOBALNESS1-NEXT:    s_mov_b32 s74, s93
 ; GLOBALNESS1-NEXT:    s_mov_b32 s75, s69
 ; GLOBALNESS1-NEXT:    s_mov_b32 s76, s93

 ; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[24:25], s[92:93], s[92:93] op_sel:[0,1]
 ; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[26:27], s[94:95], s[94:95] op_sel:[0,1]
 ; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[28:29], s[96:97], s[96:97] op_sel:[0,1]
 ; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[30:31], s[98:99], s[98:99] op_sel:[0,1]
 ; GLOBALNESS1-NEXT:    s_and_saveexec_b64 s[70:71], s[96:97]
 ; GLOBALNESS1-NEXT:    s_cbranch_execz .LBB1_26
 ; GLOBALNESS1-NEXT:  ; %bb.10: ; %bb33.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS1-NEXT:    global_load_dwordx2 v[0:1], v[32:33], off
+; GLOBALNESS1-NEXT:    global_load_dwordx2 v[0:1], v[44:45], off
 ; GLOBALNESS1-NEXT:    v_readlane_b32 s4, v40, 0
 ; GLOBALNESS1-NEXT:    v_readlane_b32 s5, v40, 1
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[72:73], s[36:37]
 ; GLOBALNESS1-NEXT:    s_mov_b32 s75, s39
 ; GLOBALNESS1-NEXT:    s_andn2_b64 vcc, exec, s[4:5]
 ; GLOBALNESS1-NEXT:    s_cbranch_vccnz .LBB1_12
 ; GLOBALNESS1-NEXT:  ; %bb.11: ; %bb39.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[2:3], 0, 0
-; GLOBALNESS1-NEXT:    global_store_dwordx2 v[2:3], v[42:43], off
+; GLOBALNESS1-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS1-NEXT:  .LBB1_12: ; %bb44.lr.ph.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS1-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v45
-; GLOBALNESS1-NEXT:    v_cndmask_b32_e32 v2, 0, v44, vcc
+; GLOBALNESS1-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v47
+; GLOBALNESS1-NEXT:    v_cndmask_b32_e32 v2, 0, v46, vcc
 ; GLOBALNESS1-NEXT:    s_waitcnt vmcnt(0)
 ; GLOBALNESS1-NEXT:    v_cmp_nlt_f64_e64 s[36:37], 0, v[0:1]
 ; GLOBALNESS1-NEXT:    v_cmp_eq_u32_e64 s[58:59], 0, v2
 ; GLOBALNESS1-NEXT:    s_branch .LBB1_15
 ; GLOBALNESS1-NEXT:  .LBB1_13: ; %Flow7
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_15 Depth=2
 ; GLOBALNESS1-NEXT:    s_or_b64 exec, exec, s[4:5]
 ; GLOBALNESS1-NEXT:  .LBB1_14: ; %bb63.i

 ; GLOBALNESS1-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[8:9], s[60:61]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS1-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS1-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS1-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v31, v41
 ; GLOBALNESS1-NEXT:    s_swappc_b64 s[30:31], s[66:67]
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[44:45], 0, 0
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[4:5], s[64:65]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[8:9], s[60:61]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS1-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS1-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS1-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v31, v41

 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS1-NEXT:    v_readlane_b32 s6, v40, 2
 ; GLOBALNESS1-NEXT:    v_readlane_b32 s7, v40, 3
 ; GLOBALNESS1-NEXT:    s_andn2_b64 vcc, exec, s[6:7]
 ; GLOBALNESS1-NEXT:    s_cbranch_vccnz .LBB1_1
 ; GLOBALNESS1-NEXT:  ; %bb.28: ; %bb69.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS1-NEXT:    global_store_dwordx2 v[32:33], v[42:43], off
+; GLOBALNESS1-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS1-NEXT:    s_branch .LBB1_1
 ; GLOBALNESS1-NEXT:  .LBB1_29: ; %bb73.i
 ; GLOBALNESS1-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS1-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS1-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS1-NEXT:    global_store_dwordx2 v[32:33], v[42:43], off
+; GLOBALNESS1-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS1-NEXT:    s_branch .LBB1_2
 ; GLOBALNESS1-NEXT:  .LBB1_30: ; %loop.exit.guard
 ; GLOBALNESS1-NEXT:    s_andn2_b64 vcc, exec, s[4:5]
 ; GLOBALNESS1-NEXT:    s_mov_b64 s[4:5], -1
 ; GLOBALNESS1-NEXT:    s_cbranch_vccz .LBB1_32
 ; GLOBALNESS1-NEXT:  ; %bb.31: ; %bb7.i.i
 ; GLOBALNESS1-NEXT:    s_add_u32 s8, s62, 40
 ; GLOBALNESS1-NEXT:    s_addc_u32 s9, s63, 0

 ; GLOBALNESS1-NEXT:    s_swappc_b64 s[30:31], s[16:17]
 ; GLOBALNESS1-NEXT:  .LBB1_34: ; %UnifiedUnreachableBlock
 ;
 ; GLOBALNESS0-LABEL: kernel:
 ; GLOBALNESS0:       ; %bb.0: ; %bb
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[54:55], s[6:7]
 ; GLOBALNESS0-NEXT:    s_load_dwordx4 s[36:39], s[8:9], 0x0
 ; GLOBALNESS0-NEXT:    s_load_dword s6, s[8:9], 0x14
-; GLOBALNESS0-NEXT:    v_mov_b32_e32 v41, v0
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v42, 0
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[0:1], 0, 0
-; GLOBALNESS0-NEXT:    global_store_dword v[0:1], v42, off
+; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[44:45], 0, 0
+; GLOBALNESS0-NEXT:    global_store_dword v[44:45], v42, off
+; GLOBALNESS0-NEXT:    v_mov_b32_e32 v41, v0
 ; GLOBALNESS0-NEXT:    s_waitcnt lgkmcnt(0)
 ; GLOBALNESS0-NEXT:    global_load_dword v0, v42, s[36:37]
 ; GLOBALNESS0-NEXT:    s_add_u32 flat_scratch_lo, s12, s17
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[62:63], s[4:5]
 ; GLOBALNESS0-NEXT:    s_load_dwordx2 s[4:5], s[8:9], 0x18
 ; GLOBALNESS0-NEXT:    s_load_dword s7, s[8:9], 0x20
 ; GLOBALNESS0-NEXT:    s_addc_u32 flat_scratch_hi, s13, 0
 ; GLOBALNESS0-NEXT:    s_add_u32 s0, s0, s17

 ; GLOBALNESS0-NEXT:    v_accvgpr_write_b32 a34, v2
 ; GLOBALNESS0-NEXT:    v_accvgpr_write_b32 a33, v1
 ; GLOBALNESS0-NEXT:    v_accvgpr_write_b32 a32, v0
 ; GLOBALNESS0-NEXT:    s_cbranch_vccnz .LBB1_30
 ; GLOBALNESS0-NEXT:  .LBB1_4: ; %bb5
 ; GLOBALNESS0-NEXT:    ; =>This Loop Header: Depth=1
 ; GLOBALNESS0-NEXT:    ; Child Loop BB1_15 Depth 2
 ; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[0:1], s[92:93], s[92:93] op_sel:[0,1]
-; GLOBALNESS0-NEXT:    flat_load_dword v44, v[0:1]
+; GLOBALNESS0-NEXT:    flat_load_dword v46, v[0:1]
 ; GLOBALNESS0-NEXT:    s_add_u32 s8, s60, 40
 ; GLOBALNESS0-NEXT:    buffer_store_dword v42, off, s[0:3], 0
-; GLOBALNESS0-NEXT:    flat_load_dword v45, v[0:1]
+; GLOBALNESS0-NEXT:    flat_load_dword v47, v[0:1]
 ; GLOBALNESS0-NEXT:    s_addc_u32 s9, s61, 0
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[4:5], s[62:63]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS0-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS0-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS0-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v31, v41

 ; GLOBALNESS0-NEXT:    ; implicit-def: $sgpr4_sgpr5
 ; GLOBALNESS0-NEXT:    s_branch .LBB1_23
 ; GLOBALNESS0-NEXT:  .LBB1_8: ; %Flow16
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS0-NEXT:    s_and_b64 vcc, exec, s[6:7]
 ; GLOBALNESS0-NEXT:    s_cbranch_vccz .LBB1_24
 ; GLOBALNESS0-NEXT:  .LBB1_9: ; %baz.exit.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS0-NEXT:    flat_load_dword v0, v[32:33]
+; GLOBALNESS0-NEXT:    flat_load_dword v0, v[44:45]
 ; GLOBALNESS0-NEXT:    s_mov_b32 s68, s93
 ; GLOBALNESS0-NEXT:    s_mov_b32 s70, s93
 ; GLOBALNESS0-NEXT:    s_mov_b32 s71, s69
 ; GLOBALNESS0-NEXT:    s_mov_b32 s72, s93
 ; GLOBALNESS0-NEXT:    s_mov_b32 s73, s69
 ; GLOBALNESS0-NEXT:    s_mov_b32 s74, s93
 ; GLOBALNESS0-NEXT:    s_mov_b32 s75, s69
 ; GLOBALNESS0-NEXT:    s_mov_b32 s76, s93

 ; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[24:25], s[92:93], s[92:93] op_sel:[0,1]
 ; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[26:27], s[94:95], s[94:95] op_sel:[0,1]
 ; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[28:29], s[96:97], s[96:97] op_sel:[0,1]
 ; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[30:31], s[98:99], s[98:99] op_sel:[0,1]
 ; GLOBALNESS0-NEXT:    s_and_saveexec_b64 s[70:71], s[96:97]
 ; GLOBALNESS0-NEXT:    s_cbranch_execz .LBB1_26
 ; GLOBALNESS0-NEXT:  ; %bb.10: ; %bb33.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS0-NEXT:    global_load_dwordx2 v[0:1], v[32:33], off
+; GLOBALNESS0-NEXT:    global_load_dwordx2 v[0:1], v[44:45], off
 ; GLOBALNESS0-NEXT:    v_readlane_b32 s4, v40, 0
 ; GLOBALNESS0-NEXT:    v_readlane_b32 s5, v40, 1
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[72:73], s[36:37]
 ; GLOBALNESS0-NEXT:    s_mov_b32 s75, s39
 ; GLOBALNESS0-NEXT:    s_andn2_b64 vcc, exec, s[4:5]
 ; GLOBALNESS0-NEXT:    s_cbranch_vccnz .LBB1_12
 ; GLOBALNESS0-NEXT:  ; %bb.11: ; %bb39.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[2:3], 0, 0
-; GLOBALNESS0-NEXT:    global_store_dwordx2 v[2:3], v[42:43], off
+; GLOBALNESS0-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS0-NEXT:  .LBB1_12: ; %bb44.lr.ph.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
-; GLOBALNESS0-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v45
-; GLOBALNESS0-NEXT:    v_cndmask_b32_e32 v2, 0, v44, vcc
+; GLOBALNESS0-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v47
+; GLOBALNESS0-NEXT:    v_cndmask_b32_e32 v2, 0, v46, vcc
 ; GLOBALNESS0-NEXT:    s_waitcnt vmcnt(0)
 ; GLOBALNESS0-NEXT:    v_cmp_nlt_f64_e64 s[36:37], 0, v[0:1]
 ; GLOBALNESS0-NEXT:    v_cmp_eq_u32_e64 s[58:59], 0, v2
 ; GLOBALNESS0-NEXT:    s_branch .LBB1_15
 ; GLOBALNESS0-NEXT:  .LBB1_13: ; %Flow7
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_15 Depth=2
 ; GLOBALNESS0-NEXT:    s_or_b64 exec, exec, s[4:5]
 ; GLOBALNESS0-NEXT:  .LBB1_14: ; %bb63.i

 ; GLOBALNESS0-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[8:9], s[64:65]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS0-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS0-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS0-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v31, v41
 ; GLOBALNESS0-NEXT:    s_swappc_b64 s[30:31], s[66:67]
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[44:45], 0, 0
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[4:5], s[62:63]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[6:7], s[54:55]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[8:9], s[64:65]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[10:11], s[34:35]
 ; GLOBALNESS0-NEXT:    s_mov_b32 s12, s56
 ; GLOBALNESS0-NEXT:    s_mov_b32 s13, s99
 ; GLOBALNESS0-NEXT:    s_mov_b32 s14, s98
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v31, v41

 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS0-NEXT:    v_readlane_b32 s6, v40, 2
 ; GLOBALNESS0-NEXT:    v_readlane_b32 s7, v40, 3
 ; GLOBALNESS0-NEXT:    s_andn2_b64 vcc, exec, s[6:7]
 ; GLOBALNESS0-NEXT:    s_cbranch_vccnz .LBB1_1
 ; GLOBALNESS0-NEXT:  ; %bb.28: ; %bb69.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS0-NEXT:    global_store_dwordx2 v[32:33], v[42:43], off
+; GLOBALNESS0-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS0-NEXT:    s_branch .LBB1_1
 ; GLOBALNESS0-NEXT:  .LBB1_29: ; %bb73.i
 ; GLOBALNESS0-NEXT:    ; in Loop: Header=BB1_4 Depth=1
 ; GLOBALNESS0-NEXT:    v_mov_b32_e32 v43, v42
-; GLOBALNESS0-NEXT:    v_pk_mov_b32 v[32:33], 0, 0
-; GLOBALNESS0-NEXT:    global_store_dwordx2 v[32:33], v[42:43], off
+; GLOBALNESS0-NEXT:    global_store_dwordx2 v[44:45], v[42:43], off
 ; GLOBALNESS0-NEXT:    s_branch .LBB1_2
 ; GLOBALNESS0-NEXT:  .LBB1_30: ; %loop.exit.guard
 ; GLOBALNESS0-NEXT:    s_andn2_b64 vcc, exec, s[4:5]
 ; GLOBALNESS0-NEXT:    s_mov_b64 s[4:5], -1
 ; GLOBALNESS0-NEXT:    s_cbranch_vccz .LBB1_32
 ; GLOBALNESS0-NEXT:  ; %bb.31: ; %bb7.i.i
 ; GLOBALNESS0-NEXT:    s_add_u32 s8, s60, 40
 ; GLOBALNESS0-NEXT:    s_addc_u32 s9, s61, 0

llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll

 ; GFX9-O0-LABEL: strict_wwm_call:
 ; GFX9-O0:       ; %bb.0:
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-O0-NEXT:    s_mov_b32 s35, s33
 ; GFX9-O0-NEXT:    s_mov_b32 s33, s32
 ; GFX9-O0-NEXT:    s_xor_saveexec_b64 s[36:37], -1
 ; GFX9-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v1, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[36:37]
 ; GFX9-O0-NEXT:    s_add_i32 s32, s32, 0x800
 ; GFX9-O0-NEXT:    ; implicit-def: $vgpr0
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s30, 0
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s31, 1
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[48:49], -1
 ; GFX9-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[48:49]

 ; GFX9-O0-NEXT:    s_add_u32 s42, s42, strict_wwm_called@rel32@lo+4
 ; GFX9-O0-NEXT:    s_addc_u32 s43, s43, strict_wwm_called@rel32@hi+12
 ; GFX9-O0-NEXT:    s_mov_b64 s[46:47], s[2:3]
 ; GFX9-O0-NEXT:    s_mov_b64 s[44:45], s[0:1]
 ; GFX9-O0-NEXT:    s_mov_b64 s[0:1], s[44:45]
 ; GFX9-O0-NEXT:    s_mov_b64 s[2:3], s[46:47]
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v2
 ; GFX9-O0-NEXT:    s_swappc_b64 s[30:31], s[42:43]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v0
+; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v0
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[48:49], -1
 ; GFX9-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[48:49]
-; GFX9-O0-NEXT:    v_add_u32_e64 v1, v1, v2
+; GFX9-O0-NEXT:    v_add_u32_e64 v2, v3, v2
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[40:41]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v1
-; GFX9-O0-NEXT:    buffer_store_dword v3, off, s[36:39], s34 offset:4
+; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v2
+; GFX9-O0-NEXT:    buffer_store_dword v1, off, s[36:39], s34 offset:4
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(1)
 ; GFX9-O0-NEXT:    v_readlane_b32 s31, v0, 1
 ; GFX9-O0-NEXT:    v_readlane_b32 s30, v0, 0
 ; GFX9-O0-NEXT:    s_xor_saveexec_b64 s[36:37], -1
 ; GFX9-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT:    buffer_load_dword v1, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[36:37]
 ; GFX9-O0-NEXT:    s_add_i32 s32, s32, 0xfffff800
 ; GFX9-O0-NEXT:    s_mov_b32 s33, s35
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O0-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-O3-LABEL: strict_wwm_call:
 ; GFX9-O3:       ; %bb.0:
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-O3-NEXT:    s_mov_b32 s38, s33
 ; GFX9-O3-NEXT:    s_mov_b32 s33, s32
 ; GFX9-O3-NEXT:    s_xor_saveexec_b64 s[34:35], -1
-; GFX9-O3-NEXT:    buffer_store_dword v3, off, s[0:3], s33 ; 4-byte Folded Spill
+; GFX9-O3-NEXT:    buffer_store_dword v4, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-O3-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
-; GFX9-O3-NEXT:    buffer_store_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
+; GFX9-O3-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
-; GFX9-O3-NEXT:    ; implicit-def: $vgpr3
+; GFX9-O3-NEXT:    ; implicit-def: $vgpr4
 ; GFX9-O3-NEXT:    s_addk_i32 s32, 0x400
-; GFX9-O3-NEXT:    v_writelane_b32 v3, s30, 0
-; GFX9-O3-NEXT:    v_writelane_b32 v3, s31, 1
+; GFX9-O3-NEXT:    v_writelane_b32 v4, s30, 0
+; GFX9-O3-NEXT:    v_writelane_b32 v4, s31, 1
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v2, s8
 ; GFX9-O3-NEXT:    s_not_b64 exec, exec
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v2, 0
 ; GFX9-O3-NEXT:    s_not_b64 exec, exec
 ; GFX9-O3-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v2
 ; GFX9-O3-NEXT:    s_getpc_b64 s[36:37]
 ; GFX9-O3-NEXT:    s_add_u32 s36, s36, strict_wwm_called@rel32@lo+4
 ; GFX9-O3-NEXT:    s_addc_u32 s37, s37, strict_wwm_called@rel32@hi+12
 ; GFX9-O3-NEXT:    s_swappc_b64 s[30:31], s[36:37]
-; GFX9-O3-NEXT:    v_mov_b32_e32 v1, v0
-; GFX9-O3-NEXT:    v_add_u32_e32 v1, v1, v2
+; GFX9-O3-NEXT:    v_mov_b32_e32 v3, v0
+; GFX9-O3-NEXT:    v_add_u32_e32 v2, v3, v2
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
-; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v1
+; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v2
 ; GFX9-O3-NEXT:    buffer_store_dword v0, off, s[4:7], 0 offset:4
-; GFX9-O3-NEXT:    v_readlane_b32 s31, v3, 1
-; GFX9-O3-NEXT:    v_readlane_b32 s30, v3, 0
+; GFX9-O3-NEXT:    v_readlane_b32 s31, v4, 1
+; GFX9-O3-NEXT:    v_readlane_b32 s30, v4, 0
 ; GFX9-O3-NEXT:    s_xor_saveexec_b64 s[34:35], -1
-; GFX9-O3-NEXT:    buffer_load_dword v3, off, s[0:3], s33 ; 4-byte Folded Reload
+; GFX9-O3-NEXT:    buffer_load_dword v4, off, s[0:3], s33 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
-; GFX9-O3-NEXT:    buffer_load_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
+; GFX9-O3-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-O3-NEXT:    s_addk_i32 s32, 0xfc00
 ; GFX9-O3-NEXT:    s_mov_b32 s33, s38
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O3-NEXT:    s_setpc_b64 s[30:31]
   %tmp107 = tail call i32 @llvm.amdgcn.set.inactive.i32(i32 %arg, i32 0)
   %tmp134 = call amdgpu_gfx i32 @strict_wwm_called(i32 %tmp107)
   %tmp136 = add i32 %tmp134, %tmp107

 define amdgpu_gfx void @strict_wwm_call_i64(<4 x i32> inreg %tmp14, i64 inreg %arg) {
 ; GFX9-O0-LABEL: strict_wwm_call_i64:
 ; GFX9-O0:       ; %bb.0:
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-O0-NEXT:    s_mov_b32 s44, s33
 ; GFX9-O0-NEXT:    s_mov_b32 s33, s32
 ; GFX9-O0-NEXT:    s_xor_saveexec_b64 s[34:35], -1
 ; GFX9-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v6, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    buffer_store_dword v8, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O0-NEXT:    buffer_store_dword v9, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v10, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v11, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT:    buffer_store_dword v4, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v4, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
-; GFX9-O0-NEXT:    buffer_store_dword v5, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v12, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v11, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v10, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT:    buffer_store_dword v11, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v8, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
+; GFX9-O0-NEXT:    buffer_store_dword v9, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-O0-NEXT:    s_add_i32 s32, s32, 0x1000
 ; GFX9-O0-NEXT:    ; implicit-def: $vgpr0
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s30, 0
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s31, 1
 ; GFX9-O0-NEXT:    s_mov_b32 s34, s8
 ; GFX9-O0-NEXT:    s_mov_b32 s36, s4
 ; GFX9-O0-NEXT:    ; kill: def $sgpr36 killed $sgpr36 def $sgpr36_sgpr37_sgpr38_sgpr39

 ; GFX9-O0-NEXT:    v_mov_b32_e32 v9, s37
 ; GFX9-O0-NEXT:    s_not_b64 exec, exec
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s34, 6
 ; GFX9-O0-NEXT:    v_writelane_b32 v0, s35, 7
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[42:43], -1
 ; GFX9-O0-NEXT:    buffer_store_dword v0, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[42:43]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v8
+; GFX9-O0-NEXT:    v_mov_b32_e32 v10, v8
 ; GFX9-O0-NEXT:    s_mov_b32 s34, 32
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr36_sgpr37
-; GFX9-O0-NEXT:    v_lshrrev_b64 v[3:4], s34, v[8:9]
+; GFX9-O0-NEXT:    v_lshrrev_b64 v[11:12], s34, v[8:9]
 ; GFX9-O0-NEXT:    s_getpc_b64 s[34:35]
 ; GFX9-O0-NEXT:    s_add_u32 s34, s34, strict_wwm_called_i64@gotpcrel32@lo+4
 ; GFX9-O0-NEXT:    s_addc_u32 s35, s35, strict_wwm_called_i64@gotpcrel32@hi+12
 ; GFX9-O0-NEXT:    s_load_dwordx2 s[34:35], s[34:35], 0x0
 ; GFX9-O0-NEXT:    s_mov_b64 s[38:39], s[2:3]
 ; GFX9-O0-NEXT:    s_mov_b64 s[36:37], s[0:1]
 ; GFX9-O0-NEXT:    s_mov_b64 s[0:1], s[36:37]
 ; GFX9-O0-NEXT:    s_mov_b64 s[2:3], s[38:39]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v2
-; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v3
+; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v10
+; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v11
 ; GFX9-O0-NEXT:    s_waitcnt lgkmcnt(0)
 ; GFX9-O0-NEXT:    s_swappc_b64 s[30:31], s[34:35]
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[42:43], -1
-; GFX9-O0-NEXT:    buffer_load_dword v6, off, s[0:3], s33 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v2, off, s[0:3], s33 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[42:43]
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT:    v_readlane_b32 s34, v6, 6
-; GFX9-O0-NEXT:    v_readlane_b32 s35, v6, 7
-; GFX9-O0-NEXT:    v_readlane_b32 s36, v6, 2
-; GFX9-O0-NEXT:    v_readlane_b32 s37, v6, 3
-; GFX9-O0-NEXT:    v_readlane_b32 s38, v6, 4
-; GFX9-O0-NEXT:    v_readlane_b32 s39, v6, 5
-; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v0
+; GFX9-O0-NEXT:    v_readlane_b32 s34, v2, 6
+; GFX9-O0-NEXT:    v_readlane_b32 s35, v2, 7
+; GFX9-O0-NEXT:    v_readlane_b32 s36, v2, 2
+; GFX9-O0-NEXT:    v_readlane_b32 s37, v2, 3
+; GFX9-O0-NEXT:    v_readlane_b32 s38, v2, 4
+; GFX9-O0-NEXT:    v_readlane_b32 s39, v2, 5
+; GFX9-O0-NEXT:    v_mov_b32_e32 v10, v0
 ; GFX9-O0-NEXT:    s_or_saveexec_b64 s[42:43], -1
 ; GFX9-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[42:43]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v1
+; GFX9-O0-NEXT:    v_mov_b32_e32 v11, v1
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr40
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr40
-; GFX9-O0-NEXT:    ; kill: def $vgpr2 killed $vgpr2 killed $exec
-; GFX9-O0-NEXT:    v_mov_b32_e32 v4, v8
-; GFX9-O0-NEXT:    v_mov_b32_e32 v5, v9
-; GFX9-O0-NEXT:    v_add_co_u32_e64 v2, s[40:41], v2, v4
-; GFX9-O0-NEXT:    v_addc_co_u32_e64 v3, s[40:41], v3, v5, s[40:41]
+; GFX9-O0-NEXT:    ; kill: def $vgpr10 killed $vgpr10 killed $exec
+; GFX9-O0-NEXT:    v_add_co_u32_e64 v8, s[40:41], v10, v8
+; GFX9-O0-NEXT:    v_addc_co_u32_e64 v9, s[40:41], v11, v9, s[40:41]
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[34:35]
-; GFX9-O0-NEXT:    v_mov_b32_e32 v6, v2
-; GFX9-O0-NEXT:    v_mov_b32_e32 v7, v3
+; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v8
+; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v9
 ; GFX9-O0-NEXT:    s_mov_b32 s34, 0
-; GFX9-O0-NEXT:    buffer_store_dwordx2 v[6:7], off, s[36:39], s34 offset:4
+; GFX9-O0-NEXT:    buffer_store_dwordx2 v[1:2], off, s[36:39], s34 offset:4
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(1)
 ; GFX9-O0-NEXT:    v_readlane_b32 s31, v0, 1
 ; GFX9-O0-NEXT:    v_readlane_b32 s30, v0, 0
 ; GFX9-O0-NEXT:    s_xor_saveexec_b64 s[34:35], -1
 ; GFX9-O0-NEXT:    buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
-; GFX9-O0-NEXT:    buffer_load_dword v6, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
 ; GFX9-O0-NEXT:    buffer_load_dword v8, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
 ; GFX9-O0-NEXT:    buffer_load_dword v9, off, s[0:3], s33 offset:16 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:20 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v10, off, s[0:3], s33 offset:20 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:24 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v11, off, s[0:3], s33 offset:24 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v4, off, s[0:3], s33 offset:28 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v12, off, s[0:3], s33 offset:28 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v11, off, s[0:3], s33 offset:32 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:36 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v10, off, s[0:3], s33 offset:36 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:40 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v11, off, s[0:3], s33 offset:40 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_nop 0
-; GFX9-O0-NEXT:    buffer_load_dword v4, off, s[0:3], s33 offset:44 ; 4-byte Folded Reload
-; GFX9-O0-NEXT:    buffer_load_dword v5, off, s[0:3], s33 offset:48 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v8, off, s[0:3], s33 offset:44 ; 4-byte Folded Reload
+; GFX9-O0-NEXT:    buffer_load_dword v9, off, s[0:3], s33 offset:48 ; 4-byte Folded Reload
 ; GFX9-O0-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-O0-NEXT:    s_add_i32 s32, s32, 0xfffff000
 ; GFX9-O0-NEXT:    s_mov_b32 s33, s44
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O0-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-O3-LABEL: strict_wwm_call_i64:
 ; GFX9-O3:       ; %bb.0:
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-O3-NEXT:    s_mov_b32 s40, s33
 ; GFX9-O3-NEXT:    s_mov_b32 s33, s32
 ; GFX9-O3-NEXT:    s_xor_saveexec_b64 s[34:35], -1
-; GFX9-O3-NEXT:    buffer_store_dword v8, off, s[0:3], s33 ; 4-byte Folded Spill
+; GFX9-O3-NEXT:    buffer_store_dword v10, off, s[0:3], s33 ; 4-byte Folded Spill
 ; GFX9-O3-NEXT:    buffer_store_dword v6, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O3-NEXT:    buffer_store_dword v7, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
-; GFX9-O3-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
-; GFX9-O3-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
-; GFX9-O3-NEXT:    buffer_store_dword v2, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
-; GFX9-O3-NEXT:    s_waitcnt vmcnt(0)
-; GFX9-O3-NEXT:    buffer_store_dword v3, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
+; GFX9-O3-NEXT:    buffer_store_dword v8, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
+; GFX9-O3-NEXT:    buffer_store_dword v9, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
-; GFX9-O3-NEXT:    ; implicit-def: $vgpr8
+; GFX9-O3-NEXT:    ; implicit-def: $vgpr10
 ; GFX9-O3-NEXT:    s_addk_i32 s32, 0x800
-; GFX9-O3-NEXT:    v_writelane_b32 v8, s30, 0
-; GFX9-O3-NEXT:    v_writelane_b32 v8, s31, 1
+; GFX9-O3-NEXT:    v_writelane_b32 v10, s30, 0
+; GFX9-O3-NEXT:    v_writelane_b32 v10, s31, 1
 ; GFX9-O3-NEXT:    s_or_saveexec_b64 s[34:35], -1
 ; GFX9-O3-NEXT:    s_getpc_b64 s[36:37]
 ; GFX9-O3-NEXT:    s_add_u32 s36, s36, strict_wwm_called_i64@gotpcrel32@lo+4
 ; GFX9-O3-NEXT:    s_addc_u32 s37, s37, strict_wwm_called_i64@gotpcrel32@hi+12
 ; GFX9-O3-NEXT:    s_load_dwordx2 s[36:37], s[36:37], 0x0
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v6, s8
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v7, s9
 ; GFX9-O3-NEXT:    s_not_b64 exec, exec
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v6, 0
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v7, 0
 ; GFX9-O3-NEXT:    s_not_b64 exec, exec
 ; GFX9-O3-NEXT:    s_or_saveexec_b64 s[38:39], -1
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v6
 ; GFX9-O3-NEXT:    v_mov_b32_e32 v1, v7
 ; GFX9-O3-NEXT:    s_waitcnt lgkmcnt(0)
 ; GFX9-O3-NEXT:    s_swappc_b64 s[30:31], s[36:37]
-; GFX9-O3-NEXT:    v_mov_b32_e32 v2, v0
-; GFX9-O3-NEXT:    v_mov_b32_e32 v3, v1
-; GFX9-O3-NEXT:    v_add_co_u32_e32 v2, vcc, v2, v6
-; GFX9-O3-NEXT:    v_addc_co_u32_e32 v3, vcc, v3, v7, vcc
+; GFX9-O3-NEXT:    v_mov_b32_e32 v8, v0
+; GFX9-O3-NEXT:    v_mov_b32_e32 v9, v1
+; GFX9-O3-NEXT:    v_add_co_u32_e32 v6, vcc, v8, v6
+; GFX9-O3-NEXT:    v_addc_co_u32_e32 v7, vcc, v9, v7, vcc
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[38:39]
-; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v2
-; GFX9-O3-NEXT:    v_mov_b32_e32 v1, v3
+; GFX9-O3-NEXT:    v_mov_b32_e32 v0, v6
+; GFX9-O3-NEXT:    v_mov_b32_e32 v1, v7
 ; GFX9-O3-NEXT:    buffer_store_dwordx2 v[0:1], off, s[4:7], 0 offset:4
-; GFX9-O3-NEXT:    v_readlane_b32 s31, v8, 1
-; GFX9-O3-NEXT:    v_readlane_b32 s30, v8, 0
+; GFX9-O3-NEXT:    v_readlane_b32 s31, v10, 1
+; GFX9-O3-NEXT:    v_readlane_b32 s30, v10, 0
 ; GFX9-O3-NEXT:    s_xor_saveexec_b64 s[34:35], -1
-; GFX9-O3-NEXT:    buffer_load_dword v8, off, s[0:3], s33 ; 4-byte Folded Reload
+; GFX9-O3-NEXT:    buffer_load_dword v10, off, s[0:3], s33 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    s_nop 0
 ; GFX9-O3-NEXT:    buffer_load_dword v6, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    s_nop 0
 ; GFX9-O3-NEXT:    buffer_load_dword v7, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    s_nop 0
-; GFX9-O3-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
-; GFX9-O3-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:16 ; 4-byte Folded Reload
-; GFX9-O3-NEXT:    s_nop 0
-; GFX9-O3-NEXT:    buffer_load_dword v2, off, s[0:3], s33 offset:20 ; 4-byte Folded Reload
-; GFX9-O3-NEXT:    s_nop 0
-; GFX9-O3-NEXT:    buffer_load_dword v3, off, s[0:3], s33 offset:24 ; 4-byte Folded Reload
+; GFX9-O3-NEXT:    buffer_load_dword v8, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
+; GFX9-O3-NEXT:    buffer_load_dword v9, off, s[0:3], s33 offset:16 ; 4-byte Folded Reload
 ; GFX9-O3-NEXT:    s_mov_b64 exec, s[34:35]
 ; GFX9-O3-NEXT:    s_addk_i32 s32, 0xf800
 ; GFX9-O3-NEXT:    s_mov_b32 s33, s40
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0)
 ; GFX9-O3-NEXT:    s_setpc_b64 s[30:31]
   %tmp107 = tail call i64 @llvm.amdgcn.set.inactive.i64(i64 %arg, i64 0)
   %tmp134 = call amdgpu_gfx i64 @strict_wwm_called_i64(i64 %tmp107)
   %tmp136 = add i64 %tmp134, %tmp107

llvm/test/CodeGen/AMDGPU/wwm-reserved.ll

 ; GFX9-NEXT: s_not_b64 exec, exec
 ; GFX9-O0-NEXT: v_mov_b32_e32 v2, s4
 ; GFX9-O3-NEXT: v_mov_b32_e32 v2, 0
 ; GFX9-NEXT: s_not_b64 exec, exec
   %tmp107 = tail call i32 @llvm.amdgcn.set.inactive.i32(i32 %arg, i32 0)
 ; GFX9: v_mov_b32_e32 v0, v2
 ; GFX9: s_swappc_b64
   %tmp134 = call i32 @called(i32 %tmp107)
-; GFX9: v_mov_b32_e32 v1, v0
-; GFX9-O3: v_add_u32_e32 v1, v1, v2
-; GFX9-O0: v_add_u32_e64 v1, v1, v2
+; GFX9: v_mov_b32_e32 v3, v0
+; GFX9-O3: v_add_u32_e32 v2, v3, v2
+; GFX9-O0: v_add_u32_e64 v2, v3, v2
   %tmp136 = add i32 %tmp134, %tmp107
   %tmp137 = tail call i32 @llvm.amdgcn.wwm.i32(i32 %tmp136)
 ; GFX9: buffer_store_dword v0
   call void @llvm.amdgcn.raw.buffer.store.i32(i32 %tmp137, <4 x i32> %tmp14, i32 4, i32 0, i32 0)
   ret void
 }
 
 ; GFX9-LABEL: {{^}}called_i64:

 ; GFX9-NEXT: s_not_b64 exec, exec
 ; GFX9-O0-NEXT: v_mov_b32_e32 v2, s4
 ; GFX9-O3-NEXT: v_mov_b32_e32 v2, 0
 ; GFX9-NEXT: s_not_b64 exec, exec
   %tmp107 = tail call i32 @llvm.amdgcn.set.inactive.i32(i32 %arg, i32 0)
 ; GFX9: v_mov_b32_e32 v0, v2
 ; GFX9: s_swappc_b64
   %tmp134 = call i32 @strict_wwm_called(i32 %tmp107)
-; GFX9: v_mov_b32_e32 v1, v0
-; GFX9-O3: v_add_u32_e32 v1, v1, v2
-; GFX9-O0: v_add_u32_e64 v1, v1, v2
+; GFX9: v_mov_b32_e32 v3, v0
+; GFX9-O3: v_add_u32_e32 v2, v3, v2
+; GFX9-O0: v_add_u32_e64 v2, v3, v2
   %tmp136 = add i32 %tmp134, %tmp107
   %tmp137 = tail call i32 @llvm.amdgcn.strict.wwm.i32(i32 %tmp136)
 ; GFX9: buffer_store_dword v0
   call void @llvm.amdgcn.raw.buffer.store.i32(i32 %tmp137, <4 x i32> %tmp14, i32 4, i32 0, i32 0)
   ret void
 }
 
 ; GFX9-LABEL: {{^}}strict_wwm_called_i64:

llvm/test/CodeGen/PowerPC/cse-despite-rounding-mode.ll

 ; The non-strictfp version of test/CodeGen/PowerPC/respect-rounding-mode.ll
 ; Without strictfp, CSE should be free to eliminate the repeated multiply
 ; and conversion instructions.
 ; RUN: llc -verify-machineinstrs --mtriple powerpc64le-unknown-linux-gnu \
-; RUN:   -mcpu=pwr8 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 2
+; RUN:   -mcpu=pwr8 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 4
 ; RUN: llc -verify-machineinstrs --mtriple powerpc-unknown-linux-gnu \
-; RUN:   -mcpu=pwr9 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 2
+; RUN:   -mcpu=pwr9 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 4
 ; RUN: llc -verify-machineinstrs --mtriple powerpc64le-unknown-linux-gnu \
-; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 2
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | grep 'xvrdpic' | count 4
 
 ; RUN: llc -verify-machineinstrs --mtriple powerpc64le-unknown-linux-gnu \
-; RUN:   -mcpu=pwr8 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 2
+; RUN:   -mcpu=pwr8 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 4
 ; RUN: llc -verify-machineinstrs --mtriple powerpc-unknown-linux-gnu \
-; RUN:   -mcpu=pwr9 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 2
+; RUN:   -mcpu=pwr9 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 4
 ; RUN: llc -verify-machineinstrs --mtriple powerpc64le-unknown-linux-gnu \
-; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 2
+; RUN:   -mcpu=pwr10 -ppc-asm-full-reg-names < %s | grep 'xvmuldp' | count 4
 @IndirectCallPtr = dso_local local_unnamed_addr global ptr null, align 8
 
 define dso_local signext i32 @func1() local_unnamed_addr #0 {
 entry:
   tail call void @directCall() #0
   %0 = tail call <2 x double> @llvm.experimental.constrained.rint.v2f64(<2 x double> <double -9.990000e+01, double 9.990000e+01>, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
   %vecext = extractelement <2 x double> %0, i32 0
   %sub = tail call double @llvm.experimental.constrained.fsub.f64(double %vecext, double -9.900000e+01, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0

llvm/test/CodeGen/PowerPC/cxx_tlscc64.ll

 define cxx_fast_tlscc nonnull ptr @_ZTW2sg() nounwind {
 ; CHECK-LABEL: _ZTW2sg:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    mflr 0
 ; CHECK-NEXT:    std 30, -16(1) # 8-byte Folded Spill
 ; CHECK-NEXT:    stdu 1, -48(1)
 ; CHECK-NEXT:    std 0, 64(1)
 ; CHECK-NEXT:    addis 3, 13, __tls_guard@tprel@ha
-; CHECK-NEXT:    lbz 4, __tls_guard@tprel@l(3)
-; CHECK-NEXT:    andi. 4, 4, 1
+; CHECK-NEXT:    lbz 3, __tls_guard@tprel@l(3)
+; CHECK-NEXT:    andi. 3, 3, 1
 ; CHECK-NEXT:    bc 12, 1, .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %init.i
+; CHECK-NEXT:    addis 3, 13, __tls_guard@tprel@ha
 ; CHECK-NEXT:    li 4, 1
 ; CHECK-NEXT:    stb 4, __tls_guard@tprel@l(3)
 ; CHECK-NEXT:    addis 3, 13, sg@tprel@ha
 ; CHECK-NEXT:    addi 30, 3, sg@tprel@l
 ; CHECK-NEXT:    mr 3, 30
 ; CHECK-NEXT:    bl _ZN1SC1Ev
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    addis 3, 2, .LC0@toc@ha

llvm/test/CodeGen/PowerPC/handle-f16-storage-type.ll

 ; P8-NEXT:    stdu r1, -32(r1)
 ; P8-NEXT:    std r0, 48(r1)
 ; P8-NEXT:    bl __gnu_f2h_ieee
 ; P8-NEXT:    nop
 ; P8-NEXT:    clrldi r3, r3, 48
 ; P8-NEXT:    bl __gnu_h2f_ieee
 ; P8-NEXT:    nop
 ; P8-NEXT:    xxlxor f0, f0, f0
+; P8-NEXT:    addis r3, r2, .LCPI20_0@toc@ha
 ; P8-NEXT:    fcmpu cr0, f1, f0
 ; P8-NEXT:    beq cr0, .LBB20_2
 ; P8-NEXT:  # %bb.1:
-; P8-NEXT:    addis r3, r2, .LCPI20_0@toc@ha
 ; P8-NEXT:    lfs f0, .LCPI20_0@toc@l(r3)
 ; P8-NEXT:  .LBB20_2:
 ; P8-NEXT:    fmr f1, f0
 ; P8-NEXT:    addi r1, r1, 32
 ; P8-NEXT:    ld r0, 16(r1)
 ; P8-NEXT:    mtlr r0
 ; P8-NEXT:    blr
 ;

llvm/test/CodeGen/SPARC/2011-01-19-DelaySlot.ll

;CHECK:  restore
   ret i32 %1
 }
 
 
 define i32 @restore_sethi(i32 %a) {
 entry:
 ;CHECK-LABEL: restore_sethi:
 ;CHECK-NOT: sethi  3
-;CHECK: restore %g0, 3072, %o0
+;CHECK: restore
   %0 = tail call i32 @bar(i32 %a) nounwind
   %1 = icmp ne i32 %0, 0
   %2 = select i1 %1, i32 3072, i32 0
   ret i32 %2
 }
 
 define i32 @restore_sethi_3bit(i32 %a) {
 entry:

llvm/test/CodeGen/X86/2008-10-27-CoalescerBug.ll

 ; REQUIRES: asserts
 ; RUN: llc < %s -mtriple=i386-apple-darwin -mattr=+sse2 -stats 2>&1 | FileCheck %s
 ; Now this test spills one register. But a reload in the loop is cheaper than
 ; the divsd so it's a win.
 
+; FIXME: MachineLICM failed to move DIVSDrr out of loop because it uses register
+; $mxcsr, it is clobbered by function call to sin. We need to model the
+; volatile / non-volatile part of $mxcsr, so DIVSDrr uses the non-volatile part
+; of $mxcsr and function call clobbers volatile part of $mxcsr, then we can
+; safely move DIVSDrr out of the loop.
+
 define fastcc void @fourn(ptr %data, i32 %isign) nounwind {
 ; CHECK: fourn
 entry:
   br label %bb
 
 bb:   ; preds = %bb, %entry
   %indvar93 = phi i32 [ 0, %entry ], [ %idim.030, %bb ]    ; <i32> [#uses=2]
   %idim.030 = add i32 %indvar93, 1    ; <i32> [#uses=1]
   %0 = add i32 %indvar93, 2   ; <i32> [#uses=1]
   %1 = icmp sgt i32 %0, 2    ; <i1> [#uses=1]
   br i1 %1, label %bb30.loopexit, label %bb
 
-; CHECK: %bb30.loopexit
+; CHECK: %bb18
 ; CHECK: divsd %xmm0
-; CHECK: movsd %xmm0, 16(%esp)
-; CHECK: %bb3
+; CHECK: movsd %xmm0, (%esp)
 bb3:    ; preds = %bb30.loopexit, %bb25, %bb3
   %2 = load i32, ptr null, align 4    ; <i32> [#uses=1]
   %3 = mul i32 %2, 0    ; <i32> [#uses=1]
   %4 = icmp slt i32 0, %3    ; <i1> [#uses=1]
   br i1 %4, label %bb18, label %bb3
 
 bb18:   ; preds = %bb3
   %5 = fdiv double %11, 0.000000e+00    ; <double> [#uses=1]

llvm/test/CodeGen/X86/fp-strict-scalar-inttofp-fp16.ll

 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movq %rdi, %rax
 ; SSE2-NEXT:    shrq %rax
 ; SSE2-NEXT:    movl %edi, %ecx
 ; SSE2-NEXT:    andl $1, %ecx
 ; SSE2-NEXT:    orq %rax, %rcx
 ; SSE2-NEXT:    testq %rdi, %rdi
 ; SSE2-NEXT:    cmovnsq %rdi, %rcx
-; SSE2-NEXT:    cvtsi2ss %rcx, %xmm0
-; SSE2-NEXT:    jns .LBB9_2
+; SSE2-NEXT:    cvtsi2ss %rcx, %xmm1
+; SSE2-NEXT:    movaps %xmm1, %xmm0
+; SSE2-NEXT:    addss %xmm1, %xmm0
+; SSE2-NEXT:    js .LBB9_2
 ; SSE2-NEXT:  # %bb.1:
-; SSE2-NEXT:    addss %xmm0, %xmm0
+; SSE2-NEXT:    movaps %xmm1, %xmm0
 ; SSE2-NEXT:  .LBB9_2:
 ; SSE2-NEXT:    pushq %rax
 ; SSE2-NEXT:    callq __truncsfhf2@PLT
 ; SSE2-NEXT:    popq %rax
 ; SSE2-NEXT:    retq
 ;
 ; F16C-LABEL: uitofp_i64tof16:
 ; F16C:       # %bb.0:

llvm/test/CodeGen/X86/half.ll

; CHECK-I686-NEXT:    retl
   ret i64 %r
 }
 
 define void @test_uitofp_i64(i64 %a, ptr %p) #0 {
 ; CHECK-LIBCALL-LABEL: test_uitofp_i64:
 ; CHECK-LIBCALL:       # %bb.0:
 ; CHECK-LIBCALL-NEXT:    pushq %rbx
 ; CHECK-LIBCALL-NEXT:    movq %rsi, %rbx
-; CHECK-LIBCALL-NEXT:    testq %rdi, %rdi
-; CHECK-LIBCALL-NEXT:    js .LBB10_1
-; CHECK-LIBCALL-NEXT:  # %bb.2:
-; CHECK-LIBCALL-NEXT:    cvtsi2ss %rdi, %xmm0
-; CHECK-LIBCALL-NEXT:    jmp .LBB10_3
-; CHECK-LIBCALL-NEXT:  .LBB10_1:
 ; CHECK-LIBCALL-NEXT:    movq %rdi, %rax
 ; CHECK-LIBCALL-NEXT:    shrq %rax
-; CHECK-LIBCALL-NEXT:    andl $1, %edi
-; CHECK-LIBCALL-NEXT:    orq %rax, %rdi
-; CHECK-LIBCALL-NEXT:    cvtsi2ss %rdi, %xmm0
+; CHECK-LIBCALL-NEXT:    movl %edi, %ecx
+; CHECK-LIBCALL-NEXT:    andl $1, %ecx
+; CHECK-LIBCALL-NEXT:    orq %rax, %rcx
+; CHECK-LIBCALL-NEXT:    cvtsi2ss %rcx, %xmm0
 ; CHECK-LIBCALL-NEXT:    addss %xmm0, %xmm0
-; CHECK-LIBCALL-NEXT:  .LBB10_3:
+; CHECK-LIBCALL-NEXT:    cvtsi2ss %rdi, %xmm1
+; CHECK-LIBCALL-NEXT:    testq %rdi, %rdi
+; CHECK-LIBCALL-NEXT:    js .LBB10_2
+; CHECK-LIBCALL-NEXT:  # %bb.1:
+; CHECK-LIBCALL-NEXT:    movaps %xmm1, %xmm0
+; CHECK-LIBCALL-NEXT:  .LBB10_2:
 ; CHECK-LIBCALL-NEXT:    callq __truncsfhf2@PLT
 ; CHECK-LIBCALL-NEXT:    pextrw $0, %xmm0, %eax
 ; CHECK-LIBCALL-NEXT:    movw %ax, (%rbx)
 ; CHECK-LIBCALL-NEXT:    popq %rbx
 ; CHECK-LIBCALL-NEXT:    retq
 ;
 ; BWON-F16C-LABEL: test_uitofp_i64:
 ; BWON-F16C:       # %bb.0:

llvm/test/CodeGen/X86/pr29112.ll

 ; CHECK-NEXT:    vunpcklps {{.*#+}} xmm5 = xmm11[0],xmm5[0],xmm11[1],xmm5[1]
 ; CHECK-NEXT:    vshufps {{.*#+}} xmm5 = xmm5[0,1],xmm2[1,3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm5 = xmm5[0,1,2],xmm3[1]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm11 = xmm7[0,1],xmm2[1],xmm7[3]
 ; CHECK-NEXT:    vblendps {{.*#+}} xmm7 = xmm11[0,1,2],xmm3[3]
 ; CHECK-NEXT:    vblendps {{.*#+}} xmm10 = xmm10[0,1,2],xmm3[3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm8 = xmm8[0,1,2],xmm3[1]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm11 = xmm11[0,1,2],xmm3[1]
-; CHECK-NEXT:    vaddps %xmm8, %xmm11, %xmm8
 ; CHECK-NEXT:    vshufps {{.*#+}} xmm2 = xmm9[0,1],xmm2[3,3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm3[2]
 ; CHECK-NEXT:    vaddps %xmm0, %xmm2, %xmm2
 ; CHECK-NEXT:    vmovaps %xmm1, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    vaddps %xmm1, %xmm12, %xmm9
-; CHECK-NEXT:    vaddps %xmm1, %xmm1, %xmm3
+; CHECK-NEXT:    vaddps %xmm8, %xmm11, %xmm3
+; CHECK-NEXT:    vaddps %xmm1, %xmm1, %xmm8
 ; CHECK-NEXT:    vaddps %xmm0, %xmm10, %xmm0
-; CHECK-NEXT:    vaddps %xmm0, %xmm8, %xmm0
+; CHECK-NEXT:    vaddps %xmm3, %xmm0, %xmm0
 ; CHECK-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; CHECK-NEXT:    vmovaps %xmm3, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    vmovaps %xmm8, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    vmovaps %xmm9, (%rsp)
 ; CHECK-NEXT:    vmovaps %xmm13, %xmm3
 ; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    callq foo@PLT
 ; CHECK-NEXT:    vmovaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm1 # 16-byte Reload
 ; CHECK-NEXT:    vaddps {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm1 # 16-byte Folded Reload
 ; CHECK-NEXT:    vaddps %xmm0, %xmm1, %xmm0
 ; CHECK-NEXT:    addq $72, %rsp

llvm/test/CodeGen/X86/pr59305.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-pc-linux < %s | FileCheck %s
 
 define double @foo(double %0) #0 {
 ; CHECK-LABEL: foo:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    pushq %rax
+; CHECK-NEXT:    subq $24, %rsp
 ; CHECK-NEXT:    movsd %xmm0, (%rsp) # 8-byte Spill
 ; CHECK-NEXT:    movl $1024, %edi # imm = 0x400
 ; CHECK-NEXT:    callq fesetround@PLT
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; CHECK-NEXT:    divsd (%rsp), %xmm0 # 8-byte Folded Reload
-; CHECK-NEXT:    movsd %xmm0, (%rsp) # 8-byte Spill
+; CHECK-NEXT:    movsd {{.*#+}} xmm1 = mem[0],zero
+; CHECK-NEXT:    divsd (%rsp), %xmm1 # 8-byte Folded Reload
+; CHECK-NEXT:    movsd %xmm1, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    movl $1024, %edi # imm = 0x400
 ; CHECK-NEXT:    callq fesetround@PLT
+; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; CHECK-NEXT:    divsd (%rsp), %xmm0 # 8-byte Folded Reload
+; CHECK-NEXT:    movsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    movl $1024, %edi # imm = 0x400
 ; CHECK-NEXT:    callq fesetround@PLT
-; CHECK-NEXT:    movsd (%rsp), %xmm0 # 8-byte Reload
+; CHECK-NEXT:    movsd {{.*#+}} xmm2 = mem[0],zero
+; CHECK-NEXT:    divsd (%rsp), %xmm2 # 8-byte Folded Reload
+; CHECK-NEXT:    movsd {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 8-byte Reload
 ; CHECK-NEXT:    # xmm0 = mem[0],zero
-; CHECK-NEXT:    movaps %xmm0, %xmm1
-; CHECK-NEXT:    movaps %xmm0, %xmm2
+; CHECK-NEXT:    movsd {{[-0-9]+}}(%r{{[sb]}}p), %xmm1 # 8-byte Reload
+; CHECK-NEXT:    # xmm1 = mem[0],zero
 ; CHECK-NEXT:    callq fma@PLT
-; CHECK-NEXT:    popq %rax
+; CHECK-NEXT:    addq $24, %rsp
 ; CHECK-NEXT:    retq
     %2 = call i32 @fesetround(i32 noundef 1024)
     %3 = call double @llvm.experimental.constrained.fdiv.f64(double 1.000000e+00, double %0, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
     %4 = call i32 @fesetround(i32 noundef 1024)
     %5 = call double @llvm.experimental.constrained.fdiv.f64(double 1.000000e+00, double %0, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
     %6 = call i32 @fesetround(i32 noundef 1024)
     %7 = call double @llvm.experimental.constrained.fdiv.f64(double 1.000000e+00, double %0, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
     %8 = call double @llvm.experimental.constrained.fma.f64(double %3, double %5, double %7, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
     ret double %8
 }
 
 declare i32 @fesetround(i32) #0
 declare double @llvm.experimental.constrained.fdiv.f64(double, double, metadata, metadata) #0
 declare double @llvm.experimental.constrained.fma.f64(double, double, double, metadata, metadata) #0
 
 attributes #0 = { nounwind strictfp }

llvm/test/CodeGen/X86/sqrt-partial.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=SSE
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx | FileCheck %s --check-prefix=AVX
 
 ; PR31455 - https://bugs.llvm.org/show_bug.cgi?id=31455
 ; We have to assume that errno can be set, so we have to make a libcall in that case.
 ; But it's better for perf to check that the argument is valid rather than the result of
 ; sqrtss/sqrtsd.
 ; Note: This is really a test of the -partially-inline-libcalls IR pass (and we have an IR test
 ; for that), but we're checking the final asm to make sure that comes out as expected too.
 
 define float @f(float %val) nounwind {
 ; SSE-LABEL: f:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    xorps %xmm1, %xmm1
-; SSE-NEXT:    ucomiss %xmm1, %xmm0
+; SSE-NEXT:    sqrtss %xmm0, %xmm1
+; SSE-NEXT:    xorps %xmm2, %xmm2
+; SSE-NEXT:    ucomiss %xmm2, %xmm0
 ; SSE-NEXT:    jb .LBB0_2
 ; SSE-NEXT:  # %bb.1: # %.split
-; SSE-NEXT:    sqrtss %xmm0, %xmm0
+; SSE-NEXT:    movaps %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ; SSE-NEXT:  .LBB0_2: # %call.sqrt
 ; SSE-NEXT:    jmp sqrtf # TAILCALL
 ;
 ; AVX-LABEL: f:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vxorps %xmm1, %xmm1, %xmm1
-; AVX-NEXT:    vucomiss %xmm1, %xmm0
+; AVX-NEXT:    vsqrtss %xmm0, %xmm0, %xmm1
+; AVX-NEXT:    vxorps %xmm2, %xmm2, %xmm2
+; AVX-NEXT:    vucomiss %xmm2, %xmm0
 ; AVX-NEXT:    jb .LBB0_2
 ; AVX-NEXT:  # %bb.1: # %.split
-; AVX-NEXT:    vsqrtss %xmm0, %xmm0, %xmm0
+; AVX-NEXT:    vmovaps %xmm1, %xmm0
 ; AVX-NEXT:    retq
 ; AVX-NEXT:  .LBB0_2: # %call.sqrt
 ; AVX-NEXT:    jmp sqrtf # TAILCALL
   %res = tail call float @sqrtf(float %val)
   ret float %res
 }
 
 define double @d(double %val) nounwind {
 ; SSE-LABEL: d:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    xorpd %xmm1, %xmm1
-; SSE-NEXT:    ucomisd %xmm1, %xmm0
+; SSE-NEXT:    sqrtsd %xmm0, %xmm1
+; SSE-NEXT:    xorpd %xmm2, %xmm2
+; SSE-NEXT:    ucomisd %xmm2, %xmm0
 ; SSE-NEXT:    jb .LBB1_2
 ; SSE-NEXT:  # %bb.1: # %.split
-; SSE-NEXT:    sqrtsd %xmm0, %xmm0
+; SSE-NEXT:    movapd %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ; SSE-NEXT:  .LBB1_2: # %call.sqrt
 ; SSE-NEXT:    jmp sqrt # TAILCALL
 ;
 ; AVX-LABEL: d:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vxorpd %xmm1, %xmm1, %xmm1
-; AVX-NEXT:    vucomisd %xmm1, %xmm0
+; AVX-NEXT:    vsqrtsd %xmm0, %xmm0, %xmm1
+; AVX-NEXT:    vxorpd %xmm2, %xmm2, %xmm2
+; AVX-NEXT:    vucomisd %xmm2, %xmm0
 ; AVX-NEXT:    jb .LBB1_2
 ; AVX-NEXT:  # %bb.1: # %.split
-; AVX-NEXT:    vsqrtsd %xmm0, %xmm0, %xmm0
+; AVX-NEXT:    vmovapd %xmm1, %xmm0
 ; AVX-NEXT:    retq
 ; AVX-NEXT:  .LBB1_2: # %call.sqrt
 ; AVX-NEXT:    jmp sqrt # TAILCALL
   %res = tail call double @sqrt(double %val)
   ret double %res
 }
 
 define double @minsize(double %x, double %y) minsize {

llvm/test/CodeGen/X86/sse-intel-ocl.ll

 ; WIN32-LABEL: testf16_inp:
 ; WIN32:       # %bb.0:
 ; WIN32-NEXT:    pushl %ebp
 ; WIN32-NEXT:    movl %esp, %ebp
 ; WIN32-NEXT:    andl $-16, %esp
 ; WIN32-NEXT:    subl $80, %esp
 ; WIN32-NEXT:    movups 72(%ebp), %xmm4
 ; WIN32-NEXT:    movups 8(%ebp), %xmm3
-; WIN32-NEXT:    addps %xmm4, %xmm3
-; WIN32-NEXT:    movups 56(%ebp), %xmm4
-; WIN32-NEXT:    movups 40(%ebp), %xmm5
-; WIN32-NEXT:    movups 24(%ebp), %xmm6
+; WIN32-NEXT:    movups 56(%ebp), %xmm5
+; WIN32-NEXT:    movups 40(%ebp), %xmm6
+; WIN32-NEXT:    movups 24(%ebp), %xmm7
 ; WIN32-NEXT:    movl %esp, %eax
-; WIN32-NEXT:    addps %xmm6, %xmm0
-; WIN32-NEXT:    addps %xmm5, %xmm1
-; WIN32-NEXT:    addps %xmm4, %xmm2
+; WIN32-NEXT:    addps %xmm7, %xmm0
+; WIN32-NEXT:    addps %xmm6, %xmm1
+; WIN32-NEXT:    addps %xmm5, %xmm2
+; WIN32-NEXT:    addps %xmm4, %xmm3
 ; WIN32-NEXT:    pushl %eax
 ; WIN32-NEXT:    calll _func_float16_ptr
 ; WIN32-NEXT:    addl $4, %esp
 ; WIN32-NEXT:    addps (%esp), %xmm0
 ; WIN32-NEXT:    addps {{[0-9]+}}(%esp), %xmm1
 ; WIN32-NEXT:    addps {{[0-9]+}}(%esp), %xmm2
 ; WIN32-NEXT:    addps {{[0-9]+}}(%esp), %xmm3
 ; WIN32-NEXT:    movl %ebp, %esp

llvm/test/CodeGen/X86/swifterror.ll

 ; CHECK-i386-NEXT:    pushl %esi
 ; CHECK-i386-NEXT:    .cfi_def_cfa_offset 12
 ; CHECK-i386-NEXT:    subl $20, %esp
 ; CHECK-i386-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-i386-NEXT:    .cfi_offset %esi, -12
 ; CHECK-i386-NEXT:    .cfi_offset %edi, -8
 ; CHECK-i386-NEXT:    movl 32(%esp), %esi
 ; CHECK-i386-NEXT:    leal 16(%esp), %edi
-; CHECK-i386-NEXT:    fld1
-; CHECK-i386-NEXT:    fstps {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Spill
 ; CHECK-i386-NEXT:  LBB2_1: ## %bb_loop
 ; CHECK-i386-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-i386-NEXT:    movl $0, 16(%esp)
 ; CHECK-i386-NEXT:    movl %edi, (%esp)
 ; CHECK-i386-NEXT:    calll _foo
 ; CHECK-i386-NEXT:    movl 16(%esp), %ecx
 ; CHECK-i386-NEXT:    testl %ecx, %ecx
 ; CHECK-i386-NEXT:    jne LBB2_4
 ; CHECK-i386-NEXT:  ## %bb.2: ## %cont
 ; CHECK-i386-NEXT:    ## in Loop: Header=BB2_1 Depth=1
-; CHECK-i386-NEXT:    flds {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Reload
+; CHECK-i386-NEXT:    fld1
 ; CHECK-i386-NEXT:    fxch %st(1)
 ; CHECK-i386-NEXT:    fucompp
 ; CHECK-i386-NEXT:    fnstsw %ax
 ; CHECK-i386-NEXT:    ## kill: def $ah killed $ah killed $ax
 ; CHECK-i386-NEXT:    sahf
 ; CHECK-i386-NEXT:    jbe LBB2_1
 ; CHECK-i386-NEXT:  ## %bb.3: ## %bb_end
 ; CHECK-i386-NEXT:    movzbl 8(%ecx), %eax
 ; CHECK-i386-NEXT:    movb %al, (%esi)
 ; CHECK-i386-NEXT:    fldz
 ; CHECK-i386-NEXT:  LBB2_4: ## %handler
 ; CHECK-i386-NEXT:    fstp %st(0)
 ; CHECK-i386-NEXT:    movl %ecx, (%esp)
 ; CHECK-i386-NEXT:    calll _free
-; CHECK-i386-NEXT:    flds {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Reload
+; CHECK-i386-NEXT:    fld1
 ; CHECK-i386-NEXT:    addl $20, %esp
 ; CHECK-i386-NEXT:    popl %esi
 ; CHECK-i386-NEXT:    popl %edi
 ; CHECK-i386-NEXT:    retl
 ; Access part of the error object and save it to error_ref
 
 entry:
   %error_ptr_ref = alloca swifterror ptr

 ; CHECK-i386-NEXT:    subl $20, %esp
 ; CHECK-i386-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-i386-NEXT:    .cfi_offset %esi, -12
 ; CHECK-i386-NEXT:    .cfi_offset %edi, -8
 ; CHECK-i386-NEXT:    flds 40(%esp)
 ; CHECK-i386-NEXT:    fstps {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Spill
 ; CHECK-i386-NEXT:    movl 36(%esp), %esi
 ; CHECK-i386-NEXT:    movl 32(%esp), %edi
-; CHECK-i386-NEXT:    fld1
-; CHECK-i386-NEXT:    fstps {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Spill
 ; CHECK-i386-NEXT:  LBB4_1: ## %bb_loop
 ; CHECK-i386-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-i386-NEXT:    testl %esi, %esi
 ; CHECK-i386-NEXT:    je LBB4_3
 ; CHECK-i386-NEXT:  ## %bb.2: ## %gen_error
 ; CHECK-i386-NEXT:    ## in Loop: Header=BB4_1 Depth=1
 ; CHECK-i386-NEXT:    movl $0, 4(%esp)
 ; CHECK-i386-NEXT:    movl $16, (%esp)
 ; CHECK-i386-NEXT:    calll _malloc
 ; CHECK-i386-NEXT:    movl %eax, (%edi)
 ; CHECK-i386-NEXT:    movb $1, 8(%eax)
 ; CHECK-i386-NEXT:  LBB4_3: ## %bb_cont
 ; CHECK-i386-NEXT:    ## in Loop: Header=BB4_1 Depth=1
+; CHECK-i386-NEXT:    fld1
 ; CHECK-i386-NEXT:    flds {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Reload
-; CHECK-i386-NEXT:    flds {{[-0-9]+}}(%e{{[sb]}}p) ## 4-byte Folded Reload
-; CHECK-i386-NEXT:    fxch %st(1)
 ; CHECK-i386-NEXT:    fucompp
 ; CHECK-i386-NEXT:    fnstsw %ax
 ; CHECK-i386-NEXT:    ## kill: def $ah killed $ah killed $ax
 ; CHECK-i386-NEXT:    sahf
 ; CHECK-i386-NEXT:    jbe LBB4_1
 ; CHECK-i386-NEXT:  ## %bb.4: ## %bb_end
 ; CHECK-i386-NEXT:    fldz
 ; CHECK-i386-NEXT:    addl $20, %esp

llvm/tools/llvm-reduce/ReducerWorkItem.cpp

for (auto &SrcMI : SrcMBB) {
         else if (DstMO.isRegMask()) {
           DstMRI->addPhysRegsUsedFromRegMask(DstMO.getRegMask());
 
           if (!ConstRegisterMasks.count(DstMO.getRegMask())) {
             uint32_t *DstMask = DstMF->allocateRegMask();
             std::memcpy(DstMask, SrcMO.getRegMask(),
                         sizeof(*DstMask) *
                             MachineOperand::getRegMaskSize(TRI->getNumRegs()));
-            DstMO.setRegMask(DstMask);
+            DstMO.setRegMask(DstMask, DstMI);
           }
         }
 
         DstMI->addOperand(DstMO);
       }
 
       cloneMemOperands(*DstMI, SrcMI, *SrcMF, *DstMF);
     }

llvm/unittests/CodeGen/MachineOperandTest.cpp

 }
 
 TEST(MachineOperandTest, PrintRegisterMask) {
   LLVMContext Ctx;
   Module Mod("Module", Ctx);
   auto MF = createMachineFunction(Ctx, Mod);
 
   uint32_t *Dummy = MF->allocateRegMask();
-  MachineOperand MO = MachineOperand::CreateRegMask(Dummy);
+  MachineOperand MO = MachineOperand::CreateRegMask(Dummy, MF.get());
 
   // Checking some preconditions on the newly created
   // MachineOperand.
   ASSERT_TRUE(MO.isRegMask());
   ASSERT_TRUE(MO.getRegMask() == Dummy);
 
   // Print a MachineOperand containing a RegMask. Here we check that without a
   // TRI and IntrinsicInfo we still print a less detailed regmask.