[MachineCopyPropagation] Extend MCP to do trivial copy backward propagation

[lkail]

Summary

This patch mainly do such transformation

$R0 = OP ...
... // No read/clobber of $R0 and $R1
$R1 = COPY $R0 // $R0 is killed

Replace $R0 with $R1 and remove the COPY, we have

$R1 = OP ...

This transformation can also expose more opportunities for existing copy elimination in MCP.

[lkail]

Gentle ping

[lkail]

ping

[lkail]

ping

[lkail]

ping

[qcolombet]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

348

Hard coding 0, seems artificial to me.
Could we find which operand defines Copy.getOperand(1).getReg() instead?

352

I find this condition hard to read.
Could you break it into smaller conditions?

E.g.,

if (!SrcOp.isRenamable())
  return false;
if (SrcOp.isImplicit())
  return false;

and so on.

385

Could you build this information as we iterate into the basic block?
I am afraid the compile time cost of this loop to be fairly large otherwise, since we are going to traverse all the instructions between two instructions again and again.

[lkail]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

385

I find that it might be non-trivial to track information that help perform backward copy propagation in current forward iteration. Would it be a proper option that separating current main loop into two parts, one performs forward copy propagation, the other performs backward copy propagation? i.e.

for (MachineBasicBlock &MBB : MF) {
  ForwardCopyPropagateBlock(MBB); // Current CopyPropagateBlock
  BackwardCopyPropagateBlock(MBB);
}

In BackwardCopyPropagateBlock, iterate instructions reversely. Cons is increasing constant complexity.

[qcolombet]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

385

That would work.
The nice thing about this approach is that we could easily stage the new optimization:

for (MachineBasicBlock &MBB : MF) {
  ForwardCopyPropagateBlock(MBB); // Current CopyPropagateBlock
  if (EnableBackwardCP)
    BackwardCopyPropagateBlock(MBB);
}

[lkail]

Address @qcolombet's comment. @qcolombet I have no idea which is preferable to pass EnableBackwardCP option, target hook or cl::opt?

[lenary]

The RISC-V test changes look good. (I do not intend to review the optimisation implementation or other backend test changes).

[lkail]

Introduce CopyTracker::invalidateRegister, original markRegUnavailable and clobberRegister are not appropriate for backward copy propagation.

[lkail]

Add test case involving RegMask.

[lkail]

Rebased and get broken tests, I'll investigate it and change this patch.

[lkail]

Rebased and fixed broken tests. Ready to be reviewed.

[lkail]

In D67794#1747301, @lenary wrote:

The RISC-V test changes look good. (I do not intend to review the optimisation implementation or other backend test changes).

Thanks @lenary .

[qcolombet]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

215

It seems to me that we could just reuse directly findAvailCopy and thus eliminate findCopyDefViaUnit.
The only difference would be the that we use a different iterator for the. forward and the backward case.

[qcolombet]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

215

Spoke too soon. I see the difference in findCopyDefViaUnit:
We check that the source of the copy is used only once, then that the reg unit defined by this copy is available.

I still need to go through the rest of that patch, but I would expect that we could do more code reuse if we were to drop from the list of available copies anything that is read or clobbered between the use and the definition. I.e., the case were more than one copy use the same source (which is basically what DefRegs.size() != 1 represents) wouldn't be kept in the Copies variable to begin with.

Anyway, not something that must be fixed here. We can refactor later.
Let me go through the whole patch first.

[lkail]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

215

I agree with that there are still some redundancies in current code. I think findAvailBackwardCopy can be combined into findAvailCopy with an additional flag to indicate if its forward or backward and reverseIterator in findAvailBackwardCopy can be replaced with Iterator.

[lkail]

Rebased and add comment for invalidateRegister.

[qcolombet]

LGTM.
One nit below.

llvm/lib/CodeGen/MachineCopyPropagation.cpp

772

Should this be an assert?

Put differently, when can we find a backward copy for MODef.getReg() when the source is not MODef.getReg()?

[lkail]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

772

Yes, you are right. I supposed following code might not be handled correctly(to illustrate, might not be perfect)

name: foo
alignment:       2
tracksRegLiveness: true
frameInfo:
  maxAlignment:    16
  maxCallFrameSize: 0
stack:
  - { id: 0, type: spill-slot, size: 16, alignment: 16 }
body: |
  bb.0.entry:
    liveins: $r0, $r1
    renamable $q0 = MVE_VLDRWU32 renamable $r0, 80, 0, $noreg :: (load 16 from %stack.0, align 4)
    renamable $s4 = COPY renamable killed $s0
    MVE_VSTRWU32 killed renamable $q1, killed renamable $r1, 0, 0, $noreg :: (store 16 into %stack.0, align 128)
    tBX_RET 14, $noreg

I thought $s0 might be propagated(thought it won't pass isBackwardPropagatableRegClassCopy later). However, in findAvailBackwardCopy we have already checked !TRI.isSubRegisterEq(AvailCopy->getOperand(1).getReg(), Reg)), thus $s0 won't be propagated.

[lkail]

This comment was removed by lkail.

llvm/lib/CodeGen/MachineCopyPropagation.cpp

 //    %R0 = COPY %R1 <<< Removed
 //
 // or
 //
 //    %R1 = COPY %R0
 //    ... // No clobber of %R0
 //    %R1 = COPY %R0 <<< Removed
 //
+// or
+//
+//    $R0 = OP ...
+//    ... // No read/clobber of $R0 and $R1
+//    $R1 = COPY $R0 // $R0 is killed
+// Replace $R0 with $R1 and remove the COPY
+//    $R1 = OP ...
+//    ...
+//
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/iterator_range.h"
for (unsigned Reg : Regs) {
       for (MCRegUnitIterator RUI(Reg, &TRI); RUI.isValid(); ++RUI) {
         auto CI = Copies.find(*RUI);
         if (CI != Copies.end())
           CI->second.Avail = false;
       }
     }
   }
 
+  /// Remove register from copy maps.
+  void invalidateRegister(unsigned Reg, const TargetRegisterInfo &TRI) {
+    // Since Reg might be a subreg of some registers, only invalidate Reg is not
+    // enough. We have to find the COPY defines Reg or registers defined by Reg
+    // and invalidate all of them.
+    DenseSet<unsigned> RegsToInvalidate{Reg};
+    for (MCRegUnitIterator RUI(Reg, &TRI); RUI.isValid(); ++RUI) {
+      auto I = Copies.find(*RUI);
+      if (I != Copies.end()) {
+        if (MachineInstr *MI = I->second.MI) {
+          RegsToInvalidate.insert(MI->getOperand(0).getReg());
+          RegsToInvalidate.insert(MI->getOperand(1).getReg());
+        }
+        RegsToInvalidate.insert(I->second.DefRegs.begin(),
+                                I->second.DefRegs.end());
+      }
+    }
+    for (unsigned InvalidReg : RegsToInvalidate)
+      for (MCRegUnitIterator RUI(InvalidReg, &TRI); RUI.isValid(); ++RUI)
+        Copies.erase(*RUI);
+  }
+
   /// Clobber a single register, removing it from the tracker's copy maps.
   void clobberRegister(unsigned Reg, const TargetRegisterInfo &TRI) {
     for (MCRegUnitIterator RUI(Reg, &TRI); RUI.isValid(); ++RUI) {
       auto I = Copies.find(*RUI);
       if (I != Copies.end()) {
         // When we clobber the source of a copy, we need to clobber everything
         // it defined.
         markRegsUnavailable(I->second.DefRegs, TRI);
MachineInstr *findCopyForUnit(unsigned RegUnit, const TargetRegisterInfo &TRI,
     auto CI = Copies.find(RegUnit);
     if (CI == Copies.end())
       return nullptr;
     if (MustBeAvailable && !CI->second.Avail)
       return nullptr;
     return CI->second.MI;
   }
 
+  MachineInstr *findCopyDefViaUnit(unsigned RegUnit,
+                                    const TargetRegisterInfo &TRI) {
+    auto CI = Copies.find(RegUnit);
+    if (CI == Copies.end())
+      return nullptr;
+    if (CI->second.DefRegs.size() != 1)
+      return nullptr;
+    MCRegUnitIterator RUI(CI->second.DefRegs[0], &TRI);
+    return findCopyForUnit(*RUI, TRI, true);
+  }
+
+  MachineInstr *findAvailBackwardCopy(MachineInstr &I, unsigned Reg,
+                                      const TargetRegisterInfo &TRI) {
+    MCRegUnitIterator RUI(Reg, &TRI);
+    MachineInstr *AvailCopy = findCopyDefViaUnit(*RUI, TRI);
+    if (!AvailCopy ||
+        !TRI.isSubRegisterEq(AvailCopy->getOperand(1).getReg(), Reg))
+      return nullptr;
+
+    Register AvailSrc = AvailCopy->getOperand(1).getReg();
+    Register AvailDef = AvailCopy->getOperand(0).getReg();
+    for (const MachineInstr &MI :
+         make_range(AvailCopy->getReverseIterator(), I.getReverseIterator()))
+      for (const MachineOperand &MO : MI.operands())
+        if (MO.isRegMask())
+          // FIXME: Shall we simultaneously invalidate AvailSrc or AvailDef?
+          if (MO.clobbersPhysReg(AvailSrc) || MO.clobbersPhysReg(AvailDef))
+            return nullptr;
+
+    return AvailCopy;
+  }
+
   MachineInstr *findAvailCopy(MachineInstr &DestCopy, unsigned Reg,
                               const TargetRegisterInfo &TRI) {
     // We check the first RegUnit here, since we'll only be interested in the
     // copy if it copies the entire register anyway.
     MCRegUnitIterator RUI(Reg, &TRI);
     MachineInstr *AvailCopy =
         findCopyForUnit(*RUI, TRI, /*MustBeAvailable=*/true);
     if (!AvailCopy ||
public:
   }
 
 private:
   typedef enum { DebugUse = false, RegularUse = true } DebugType;
 
   void ClobberRegister(unsigned Reg);
   void ReadRegister(unsigned Reg, MachineInstr &Reader,
                     DebugType DT);
-  void CopyPropagateBlock(MachineBasicBlock &MBB);
+  void ForwardCopyPropagateBlock(MachineBasicBlock &MBB);
+  void BackwardCopyPropagateBlock(MachineBasicBlock &MBB);
   bool eraseIfRedundant(MachineInstr &Copy, unsigned Src, unsigned Def);
   void forwardUses(MachineInstr &MI);
+  void propagateDefs(MachineInstr &MI);
   bool isForwardableRegClassCopy(const MachineInstr &Copy,
                                  const MachineInstr &UseI, unsigned UseIdx);
+  bool isBackwardPropagatableRegClassCopy(const MachineInstr &Copy,
+                                          const MachineInstr &UseI,
+                                          unsigned UseIdx);
   bool hasImplicitOverlap(const MachineInstr &MI, const MachineOperand &Use);
 
   /// Candidates for deletion.
   SmallSetVector<MachineInstr *, 8> MaybeDeadCopies;
 
   /// Multimap tracking debug users in current BB
   DenseMap<MachineInstr*, SmallVector<MachineInstr*, 2>> CopyDbgUsers;
 
static bool isNopCopy(const MachineInstr &PreviousCopy, unsigned Src,
   if (!TRI->isSubRegister(PreviousSrc, Src))
     return false;
   unsigned SubIdx = TRI->getSubRegIndex(PreviousSrc, Src);
   return SubIdx == TRI->getSubRegIndex(PreviousDef, Def);
 }
 
 /// Remove instruction \p Copy if there exists a previous copy that copies the
 /// register \p Src to the register \p Def; This may happen indirectly by
 /// copying the super registers.
 bool MachineCopyPropagation::eraseIfRedundant(MachineInstr &Copy, unsigned Src,
                                               unsigned Def) {
   // Avoid eliminating a copy from/to a reserved registers as we cannot predict
   // the value (Example: The sparc zero register is writable but stays zero).
   if (MRI->isReserved(Src) || MRI->isReserved(Def))
     return false;
 
   // Search for an existing copy.
   MachineInstr *PrevCopy = Tracker.findAvailCopy(Copy, Def, *TRI);
   if (!PrevCopy)
     return false;
 
for (MachineInstr &MI :
     MI.clearRegisterKills(CopyDef, TRI);
 
   Copy.eraseFromParent();
   Changed = true;
   ++NumDeletes;
   return true;
 }
 
+bool MachineCopyPropagation::isBackwardPropagatableRegClassCopy(
+    const MachineInstr &Copy, const MachineInstr &UseI, unsigned UseIdx) {
+  Register Def = Copy.getOperand(0).getReg();
+
+  if (const TargetRegisterClass *URC =
+          UseI.getRegClassConstraint(UseIdx, TII, TRI))
+    return URC->contains(Def);
+
+  // We don't process further if UseI is a COPY, since forward copy propagation
+  // should handle that.
+  return false;
+}
+
 /// Decide whether we should forward the source of \param Copy to its use in
 /// \param UseI based on the physical register class constraints of the opcode
 /// and avoiding introducing more cross-class COPYs.
 bool MachineCopyPropagation::isForwardableRegClassCopy(const MachineInstr &Copy,
                                                        const MachineInstr &UseI,
                                                        unsigned UseIdx) {
 
   Register CopySrcReg = Copy.getOperand(1).getReg();
for (MachineInstr &KMI :
          make_range(Copy->getIterator(), std::next(MI.getIterator())))
       KMI.clearRegisterKills(CopySrcReg, TRI);
 
     ++NumCopyForwards;
     Changed = true;
   }
 }
 
-void MachineCopyPropagation::CopyPropagateBlock(MachineBasicBlock &MBB) {
-  LLVM_DEBUG(dbgs() << "MCP: CopyPropagateBlock " << MBB.getName() << "\n");
+void MachineCopyPropagation::ForwardCopyPropagateBlock(MachineBasicBlock &MBB) {
+  LLVM_DEBUG(dbgs() << "MCP: ForwardCopyPropagateBlock " << MBB.getName()
+                    << "\n");
 
   for (MachineBasicBlock::iterator I = MBB.begin(), E = MBB.end(); I != E; ) {
     MachineInstr *MI = &*I;
     ++I;
 
     // Analyze copies (which don't overlap themselves).
     if (MI->isCopy() && !TRI->regsOverlap(MI->getOperand(0).getReg(),
                                           MI->getOperand(1).getReg())) {
if (MBB.succ_empty()) {
     }
   }
 
   MaybeDeadCopies.clear();
   CopyDbgUsers.clear();
   Tracker.clear();
 }
 
+static bool isBackwardPropagatableCopy(MachineInstr &MI,
+                                       const MachineRegisterInfo &MRI) {
+  assert(MI.isCopy() && "MI is expected to be a COPY");
+  Register Def = MI.getOperand(0).getReg();
+  Register Src = MI.getOperand(1).getReg();
+
+  if (MRI.isReserved(Def) || MRI.isReserved(Src))
+    return false;
+
+  return MI.getOperand(1).isRenamable() && MI.getOperand(1).isKill();
+}
+
+void MachineCopyPropagation::propagateDefs(MachineInstr &MI) {
+  if (!Tracker.hasAnyCopies())
+    return;
+
+  for (unsigned OpIdx = 0, OpEnd = MI.getNumOperands(); OpIdx != OpEnd;
+       ++OpIdx) {
+    MachineOperand &MODef = MI.getOperand(OpIdx);
+
+    if (!MODef.isReg() || MODef.isUse())
+      continue;
+
+    // Ignore non-trivial cases.
+    if (MODef.isTied() || MODef.isUndef() || MODef.isImplicit())
+      continue;
+
+    // We only handle if the register comes from a vreg.
+    if (!MODef.isRenamable())
+      continue;
+
+    MachineInstr *Copy =
+        Tracker.findAvailBackwardCopy(MI, MODef.getReg(), *TRI);
+    if (!Copy)
+      continue;
+
+    Register Def = Copy->getOperand(0).getReg();
+    Register Src = Copy->getOperand(1).getReg();
+
+    if (MODef.getReg() != Src)
+      continue;
+
+    if (!isBackwardPropagatableRegClassCopy(*Copy, MI, OpIdx))
+      continue;
+
+    if (hasImplicitOverlap(MI, MODef))
+      continue;
+
+    LLVM_DEBUG(dbgs() << "MCP: Replacing " << printReg(MODef.getReg(), TRI)
+                      << "\n     with " << printReg(Def, TRI) << "\n     in "
+                      << MI << "     from " << *Copy);
+
+    MODef.setReg(Def);
+    MODef.setIsRenamable(Copy->getOperand(0).isRenamable());
+
+    LLVM_DEBUG(dbgs() << "MCP: After replacement: " << MI << "\n");
+    MaybeDeadCopies.insert(Copy);
+    Changed = true;
+  }
+}
+
+void MachineCopyPropagation::BackwardCopyPropagateBlock(
+    MachineBasicBlock &MBB) {
+  LLVM_DEBUG(dbgs() << "MCP: BackwardCopyPropagateBlock " << MBB.getName()
+                    << "\n");
+
+  for (MachineBasicBlock::reverse_iterator I = MBB.rbegin(), E = MBB.rend();
+       I != E;) {
+    MachineInstr *MI = &*I;
+    ++I;
+
+    // Ignore non-trivial COPYs.
+    if (MI->isCopy() && MI->getNumOperands() == 2 &&
+        !TRI->regsOverlap(MI->getOperand(0).getReg(),
+                          MI->getOperand(1).getReg())) {
+
+      Register Def = MI->getOperand(0).getReg();
+      Register Src = MI->getOperand(1).getReg();
+
+      // Unlike forward cp, we don't invoke propagateDefs here,
+      // just let forward cp do COPY-to-COPY propagation.
+      if (isBackwardPropagatableCopy(*MI, *MRI)) {
+        Tracker.invalidateRegister(Src, *TRI);
+        Tracker.invalidateRegister(Def, *TRI);
+        Tracker.trackCopy(MI, *TRI);
+        continue;
+      }
+    }
+
+    // Invalidate any earlyclobber regs first.
+    for (const MachineOperand &MO : MI->operands())
+      if (MO.isReg() && MO.isEarlyClobber()) {
+        Register Reg = MO.getReg();
+        if (!Reg)
+          continue;
+        Tracker.invalidateRegister(Reg, *TRI);
+      }
+
+    propagateDefs(*MI);
+    for (const MachineOperand &MO : MI->operands()) {
+      if (!MO.isReg())
+        continue;
+
+      if (!MO.getReg())
+        continue;
+
+      if (MO.isDef())
+        Tracker.invalidateRegister(MO.getReg(), *TRI);
+
+      if (MO.readsReg())
+        Tracker.invalidateRegister(MO.getReg(), *TRI);
+    }
+  }
+
+  for (auto *Copy : MaybeDeadCopies)
+    Copy->eraseFromParent();
+
+  MaybeDeadCopies.clear();
+  CopyDbgUsers.clear();
+  Tracker.clear();
+}
+
 bool MachineCopyPropagation::runOnMachineFunction(MachineFunction &MF) {
   if (skipFunction(MF.getFunction()))
     return false;
 
   Changed = false;
 
   TRI = MF.getSubtarget().getRegisterInfo();
   TII = MF.getSubtarget().getInstrInfo();
   MRI = &MF.getRegInfo();
 
-  for (MachineBasicBlock &MBB : MF)
-    CopyPropagateBlock(MBB);
+  for (MachineBasicBlock &MBB : MF) {
+    BackwardCopyPropagateBlock(MBB);
+    ForwardCopyPropagateBlock(MBB);
+  }
 
   return Changed;
 }

llvm/test/CodeGen/PowerPC/machine-backward-cp.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -O3 -verify-machineinstrs -mtriple=powerpc64le-unknown-unknown \
 # RUN: -mcpu=pwr9 -simplify-mir -run-pass=machine-cp %s -o - | FileCheck %s
 
 
 # Normal case
 ---
 name: test0
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     ; CHECK-LABEL: name: test0
-    ; CHECK: renamable $x4 = LI8 1024
-    ; CHECK: $x3 = COPY killed renamable $x4
+    ; CHECK: $x3 = LI8 1024
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x4 = LI8 1024
     $x3 = COPY renamable killed $x4
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 # Not in terminal BBs
 ---
 name: test1
 alignment: 4
 tracksRegLiveness: true
 body: |
   ; CHECK-LABEL: name: test1
   ; CHECK: bb.0.entry:
-  ; CHECK:   renamable $x5 = LI8 42
-  ; CHECK:   renamable $x4 = COPY killed renamable $x5
+  ; CHECK:   renamable $x4 = LI8 42
   ; CHECK:   B %bb.1
   ; CHECK: bb.1:
   ; CHECK:   liveins: $x4
   ; CHECK:   $x3 = COPY killed renamable $x4
   ; CHECK:   BLR8 implicit $lr8, implicit undef $rm, implicit $x3
   bb.0.entry:
     successors: %bb.1
 

 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x5
 
     ; CHECK-LABEL: name: iterative_deletion
     ; CHECK: liveins: $x5
-    ; CHECK: renamable $x6 = ADDI8 killed renamable $x5, 1
-    ; CHECK: $x3 = COPY $x6
+    ; CHECK: renamable $x4 = ADDI8 killed renamable $x5, 1
+    ; CHECK: $x3 = COPY $x4
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x6 = ADDI8 renamable killed $x5, 1
     renamable $x4 = COPY renamable killed $x6
     renamable $x7 = COPY renamable killed $x4
     $x3 = COPY renamable killed $x7
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 ---
 name: Enter
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x4, $x7
     ; CHECK-LABEL: name: Enter
     ; CHECK: liveins: $x4, $x7
     ; CHECK: renamable $x5 = COPY killed renamable $x7
-    ; CHECK: renamable $x6 = ADDI8 killed renamable $x4, 1
-    ; CHECK: $x3 = ADD8 killed renamable $x5, $x6
+    ; CHECK: renamable $x7 = ADDI8 killed renamable $x4, 1
+    ; CHECK: $x3 = ADD8 killed renamable $x5, killed renamable $x7
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x5 = COPY killed renamable $x7
     renamable $x6 = ADDI8 killed renamable $x4, 1
     renamable $x7 = COPY killed renamable $x6
     $x3 = ADD8 renamable killed $x5, renamable killed $x7
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 ---
 name: foo
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x4, $x7
     ; CHECK-LABEL: name: foo
     ; CHECK: liveins: $x4, $x7
     ; CHECK: renamable $x5 = COPY killed renamable $x7
-    ; CHECK: renamable $x6 = ADDI8 renamable $x4, 1
-    ; CHECK: renamable $x7 = COPY killed renamable $x6
-    ; CHECK: renamable $x8 = ADDI8 killed $x4, 2
-    ; CHECK: $x3 = ADD8 killed renamable $x5, $x8
+    ; CHECK: renamable $x7 = ADDI8 renamable $x4, 1
+    ; CHECK: renamable $x6 = ADDI8 killed $x4, 2
+    ; CHECK: $x3 = ADD8 killed renamable $x5, killed renamable $x6
     ; CHECK: $x3 = ADD8 $x3, killed renamable $x7
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x5 = COPY killed renamable $x7
     renamable $x6 = ADDI8 renamable $x4, 1
     renamable $x7 = COPY killed renamable $x6
     renamable $x8 = ADDI8 killed $x4, 2
     renamable $x6 = COPY killed renamable $x8
     $x3 = ADD8 renamable killed $x5, renamable killed $x6
     $x3 = ADD8 $x3, renamable killed $x7
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 ---
 name: bar
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x4, $x7
     ; CHECK-LABEL: name: bar
     ; CHECK: liveins: $x4, $x7
     ; CHECK: renamable $x5 = COPY killed renamable $x7
-    ; CHECK: renamable $x6 = ADDI8 renamable $x4, 1
-    ; CHECK: renamable $x8 = COPY $x6
-    ; CHECK: renamable $x6 = ADDI8 renamable $x5, 2
-    ; CHECK: $x3 = ADD8 killed renamable $x5, $x6
+    ; CHECK: renamable $x7 = ADDI8 renamable $x4, 1
+    ; CHECK: renamable $x8 = COPY killed renamable $x7
+    ; CHECK: renamable $x7 = ADDI8 renamable $x5, 2
+    ; CHECK: $x3 = ADD8 killed renamable $x5, killed renamable $x7
     ; CHECK: $x3 = ADD8 $x3, killed renamable $x8
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x5 = COPY killed renamable $x7
     renamable $x6 = ADDI8 renamable $x4, 1
     renamable $x7 = COPY killed renamable $x6
     renamable $x8 = COPY killed renamable $x7
     renamable $x6 = ADDI8 renamable $x5, 2
     renamable $x7 = COPY killed renamable $x6
     $x3 = ADD8 renamable killed $x5, renamable killed $x7
     $x3 = ADD8 $x3, renamable killed $x8
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 ---
 name: bogus
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x7
     ; CHECK-LABEL: name: bogus
     ; CHECK: liveins: $x7
     ; CHECK: renamable $x5 = COPY renamable $x7
-    ; CHECK: renamable $x6 = ADDI8 $x7, 1
-    ; CHECK: renamable $x7 = COPY $x6
+    ; CHECK: renamable $x4 = ADDI8 $x7, 1
     ; CHECK: renamable $x6 = ADDI8 renamable $x5, 2
-    ; CHECK: $x3 = ADD8 $x7, killed renamable $x5
+    ; CHECK: $x3 = ADD8 killed renamable $x4, killed renamable $x5
     ; CHECK: $x3 = ADD8 $x3, killed renamable $x6
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x5 = COPY killed renamable $x7
     renamable $x6 = ADDI8 renamable $x5, 1
     renamable $x4 = COPY killed renamable $x6
     renamable $x7 = COPY killed renamable $x4
     renamable $x6 = ADDI8 renamable $x5, 2
     renamable $x4 = COPY killed renamable $x7
     $x3 = ADD8 renamable killed $x4, renamable killed $x5
     $x3 = ADD8 $x3, renamable killed $x6
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
 
 ---
 name: foobar
 alignment: 4
 tracksRegLiveness: true
 body: |
   bb.0.entry:
     liveins: $x7
     ; CHECK-LABEL: name: foobar
     ; CHECK: liveins: $x7
-    ; CHECK: renamable $x6 = ADDI8 $x7, 1
-    ; CHECK: renamable $x8 = COPY $x6
-    ; CHECK: renamable $x6 = ADDI8 $x7, 2
-    ; CHECK: $x3 = ADD8 $x6, $x7
+    ; CHECK: renamable $x4 = ADDI8 $x7, 1
+    ; CHECK: renamable $x8 = COPY killed renamable $x4
+    ; CHECK: renamable $x4 = ADDI8 $x7, 2
+    ; CHECK: $x3 = ADD8 killed renamable $x4, $x7
     ; CHECK: $x3 = ADD8 $x3, killed renamable $x8
     ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
     renamable $x5 = COPY killed renamable $x7
     renamable $x6 = ADDI8 renamable $x5, 1
     renamable $x4 = COPY killed renamable $x6
     renamable $x8 = COPY killed renamable $x4
     renamable $x6 = ADDI8 renamable $x5, 2
     renamable $x4 = COPY killed renamable $x6
     $x3 = ADD8 renamable killed $x4, renamable killed $x5
     $x3 = ADD8 $x3, renamable killed $x8
     BLR8 implicit $lr8, implicit undef $rm, implicit $x3
 
 ...
+
+---
+name: cross_call
+alignment: 4
+tracksRegLiveness: true
+body: |
+  bb.0.entry:
+    liveins: $x2, $x3, $x20
+    ; CHECK-LABEL: name: cross_call
+    ; CHECK: liveins: $x2, $x3, $x20
+    ; CHECK: renamable $x20 = LI8 1024
+    ; CHECK: BL8_NOP @foo, csr_svr464_altivec, implicit-def $lr8, implicit $rm, implicit $x3, implicit-def $x3, implicit $x2
+    ; CHECK: $x3 = COPY killed renamable $x20
+    ; CHECK: BLR8 implicit $lr8, implicit undef $rm, implicit $x3
+    renamable $x20 = LI8 1024
+    BL8_NOP @foo, csr_svr464_altivec, implicit-def $lr8, implicit $rm, implicit $x3, implicit-def $x3, implicit $x2
+    $x3 = COPY renamable killed $x20
+    BLR8 implicit $lr8, implicit undef $rm, implicit $x3
+...

llvm/test/CodeGen/PowerPC/redundant-copy-after-tail-dup.ll

 ; CHECK-P9-NEXT:    bc 12, 4*cr5+lt, .LBB0_4
 ; CHECK-P9-NEXT:  # %bb.1: # %land.lhs.true
 ; CHECK-P9-NEXT:    bc 12, 4*cr5+lt, .LBB0_5
 ; CHECK-P9-NEXT:  .LBB0_2: # %for.inc
 ; CHECK-P9-NEXT:    lhz r3, 5308(r3)
 ; CHECK-P9-NEXT:    cmplwi r3, 2
 ; CHECK-P9-NEXT:    bge- cr0, .LBB0_6
 ; CHECK-P9-NEXT:  # %bb.3: # %land.lhs.true.1
-; CHECK-P9-NEXT:    li r5, 0
-; CHECK-P9-NEXT:    mr r3, r5
+; CHECK-P9-NEXT:    li r3, 0
 ; CHECK-P9-NEXT:    blr
 ; CHECK-P9-NEXT:  .LBB0_4: # %lor.lhs.false
 ; CHECK-P9-NEXT:    cmplwi cr0, r4, 0
 ; CHECK-P9-NEXT:    bne cr0, .LBB0_2
 ; CHECK-P9-NEXT:  .LBB0_5: # %cleanup16
 ; CHECK-P9-NEXT:    mr r3, r5
 ; CHECK-P9-NEXT:    blr
 ; CHECK-P9-NEXT:  .LBB0_6: # %lor.lhs.false.1

llvm/test/CodeGen/RISCV/alu64.ll

 ; RV32I-NEXT:    ret
 ; RV32I-NEXT:  .LBB11_2:
 ; RV32I-NEXT:    sll a1, a1, a2
 ; RV32I-NEXT:    addi a3, zero, 31
 ; RV32I-NEXT:    sub a3, a3, a2
 ; RV32I-NEXT:    srli a4, a0, 1
 ; RV32I-NEXT:    srl a3, a4, a3
 ; RV32I-NEXT:    or a1, a1, a3
-; RV32I-NEXT:    sll a2, a0, a2
-; RV32I-NEXT:    mv a0, a2
+; RV32I-NEXT:    sll a0, a0, a2
 ; RV32I-NEXT:    ret
   %1 = shl i64 %a, %b
   ret i64 %1
 }
 
 define i64 @slt(i64 %a, i64 %b) nounwind {
 ; RV64I-LABEL: slt:
 ; RV64I:       # %bb.0:

 ; RV32I-NEXT:    ret
 ; RV32I-NEXT:  .LBB15_2:
 ; RV32I-NEXT:    srl a0, a0, a2
 ; RV32I-NEXT:    addi a3, zero, 31
 ; RV32I-NEXT:    sub a3, a3, a2
 ; RV32I-NEXT:    slli a4, a1, 1
 ; RV32I-NEXT:    sll a3, a4, a3
 ; RV32I-NEXT:    or a0, a0, a3
-; RV32I-NEXT:    srl a2, a1, a2
-; RV32I-NEXT:    mv a1, a2
+; RV32I-NEXT:    srl a1, a1, a2
 ; RV32I-NEXT:    ret
   %1 = lshr i64 %a, %b
   ret i64 %1
 }
 
 define i64 @sra(i64 %a, i64 %b) nounwind {
 ; RV64I-LABEL: sra:
 ; RV64I:       # %bb.0:

llvm/test/CodeGen/RISCV/shifts.ll

 ; RV32I-NEXT:    ret
 ; RV32I-NEXT:  .LBB0_2:
 ; RV32I-NEXT:    srl a0, a0, a2
 ; RV32I-NEXT:    addi a3, zero, 31
 ; RV32I-NEXT:    sub a3, a3, a2
 ; RV32I-NEXT:    slli a4, a1, 1
 ; RV32I-NEXT:    sll a3, a4, a3
 ; RV32I-NEXT:    or a0, a0, a3
-; RV32I-NEXT:    srl a2, a1, a2
-; RV32I-NEXT:    mv a1, a2
+; RV32I-NEXT:    srl a1, a1, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV64I-LABEL: lshr64:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    srl a0, a0, a1
 ; RV64I-NEXT:    ret
   %1 = lshr i64 %a, %b
   ret i64 %1

 ; RV32I-NEXT:    ret
 ; RV32I-NEXT:  .LBB4_2:
 ; RV32I-NEXT:    sll a1, a1, a2
 ; RV32I-NEXT:    addi a3, zero, 31
 ; RV32I-NEXT:    sub a3, a3, a2
 ; RV32I-NEXT:    srli a4, a0, 1
 ; RV32I-NEXT:    srl a3, a4, a3
 ; RV32I-NEXT:    or a1, a1, a3
-; RV32I-NEXT:    sll a2, a0, a2
-; RV32I-NEXT:    mv a0, a2
+; RV32I-NEXT:    sll a0, a0, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV64I-LABEL: shl64:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    sll a0, a0, a1
 ; RV64I-NEXT:    ret
   %1 = shl i64 %a, %b
   ret i64 %1

 ; RV64I-NEXT:    ret
 ; RV64I-NEXT:  .LBB6_2:
 ; RV64I-NEXT:    srl a0, a0, a2
 ; RV64I-NEXT:    addi a3, zero, 63
 ; RV64I-NEXT:    sub a3, a3, a2
 ; RV64I-NEXT:    slli a4, a1, 1
 ; RV64I-NEXT:    sll a3, a4, a3
 ; RV64I-NEXT:    or a0, a0, a3
-; RV64I-NEXT:    srl a2, a1, a2
-; RV64I-NEXT:    mv a1, a2
+; RV64I-NEXT:    srl a1, a1, a2
 ; RV64I-NEXT:    ret
   %1 = lshr i128 %a, %b
   ret i128 %1
 }
 
 define i128 @ashr128(i128 %a, i128 %b) nounwind {
 ; RV32I-LABEL: ashr128:
 ; RV32I:       # %bb.0:

 ; RV64I-NEXT:    ret
 ; RV64I-NEXT:  .LBB8_2:
 ; RV64I-NEXT:    sll a1, a1, a2
 ; RV64I-NEXT:    addi a3, zero, 63
 ; RV64I-NEXT:    sub a3, a3, a2
 ; RV64I-NEXT:    srli a4, a0, 1
 ; RV64I-NEXT:    srl a3, a4, a3
 ; RV64I-NEXT:    or a1, a1, a3
-; RV64I-NEXT:    sll a2, a0, a2
-; RV64I-NEXT:    mv a0, a2
+; RV64I-NEXT:    sll a0, a0, a2
 ; RV64I-NEXT:    ret
   %1 = shl i128 %a, %b
   ret i128 %1
 }

llvm/test/CodeGen/X86/fp128-i128.ll

 ; AVX-NEXT:    callq __gttf2
 ; AVX-NEXT:    movl %eax, %ebp
 ; AVX-NEXT:    vmovaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 16-byte Reload
 ; AVX-NEXT:    vmovaps %xmm0, %xmm1
 ; AVX-NEXT:    callq __subtf3
 ; AVX-NEXT:    testl %ebp, %ebp
 ; AVX-NEXT:    jle .LBB10_1
 ; AVX-NEXT:  # %bb.2: # %if.then
-; AVX-NEXT:    vandps {{.*}}(%rip), %xmm0, %xmm1
+; AVX-NEXT:    vandps {{.*}}(%rip), %xmm0, %xmm2
 ; AVX-NEXT:    vmovaps (%rsp), %xmm0 # 16-byte Reload
-; AVX-NEXT:    vmovaps %xmm1, %xmm2
 ; AVX-NEXT:    jmp .LBB10_3
 ; AVX-NEXT:  .LBB10_1:
 ; AVX-NEXT:    vmovaps (%rsp), %xmm2 # 16-byte Reload
 ; AVX-NEXT:  .LBB10_3: # %cleanup
 ; AVX-NEXT:    vmovaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm1 # 16-byte Reload
 ; AVX-NEXT:    vandps {{.*}}(%rip), %xmm1, %xmm1
 ; AVX-NEXT:    vandps {{.*}}(%rip), %xmm0, %xmm0
 ; AVX-NEXT:    vorps %xmm1, %xmm0, %xmm0

llvm/test/CodeGen/X86/fshr.ll

 ; X86-SLOW-NEXT:    shrl %cl, %edi
 ; X86-SLOW-NEXT:    testb %ch, %ch
 ; X86-SLOW-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-SLOW-NEXT:    je .LBB4_2
 ; X86-SLOW-NEXT:  # %bb.1:
 ; X86-SLOW-NEXT:    orl %edi, %edx
 ; X86-SLOW-NEXT:    movl %edx, (%esp) # 4-byte Spill
 ; X86-SLOW-NEXT:  .LBB4_2:
-; X86-SLOW-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-SLOW-NEXT:    movl %ecx, %edx
+; X86-SLOW-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-SLOW-NEXT:    movl %ebx, %ecx
 ; X86-SLOW-NEXT:    shrl %cl, %edx
 ; X86-SLOW-NEXT:    movb %bl, %ah
 ; X86-SLOW-NEXT:    andb $31, %ah
 ; X86-SLOW-NEXT:    movb %ah, %cl
 ; X86-SLOW-NEXT:    negb %cl
 ; X86-SLOW-NEXT:    movl %ebp, %edi
 ; X86-SLOW-NEXT:    shll %cl, %edi

llvm/test/CodeGen/X86/i128-mul.ll

 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl (%eax,%ebp,8), %esi
 ; X86-NEXT:    movl 4(%eax,%ebp,8), %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %esi, %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movl %edx, %edi
-; X86-NEXT:    mull %edx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    movl %eax, %ebx
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
 ; X86-NEXT:    adcl $0, %ecx

llvm/test/CodeGen/X86/mul-i512.ll

 ; X32-NEXT:    addl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; X32-NEXT:    adcl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X32-NEXT:    movzbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 1-byte Folded Reload
 ; X32-NEXT:    adcl %ecx, %eax
 ; X32-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X32-NEXT:    adcl $0, %edx
 ; X32-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl 8(%esi), %ebx
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X32-NEXT:    movl 8(%ecx), %ebx
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X32-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebp
 ; X32-NEXT:    movl 12(%ecx), %ecx
 ; X32-NEXT:    movl %ecx, %eax

llvm/test/CodeGen/X86/shift-i128.ll

 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    subl $20, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %ebx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shll %cl, %ebx
 ; X86-NEXT:    movl %ebp, %esi
 ; X86-NEXT:    shll %cl, %esi
 ; X86-NEXT:    movl %edi, %edx
 ; X86-NEXT:    shldl %cl, %ebp, %edx
 ; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; X86-NEXT:    testb $32, %al

 ; X86-NEXT:    shrdl %cl, %ebp, %esi
 ; X86-NEXT:    testb $32, %dl
 ; X86-NEXT:    jne .LBB6_9
 ; X86-NEXT:  # %bb.8: # %entry
 ; X86-NEXT:    movl %esi, %ebx
 ; X86-NEXT:  .LBB6_9: # %entry
 ; X86-NEXT:    movl %edi, %esi
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %ebp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    shrl %cl, %ebp
 ; X86-NEXT:    testb $32, %cl
 ; X86-NEXT:    movl $0, %ecx
 ; X86-NEXT:    jne .LBB6_11
 ; X86-NEXT:  # %bb.10: # %entry
 ; X86-NEXT:    movl %ebp, %ecx
 ; X86-NEXT:  .LBB6_11: # %entry

 ; X86-NEXT:    testb $32, %al
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    jne .LBB7_5
 ; X86-NEXT:  # %bb.4: # %entry
 ; X86-NEXT:    movl %edi, %ebx
 ; X86-NEXT:  .LBB7_5: # %entry
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %ebp
-; X86-NEXT:    movl %ecx, %edi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; X86-NEXT:    movl %ebp, %edi
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    sarl %cl, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    shrl %cl, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sarl $31, %ebp
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    subl $64, %ecx

 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    shrdl %cl, %ebp, %esi
 ; X86-NEXT:    testb $32, %dl
 ; X86-NEXT:    jne .LBB7_9
 ; X86-NEXT:  # %bb.8: # %entry
 ; X86-NEXT:    movl %esi, %edi
 ; X86-NEXT:  .LBB7_9: # %entry
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %esi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sarl %cl, %esi
 ; X86-NEXT:    testb $32, %cl
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    jne .LBB7_11
 ; X86-NEXT:  # %bb.10: # %entry
 ; X86-NEXT:    movl %esi, %ecx
 ; X86-NEXT:  .LBB7_11: # %entry
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movb $64, %cl
 ; X86-NEXT:    subb %dl, %cl
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movl %ebx, %ebp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    shldl %cl, %ebx, %ebp
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ebx, %ebp
 ; X86-NEXT:    shll %cl, %ebp
 ; X86-NEXT:    testb $32, %cl
 ; X86-NEXT:    movb $64, %bl
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill

 ; X86-LABEL: test_shl_v2i128:
 ; X86:       # %bb.0: # %entry
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    subl $72, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    movl %ebx, %ecx
 ; X86-NEXT:    shll %cl, %ebp
-; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    shll %cl, %esi
 ; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    subl $64, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    sbbl $0, %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    sbbl $0, %eax

 ; X86-NEXT:    jne .LBB8_9
 ; X86-NEXT:  # %bb.8: # %entry
 ; X86-NEXT:    movl %edi, %ebp
 ; X86-NEXT:  .LBB8_9: # %entry
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, %ebp
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    shll %cl, %ebp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %esi
-; X86-NEXT:    movl %edx, %ecx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    shll %cl, %esi
 ; X86-NEXT:    testb $32, %dl
 ; X86-NEXT:    movl $0, %edi
 ; X86-NEXT:    movl $0, %ecx
 ; X86-NEXT:    jne .LBB8_11
 ; X86-NEXT:  # %bb.10: # %entry
 ; X86-NEXT:    movl %esi, %edi
 ; X86-NEXT:    movl %ebp, %ecx

 ; X86-NEXT:    shldl %cl, %ebp, %edi
 ; X86-NEXT:    testb $32, %cl
 ; X86-NEXT:    movl %ebx, %ecx
 ; X86-NEXT:    jne .LBB8_23
 ; X86-NEXT:  # %bb.22: # %entry
 ; X86-NEXT:    movl %edi, %ecx
 ; X86-NEXT:  .LBB8_23: # %entry
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %edi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shll %cl, %edi
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    testb $32, %al
 ; X86-NEXT:    movl $0, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
 ; X86-NEXT:    jne .LBB8_25
 ; X86-NEXT:  # %bb.24: # %entry

llvm/test/CodeGen/X86/umulo-128-legalisation-lowering.ll

 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    mull %ecx
 ; X86-NEXT:    movl %edx, %ebp
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    addl %esi, %ecx
 ; X86-NEXT:    adcl $0, %ebp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    mull %esi
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    mull %edx
 ; X86-NEXT:    movl %edx, %esi
 ; X86-NEXT:    addl %ecx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    adcl %ebp, %esi
 ; X86-NEXT:    setb %cl
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl %ebp, %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)