[PowerPC] Eliminate redundant register copys after register allocation

[inouehrs]

Summary

This patch add a PPC-specific path after register allocation to eliminate redundancy related register copies among physical registers.

So far, we eliminate the following two types of redundant register copys.

1. intra-BB redundant register copy

li Y, 0          li X, 0  (any instruction, not limited to li)
mr X, Y    =>    (erase mr)
..               ..

2. inter-BB partially redundant register copy

          BB1--------                             BB1--------
          | ..      |                             | ..      |
          | mr Y, X |                             | (erase) |
          | ..      |                             | ..      |
  with    -----------                    with     -----------
  1 pred /     |                         1 pred  /     |
BB--------     |      BB--------      BB---------      |      BB---------
| ..     |     |      | ..     |  =>  | mr Y, X |      |      | ..      |
| ..     |     |      | ..     |      | ..      |      |      | mr X, Y |
----------     |      ----------      -----------      |      -----------
               |     /  with                           |     /  with
          BB2--------   1 succ                    BB2--------   1 succ
          | ..      |                             | ..      |
          | mr X, Y |                             | (erase) |
          | ..      |                             | ..      |
          -----------                             -----------

[nemanjai]

I think that the main loop definitely needs to be re-written and made more readable and easier to follow. To me, it seems like it does too much. It spans nearly 250 lines and does a whole bunch of analyses and transformations.
Could you extract functionality into easy-to-read functions that perform individual things so that the flow is easier to follow?

Also, would it be appropriate to make this target-independent and do it perhaps in MachineCopyPropagation? Perhaps Geoff can comment on that since he's done something similar that he is still working on improvements for in order to finally commit it.

lib/Target/PowerPC/PPCPostRAPeephole.cpp

61

If we're adding a new peephole, it likely won't only be used for this one purpose. A more general name might be in order.

100

This seems a bit fragile. I'm not sure there's any hard guarantee that the flag will be on the last source operand.

142

These loops as well as a check for whether the instruction is a call seem a little odd. Why is it inadequate to use MachineInstr::readsRegister() and MachineInstr::modifiesRegister() for these purposes. Given the TRI, they should return the correct value for any instruction (including calls).

[inouehrs]

• changed a new pass name
• code simplified

[inouehrs]

If we're adding a new peephole, it likely won't only be used for this one purpose. A more general name might be in order.

I changed the pass name. Do you have any suggestion on an alternative name?

This seems a bit fragile. I'm not sure there's any hard guarantee that the flag will be on the last source operand.

I modified to check the flag for both input operands.

These loops as well as a check for whether the instruction is a call seem a little odd. Why is it inadequate to use MachineInstr::readsRegister() and MachineInstr::modifiesRegister() for these purposes. Given the TRI, they should return the correct value for any instruction (including calls).

readsRegister checks whether the instruction reads the specified register or its super register, but it does not check about sub register access. I need to check sub register access for safety.

[nemanjai]

In D39536#917677, @inouehrs wrote:

If we're adding a new peephole, it likely won't only be used for this one purpose. A more general name might be in order.

I changed the pass name. Do you have any suggestion on an alternative name?

If you agree that this would likely be used for other purposes, why not just name it PPCPostRAPeephole? Otherwise if you think this will likely only be used for this purpose, I suppose the name is fine.

This seems a bit fragile. I'm not sure there's any hard guarantee that the flag will be on the last source operand.

I modified to check the flag for both input operands.

Thanks.

These loops as well as a check for whether the instruction is a call seem a little odd. Why is it inadequate to use MachineInstr::readsRegister() and MachineInstr::modifiesRegister() for these purposes. Given the TRI, they should return the correct value for any instruction (including calls).

readsRegister checks whether the instruction reads the specified register or its super register, but it does not check about sub register access. I need to check sub register access for safety.

Hmm.. that certainly doesn't seem to be the case by just looking at the implementation. There's a check for Found = TRI->regsOverlap(MOReg, Reg); and I would assume this would return true for any overlap. But of course, this is based on just a cursory look at the code and I may be missing something.

[gberry]

I'm not sure adding a new pass that relies on kill flags is a good idea, since I believe they are slowly being phased out.
Also, if you stop using kill flags I believe you will need something like https://reviews.llvm.org/D39400 in order to avoid removing copies whose defs are needed for ABI/ISA reasons that aren't captured in the MI representation.

[inouehrs]

• rewrote without using kill flags as @gberry suggested.
• slightly increased coverage to support opportunity addressed by D39785.
• refactored to reduce duplicated code.

[inouehrs]

@gberry Thank you for the suggestions. I updated the implementation not to use kill flags.
Also, I updated not to optimize reg copies around ABI-dependent special registers such as stack pointer.
Additionally, the current version avoids optimization for specific ABI-dependent opcode (GETtlsldADDR), but I will use isRenamable flag after D39400 is merged.

[MatzeB]

Is this like the existing MachineCopyPropagation pass but working accross multiple blocks? Would D30751 solve your problem as well?

[inouehrs]

In D39536#926354, @MatzeB wrote:

Is this like the existing MachineCopyPropagation pass but working accross multiple blocks? Would D30751 solve your problem as well?

The two optimization are mostly disjoint.
Copy source forwarding optimizes

mr 3, 4
addi 5, 3, 1

to

mr 3, 4
addi 5, 4, 1

but this optimization intends to rewrite

addi 4, 3, 1 
mr 3, 4

to

addi 3, 3, 1

or eliminate

mr 4, 3
mr 3, 4

[gberry]

In D39536#927450, @inouehrs wrote:

In D39536#926354, @MatzeB wrote:

Is this like the existing MachineCopyPropagation pass but working accross multiple blocks? Would D30751 solve your problem as well?

The two optimization are mostly disjoint.
Copy source forwarding optimizes

mr 3, 4
addi 5, 3, 1

to

mr 3, 4
addi 5, 4, 1

but this optimization intends to rewrite

addi 4, 3, 1 
mr 3, 4

to

addi 3, 3, 1

or eliminate

mr 4, 3
mr 3, 4

FWIW, the MachineCopyPropagation already handles eliminating the second COPY in the second case above, and could fairly easily be extended to handle the other cases once the change to mark MachineOperands as renamable lands. It is currently limited to single block though. It also only currently deals with instructions whose opcode is COPY (i.e. it doesn't handle other target specific copy operations). I don't see any reason why all of these things couldn't be added to MachineCopyPropagation eventually, but I don't want to hold up your change because it might be made redundant later, especially since I can;t commit to making these extensions to MachineCopyProp.

[inouehrs]

@gberry Thank you for the comment!
Many redundant register copies (e.g. about 2/3 of the intra-BB redundancies) are generated by optimizers executed after COPYs are expanded to real instructions. To catch them, this optimization is run just before code emission.
So, MachineCopyPropagation, which is executed just after register allocation, may not be able to catch these opportunities.
(For example, when I run this pass just after pseudo instruction expansion, optimizations happen 7780 times during the bootstrap while they happen 10255 times if the pass is executed at the very last of the compilation.)

[inouehrs]

minor touch up in comments and formatting

[jtony]

lib/Target/PowerPC/PPCRegCopyElim.cpp

101 ↗

(On Diff #123880)

There are some other instructions which are equivalent to MR, do we want to handle them too in this patch? Some examples: ADDI Rx,Ry, 0 and ORI Rx,Ry,0

test/CodeGen/PowerPC/remove-cyclic-mr.ll

47 ↗

(On Diff #123880)

Minor nit: I know this is auto generated test cases, but this line can be simplified away. Similarly line 1 and line 54.

[inouehrs]

lib/Target/PowerPC/PPCRegCopyElim.cpp

101 ↗

(On Diff #123880)

I picked these instructions since they are emitted for PPC::COPY. Do you see ADDI Rx,Ry, 0 or ORI Rx,Ry,0 frequently? If so, I will add that opcode here.

[hfinkel]

Is this something that RDF copy propagation would do for us as well? @kparzysz might know. FWIW, if this is something that RDF would help with, I'd rather see our efforts put there rather than recreating a less-powerful version of that infrastructure just for this specific case.

lib/Target/PowerPC/PPCRegCopyElim.cpp

267 ↗

(On Diff #123880)

X2 is not always special (in leaf functions, it might be allocatable). You can check using:

MF->getRegInfo().isAllocatable(PPC::X2) or !getReservedRegs(*MF).test(PPC::X2) or similar.

[nemanjai]

Now that https://reviews.llvm.org/rL323991 has actually landed, can you re-evaluate whether this patch is still applicable and if so, whether it needs any changes. Also, have you looked into Hal's suggestion?

[inouehrs]

• check the reserved register using getReservedRegs instead of own check.
• rebased to the latest code

[inouehrs]

In D39536#996106, @nemanjai wrote:

Now that https://reviews.llvm.org/rL323991 has actually landed, can you re-evaluate whether this patch is still applicable and if so, whether it needs any changes. Also, have you looked into Hal's suggestion?

As long as I tested, COPY source forwarding (rL323991) does not affect this optimization. The number of optimizations happen in this pass are mostly unchanged by enabling/disabling the COPY source forwarding (11730 with COPY source forwarding and 11242 without COPY source forwarding during the bootstrap test).

Also, I updated the patch to address Hal's suggestion.

[inouehrs]

lib/Target/PowerPC/PPCRegCopyElim.cpp

267 ↗

(On Diff #123880)

Thank you so much for the suggestion. I totally replaced my own check with getReservedRegs (not only for X2). It makes the code cleaner.

[syzaara]

lib/Target/PowerPC/PPCRegCopyElim.cpp

12 ↗

(On Diff #133166)

s/copys/copies

87 ↗

(On Diff #133166)

Is this an old comment you forgot to remove? I don't see anything related to IsKill in this function.

121 ↗

(On Diff #133166)

I think this flow would be more clear if written like:

if (MO.isReg() && TRI->isSuperOrSubRegisterEq(MO.getReg(), Reg)) {
    if (MO.isUse())
        return false;
    else if(MO.isDef() ||  (MO.isRegMask() && MO.clobbersPhysReg(Reg)))
        SrcKilled = true; // we may still have use of Reg in this MI
}

134 ↗

(On Diff #133166)

s/seccessor/successor

173 ↗

(On Diff #133166)

Since findRegisterDefOperandIdx already checks for SrcReg, do we need to check DefMO.getReg() == SrcReg here?

196 ↗

(On Diff #133166)

MO.isUse() would be more clear

210 ↗

(On Diff #133166)

It seems confusing that we check for registers being equal with isSuperOrSubRegisterEq and then again with if (MO.getReg() == SrcReg).
Can we rewrite to something like:

if (MO.getReg() == SrcReg)
   TmpOperands.push_back(&MO);
else if (TRI->isSuperOrSubRegisterEq(MO.getReg(), SrcReg)) {
   IsClobbered = true;
   return nullptr;
}

314 ↗

(On Diff #133166)

This is repeated below for the second case on line 443
However, the second case also sets Simplified = CurrentInstrErased which doesn't seem to be set here.
Can this be rewritten in a way that we only have to do this in one place for both cases?

339 ↗

(On Diff #133166)

Can we put these checks of pred_size/DstLive/control flow for which we just continue into a separate function and do something like:

if (!controlFlowSupported())
    continue;

[syzaara]

lib/Target/PowerPC/PPCRegCopyElim.cpp

285 ↗

(On Diff #133166)

I'm finding this difficult to follow since there is a lot happening in the loop. Can we extract out more functionality into functions? From what I understand, maybe we can do something like:

if (DefMI)
   Simplified |= eliminateIntraBBCopy
else if (controlFlowSupported())
   Simplified |= elimanteInterBBCopy

[inouehrs]

• addressed comments from @syzaara
• rebased to the lates tcode and fix a failing test case

[inouehrs]

lib/Target/PowerPC/PPCRegCopyElim.cpp

121 ↗

(On Diff #133166)

They are calling different methods isSuperOrSubRegisterEq and isSubRegisterEq, but I gave a try to make the control flow clearer.

173 ↗

(On Diff #133166)

Yes, it intends to confirm that it is not sub/super register.

210 ↗

(On Diff #133166)

Fixed.
Originally, I intended to make the if-statement parallel as

• isSuperOrSubRegisterEq for DstReg
• isSuperOrSubRegisterEq for SrcReg

285 ↗

(On Diff #133166)

I hope it becomes easier to follow now.

[syzaara]

lib/Target/PowerPC/PPCRegCopyElim.cpp

80 ↗

(On Diff #144073)

I don't see any uses of TII, but there are uses of TRI. Maybe replace this with TRI?

430 ↗

(On Diff #144073)

Can we change this to

if (!isRegCopy(CopyMI, SrcMO, DstMO))
   continue;

so that we don't have to nest the rest of the code.

454 ↗

(On Diff #144073)

It seems a little messy to me for this function to have a parameter for OperandsToRewrite1. I wonder if it would be better to have another function like getOperandsToRewrite which we can call on line 482 after we know that we have erased an instruction.

[inouehrs]

• addresses @syzaara 's comments

[inouehrs]

lib/Target/PowerPC/PPCRegCopyElim.cpp

80 ↗

(On Diff #144073)

Removed TII. I do not add TRI as a member since TRI must be passed among static functions anyway.

454 ↗

(On Diff #144073)

To implement getOperandsToRewrite helper, we need to do almost same (potentially costly) iteration again in the helper. So I have not do this yet.

lib/Target/PowerPC/CMakeLists.txt

add_llvm_target(PowerPCCodeGen
   PPCISelLowering.cpp
   PPCEarlyReturn.cpp
   PPCFastISel.cpp
   PPCFrameLowering.cpp
   PPCLoopPreIncPrep.cpp
   PPCMCInstLower.cpp
   PPCMachineFunctionInfo.cpp
   PPCMIPeephole.cpp
+  PPCPostRAPeephole.cpp
   PPCRegisterInfo.cpp
   PPCQPXLoadSplat.cpp
   PPCSubtarget.cpp
   PPCTargetMachine.cpp
   PPCTargetObjectFile.cpp
   PPCTargetTransformInfo.cpp
   PPCTOCRegDeps.cpp
   PPCTLSDynamicCall.cpp

lib/Target/PowerPC/PPC.h

 #endif
   FunctionPass *createPPCLoopPreIncPrepPass(PPCTargetMachine &TM);
   FunctionPass *createPPCTOCRegDepsPass();
   FunctionPass *createPPCEarlyReturnPass();
   FunctionPass *createPPCVSXCopyPass();
   FunctionPass *createPPCVSXFMAMutatePass();
   FunctionPass *createPPCVSXSwapRemovalPass();
   FunctionPass *createPPCMIPeepholePass();
+  FunctionPass *createPPCPostRAPeepholePass();
   FunctionPass *createPPCBranchSelectionPass();
   FunctionPass *createPPCBranchCoalescingPass();
   FunctionPass *createPPCQPXLoadSplatPass();
   FunctionPass *createPPCISelDag(PPCTargetMachine &TM, CodeGenOpt::Level OL);
   FunctionPass *createPPCTLSDynamicCallPass();
   FunctionPass *createPPCBoolRetToIntPass();
   FunctionPass *createPPCExpandISELPass();
   void LowerPPCMachineInstrToMCInst(const MachineInstr *MI, MCInst &OutMI,
                                     AsmPrinter &AP, bool isDarwin);
   bool LowerPPCMachineOperandToMCOperand(const MachineOperand &MO,
                                          MCOperand &OutMO, AsmPrinter &AP,
                                          bool isDarwin);
 
   void initializePPCVSXFMAMutatePass(PassRegistry&);
   void initializePPCBoolRetToIntPass(PassRegistry&);
   void initializePPCExpandISELPass(PassRegistry &);
   void initializePPCTLSDynamicCallPass(PassRegistry &);
+  void initializePPCPostRAPeepholePass(PassRegistry &);
   extern char &PPCVSXFMAMutateID;
 
   namespace PPCII {
     
   /// Target Operand Flag enum.
   enum TOF {
     //===------------------------------------------------------------------===//
     // PPC Specific MachineOperand flags.

lib/Target/PowerPC/PPCPostRAPeephole.cpp

+//===-------------- PPCPostRAPeephole.cpp - MI Peephole Cleanups ---------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+//
+// This pass aims to eliminate redundancy related to physical registers
+// after the register allocation.
+// So far, we eliminate the following two types of redundant register copys.
+//
+// 1) intra-BB redundant register copy
+//   li Y, 0          li X, 0
+//   mr X, Y    =>    (erase mr)
+//   ..               ..
+// 2) inter-BB partially redundant register copy
+//            BB1--------                             BB1--------
+//            | ..      |                             | ..      |
+//            | mr Y, X |                             | (erase) |
+//            | ..      |                             | ..      |
+//    with    -----------                    with     -----------
+//    1 pred /     |                         1 pred  /     |
+//  BB--------     |      BB--------      BB---------      |      BB---------
+//  | ..     |     |      | ..     |  =>  | mr Y, X |      |      | ..      |
+//  | ..     |     |      | ..     |      | ..      |      |      | mr X, Y |
+//  ----------     |      ----------      -----------      |      -----------
+//                 |     /  with                           |     /  with
+//            BB2--------   1 succ                    BB2--------   1 succ
+//            | ..      |                             | ..      |
+//            | mr X, Y |                             | (erase) |
+//            | ..      |                             | ..      |
+//            -----------                             -----------
+//
+//===---------------------------------------------------------------------===//
+
+#include "PPCTargetMachine.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "ppc-postra-peepholes"
+
+namespace {
+
+struct PPCPostRAPeephole : public MachineFunctionPass {
+
+  static char ID;
+  const PPCInstrInfo *TII;
+  MachineFunction *MF;
+
+  PPCPostRAPeephole() : MachineFunctionPass(ID) {
+    initializePPCPostRAPeepholePass(*PassRegistry::getPassRegistry());
+  }
+
+private:
+  // Initialize class variables.
+  void initialize(MachineFunction &MFParm);
+
+  // Perform peepholes.
+  bool eliminateRedundantRegCopy(void);
+
+public:
+  // Main entry point for this pass.
+  bool runOnMachineFunction(MachineFunction &MF) override {
+    if (skipFunction(*MF.getFunction()))
+      return false;
+    initialize(MF);
+    bool Simplified = false;
+
+    Simplified |= eliminateRedundantRegCopy();
+    return Simplified;
+  }
+};
+
+// Initialize class variables.
+void PPCPostRAPeephole::initialize(MachineFunction &MFParm) {
+  MF = &MFParm;
+  TII = MF->getSubtarget<PPCSubtarget>().getInstrInfo();
+  DEBUG(dbgs() << "*** PowerPC Post RA peephole pass ***\n\n");
+}
+
+// If MI is a register copy, this method returns true and set
+// source and destination operands in SrcOperand and DstOperand.
+static bool isRegCopy(MachineInstr &MI, MachineOperand* &SrcOperand,
+                      MachineOperand* &DstOperand) {
+  // Refer PPCInstrInfo::copyPhysReg to find opcodes used for copying
+  // the value in a physical register for each register class.
+  // Currently we do not optimize VSX registers (copied by xxlor),
+  // which may involve different register types, i.e. FPR and VRF.
+  unsigned Opc = MI.getOpcode();
+  if (Opc == PPC::FMR) {
+    DstOperand = &MI.getOperand(0);
+    SrcOperand = &MI.getOperand(1);
+    return true;
+  }
+  if ((Opc == PPC::OR8 || Opc == PPC::OR || Opc == PPC::VOR) &&
+      MI.getOperand(1).getReg() == MI.getOperand(2).getReg()) {
+    DstOperand = &MI.getOperand(0);
+    // We return the last operand as src to access kill flag.
+    SrcOperand = &MI.getOperand(2);
+    return true;
+  }
+  return false;
+}
+
+bool PPCPostRAPeephole::eliminateRedundantRegCopy(void) {
+  const PPCSubtarget *PPCSubTarget = &MF->getSubtarget<PPCSubtarget>();
+  const TargetRegisterInfo *TRI = PPCSubTarget->getRegisterInfo();
+  bool Simplified = false;
+  for (MachineBasicBlock &MBB : *MF) {
+    bool CurrentInstrErased = false;
+    do {
+      CurrentInstrErased = false;
+      for (MachineInstr &CopyMI : MBB) {
+        if (CopyMI.isDebugValue())
+          continue;
+
+        MachineOperand *SrcMO = nullptr, *DstMO = nullptr;
+        if (isRegCopy(CopyMI, SrcMO, DstMO) && SrcMO->isKill()) {
+          unsigned DstReg = DstMO->getReg();
+          unsigned SrcReg = SrcMO->getReg();
+
+          MachineInstr *DefMI = nullptr;
+          bool Found = false;
+          SmallVector<MachineOperand*, 8> OperandsToRewrite1;
+
+          // We iterate instructions backward to find the def of the source
+          // of register copy or instructions conflicting with register copy
+          // (e.g. instructions that use destination reg of copy).
+          MachineBasicBlock::reverse_iterator MBBRI = CopyMI,
+                                              MBBRIE = MBB.rend();
+          for (MBBRI++; MBBRI != MBBRIE && !Found; MBBRI++) {
+            // We conservatively assume function call may crobber all regs.
+            if (MBBRI->isCall()) {
+              Found = true;
+              break;
+            }
+
+            // If this instruction defines source reg of copy, this is the
+            // target for our optimization.
+            for (auto DefMO: MBBRI->defs())
+              if (TRI->isSuperOrSubRegisterEq(DefMO.getReg(), SrcReg)) {
+                if (!DefMO.isTied())
+                  DefMI = &(*MBBRI);
+                Found = true;
+                break;
+              }
+            if (Found)
+              break;
+
+            // If this instr conflicts with the reg copy, we give up here.
+            for (auto MO: MBBRI->operands())
+              if (MO.isReg() &&
+                  TRI->isSuperOrSubRegisterEq(MO.getReg(), DstReg)) {
+                Found = true;
+                break;
+              }
+
+            const MCInstrDesc &MCID = TII->get(MBBRI->getOpcode());
+            if (MCID.ImplicitDefs && !Found)
+              for (const MCPhysReg *ImpDef = MCID.getImplicitDefs(); *ImpDef;
+                   ImpDef++)
+                if (TRI->isSuperOrSubRegisterEq(*ImpDef, SrcReg) ||
+                    TRI->isSuperOrSubRegisterEq(*ImpDef, DstReg)) {
+                  Found = true;
+                  break;
+                }
+            if (Found)
+              break;
+
+            // If this instruction uses source register of register copy,
+            // we need to remember the operand to replace it with destination
+            // register of copy. For example:
+            //   li r4, 1            li r3, 1
+            //   cmplwi r4, 0   ->   cmplwi r3, 0
+            //   mr r3, r4           (erase mr)
+            for (unsigned I = MCID.getNumDefs(); I < MCID.getNumOperands();
+                 I++)
+              if (MBBRI->getOperand(I).isReg()) {
+                unsigned Operand = MBBRI->getOperand(I).getReg();
+                if (Operand == SrcReg)
+                  OperandsToRewrite1.push_back(&MBBRI->getOperand(I));
+                else if (TRI->isSuperOrSubRegisterEq(Operand, SrcReg)) {
+                  Found = true;
+                  break;
+                }
+              }
+          }
+
+          // Def for the source of the register copy is found in the BB
+          // and can be optimized within this block.
+          if (DefMI && DefMI->getOperand(0).getReg() == SrcReg) {
+            DEBUG(dbgs()
+                  << "Eliminating redundant register copy:\n");
+            DEBUG(DefMI->dump());
+            DEBUG(CopyMI.dump());
+            DefMI->getOperand(0).setReg(DstReg);
+            CopyMI.eraseFromParent();
+            for (auto &MO: OperandsToRewrite1)
+              MO->setReg(DstReg);
+            CurrentInstrErased = true;
+            break;
+          }
+
+          // If there is conflicting instruction (e.g. uses source reg of copy)
+          // between def and reg copy, we cannot optimize.
+          if (Found)
+            continue;
+
+          // If no def and no conflicting instruction is found in this BB,
+          // we check the predecessors.
+          // So far, we do not optimize if this BB has more than two
+          // predecessors not to increase the number of total instructions.
+          if (MBB.pred_size() != 2) continue;
+
+          // From here, we handle redundancy among multiple BBs.
+          // We currently support following CFGs.
+          //
+          //         Pred1MBB
+          //          /  |    /
+          //   (SideMBB) | Pred2MBB
+          //        /    |  /
+          //           MBB (including CopyMI)
+          //
+          // - MBB doesn't have more than two predecessors.
+          // - Pred1MBB doesn't have more than two successors.
+          // - SideMBB is optional.
+          // - SideMBB has only one predecessor (Pred1MBB).
+          // - Pred2MBB has only one successor (MBB).
+          // - SideMBB and Pred2MBB may be the same BB.
+          auto FirstPredBB = MBB.pred_begin();
+          MachineBasicBlock *Pred2MBB = nullptr, *SideMBB = nullptr;
+          for (auto &Pred1MBB: MBB.predecessors()) {
+            // If the BBs does not match the above patterns we support,
+            // we give up before iterating instructions.
+            Pred2MBB = (Pred1MBB == *FirstPredBB) ? *(FirstPredBB + 1):
+                                                    * FirstPredBB;
+            if (Pred1MBB->succ_size() > 2 || Pred2MBB->succ_size() > 1)
+              continue;
+
+            if (Pred1MBB->succ_size() == 2) {
+              auto FirstSuccBB = Pred1MBB->succ_begin();
+              SideMBB = (&MBB == *FirstSuccBB) ? *(FirstSuccBB + 1):
+                                                 * FirstSuccBB;
+              if (SideMBB->pred_size() > 1)
+                continue;
+            }
+
+            MachineInstr *DefMI = nullptr;
+            bool Found = false;
+            SmallVector<MachineOperand*, 8> OperandsToRewrite2;
+            // As we did for the current MBB, we iterate instructions backward
+            // to find the def of the source of register copy, starting from
+            // the end of the BB.
+            MachineBasicBlock::reverse_iterator MBBRI = Pred1MBB->rbegin(),
+                                                MBBRIE = Pred1MBB->rend();
+            for (; MBBRI != MBBRIE && !Found; MBBRI++) {
+              if (MBBRI->isCall()) {
+                Found = true;
+                break;
+              }
+
+              // If this instruction defines source reg of copy, this is the
+              // target for our optimization.
+              for (auto DefMO: MBBRI->defs())
+                if (TRI->isSuperOrSubRegisterEq(DefMO.getReg(), SrcReg)) {
+                  if (!DefMO.isTied())
+                    DefMI = &(*MBBRI);
+                  Found = true;
+                  break;
+                }
+              if (Found)
+                break;
+
+              // If this instruction conflicts with the reg copy, we give up.
+              for (auto DefMO: MBBRI->defs())
+                if (TRI->isSuperOrSubRegisterEq(DefMO.getReg(), DstReg)) {
+                  Found = true;
+                  break;
+                }
+
+              const MCInstrDesc &MCID = TII->get(MBBRI->getOpcode());
+              if (MCID.ImplicitDefs && !Found)
+                for (const MCPhysReg *ImpDef = MCID.getImplicitDefs(); *ImpDef;
+                    ImpDef++)
+                  if (TRI->isSuperOrSubRegisterEq(*ImpDef, SrcReg) ||
+                      TRI->isSuperOrSubRegisterEq(*ImpDef, DstReg)) {
+                    Found = true;
+                    break;
+                  }
+              if (Found)
+                break;
+
+              for (unsigned I = MCID.getNumDefs(); I < MCID.getNumOperands();
+                   I++)
+                if (MBBRI->getOperand(I).isReg()) {
+                  unsigned Operand = MBBRI->getOperand(I).getReg();
+                  if (Operand == SrcReg)
+                    OperandsToRewrite2.push_back(&MBBRI->getOperand(I));
+                  else if (TRI->isSuperOrSubRegisterEq(Operand, SrcReg)) {
+                    Found = true;
+                    break;
+                  }
+                }
+            }
+            // We keep going iff the def of the reg copy is another reg copy
+            // with reversed src and dst.
+            if (!(DefMI && isRegCopy(*DefMI, SrcMO, DstMO) &&
+                  DefMI->getOpcode() == CopyMI.getOpcode() &&
+                  SrcReg == DstMO->getReg() && DstReg == SrcMO->getReg()))
+              continue;
+
+            DEBUG(dbgs()
+                  << "Optimizing partially redundant reg copy pair:\n");
+            DEBUG(DefMI->dump());
+            DEBUG(CopyMI.dump());
+
+            // Here, we find a partially redundant register copy pair.
+            // If we have SideMBB, we copy the first reg copy to the top of it.
+            // Otherwise, we can erase it.
+            if (SideMBB) {
+              SideMBB->splice(SideMBB->getFirstNonDebugInstr(), Pred1MBB,
+                              MachineBasicBlock::iterator(DefMI));
+              SideMBB->removeLiveIn(SrcReg);
+              SideMBB->addLiveIn(DstReg);
+            }
+            else
+              DefMI->eraseFromParent();
+
+            // We move the second reg copy to Pred2MBB by inserting before
+            // the first terminator instructions (or at the end if no
+            // terminator in the BB).
+            MachineBasicBlock::iterator To = Pred2MBB->getFirstTerminator();
+            if (To == Pred2MBB->end())
+              Pred2MBB->push_back(MBB.remove(&CopyMI));
+            else
+              Pred2MBB->splice(To, &MBB, MachineBasicBlock::iterator(CopyMI));
+            MBB.removeLiveIn(SrcReg);
+            MBB.addLiveIn(DstReg);
+
+            // We touch up some operands if we have found any.
+            for (auto &MO: OperandsToRewrite1)
+              MO->setReg(DstReg);
+            for (auto &MO: OperandsToRewrite2)
+              MO->setReg(DstReg);
+
+            CurrentInstrErased = true;
+            break;
+          }
+
+          Simplified = CurrentInstrErased;
+          // If the current instruction has been eliminated,
+          // we cannot continue iteration, so restart it.
+          if (CurrentInstrErased)
+            break;
+        }
+      }
+    } while(CurrentInstrErased);
+  }
+  return Simplified;
+}
+
+} // end default namespace
+
+INITIALIZE_PASS_BEGIN(PPCPostRAPeephole, DEBUG_TYPE,
+                      "PowerPC Post RA Peephole Optimization", false, false)
+INITIALIZE_PASS_END(PPCPostRAPeephole, DEBUG_TYPE,
+                    "PowerPC Post RA Peephole Optimization", false, false)
+
+char PPCPostRAPeephole::ID = 0;
+FunctionPass*
+llvm::createPPCPostRAPeepholePass() { return new PPCPostRAPeephole(); }

lib/Target/PowerPC/PPCTargetMachine.cpp

 static cl::
 opt<bool> DisableQPXLoadSplat("disable-ppc-qpx-load-splat", cl::Hidden,
                               cl::desc("Disable QPX load splat simplification"));
 
 static cl::
 opt<bool> DisableMIPeephole("disable-ppc-peephole", cl::Hidden,
                             cl::desc("Disable machine peepholes for PPC"));
 
+static cl::
+opt<bool> DisablePostRAPeephole("disable-ppc-postra-peephole", cl::Hidden,
+                            cl::desc("Disable post RA peepholes for PPC"));
+
 static cl::opt<bool>
 EnableGEPOpt("ppc-gep-opt", cl::Hidden,
              cl::desc("Enable optimizations on complex GEPs"),
              cl::init(true));
 
 static cl::opt<bool>
 EnablePrefetch("enable-ppc-prefetching",
                   cl::desc("disable software prefetching on PPC"),
extern "C" void LLVMInitializePowerPCTarget() {
   RegisterTargetMachine<PPCTargetMachine> A(getThePPC32Target());
   RegisterTargetMachine<PPCTargetMachine> B(getThePPC64Target());
   RegisterTargetMachine<PPCTargetMachine> C(getThePPC64LETarget());
 
   PassRegistry &PR = *PassRegistry::getPassRegistry();
   initializePPCBoolRetToIntPass(PR);
   initializePPCExpandISELPass(PR);
   initializePPCTLSDynamicCallPass(PR);
+  initializePPCPostRAPeepholePass(PR);
 }
 
 /// Return the datalayout string of a subtarget.
 static std::string getDataLayoutString(const Triple &T) {
   bool is64Bit = T.getArch() == Triple::ppc64 || T.getArch() == Triple::ppc64le;
   std::string Ret;
 
   // Most PPC* platforms are big endian, PPC64LE is little endian.
if (getPPCTargetMachine().isPositionIndependent()) {
     addPass(createPPCTLSDynamicCallPass());
   }
   if (EnableExtraTOCRegDeps)
     addPass(createPPCTOCRegDepsPass());
 }
 
 void PPCPassConfig::addPreSched2() {
   if (getOptLevel() != CodeGenOpt::None) {
+    // Target-specific peephole optimization after register allocation.
+    if (!DisablePostRAPeephole)
+      addPass(createPPCPostRAPeepholePass());
+
     addPass(&IfConverterID);
 
     // This optimization must happen after anything that might do store-to-load
     // forwarding. Here we're after RA (and, thus, when spills are inserted)
     // but before post-RA scheduling.
     if (!DisableQPXLoadSplat)
       addPass(createPPCQPXLoadSplatPass());
   }

test/CodeGen/PowerPC/aggressive-anti-dep-breaker-subreg.ll

 ; RUN: llc -verify-machineinstrs %s -mtriple=powerpc64-unknown-linux-gnu -O2 -o - -optimize-regalloc=false -regalloc=fast | FileCheck %s
 
 declare void @func(i8*, i64, i64)
 
 define void @test(i8* %context, i32** %elementArrayPtr, i32 %value) {
 entry:
   %cmp = icmp eq i32 %value, 0
   br i1 %cmp, label %lreturn, label %lnext
 
 lnext:
   %elementArray = load i32*, i32** %elementArrayPtr, align 8
 ; CHECK: lwz [[LDREG:[0-9]+]], 124(1)                   # 4-byte Folded Reload
-; CHECK: # implicit-def: %X[[TEMPREG:[0-9]+]]
   %element = load i32, i32* %elementArray, align 4
-; CHECK: mr [[TEMPREG]], [[LDREG]]
-; CHECK: clrldi   4, [[TEMPREG]], 32
+; CHECK: clrldi   4, [[LDREG]], 32
   %element.ext = zext i32 %element to i64
   %value.ext = zext i32 %value to i64
   call void @func(i8* %context, i64 %value.ext, i64 %element.ext)
   br label %lreturn
 
 lreturn:
   ret void
 }

test/CodeGen/PowerPC/ppc64-byval-align.ll

 
 declare signext i32 @test1(i32 signext, %struct.test* byval align 16, i32 signext)
 define void @caller1(i32 signext %z) {
 entry:
   %call = tail call signext i32 @test1(i32 signext 0, %struct.test* byval align 16 @gt, i32 signext %z)
   ret void
 }
 ; CHECK-LABEL: @caller1
-; CHECK: mr [[REG:[0-9]+]], 3
-; CHECK: mr 7, [[REG]]
+; CHECK: mr 7, 3
 ; CHECK: bl test1
 
 define i64 @callee2(%struct.pad* byval nocapture readnone %x, i32 signext %y, %struct.test* byval align 16 nocapture readonly %z) {
 entry:
   %x1 = getelementptr inbounds %struct.test, %struct.test* %z, i64 0, i32 0
   %0 = load i64, i64* %x1, align 16
   ret i64 %0
 }

test/CodeGen/PowerPC/redundant_regcopy.ll

+; RUN: llc < %s -O2 -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu | FileCheck %s
+
+define i8* @func1(i8* %a, i1 %b) {
+; CHECK-LABEL: @func1
+; CHECK: bc
+; CHECK: # BB#1
+; CHECK: mr 30, 3
+; CHECK: bl callee
+; CHECK: mr 3, 30
+; CHECK: .LBB0_2:
+; CHECK:  blr
+entry:
+  br i1 %b, label %exit, label %foo
+
+foo:
+  call void @callee()
+  br label %exit
+
+exit:
+  ret i8* %a
+}
+
+declare void @callee()

test/CodeGen/PowerPC/redundant_regcopy_1.mir

+# test for BB-local register copy elimination
+# RUN: llc -run-pass ppc-postra-peepholes -verify-machineinstrs -o - %s | FileCheck %s
+
+--- |
+  target datalayout = "E-m:e-i64:64-n32:64"
+  target triple = "powerpc64le-unknown-linux-gnu"
+  define signext i32 @func(i32 signext %i) {
+  entry:
+    ret i32 %i
+  }
+
+...
+---
+name:            func
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+tracksRegLiveness: true
+liveins:
+  - { reg: '%x3' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       0
+  offsetAdjustment: 0
+  maxAlignment:    0
+  adjustsStack:    false
+  hasCalls:        false
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+body:             |
+  bb.0.entry:
+    liveins: %x3
+
+    ; CHECK-LABEL: bb.0.entry:
+    ; CHECK: %x3 = ADDI8 killed %x3, 1
+    ; CHECK-NOT: OR8
+    %x4 = ADDI8 killed %x3, 1
+    %x3 = OR8 %x4, killed %x4
+
+...
+

test/CodeGen/PowerPC/redundant_regcopy_2.mir

+# test for BB-local register copy elimination
+# RUN: llc -run-pass ppc-postra-peepholes -verify-machineinstrs -o - %s | FileCheck %s
+
+--- |
+  target datalayout = "E-m:e-i64:64-n32:64"
+  target triple = "powerpc64le-unknown-linux-gnu"
+  define i8* @func(i8* %a, i1 %b) {
+  entry:
+    br i1 %b, label %exit, label %foo
+  
+  foo:                                              ; preds = %entry
+    br label %exit
+  
+  exit:                                             ; preds = %foo, %entry
+    ret i8* %a
+  }
+  
+...
+---
+name:            func
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+tracksRegLiveness: true
+liveins:
+  - { reg: '%x3' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       0
+  offsetAdjustment: 0
+  maxAlignment:    0
+  adjustsStack:    false
+  hasCalls:        false
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+body:             |
+  bb.0.entry:
+    successors: %bb.2.exit(0x40000000), %bb.1.foo(0x40000000)
+    liveins: %x3
+
+    ; CHECK-LABEL: bb.0.entry:
+    ; CHECK-NOT: OR8
+    ; CHECK: %cr0 = CMPLDI %x3, 0
+    %x4 = OR8 %x3, killed %x3
+    %cr0 = CMPLDI %x4, 0
+    BCC 76, killed %cr0, %bb.2.exit
+    B %bb.1.foo
+    
+  bb.1.foo:
+    liveins: %x4
+
+    ; CHECK-LABEL: bb.1.foo:
+    ; CHECK: %x4 = OR8 %x3, killed %x3
+    ; CHECK: %x4 = ADDI8 %x4, 123
+    ; CHECK: %x3 = OR8 %x4, killed %x4
+    %x4 = ADDI8 %x4, 123
+    B %bb.2.exit
+
+  bb.2.exit:
+    liveins: %x4
+    ; CHECK-LABEL: bb.2.exit:
+    ; CHECK-NOT: OR8
+    ; CHECK: BLR8
+    %x3 = OR8 %x4, killed %x4
+    BLR8 implicit %lr8, implicit %rm
+
+...
+