[ModuloSchedule] Fix terminator update when peeling.

[hgreving]

Summary

Fixes a bug of us not correctly updating the terminator of the loop's
preheader, if multiple terminating branch instructions are present.

This is tested through existing tests. The bug itself is hard or not
possible to get exposed with the upstream Hexagon backend, because
the machine pipeliner checks for an existing preheader, which is
defined as a block with only 1 edge into the header.

The condition of this bug is a block into the loop with more than 1
edge, and not every downstream target checks for an existing preheader.

[hgreving]

There are a few lit failures, I will look at those first.

[hgreving]

Updates patch. Let's do it straight forward and support cases where no branch is needed to the fallthrough.

[hgreving]

In D126386#3537423, @hgreving wrote:

There are a few lit failures, I will look at those first.

Fixed.

[hgreving]

Added more reviewers with history of the modulo expansion code, since I believe there is no feasible test for upstream targets, see commit message.

llvm/lib/CodeGen/MachineLoopUtils.cpp

if (Direction == LPD_Front) {
       MI.getOperand(LoopRegIdx).setReg(LoopReg);
       MI.removeOperand(InitRegIdx + 1);
       MI.removeOperand(InitRegIdx + 0);
     }
   }
 
   DebugLoc DL;
   if (Direction == LPD_Front) {
-    Preheader->replaceSuccessor(Loop, NewBB);
+    Preheader->ReplaceUsesOfBlockWith(Loop, NewBB);
     NewBB->addSuccessor(Loop);
     Loop->replacePhiUsesWith(Preheader, NewBB);
-    if (TII->removeBranch(*Preheader) > 0)
-      TII->insertBranch(*Preheader, NewBB, nullptr, {}, DL);
+    Preheader->updateTerminator(Loop);
     TII->removeBranch(*NewBB);
     TII->insertBranch(*NewBB, Loop, nullptr, {}, DL);
   } else {
     Loop->replaceSuccessor(Exit, NewBB);
     Exit->replacePhiUsesWith(Loop, NewBB);
     NewBB->addSuccessor(Exit);
 
     MachineBasicBlock *TBB = nullptr, *FBB = nullptr;

llvm/lib/CodeGen/ModuloSchedule.cpp

void PeelingModuloScheduleExpander::peelPrologAndEpilogs() {
   // Create a list of all blocks in order.
   SmallVector<MachineBasicBlock *, 8> Blocks;
   llvm::copy(PeeledFront, std::back_inserter(Blocks));
   Blocks.push_back(BB);
   llvm::copy(PeeledBack, std::back_inserter(Blocks));
 
   // Iterate in reverse order over all instructions, remapping as we go.
   for (MachineBasicBlock *B : reverse(Blocks)) {
-    for (auto I = B->getFirstInstrTerminator()->getReverseIterator();
+    for (auto I = B->instr_rbegin();
          I != std::next(B->getFirstNonPHI()->getReverseIterator());) {
-      MachineInstr *MI = &*I++;
-      rewriteUsesOf(MI);
+      MachineBasicBlock::reverse_instr_iterator MI = I++;
+      rewriteUsesOf(&*MI);
     }
   }
   for (auto *MI : IllegalPhisToDelete) {
     if (LIS)
       LIS->RemoveMachineInstrFromMaps(*MI);
     MI->eraseFromParent();
   }
   IllegalPhisToDelete.clear();