[NFC] Add the infrastructure to forward the scheduled state between MBB

[qiucf]

Summary

Forward the scheduled state from the single pred MBB, as they are belong to the the same window from pipeline view.

[lei]

llvm/include/llvm/CodeGen/MachineScheduler.h

228

Do you mean Return true if scheduled state is saved successfully before leaving the MBB?

234

Does pred mean predecessor? I think it is best to use the full word for code documentation.

234

Maybe this is more clear?

/// Return predecessor if MBB have only a single predecessor.

[fhahn]

llvm/include/llvm/CodeGen/MachineScheduler.h

304

IIUC this would drop the way to specify this via the scheduling policy and could change behavior for existing policies around.

It might make sense to have state forwarding imply top down here (SchedImpl->doMBBSchedRegionsTopDown() || MF.getSubtarget().forwardScheduledState())

llvm/include/llvm/CodeGen/TargetSubtargetInfo.h

197

Is there a benefit of putting this setting in here, rather than in the scheduling policy, like similar settings? Unless I am missing something, I think it would be more suitable in the scheduling policy, where we already have similar options.

[steven.zhang]

Address reviewer's comments. Thank you!

llvm/include/llvm/CodeGen/MachineScheduler.h

234

Thank you for the comments. I will update it.

234

Yes, it is. I will use the full item.

304

ok.

llvm/include/llvm/CodeGen/TargetSubtargetInfo.h

197

Sounds good.

[jonpa]

please rebase

llvm/include/llvm/CodeGen/MachineScheduler.h

public:
   /// initializing this strategy. Called after initPolicy.
   virtual bool shouldTrackPressure() const { return true; }
 
   /// Returns true if lanemasks should be tracked. LaneMask tracking is
   /// necessary to reorder independent subregister defs for the same vreg.
   /// This has to be enabled in combination with shouldTrackPressure().
   virtual bool shouldTrackLaneMasks() const { return false; }
 
-  // If this method returns true, handling of the scheduling regions
-  // themselves (in case of a scheduling boundary in MBB) will be done
-  // beginning with the topmost region of MBB.
-  virtual bool doMBBSchedRegionsTopDown() const { return false; }
-
   /// Initialize the strategy after building the DAG for a new region.
   virtual void initialize(ScheduleDAGMI *DAG) = 0;
 
   /// Tell the strategy that MBB is about to be processed.
   virtual void enterMBB(MachineBasicBlock *MBB) {};
 
   /// Tell the strategy that current MBB is done.
   virtual void leaveMBB() {};
 
+  /// Return true if save the scheduled state before leaving the MBB
+  /// successfully.
+  virtual bool saveScheduledState(MachineBasicBlock *MBB) { return false; }
+
+  /// Return true if restore the scheduled state from pred MBB successfully.
+  virtual bool restoreScheduledState(MachineBasicBlock *MBB) { return false; }
+
+  /// Return the single pred MBB if it has.
+  virtual MachineBasicBlock *getScheduledPredMBB(MachineBasicBlock *MBB);
+
   /// Notify this strategy that all roots have been released (including those
   /// that depend on EntrySU or ExitSU).
   virtual void registerRoots() {}
 
   /// Pick the next node to schedule, or return NULL. Set IsTopNode to true to
   /// schedule the node at the top of the unscheduled region. Otherwise it will
   /// be scheduled at the bottom.
   virtual SUnit *pickNode(bool &IsTopNode) = 0;
public:
 
   // Provide a vtable anchor
   ~ScheduleDAGMI() override;
 
   /// If this method returns true, handling of the scheduling regions
   /// themselves (in case of a scheduling boundary in MBB) will be done
   /// beginning with the topmost region of MBB.
   bool doMBBSchedRegionsTopDown() const override {
-    return SchedImpl->doMBBSchedRegionsTopDown();
+    return MF.getSubtarget().forwardScheduledState();
   }
 
   // Returns LiveIntervals instance for use in DAG mutators and such.
   LiveIntervals *getLIS() const { return LIS; }
 
   /// Return true if this DAG supports VReg liveness and RegPressure.
   virtual bool hasVRegLiveness() const { return false; }
 

llvm/include/llvm/CodeGen/TargetSubtargetInfo.h

public:
   /// scheduler (though see below for an option to turn this off and use the
   /// TargetLowering preference). It does not yet disable the postRA scheduler.
   virtual bool enableMachineScheduler() const;
 
   /// True if the machine scheduler should disable the TLI preference
   /// for preRA scheduling with the source level scheduler.
   virtual bool enableMachineSchedDefaultSched() const { return true; }
 
+  /// Return true if scheduled state is forwarded between MBBs. 
+  virtual bool forwardScheduledState() const { return false; }
+
   /// True if the subtarget should enable joining global copies.
   ///
   /// By default this is enabled if the machine scheduler is enabled, but
   /// can be overridden.
   virtual bool enableJoinGlobalCopies() const;
 
   /// True if the subtarget should run a scheduler after register allocation.
   ///

llvm/lib/CodeGen/MachineScheduler.cpp

 
 #include "llvm/CodeGen/MachineScheduler.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/BitVector.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/PriorityQueue.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/CodeGen/LiveInterval.h"
 #include "llvm/CodeGen/LiveIntervals.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
 #include "llvm/CodeGen/MachineDominators.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"

 #include <tuple>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "machine-scheduler"
 
+STATISTIC(NumFwdSchedState, "Number of scheduled state forwarded");
+
 namespace llvm {
 
 cl::opt<bool> ForceTopDown("misched-topdown", cl::Hidden,
                            cl::desc("Force top-down list scheduling"));
 cl::opt<bool> ForceBottomUp("misched-bottomup", cl::Hidden,
                             cl::desc("Force bottom-up list scheduling"));
 cl::opt<bool>
 DumpCriticalPathLength("misched-dcpl", cl::Hidden,

 MachineSchedContext::MachineSchedContext() {
   RegClassInfo = new RegisterClassInfo();
 }
 
 MachineSchedContext::~MachineSchedContext() {
   delete RegClassInfo;
 }
 
+MachineBasicBlock *MachineSchedStrategy::getScheduledPredMBB(
+  MachineBasicBlock *MBB) {
+  return MBB->pred_size() == 1 ? *MBB->pred_begin() : nullptr;
+}
+
 namespace {
 
 /// Base class for a machine scheduler class that can run at any point.
 class MachineSchedulerBase : public MachineSchedContext,
                              public MachineFunctionPass {
 public:
   MachineSchedulerBase(char &ID): MachineFunctionPass(ID) {}
 

 void ScheduleDAGMI::releasePredecessors(SUnit *SU) {
   for (SDep &Pred : SU->Preds)
     releasePred(SU, &Pred);
 }
 
 void ScheduleDAGMI::startBlock(MachineBasicBlock *bb) {
   ScheduleDAGInstrs::startBlock(bb);
   SchedImpl->enterMBB(bb);
+
+  // If current BB has single pred MBB, forward its scheduled
+  // state to current MBB.
+  if (MF.getSubtarget().forwardScheduledState()) {
+    MachineBasicBlock *PredMBB = SchedImpl->getScheduledPredMBB(bb);
+    if (PredMBB && SchedImpl->restoreScheduledState(PredMBB)) {
+      NumFwdSchedState ++;
+      LLVM_DEBUG(dbgs() << "** Initialize scheduled state from "
+                 << printMBBReference(*PredMBB) << "\n";);
+    }
+  }
 }
 
 void ScheduleDAGMI::finishBlock() {
+  if (MF.getSubtarget().forwardScheduledState())
+    SchedImpl->saveScheduledState(BB);
   SchedImpl->leaveMBB();
   ScheduleDAGInstrs::finishBlock();
 }
 
 /// enterRegion - Called back from MachineScheduler::runOnMachineFunction after
 /// crossing a scheduling boundary. [begin, end) includes all instructions in
 /// the region, including the boundary itself and single-instruction regions
 /// that don't get scheduled.

llvm/lib/Target/SystemZ/SystemZMachineScheduler.h

public:
   /// Called for a region before scheduling.
   void initPolicy(MachineBasicBlock::iterator Begin,
                   MachineBasicBlock::iterator End,
                   unsigned NumRegionInstrs) override;
 
   /// PostRA scheduling does not track pressure.
   bool shouldTrackPressure() const override { return false; }
 
-  // Process scheduling regions top-down so that scheduler states can be
-  // transferrred over scheduling boundaries.
-  bool doMBBSchedRegionsTopDown() const override { return true; }
-
   void initialize(ScheduleDAGMI *dag) override;
 
   /// Tell the strategy that MBB is about to be processed.
   void enterMBB(MachineBasicBlock *NextMBB) override;
 
   /// Tell the strategy that current MBB is done.
   void leaveMBB() override;
 

llvm/lib/Target/SystemZ/SystemZSubtarget.h

const SelectionDAGTargetInfo *getSelectionDAGInfo() const override {
     return &TSInfo;
   }
 
   // True if the subtarget should run MachineScheduler after aggressive
   // coalescing. This currently replaces the SelectionDAG scheduler with the
   // "source" order scheduler.
   bool enableMachineScheduler() const override { return true; }
 
+  // Copy & continue the scheduler states over scheduling boundaries.
+  bool forwardScheduledState() const override { return true; }
+
   // This is important for reducing register pressure in vector code.
   bool useAA() const override { return true; }
 
   // Always enable the early if-conversion pass.
   bool enableEarlyIfConversion() const override { return true; }
 
   // Enable tracking of subregister liveness in register allocator.
   bool enableSubRegLiveness() const override;