[AMDGPU] Skip unclusterd rescheduling w/o ld/st

[rampitec]

Summary

We are attempting rescheduling without load store clustering
if occupancy limits were not met with clustering. Skip this
for regions which do not have any loads or stores at all.

In a set of kernels I am experimenting with this improves
scheduling time by ~30%.

[rampitec]

Check for actual clusters in the region. This is a more precise method adding a bit more of speed.

[arsenm]

llvm/lib/Target/AMDGPU/GCNSchedStrategy.h

53 ↗

(On Diff #326473)

Needs comment

[rampitec]

Added comment.

[vpykhtin]

Looks good, but should we use just a single dedicated pass over SUs to check if there're clustered ops after first scheduling to make the logic slightly easier?

[rampitec]

In D97342#2590002, @vpykhtin wrote:

Looks good, but should we use just a single dedicated pass over SUs to check if there're clustered ops after first scheduling to make the logic slightly easier?

My problem with that it has to be done in the schedule() method or somewhere else within GCNScheduleDAGMILive. The only way to get an SUnit there is to call getSUnit() passing a MachineInstr and that is a map lookup. I.e. it is simply slower and I am trying to squeeze as much speed as I could.

[rampitec]

In D97342#2590737, @rampitec wrote:

In D97342#2590002, @vpykhtin wrote:

Looks good, but should we use just a single dedicated pass over SUs to check if there're clustered ops after first scheduling to make the logic slightly easier?

My problem with that it has to be done in the schedule() method or somewhere else within GCNScheduleDAGMILive. The only way to get an SUnit there is to call getSUnit() passing a MachineInstr and that is a map lookup. I.e. it is simply slower and I am trying to squeeze as much speed as I could.

Actually it is not even possible. The place where I can do it does not have mutations applied yet.

[rampitec]

In D97342#2590876, @rampitec wrote:

In D97342#2590737, @rampitec wrote:

In D97342#2590002, @vpykhtin wrote:

Looks good, but should we use just a single dedicated pass over SUs to check if there're clustered ops after first scheduling to make the logic slightly easier?

My problem with that it has to be done in the schedule() method or somewhere else within GCNScheduleDAGMILive. The only way to get an SUnit there is to call getSUnit() passing a MachineInstr and that is a map lookup. I.e. it is simply slower and I am trying to squeeze as much speed as I could.

Actually it is not even possible. The place where I can do it does not have mutations applied yet.

In fact I did the experiment, it may look better but I've got 3.6% slower scheduling with that separate loop.

[vpykhtin]

LGTM

llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp

void GCNScheduleDAGMILive::schedule() {
   if (Stage == Collect) {
     // Just record regions at the first pass.
     Regions.push_back(std::make_pair(RegionBegin, RegionEnd));
     return;
   }
 
   std::vector<MachineInstr*> Unsched;
   Unsched.reserve(NumRegionInstrs);
+  bool SeenLdSt = false;
   for (auto &I : *this) {
+    SeenLdSt |= I.mayLoadOrStore();
     Unsched.push_back(&I);
   }
 
   GCNRegPressure PressureBefore;
   if (LIS) {
     PressureBefore = Pressure[RegionIdx];
 
     LLVM_DEBUG(dbgs() << "Pressure before scheduling:\nRegion live-ins:";
void GCNScheduleDAGMILive::schedule() {
   if (WavesAfter >= MinOccupancy) {
     if (Stage == UnclusteredReschedule &&
         !PressureAfter.less(ST, PressureBefore)) {
       LLVM_DEBUG(dbgs() << "Unclustered reschedule did not help.\n");
     } else if (WavesAfter > MFI.getMinWavesPerEU() ||
         PressureAfter.less(ST, PressureBefore) ||
         !RescheduleRegions[RegionIdx]) {
       Pressure[RegionIdx] = PressureAfter;
+      if (!SeenLdSt && (Stage + 1) == UnclusteredReschedule)
+        RescheduleRegions[RegionIdx] = false;
       return;
     } else {
       LLVM_DEBUG(dbgs() << "New pressure will result in more spilling.\n");
     }
   }
 
   LLVM_DEBUG(dbgs() << "Attempting to revert scheduling.\n");
-  RescheduleRegions[RegionIdx] = true;
+  RescheduleRegions[RegionIdx] = SeenLdSt ||
+                                 (Stage + 1) != UnclusteredReschedule;
   RegionEnd = RegionBegin;
   for (MachineInstr *MI : Unsched) {
     if (MI->isDebugInstr())
       continue;
 
     if (MI->getIterator() != RegionEnd) {
       BB->remove(MI);
       BB->insert(RegionEnd, MI);