[AMDGPU] Strengthen export cluster ordering

[critson]

Summary

When removing barrier edges on exports then dependencies need to
be propagated.

[foad]

This is OK, and I can confirm it fixes some Vulkan CTS failures we were seeing.

For bonus marks:

1. removeExportDependencies iterates the preds of every node. I think it could be a lot faster if it only iterated the preds and succs of export nodes.
2. I still wonder if we could get the barriers right in the first place, perhaps by pretending that each kind of export instruction writes to a fake address space and teaching alias analysis that those address spaces are disjoint.

[critson]

Add test case.

[critson]

Address (1) by only processing on export nodes.
Since there is no way to remove a successor directly, use the
existing removeExportDependencies code and simply apply it to
successors.

[foad]

In D79855#2033855, @critson wrote:

Address (1) by only processing on export nodes.
Since there is no way to remove a successor directly, use the
existing removeExportDependencies code and simply apply it to
successors.

Thanks. For a future cleanup it should be pretty easy to implement SUnit::removeSucc.

llvm/lib/Target/AMDGPU/AMDGPUExportClustering.cpp

for (unsigned Idx = 0, End = Exports.size() - 1; Idx < End; ++Idx) {
 
     // New barrier edge ordering exports
     DAG->addEdge(SUb, SDep(SUa, SDep::Barrier));
     // Also add cluster edge
     DAG->addEdge(SUb, SDep(SUa, SDep::Cluster));
   }
 }
 
+static void removeExportDependencies(ScheduleDAGInstrs *DAG, SUnit &SU) {
+  SmallVector<SDep, 2> ToAdd, ToRemove;
+
+  for (const SDep &Pred : SU.Preds) {
+    SUnit *PredSU = Pred.getSUnit();
+    if (Pred.isBarrier() && isExport(*PredSU)) {
+      ToRemove.push_back(Pred);
+      if (isExport(SU))
+        continue;
+
+      // If we remove a barrier we need to copy dependencies
+      // from the predecessor to maintain order.
+      for (const SDep &ExportPred : PredSU->Preds) {
+        SUnit *ExportPredSU = ExportPred.getSUnit();
+        if (ExportPred.isBarrier() && !isExport(*ExportPredSU))
+          ToAdd.push_back(SDep(ExportPredSU, SDep::Barrier));
+      }
+    }
+  }
+
+  for (SDep Pred : ToRemove)
+    SU.removePred(Pred);
+  for (SDep Pred : ToAdd)
+    DAG->addEdge(&SU, Pred);
+}
+
 void ExportClustering::apply(ScheduleDAGInstrs *DAG) {
   const SIInstrInfo *TII = static_cast<const SIInstrInfo *>(DAG->TII);
 
   SmallVector<SUnit *, 8> Chain;
 
   // Pass through DAG gathering a list of exports and removing barrier edges
   // creating dependencies on exports. Freeing exports of successor edges
   // allows more scheduling freedom, and nothing should be order dependent
   // on exports.  Edges will be added later to order the exports.
   unsigned PosCount = 0;
   for (SUnit &SU : DAG->SUnits) {
     if (isExport(SU)) {
       Chain.push_back(&SU);
       if (isPositionExport(TII, &SU))
         PosCount++;
     }
-
-    SmallVector<SDep, 2> ToRemove;
-    for (const SDep &Pred : SU.Preds) {
-      SUnit *PredSU = Pred.getSUnit();
-      if (Pred.isBarrier() && isExport(*PredSU))
-        ToRemove.push_back(Pred);
-    }
-    for (SDep Pred : ToRemove)
-      SU.removePred(Pred);
+    removeExportDependencies(DAG, SU);
   }
 
   // Apply clustering if there are multiple exports
   if (Chain.size() > 1) {
     sortChain(TII, Chain, PosCount);
     buildCluster(Chain, DAG);
   }
 }