[MachineSink] Clear kill flags on operands outside loop

[critson]

Summary

If an instruction is sunk into a loop then any kill flags on
operands declared outside the loop must be cleared as these
will be live for all loop iterations.

Fixes #46827

[fhahn]

Could the test case by just a MIR test case that only runs MachineSink?

[MatzeB]

llvm/lib/CodeGen/MachineSink.cpp

1496–1504

What about this code that already exists, is it not working for this example? If so, can it be fixed?

[critson]

• Address reviewer comments
• Change to MIR test

[critson]

In D126754#3560211, @fhahn wrote:

Could the test case by just a MIR test case that only runs MachineSink?

Yes, I'll do this.

llvm/lib/CodeGen/MachineSink.cpp

1496–1504

RegsToClearKillFlags is only populated by SinkInstruction, the code path for SinkIntoCycle is different.
However I agree this should generally use the same methodology for clearing the flags, so I have changed the code to populate RegsToClearKillFlags as part of SinkIntoCycle instead of what I initially wrote.

[MatzeB]

LGTM

[MatzeB]

llvm/lib/CodeGen/MachineSink.cpp

1294

Change this to MO.readsReg() As we don't need to deal with things like undef uses or internal bundle reads.

[critson]

• readsReg() instead of isUse()

[critson]

• Rebase before commit

llvm/lib/CodeGen/MachineSink.cpp

LLVM_DEBUG(
         dbgs() << "CycleSink: Not Sinking, block too large to analyse.\n");
     return false;
   }
 
   LLVM_DEBUG(dbgs() << "CycleSink: Sinking instruction!\n");
   SinkBlock->splice(SinkBlock->SkipPHIsAndLabels(SinkBlock->begin()), Preheader,
                     I);
 
+  // Conservatively clear any kill flags on uses of sunk instruction
+  for (MachineOperand &MO : I.operands()) {
+    if (MO.isReg() && MO.readsReg())
+      RegsToClearKillFlags.insert(MO.getReg());
+  }
+
   // The instruction is moved from its basic block, so do not retain the
   // debug information.
   assert(!I.isDebugInstr() && "Should not sink debug inst");
   I.setDebugLoc(DebugLoc());
   return true;
 }
 
 /// Return true if a target defined block prologue instruction interferes
bool MachineSinking::SinkInstruction(MachineInstr &MI, bool &SawStore,
   // After sinking, some debug users may not be dominated any more. If possible,
   // copy-propagate their operands. As it's expensive, don't do this if there's
   // no debuginfo in the program.
   if (MI.getMF()->getFunction().getSubprogram() && MI.isCopy())
     SalvageUnsunkDebugUsersOfCopy(MI, SuccToSinkTo);
 
   performSink(MI, *SuccToSinkTo, InsertPos, DbgUsersToSink);
 
   // Conservatively, clear any kill flags, since it's possible that they are no
   // longer correct.
   // Note that we have to clear the kill flags for any register this instruction
   // uses as we may sink over another instruction which currently kills the
   // used registers.
   for (MachineOperand &MO : MI.operands()) {
     if (MO.isReg() && MO.isUse())
       RegsToClearKillFlags.insert(MO.getReg()); // Remember to clear kill flags.
   }
 
   return true;
 }
 
 void MachineSinking::SalvageUnsunkDebugUsersOfCopy(
     MachineInstr &MI, MachineBasicBlock *TargetBlock) {
   assert(MI.isCopy());
   assert(MI.getOperand(1).isReg());

llvm/test/CodeGen/SystemZ/machinelicm-sunk-kill-flags.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=s390x-linux-gnu -sink-insts-to-avoid-spills -verify-machineinstrs -run-pass=machine-sink -o - %s | FileCheck %s
+
+# Test kill flags are cleared after sinking operations into cycle during MachineLICM.
+
+--- |
+  target datalayout = "E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-a:8:16-n32:64"
+  target triple = "s390x-unknown-linux-gnu"
+
+  %0 = type <{ i32, [3 x i8] }>
+
+  @b = external dso_local global [1 x %0], align 2
+
+  define dso_local void @c() local_unnamed_addr {
+    ret void
+  }
+
+...
+---
+name:            c
+tracksRegLiveness: true
+body:             |
+  ; CHECK-LABEL: name: c
+  ; CHECK: bb.0:
+  ; CHECK-NEXT:   successors: %bb.1(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[LARL:%[0-9]+]]:addr64bit = LARL @b
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.1:
+  ; CHECK-NEXT:   successors: %bb.1(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[LA:%[0-9]+]]:gr64bit = LA [[LARL]], 49, $noreg
+  ; CHECK-NEXT:   [[LGHI:%[0-9]+]]:gr64bit = LGHI 7
+  ; CHECK-NEXT:   [[DEF:%[0-9]+]]:gr64bit = IMPLICIT_DEF
+  ; CHECK-NEXT:   [[DEF1:%[0-9]+]]:gr64bit = IMPLICIT_DEF
+  ; CHECK-NEXT:   ADJCALLSTACKDOWN 0, 0
+  ; CHECK-NEXT:   $r2d = COPY [[DEF]]
+  ; CHECK-NEXT:   $r3d = COPY [[LA]]
+  ; CHECK-NEXT:   $r4d = COPY [[LGHI]]
+  ; CHECK-NEXT:   CallBRASL &memcpy, $r2d, $r3d, $r4d, csr_systemz_elf, implicit-def dead $r14d, implicit-def dead $cc, implicit $fpc, implicit-def $r2d
+  ; CHECK-NEXT:   ADJCALLSTACKUP 0, 0
+  ; CHECK-NEXT:   ADJCALLSTACKDOWN 0, 0
+  ; CHECK-NEXT:   $r2d = COPY [[DEF1]]
+  ; CHECK-NEXT:   $r3d = COPY [[LA]]
+  ; CHECK-NEXT:   $r4d = COPY [[LGHI]]
+  ; CHECK-NEXT:   CallBRASL &memcpy, $r2d, $r3d, $r4d, csr_systemz_elf, implicit-def dead $r14d, implicit-def dead $cc, implicit $fpc, implicit-def $r2d
+  ; CHECK-NEXT:   ADJCALLSTACKUP 0, 0
+  ; CHECK-NEXT:   J %bb.1
+  bb.0:
+    successors: %bb.1(0x80000000)
+
+    %1:addr64bit = LARL @b
+    %0:gr64bit = LA killed %1, 49, $noreg
+    %2:gr64bit = LGHI 7
+    %3:gr64bit = IMPLICIT_DEF
+    %5:gr64bit = IMPLICIT_DEF
+
+  bb.1:
+    successors: %bb.1(0x80000000)
+
+    ADJCALLSTACKDOWN 0, 0
+    $r2d = COPY %3
+    $r3d = COPY %0
+    $r4d = COPY %2
+    CallBRASL &memcpy, $r2d, $r3d, $r4d, csr_systemz_elf, implicit-def dead $r14d, implicit-def dead $cc, implicit $fpc, implicit-def $r2d
+    ADJCALLSTACKUP 0, 0
+    ADJCALLSTACKDOWN 0, 0
+    $r2d = COPY %5
+    $r3d = COPY %0
+    $r4d = COPY %2
+    CallBRASL &memcpy, $r2d, $r3d, $r4d, csr_systemz_elf, implicit-def dead $r14d, implicit-def dead $cc, implicit $fpc, implicit-def $r2d
+    ADJCALLSTACKUP 0, 0
+    J %bb.1
+
+...