[DebugInfo][InstrRef] Track instructions that write-to-stack after having a spill fused into them

[jmorse]

Summary

During register allocation, some instructions can have stack spills fused into them. It means that when vregs are allocated on the stack we can convert:

SETCCr %0
DBG_VALUE $0

to

SETCCm %stack.0
DBG_VALUE %stack.0

for example. DBG_VALUE instructions can simply have their vreg operands replaced with a stack location, and it all Just Works (TM). Unfortunately instruction referencing finds this harder: a store to the stack doesn't have a specific operand number, therefore we don't substitute the old operand for a new operand, and the location is dropped.

This patch implements a solution: just recognise the memory operand attached to an instruction with a Special Number (TM), and record a substitution between the old value and the new one. Thus we would have:

SETCCr %0 [...] debug-instr-number 1 :: (store (s32) into %stack.0)
DBG_INSTR_REF 1, 0

become

SETCCm %stack.0 [...] debug-instr-number 2 :: (store (s32) into %stack.0)
DBG_INSTR_REF 1, 0

With a substitution from operand (1, 0) to (2, MemoryOperand). In LiveDebugValues, we can identify the stack slot written to just like a spill instruction, and track its value from that point onwards.

This patch installs the substitution code into InlineSpiller: it's not that interesting, just adds a substitution. It's not completely clear to me what all the circumstances that the spiller runs with are, so I've filtered to only substitute in common cases, see the two MIR tests added.

In InstrRefBasedLDV, we now:

• Recognise instructions that write to stack slots as def'ing that stack slot,
• Allow DBG_INSTR_REFs that refer to a memory operand to "read" the value defined by that instruction, in the stack slot.

[jmorse]

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h

911

Should be !isAliased lost the inversion in copy+paste, happily the test catches this.

[djtodoro]

(from the commit message)

`becomes

SETCCm %stack.0 [...] debug-instr-number 2 :: (store (s32) into %stack.0)
DBG_INSTR_REF 1, 0

`
this actually becomes DBG_INSTR_REF 2, 0, right?

[jmorse]

In D111317#3057344, @djtodoro wrote:

this actually becomes DBG_INSTR_REF 2, 0, right?

Actually no -- when optimisations are occurring with instruction referencing, I'm recording them in the "debugValueSubstitutions" table, which maps "old" instruction numbers onto new. Take a look at the MIR in memory-operand-tracking.mir which is added by this patch; there's a substitution from "2, 0" to "3, 1000000" (the large number signifying the memory operand).

Because instruction referencing gets rid of virtual register references, there's no efficient way to find the debug instructions that use a particular instruction number; perhaps we could make that happen, but I don't think we would gain anything from it. That's why I'm storing records of substitutions in a side-table, they're used to "patch up" the DBG_INSTR_REFs at the end of compilation.

(One downside of this approach is it adds extra steps for humans interpreting the debug-info, because they might have to interpret the substitutions; we could "patch up" DBG_INSTR_REFs as part of the printing process if that becomes a burden).

[StephenTozer]

Aside from the current comments, LGTM.

llvm/lib/CodeGen/InlineSpiller.cpp

935

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

1321

Just out of curiosity, does this represent a bug prior to this patch, or is this just needed for the new functionality?

llvm/lib/CodeGen/MachineFunction.cpp

976

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-folding-tieddef.mir

37

Nit, the unnecessary metadata (some of the long directories and version strings) can be removed.

[jmorse]

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

1321

I don't thiiiiink so: I've added this as an act of conservativeness, in case the target hooks recognise any stack write as being a stack spill. I suspect this doesn't happen for x86; perhaps it happens for other architectures though.

[djtodoro]

In D111317#3057563, @jmorse wrote:

In D111317#3057344, @djtodoro wrote:

this actually becomes DBG_INSTR_REF 2, 0, right?

Actually no -- when optimisations are occurring with instruction referencing, I'm recording them in the "debugValueSubstitutions" table, which maps "old" instruction numbers onto new. Take a look at the MIR in memory-operand-tracking.mir which is added by this patch; there's a substitution from "2, 0" to "3, 1000000" (the large number signifying the memory operand).

Because instruction referencing gets rid of virtual register references, there's no efficient way to find the debug instructions that use a particular instruction number; perhaps we could make that happen, but I don't think we would gain anything from it. That's why I'm storing records of substitutions in a side-table, they're used to "patch up" the DBG_INSTR_REFs at the end of compilation.

(One downside of this approach is it adds extra steps for humans interpreting the debug-info, because they might have to interpret the substitutions; we could "patch up" DBG_INSTR_REFs as part of the printing process if that becomes a burden).

I see... Thanks for the explanation. Improving the MIR output could be a future-work, but I don't want to block anything here.

[jmorse]

Update for Stephens nits -- but also to rebase on top of the the track-subregisters-on-the-stack patch, D112133. That exposes an interesting circumstance, which probably wants a little more review.

Now that there are more "indexes" within a spill slot that we can refer to, if we have:

INC32m $rsp, 1, $noreg, 4, [...]debug-instr-number 1 :: (store (s32) into %stack.0)
DBG_INSTR_REF 1, 1000000

Where the DBG_INSTR_REF refers to the memory operand of the INC32m: which stack index should this refer to? It could be 8/16/32/64 bits large. This is not a pointless question, because if the value is loaded off the stack, we need to try and follow it in the correct register.

Happily we can look at the memory operand on the INC32m and determine it's size; that then tells us which index on the stack the DBG_INSTR_REF should refer to, which is what I've added in this revision, in findLocationForMemOperand.

[jmorse]

... and while I'm here, we should substitute instruction numbers when loads are folded into an instruction. See the added test memory-operand-load-folding.mir where the CVTTSS2SIrr (convert single precision number to signed int?) has a load folded into it. The instruction number should be preserved (with a substitution) for the sake of the value the instruction defines.

[Orlando]

LGTM, though I'm not sure I feel qualified to press the green button myself in this instance. I left a few inline nits, plus please make the clang-format bot happy.

llvm/include/llvm/CodeGen/MachineFunction.h

545

Can you use const/constexpr here so that you can include the definition here in the header? Or is there some reason that this isn't const?

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

1372–1377

This is repeated a few lines above, between 1442-1445 (patch juggling?)

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-tracking.mir

40

Is there a way to make the IR printer/reader to print/read MemOperand or something instead of 1000000 (if yes and you agree - this could be a separate patch)?

55–58

What is the purpose of this part of the test?

[jmorse]

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-tracking.mir

55–58

It's testing that the old value, DBG_PHI $rax, 1, can't be found anywhere. It's moderately important as the INC32m's write to the stack should terminate any earlier values being tracked.

Thinking about it, this test should also check that variables _currently_ located on the stack get terminated if it's overwritten, i.e.:

SOMEINST debug-instr-number 1 ::(store 8 into %stack.0)
DBG_INSTR_REF 1, 1000000
OTHERINST ::(store 8 into %stack.0)

needs an explicit DBG_VALUE $noreg after OTHERINST overwrites the stack locataion. DbgEntityHistoryCalculator isn't aware of memory stores.

55–58

(To uh, be clear, I'll fold better explanations into this).

[jmorse]

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

1372–1377

Rats, missed this, and you're right it's patch juggling. I'll whip it out in an NFC commit later.

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-tracking.mir

40

I think for it to be well-formed JSON, we'd need some kind of flag indicating whether it's a "normal" operand or a memory reference, then make dstop optional. While that's a pain, it's probably easier to read; will need to be a later patch though.

[thakis]

Looks like this breaks tests on arm macs: http://45.33.8.238/macm1/20568/step_11.txt (...or possibly on machines that default to non-intel triples in general?)

Please take a look!

[jmorse]

In D111317#3084642, @thakis wrote:

Looks like this breaks tests on arm macs: http://45.33.8.238/macm1/20568/step_11.txt (...or possibly on machines that default to non-intel triples in general?)

Please take a look!

Argh, yep, that's another "all the world's an x86" fault for me there, fix up in a moment.

[thakis]

Thanks for the fix!

llvm/include/llvm/CodeGen/MachineFunction.h

public:
   /// enough information for every DBG_INSTR_REF to point at an instruction
   /// (or DBG_PHI).
   void finalizeDebugInstrRefs();
 
   /// Returns true if the function's variable locations should be tracked with
   /// instruction referencing.
   bool useDebugInstrRef() const;
 
+  /// A reserved operand number representing the instructions memory operand,
+  /// for instructions that have a stack spill fused into them.
+  static unsigned int DebugOperandMemNumber;
+
   MachineFunction(Function &F, const LLVMTargetMachine &Target,
                   const TargetSubtargetInfo &STI, unsigned FunctionNum,
                   MachineModuleInfo &MMI);
   MachineFunction(const MachineFunction &) = delete;
   MachineFunction &operator=(const MachineFunction &) = delete;
   ~MachineFunction();
 
   /// Reset the instance as if it was just created.

llvm/lib/CodeGen/InlineSpiller.cpp

foldMemoryOperand(ArrayRef<std::pair<MachineInstr *, unsigned>> Ops,
   int FI;
   if (TII.isStoreToStackSlot(*MI, FI) &&
       HSpiller.rmFromMergeableSpills(*MI, FI))
     --NumSpills;
   LIS.ReplaceMachineInstrInMaps(*MI, *FoldMI);
   // Update the call site info.
   if (MI->isCandidateForCallSiteEntry())
     MI->getMF()->moveCallSiteInfo(MI, FoldMI);
+
+  // If we've folded a store into an instruction labelled with debug-info,
+  // record a substitution from the old operand to the memory operand. Handle
+  // the simple common case where operand 0 is the one being folded, plus when
+  // the destination operand is also a tied def. More values could bex
+  // substituted / preserved with more analysis.
+  if (MI->peekDebugInstrNum() && Ops[0].second == 0) {
+    // Helper lambda.
+    auto MakeSubstitution = [this,FoldMI,MI,&Ops]() {
+      // Substitute old operand zero to the new instructions memory operand.
+      unsigned OldOperandNum = Ops[0].second;
+      unsigned NewNum = FoldMI->getDebugInstrNum();
+      unsigned OldNum = MI->getDebugInstrNum();
+      MF.makeDebugValueSubstitution({OldNum, OldOperandNum},
+                         {NewNum, MachineFunction::DebugOperandMemNumber});
+    };
+
+    const MachineOperand &Op0 = MI->getOperand(Ops[0].second);
+    if (Ops.size() == 1 && Op0.isDef()) {
+      MakeSubstitution();
+    } else if (Ops.size() == 2 && Op0.isDef() && MI->getOperand(1).isTied() &&
+               Op0.getReg() == MI->getOperand(1).getReg()) {
+      MakeSubstitution();
+    }
+  }
+
   MI->eraseFromParent();
 
   // Insert any new instructions other than FoldMI into the LIS maps.
   assert(!MIS.empty() && "Unexpected empty span of instructions!");
   for (MachineInstr &MI : MIS)
     if (&MI != FoldMI)
       LIS.InsertMachineInstrInMaps(MI);
 

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h

 public:
   /// Default construct and initialize the pass.
   InstrRefBasedLDV();
 
   LLVM_DUMP_METHOD
   void dump_mloc_transfer(const MLocTransferMap &mloc_transfer) const;
 
   bool isCalleeSaved(LocIdx L) const;
+
+  bool hasFoldedStackStore(const MachineInstr &MI) {
+    // Instruction must have a memory operand that's a stack slot, and isn't
+    // aliased, meaning it's a spill from regalloc instead of a variable.
+    // If it's aliased, we can't guarantee its value.
+    if (!MI.hasOneMemOperand())
+      return false;
+    auto *MemOperand = *MI.memoperands_begin();
+    return MemOperand->isStore() &&
+           MemOperand->getPseudoValue() &&
+           MemOperand->getPseudoValue()->kind() == PseudoSourceValue::FixedStack
+           && MemOperand->getPseudoValue()->isAliased(MFI);
+  }
 };
 
 } // namespace LiveDebugValues
 
 
 #endif /* LLVM_LIB_CODEGEN_LIVEDEBUGVALUES_INSTRREFBASEDLDV_H */

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

bool InstrRefBasedLDV::transferDebugInstrRef(MachineInstr &MI,
   // that it defines. It could be an instruction, or a PHI.
   auto InstrIt = DebugInstrNumToInstr.find(InstNo);
   auto PHIIt = std::lower_bound(DebugPHINumToValue.begin(),
                                 DebugPHINumToValue.end(), InstNo);
   if (InstrIt != DebugInstrNumToInstr.end()) {
     const MachineInstr &TargetInstr = *InstrIt->second.first;
     uint64_t BlockNo = TargetInstr.getParent()->getNumber();
 
-    // Pick out the designated operand.
+    // Pick out the designated operand. It might be a memory reference, if
+    // a register def was folded into a stack store.
+    if (OpNo == MachineFunction::DebugOperandMemNumber &&
+        TargetInstr.hasOneMemOperand()) {
+      auto SpillLoc =  extractSpillBaseRegAndOffset(TargetInstr);
+      auto SpillLocIdx = MTracker->getSpillMLoc(SpillLoc);
+      if (SpillLocIdx)
+        NewID = ValueIDNum(BlockNo, InstrIt->second.second, *SpillLocIdx);
+    } else if (OpNo != MachineFunction::DebugOperandMemNumber) {
       assert(OpNo < TargetInstr.getNumOperands());
       const MachineOperand &MO = TargetInstr.getOperand(OpNo);
 
       // Today, this can only be a register.
       assert(MO.isReg() && MO.isDef());
 
       unsigned LocID = MTracker->getLocID(MO.getReg(), false);
       LocIdx L = MTracker->LocIDToLocIdx[LocID];
       NewID = ValueIDNum(BlockNo, InstrIt->second.second, L);
+    }
+    // else: NewID is left as None.
   } else if (PHIIt != DebugPHINumToValue.end() && PHIIt->InstrNum == InstNo) {
     // It's actually a PHI value. Which value it is might not be obvious, use
     // the resolver helper to find out.
     NewID = resolveDbgPHIs(*MI.getParent()->getParent(), MLiveOuts, MLiveIns,
                            MI, InstNo);
   }
 
   // Apply any subregister extractions, in reverse. We might have seen code
void InstrRefBasedLDV::transferRegisterDef(MachineInstr &MI) {
 
   // Tell MLocTracker about all definitions, of regmasks and otherwise.
   for (uint32_t DeadReg : DeadRegs)
     MTracker->defReg(DeadReg, CurBB, CurInst);
 
   for (auto *MO : RegMaskPtrs)
     MTracker->writeRegMask(MO, CurBB, CurInst);
 
+  // If this instruction writes to a spill slot, def that slot.
+  if (hasFoldedStackStore(MI)) {
+    SpillLoc Spill = extractSpillBaseRegAndOffset(MI);
+    LocIdx L = MTracker->getOrTrackSpillLoc(Spill);
+    MTracker->setSpill(Spill, ValueIDNum(CurBB, CurInst, L));
+  }
+
   if (!TTracker)
     return;
 
   // When committing variable values to locations: tell transfer tracker that
   // we've clobbered things. It may be able to recover the variable from a
   // different location.
 
   // Inform TTracker about any direct clobbers.
for (auto L : MTracker->locations()) {
     if (MTracker->isSpill(L.Idx))
       continue;
 
     Register Reg = MTracker->LocIdxToLocID[L.Idx];
     for (auto *MO : RegMaskPtrs)
       if (MO->clobbersPhysReg(Reg))
         TTracker->clobberMloc(L.Idx, MI.getIterator(), false);
   }
+
+  // Tell TTracker about any folded stack store.
+  if (hasFoldedStackStore(MI)) {
+    SpillLoc Spill = extractSpillBaseRegAndOffset(MI);
+    LocIdx L = MTracker->getOrTrackSpillLoc(Spill);
+    TTracker->clobberMloc(L, MI.getIterator(), true);
+  }
 }
 
 void InstrRefBasedLDV::performCopy(Register SrcRegNum, Register DstRegNum) {
   ValueIDNum SrcValue = MTracker->readReg(SrcRegNum);
 
   MTracker->setReg(DstRegNum, SrcValue);
 
   // In all circumstances, re-def the super registers. It's definitely a new

 }
 
 bool InstrRefBasedLDV::isSpillInstruction(const MachineInstr &MI,
                                           MachineFunction *MF) {
   // TODO: Handle multiple stores folded into one.
   if (!MI.hasOneMemOperand())
     return false;
 
+  // Reject any memory operand that's aliased -- we can't guarantee its value.
+  auto MMOI = MI.memoperands_begin();
+  const PseudoSourceValue *PVal = (*MMOI)->getPseudoValue();
+  if (PVal->isAliased(MFI))
+    return false;
+
   if (!MI.getSpillSize(TII) && !MI.getFoldedSpillSize(TII))
     return false; // This is not a spill instruction, since no valid size was
                   // returned from either function.
 
   return true;
 }
 
 bool InstrRefBasedLDV::isLocationSpill(const MachineInstr &MI,
if (EmulateOldLDV)
     return false;
 
   MachineFunction *MF = MI.getMF();
   unsigned Reg;
   Optional<SpillLoc> Loc;
 
   LLVM_DEBUG(dbgs() << "Examining instruction: "; MI.dump(););
 
+  // Strictly limit ourselves to plain loads and stores, not all instructions
+  // that can access the stack.
+  int FIDummy;
+  if (!TII->isStoreToStackSlotPostFE(MI, FIDummy) &&
+      !TII->isLoadFromStackSlotPostFE(MI, FIDummy))
+    return false;
+
   // First, if there are any DBG_VALUEs pointing at a spill slot that is
   // written to, terminate that variable location. The value in memory
   // will have changed. DbgEntityHistoryCalculator doesn't try to detect this.
   if (isSpillInstruction(MI, MF)) {
     Loc = extractSpillBaseRegAndOffset(MI);
 
     if (TTracker) {
       Optional<LocIdx> MLoc = MTracker->getSpillMLoc(*Loc);

llvm/lib/CodeGen/MachineFunction.cpp

void MachineFunction::setDebugInstrNumberingCount(unsigned Num) {
   DebugInstrNumberingCount = Num;
 }
 
 void MachineFunction::makeDebugValueSubstitution(DebugInstrOperandPair A,
                                                  DebugInstrOperandPair B,
                                                  unsigned Subreg) {
   // Catch any accidental self-loops.
   assert(A.first != B.first);
+  // Don't allow any substituitions _from_ the memory operand number.
+  assert(A.second != DebugOperandMemNumber);
+
   DebugValueSubstitutions.push_back({A, B, Subreg});
 }
 
 void MachineFunction::substituteDebugValuesForInst(const MachineInstr &Old,
                                                    MachineInstr &New,
                                                    unsigned MaxOperand) {
   // If the Old instruction wasn't tracked at all, there is no work to do.
   unsigned OldInstrNum = Old.peekDebugInstrNum();
if (F.hasFnAttribute(Attribute::OptimizeNone))
     return false;
 
   if (getTarget().Options.ValueTrackingVariableLocations)
     return true;
 
   return false;
 }
 
+// Use one million as a high / reserved number.
+unsigned MachineFunction::DebugOperandMemNumber = 1000000;
+
 /// \}
 
 //===----------------------------------------------------------------------===//
 //  MachineJumpTableInfo implementation
 //===----------------------------------------------------------------------===//
 
 /// Return the size of each entry in the jump table.
 unsigned MachineJumpTableInfo::getEntrySize(const DataLayout &TD) const {

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-folding-tieddef.mir

+# RUN: llc %s -o - -experimental-debug-variable-locations \
+# RUN:    -start-before=x86-flags-copy-lowering -stop-after=virtregrewriter \
+# RUN: | FileCheck %s
+#
+# This test is for stack spill folding -- the INC32r near the end of the MIR
+# below show be morphed into an INC32m by the register allocator, making it
+# load-operate-store to %stack.0. We should track this fact in the substitution
+# table, by adding a substitution to the memory-operand operand number.
+#
+# The INC32r is a tied-def instruction.
+#
+# NB: This test would more ideally start at phi-node-elimination, where
+# register allocation begins, however for some reason the INC32r gets
+# transformed into a different instruction if we do that. Start at the last
+# optimisation pass before regalloc instead.
+#
+# CHECK:      debugValueSubstitutions:
+# CHECK-NEXT:  - { srcinst: 1, srcop: 0, dstinst: 2, dstop: 1000000, subreg: 0 }
+# CHECK-LABEL: bb.5:
+# CHECK: INC32m %stack.0, {{.*}} debug-instr-number 2,
+--- |
+  ; ModuleID = 'reduced.ll'
+  source_filename = "reduced.ll"
+  target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+  
+  %"class.llvm::APInt" = type { i32, %union.anon }
+  %union.anon = type { i64 }
+  
+  define void @_ZNK4llvm5APInt5magicEv() local_unnamed_addr align 2 !dbg !7 {
+    ret void
+  }
+
+  !llvm.dbg.cu = !{!0}
+  !llvm.module.flags = !{!3, !4, !5}
+  !llvm.ident = !{!6}
+  
+  !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 12.0.0 ([e-mail]:llvm/llvm-project 79a35789efdf2378f97642ae4a5f3099b9087a11)", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, splitDebugInlining: false, nameTableKind: None)
+  !1 = !DIFile(filename: "test.c", directory: "/fast/fs/llvm3/llvm/test/DebugInfo/MIR/InstrRef")
+  !2 = !{}
+  !3 = !{i32 7, !"Dwarf Version", i32 4}
+  !4 = !{i32 2, !"Debug Info Version", i32 3}
+  !5 = !{i32 1, !"wchar_size", i32 4}
+  !6 = !{!"clang version 12.0.0 ([e-mail]:llvm/llvm-project 79a35789efdf2378f97642ae4a5f3099b9087a11)"}
+  !7 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 3, type: !8, scopeLine: 3, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+  !8 = !DISubroutineType(types: !9)
+  !9 = !{!10, !11, !11}
+  !10 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+  !11 = !DIBasicType(name: "short", size: 16, encoding: DW_ATE_signed)
+  !12 = !DILocalVariable(name: "bar", arg: 1, scope: !7, file: !1, line: 3, type: !11)
+  !13 = !DILocation(line: 0, scope: !7)
+  !14 = !DILocalVariable(name: "baz", arg: 2, scope: !7, file: !1, line: 3, type: !11)
+  !15 = distinct !DILexicalBlock(scope: !7, file: !1, line: 8, column: 7)
+
+...
+---
+name:            _ZNK4llvm5APInt5magicEv
+alignment:       16
+tracksRegLiveness: true
+registers:
+  - { id: 0, class: gr64 }
+  - { id: 1, class: gr32 }
+  - { id: 2, class: gr64 }
+  - { id: 3, class: gr32 }
+  - { id: 4, class: gr64 }
+  - { id: 5, class: gr32 }
+  - { id: 6, class: gr32 }
+  - { id: 7, class: gr32 }
+  - { id: 8, class: gr64 }
+  - { id: 9, class: gr32 }
+  - { id: 10, class: gr64 }
+  - { id: 11, class: gr32 }
+  - { id: 12, class: gr64 }
+  - { id: 13, class: gr32 }
+  - { id: 14, class: gr64 }
+  - { id: 15, class: gr32 }
+  - { id: 16, class: gr64 }
+  - { id: 17, class: gr64 }
+  - { id: 18, class: gr64 }
+  - { id: 19, class: gr32 }
+  - { id: 20, class: gr8 }
+  - { id: 21, class: gr32 }
+  - { id: 22, class: gr64 }
+  - { id: 23, class: gr64 }
+  - { id: 24, class: gr64 }
+  - { id: 25, class: gr64 }
+  - { id: 26, class: gr64 }
+  - { id: 27, class: gr32 }
+  - { id: 28, class: gr8 }
+  - { id: 29, class: gr64 }
+  - { id: 30, class: gr64 }
+  - { id: 31, class: gr64 }
+  - { id: 32, class: gr64 }
+  - { id: 33, class: gr64 }
+  - { id: 34, class: gr64 }
+  - { id: 35, class: gr64 }
+  - { id: 36, class: gr64 }
+  - { id: 37, class: gr64 }
+  - { id: 38, class: gr8 }
+frameInfo:
+  maxAlignment:    1
+  hasCalls:        true
+machineFunctionInfo: {}
+body:             |
+  bb.0:
+    %15:gr32 = IMPLICIT_DEF
+    %14:gr64 = IMPLICIT_DEF
+    %17:gr64 = IMPLICIT_DEF
+    %18:gr64 = IMPLICIT_DEF
+    %19:gr32 = MOV32r0 implicit-def dead $eflags
+    %22:gr64 = IMPLICIT_DEF
+    %23:gr64 = IMPLICIT_DEF
+    %24:gr64 = IMPLICIT_DEF
+    %25:gr64 = IMPLICIT_DEF
+    %26:gr64 = IMPLICIT_DEF
+    %30:gr64 = IMPLICIT_DEF
+    %31:gr64 = IMPLICIT_DEF
+    %32:gr64 = IMPLICIT_DEF
+    %33:gr64 = IMPLICIT_DEF
+    %34:gr64 = IMPLICIT_DEF
+    %35:gr64 = IMPLICIT_DEF
+    %36:gr64 = IMPLICIT_DEF
+    %37:gr64 = IMPLICIT_DEF
+    %21:gr32 = IMPLICIT_DEF
+  
+  bb.1:
+    %0:gr64 = PHI %14, %bb.0, %12, %bb.5
+    %1:gr32 = PHI %15, %bb.0, %11, %bb.5
+    %2:gr64 = PHI %14, %bb.0, %10, %bb.5
+    %3:gr32 = PHI %15, %bb.0, %9, %bb.5
+    %4:gr64 = PHI %14, %bb.0, %8, %bb.5
+    %5:gr32 = PHI %15, %bb.0, %7, %bb.5, debug-location !13
+    %6:gr32 = PHI %15, %bb.0, %13, %bb.5, debug-location !13
+    %16:gr64 = ADD64rr %4, %4, implicit-def dead $eflags, debug-location !13
+    MOV32mr %17, 1, $noreg, 0, $noreg, %5, debug-location !13 :: (store (s32) into `i32* undef`, align 8)
+    MOV64mr %18, 1, $noreg, 0, $noreg, killed %16, debug-location !13 :: (store (s64) into `i64* undef`)
+    %20:gr8 = COPY %19.sub_8bit
+    TEST8rr %20, %20, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.3, 5, implicit $eflags, debug-location !13
+    JMP_1 %bb.2, debug-location !13
+  
+  bb.2:
+  
+  bb.3:
+    successors: %bb.4, %bb.5
+  
+    %7:gr32 = PHI %5, %bb.1, %21, %bb.2, debug-location !13
+    MOV32mr %22, 1, $noreg, 0, $noreg, %7, debug-location !13 :: (store (s32) into `i32* undef`, align 8)
+    %8:gr64 = MOV64rm %23, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s64) from `i64* undef`)
+    MOV32mr %24, 1, $noreg, 0, $noreg, %3, debug-location !13 :: (store (s32) into `i32* undef`, align 8)
+    MOV64mi32 %25, 1, $noreg, 0, $noreg, 0, debug-location !13 :: (store (s64) into `i64* undef`)
+    %28:gr8 = COPY %19.sub_8bit
+    TEST8rr %28, %28, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.5, 5, implicit $eflags, debug-location !13
+    JMP_1 %bb.4, debug-location !13
+  
+  bb.4:
+    %29:gr64 = ADD64rr %2, %2, implicit-def dead $eflags, debug-location !13
+    MOV64mr %30, 1, $noreg, 0, $noreg, killed %29, debug-location !13 :: (store (s64) into `i64* undef`)
+  
+  bb.5:
+    %9:gr32 = MOV32rm %26, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s32) from `i32* undef`, align 8)
+    %10:gr64 = MOV64rm %31, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s64) from `i64* undef`)
+    %12:gr64 = ADD64rr %0, %0, implicit-def dead $eflags, debug-location !13
+    MOV32mr %32, 1, $noreg, 0, $noreg, %1, debug-location !13 :: (store (s32) into `i32* undef`, align 8)
+    MOV64mr %33, 1, $noreg, 0, $noreg, %12, debug-location !13 :: (store (s64) into `i64* undef`)
+    %11:gr32 = MOV32rm %34, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s32) from `i32* undef`, align 8)
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+    $rdi = COPY %35, debug-location !13
+    $rsi = COPY %36, debug-location !13
+    CALL64r %37, csr_64, implicit $rsp, implicit $ssp, implicit $rdi, implicit $rsi, implicit-def $rsp, implicit-def $ssp, implicit-def $al, debug-location !13
+    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+    %13:gr32 = INC32r %6, implicit-def dead $eflags, debug-instr-number 1, debug-location !13
+    DBG_INSTR_REF 1, 0, !12, !DIExpression(), debug-location !13
+    JMP_1 %bb.1, debug-location !13
+...

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-folding.mir

+# RUN: llc %s -o - -experimental-debug-variable-locations \
+# RUN:    -start-before=x86-flags-copy-lowering -stop-after=virtregrewriter \
+# RUN: | FileCheck %s
+#
+# This test is for stack spill folding -- the SETCC near the start of the MIR
+# below show be morphed into an SETCCm by the register allocator, making it
+# store to %stack.0. We should track this fact in the substitution table, by
+# adding a substitution to the memory-operand operand number.
+#
+# This is a single operand spill -- there's a separate test for tied def ones.
+#
+# Ideally this test would be shorter; however, it needs to be sufficiently
+# complex to force the register allocator to spill something, so there's a
+# limit.
+#
+# CHECK:      debugValueSubstitutions:
+# CHECK-NEXT:  - { srcinst: 1, srcop: 0, dstinst: 2, dstop: 1000000, subreg: 0 }
+# CHECK-LABEL: bb.0:
+# CHECK: SETCCm %stack.0, {{.*}} debug-instr-number 2
+--- |
+  target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+  
+  define internal fastcc void @beans(i32 %Kind) unnamed_addr align 2 !dbg !7  {
+    ret void
+  }
+   
+  !llvm.dbg.cu = !{!0}
+  !llvm.module.flags = !{!3, !4, !5}
+  !llvm.ident = !{!6}
+  
+  !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 12.0.0 ([e-mail]:llvm/llvm-project 79a35789efdf2378f97642ae4a5f3099b9087a11)", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, splitDebugInlining: false, nameTableKind: None)
+  !1 = !DIFile(filename: "test.c", directory: "/fast/fs/llvm3/llvm/test/DebugInfo/MIR/InstrRef")
+  !2 = !{}
+  !3 = !{i32 7, !"Dwarf Version", i32 4}
+  !4 = !{i32 2, !"Debug Info Version", i32 3}
+  !5 = !{i32 1, !"wchar_size", i32 4}
+  !6 = !{!"clang version 12.0.0 ([e-mail]:llvm/llvm-project 79a35789efdf2378f97642ae4a5f3099b9087a11)"}
+  !7 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 3, type: !8, scopeLine: 3, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+  !8 = !DISubroutineType(types: !9)
+  !9 = !{!10, !11, !11}
+  !10 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+  !11 = !DIBasicType(name: "short", size: 16, encoding: DW_ATE_signed)
+  !12 = !DILocalVariable(name: "bar", arg: 1, scope: !7, file: !1, line: 3, type: !11)
+  !13 = !DILocation(line: 0, scope: !7)
+  !14 = !DILocalVariable(name: "baz", arg: 2, scope: !7, file: !1, line: 3, type: !11)
+  !15 = distinct !DILexicalBlock(scope: !7, file: !1, line: 8, column: 7)
+
+
+...
+---
+name:            beans
+alignment:       16
+tracksRegLiveness: true
+registers:
+  - { id: 0, class: gr8 }
+  - { id: 1, class: gr8 }
+  - { id: 2, class: gr8 }
+  - { id: 3, class: gr32 }
+  - { id: 4, class: gr64 }
+  - { id: 5, class: gr8 }
+  - { id: 6, class: gr32 }
+  - { id: 7, class: gr32 }
+  - { id: 8, class: gr32 }
+  - { id: 9, class: gr32 }
+  - { id: 10, class: gr32 }
+  - { id: 11, class: gr64 }
+  - { id: 12, class: gr64 }
+  - { id: 13, class: gr32 }
+  - { id: 14, class: gr8 }
+  - { id: 15, class: gr8 }
+  - { id: 16, class: gr32 }
+  - { id: 17, class: gr64 }
+  - { id: 18, class: gr8 }
+  - { id: 19, class: gr8 }
+  - { id: 20, class: gr32 }
+  - { id: 21, class: gr64 }
+  - { id: 22, class: gr32 }
+  - { id: 23, class: gr32 }
+  - { id: 24, class: gr64 }
+  - { id: 25, class: gr32 }
+  - { id: 26, class: gr64 }
+  - { id: 27, class: gr8 }
+  - { id: 28, class: gr64_nosp }
+  - { id: 29, class: gr32 }
+  - { id: 30, class: gr8 }
+  - { id: 31, class: gr8 }
+  - { id: 32, class: gr64 }
+  - { id: 33, class: gr8 }
+  - { id: 34, class: gr64 }
+  - { id: 35, class: gr64 }
+  - { id: 36, class: gr8 }
+  - { id: 37, class: gr64 }
+  - { id: 38, class: gr8 }
+  - { id: 39, class: gr32 }
+  - { id: 40, class: gr32 }
+  - { id: 41, class: gr8 }
+  - { id: 42, class: gr8 }
+  - { id: 43, class: gr8 }
+liveins:
+  - { reg: '$edi', virtual-reg: '%7' }
+frameInfo:
+  maxAlignment:    1
+  hasCalls:        true
+machineFunctionInfo: {}
+jumpTable:
+  kind:            block-address
+  entries:
+    - id:              0
+      blocks:          [ '%bb.11', '%bb.7', '%bb.11', '%bb.7', '%bb.12', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.13', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', 
+                         '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', '%bb.7', 
+                         '%bb.8' ]
+body:             |
+  bb.0:
+    successors: %bb.2(0x20000000), %bb.14(0x60000000)
+    liveins: $edi
+  
+    %7:gr32 = COPY $edi
+    CMP32ri8 %7, 4, implicit-def $eflags
+    %0:gr8 = SETCCr 4, implicit $eflags, debug-instr-number 1
+    CMP32ri8 %7, 2, implicit-def $eflags
+    %1:gr8 = SETCCr 4, implicit $eflags
+    CMP32ri8 %7, 1, implicit-def $eflags
+    %2:gr8 = SETCCr 4, implicit $eflags
+    %11:gr64 = IMPLICIT_DEF
+    %12:gr64 = IMPLICIT_DEF
+    %5:gr8 = MOV8rm %12, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s8) from `i8* undef`, align 8)
+    %13:gr32 = MOV32r0 implicit-def dead $eflags
+    %14:gr8 = COPY %13.sub_8bit
+    TEST8rr %14, %14, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.2, 5, implicit $eflags, debug-location !13
+    JMP_1 %bb.14, debug-location !13
+  
+  bb.14:
+    successors: %bb.2(0x2aaaaaab), %bb.1(0x55555555)
+  
+    CMP8ri %5, 70, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.2, 5, implicit $eflags, debug-location !13
+    JMP_1 %bb.1, debug-location !13
+  
+  bb.1:
+  
+  bb.2:
+    successors: %bb.4(0x20000000), %bb.15(0x60000000)
+  
+    %4:gr64 = MOV64rm %11, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s64) from `%"class.llvm::Instruction"** undef`)
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+    %16:gr32 = IMPLICIT_DEF
+    $edi = COPY %16, debug-location !13
+    %17:gr64 = IMPLICIT_DEF
+    CALL64r killed %17, csr_64, implicit $rsp, implicit $ssp, implicit $edi, implicit-def $rsp, implicit-def $ssp, implicit-def $al, debug-location !13
+    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+    %18:gr8 = COPY $al, debug-location !13
+    TEST8ri %18, 1, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.4, 5, implicit $eflags, debug-location !13
+    JMP_1 %bb.15, debug-location !13
+  
+  bb.15:
+    successors: %bb.4(0x2aaaaaab), %bb.3(0x55555555)
+  
+    CMP8ri %5, 70, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.4, 4, implicit $eflags, debug-location !13
+    JMP_1 %bb.3, debug-location !13
+  
+  bb.3:
+  
+  bb.4:
+    successors: %bb.5, %bb.6
+  
+    %21:gr64 = IMPLICIT_DEF
+    %20:gr32 = MOVZX32rm8 killed %21, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s8) from `i32* undef`, align 8)
+    %6:gr32 = nsw DEC32r %20, implicit-def dead $eflags, debug-location !13
+    CMP32ri8 %6, 5, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.6, 7, implicit $eflags, debug-location !13
+    JMP_1 %bb.5, debug-location !13
+  
+  bb.5:
+    %24:gr64 = IMPLICIT_DEF
+    %23:gr32 = MOVZX32rm8 %24, 1, $noreg, 0, $noreg, debug-location !13 :: (load (s8) from `i8* undef`, align 8)
+    %25:gr32 = nsw ADD32ri8 %23, -22, implicit-def dead $eflags, debug-location !13
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+    $edi = COPY %25, debug-location !13
+    %26:gr64 = IMPLICIT_DEF
+    CALL64r %26, csr_64, implicit $rsp, implicit $ssp, implicit $edi, implicit-def $rsp, implicit-def $ssp, implicit-def $al, debug-location !13
+    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, debug-location !13
+  
+  bb.6:
+    successors: %bb.7(0x0aaaaaab), %bb.16(0x75555555)
+  
+    %31:gr8 = IMPLICIT_DEF
+    CMP8ri %31, 40, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.7, 7, implicit $eflags, debug-location !13
+  
+  bb.16:
+    successors: %bb.11(0x2e8ba2ea), %bb.7(0x0ba2e8ba), %bb.12(0x1745d174), %bb.13(0x1745d174), %bb.8(0x1745d174)
+  
+    %29:gr32 = MOV32r0 implicit-def dead $eflags
+    %28:gr64_nosp = SUBREG_TO_REG 0, %29, %subreg.sub_32bit
+    JMP64m $noreg, 8, %28, %jump-table.0, $noreg :: (load (s64) from jump-table)
+  
+  bb.7:
+    %43:gr8 = IMPLICIT_DEF
+    $al = COPY %43
+    RET 0, $al
+  
+  bb.8:
+    successors: %bb.10(0x20000000), %bb.17(0x60000000)
+  
+    CMP32ri8 %6, 5, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.10, 7, implicit $eflags, debug-location !13
+    JMP_1 %bb.17, debug-location !13
+  
+  bb.17:
+    successors: %bb.10(0x2aaaaaab), %bb.9(0x55555555)
+  
+    %3:gr32 = ADD32ri8 %7, -7, implicit-def dead $eflags
+    CMP32ri8 %3, 3, implicit-def $eflags, debug-location !13
+    JCC_1 %bb.10, 2, implicit $eflags, debug-location !13
+    JMP_1 %bb.9, debug-location !13
+  
+  bb.9:
+    %41:gr8 = IMPLICIT_DEF
+    $al = COPY %41
+    RET 0, $al
+  
+  bb.10:
+    %42:gr8 = IMPLICIT_DEF
+    $al = COPY %42
+    RET 0, $al
+  
+  bb.11:
+    %37:gr64 = IMPLICIT_DEF
+    MOV8mr killed %37, 1, $noreg, 0, $noreg, %2, debug-location !13 :: (store (s8) into `i8* undef`, align 8)
+    %38:gr8 = IMPLICIT_DEF
+    $al = COPY %38
+    RET 0, $al
+  
+  bb.12:
+    %34:gr64 = IMPLICIT_DEF
+    MOV8mr %34, 1, $noreg, 0, $noreg, %1, debug-location !13 :: (store (s8) into `i8* undef`, align 8)
+    %35:gr64 = IMPLICIT_DEF
+    MOV64mr %35, 1, $noreg, 0, $noreg, %4, debug-location !13 :: (store (s64) into `%"class.llvm::Instruction"** undef`)
+    %36:gr8 = IMPLICIT_DEF
+    $al = COPY %36
+    RET 0, $al
+  
+  bb.13:
+    DBG_INSTR_REF 1, 0, !12, !DIExpression(), debug-location !13
+    %32:gr64 = IMPLICIT_DEF
+    MOV8mr killed %32, 1, $noreg, 0, $noreg, %0, debug-location !13 :: (store (s8) into `i8* undef`, align 8)
+    %33:gr8 = IMPLICIT_DEF
+    $al = COPY %33
+    RET 0, $al
+
+...

llvm/test/DebugInfo/MIR/InstrRef/memory-operand-tracking.mir

+# RUN: llc %s -march=x86-64 -run-pass=livedebugvalues -experimental-debug-variable-locations -o - 2>&1 | FileCheck %s
+#
+# Test that memory operands of instructions are interpreted by LiveDebugValues:
+# if an instruction reference is substituted to a memory operand, we should be
+# able to emit a DBG_VALUE referring to its slot.
+#
+# In addition, further instructions that write to the same stack slot should
+# be recognised as clobbering the value in that slot.
+--- |
+  define i8 @test(i32 %bar) local_unnamed_addr !dbg !7 {
+  entry:
+    ret i8 0, !dbg !12
+  }
+
+  declare dso_local void @ext(i64)
+
+  !llvm.dbg.cu = !{!0}
+  !llvm.module.flags = !{!3, !4, !5, !6}
+  !0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, file: !1, producer: "", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug)
+  !1 = !DIFile(filename: "foo.cpp", directory: ".")
+  !2 = !DIBasicType(name: "int", size: 8, encoding: DW_ATE_signed)
+  !3 = !{i32 2, !"Dwarf Version", i32 4}
+  !4 = !{i32 2, !"Debug Info Version", i32 3}
+  !5 = !{i32 1, !"wchar_size", i32 2}
+  !6 = !{i32 7, !"PIC Level", i32 2}
+  !7 = distinct !DISubprogram(name: "foo", linkageName: "foo", scope: !1, file: !1, line: 6, type: !8, scopeLine: 6, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !10)
+  !8 = !DISubroutineType(types: !9)
+  !9 = !{!2, !2}
+  !10 = !{!11}
+  !11 = !DILocalVariable(name: "baz", scope: !7, file: !1, line: 7, type: !2)
+  !12 = !DILocation(line: 10, scope: !7)
+...
+---
+name: test
+tracksRegLiveness: true
+liveins:
+  - { reg: '$rdi', virtual-reg: '' }
+debugValueSubstitutions:
+- { srcinst: 2, srcop: 0, dstinst: 3, dstop: 1000000, subreg: 0 }
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: default, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+body:  |
+  bb.0:
+  liveins: $rdi, $rax
+    DBG_PHI $rax, 1
+    MOV64mr $rsp, 1, $noreg, 16, $noreg, $rdi :: (store 8 into %stack.0)
+    $rax = MOV64ri 0, debug-location !12
+    INC32m $rsp, 1, $noreg, 4, $noreg, implicit-def dead $eflags, debug-instr-number 3, debug-location !DILocation(line: 0, scope: !7) :: (store (s32) into %stack.0)
+    DBG_INSTR_REF 2, 0, !11, !DIExpression(), debug-location !12
+    ; CHECK:      DBG_INSTR_REF 2, 0
+    ; CHECK-NEXT: DBG_VALUE $rsp
+    ;; We should _not_ be able to find the old value.
+    DBG_INSTR_REF 1, 0, !11, !DIExpression(), debug-location !12
+    ; CHECK:      DBG_INSTR_REF 1, 0
+    ; CHECK-NEXT: DBG_VALUE $noreg
+    INC32m $rsp, 1, $noreg, 4, $noreg, implicit-def dead $eflags, debug-location !12 :: (store (s32) into %stack.0)
+    ;; The above INC32m should be detected as clobbering the stack location,
+    ;; even though it isn't debug labelled.
+    DBG_INSTR_REF 2, 0, !11, !DIExpression(), debug-location !12
+    ; CHECK:      DBG_INSTR_REF 2, 0
+    ; CHECK-NEXT: DBG_VALUE $noreg
+    $rax = MOV64rm $rsp, 1, $noreg, 8, $noreg :: (load 8 from %stack.0)
+    RETQ $rax, debug-location !12
+...