[DebugInfo] Add support for variadic DBG_INSTR_REFs in LiveDebugValues

[StephenTozer]

Summary

Following support from the previous patches in this stack being added for variadic DBG_INSTR_REFs to exist, this patch modifies LiveDebugValues to handle those instructions. Support already exists for DBG_VALUE_LISTs, which covers most of the work needed to handle these instructions; this patch only modifies the transferDebugInstrRef function to correctly track them.

[jmorse]

Looks good with some tiny comments -- as ever it'll want tests, which should be folded into this patch, could you add those please.

This is also reassuringly small, which is encouraging and suggest it composes well with the other variadic work.

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

1647–1657

I believe DenseMap does an initial allocation, and the common case for DBG_INSTR_REF is going to be single value -- could I recommend using a SmallDenseMap instead? It'll expand to be larger when needed.

1679–1691

Aliasing the type name (DbgOp DbgOp) is going to confuse someone at some point, just "Op" perhaps?

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h

1198

This will want a docu-comment

[StephenTozer]

Addressed non-test review comments.

[StephenTozer]

Add some lit test coverage, fix a bug with memoization of DBG_PHIs and remove an out-of-date comment.

[jmorse]

Looks good, but with what feels like low test coverage: remind me, the plumbing required to track variadic locations in LiveDebugValues already landed, and we have tests to cover those new code paths, yes?

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

639–640

Definitely wants a comment as to the intention behind this, I'm struggling to work it out.

1650

Recommend const DbgOp & to avoid unnecessary copies cropping up

llvm/test/DebugInfo/MIR/InstrRef/variadic-instr-ref.mir

1–3 ↗

(On Diff #470530)

Super-nit -- I don't like the file name, ideally there would be some kind of indicator that it's touching LiveDebugValues in some way.

50 ↗

(On Diff #470530)

Sounds like a DbgEntityHistoryCalculator matter rather than one for LiveDebugValues, right?

154 ↗

(On Diff #470530)

Ideally we'd remove these redundant keys (all the false / zero ones) below

[StephenTozer]

Rebased onto latest master; addressed review comments. The section of code that turned variadic expressions into non-variadic expressions so that they could be recovered as entry values has been extracted into a separate function in DebugInfoMetadata, and another instance of that logic in AsmPrinter has been updated to use this function. If you think that change should be rolled into a separate patch then I'm happy to do so!

[StephenTozer]

In D133927#4007888, @jmorse wrote:

Looks good, but with what feels like low test coverage: remind me, the plumbing required to track variadic locations in LiveDebugValues already landed, and we have tests to cover those new code paths, yes?

Yes - after all the input debug value instructions are converted to the internal representation (DbgValue) in the instruction referencing live debug values, variadic DBG_INSTR_REFs follow the same code paths as DBG_VALUE_LISTs, which had test coverage added when they were introduced. Notably those patches added unit tests, and this patch has nothing to add on top of them. If you think test coverage could be expanded though then I should be able to generate some more cases, it might not be a bad idea to cast a wide net in case there are any rare or insidious errors.

llvm/test/DebugInfo/MIR/InstrRef/variadic-instr-ref.mir

50 ↗

(On Diff #470530)

Indeed - there is some clobbering that's handled in LiveDebugValues, and I thought this should be one such case, but after checking over it again it's firmly out of LiveDebugValues' wheelhouse.

[jmorse]

In D133927#4023568, @StephenTozer wrote:

Yes - after all the input debug value instructions are converted to the internal representation (DbgValue) in the instruction referencing live debug values, variadic DBG_INSTR_REFs follow the same code paths as DBG_VALUE_LISTs, which had test coverage added when they were introduced. Notably those patches added unit tests, and this patch has nothing to add on top of them. If you think test coverage could be expanded though then I should be able to generate some more cases, it might not be a bad idea to cast a wide net in case there are any rare or insidious errors.

SGTM

[StephenTozer]

Rebased onto latest main, slightly updated and added unit tests for convertToNonVariadicExpression.

llvm/include/llvm/IR/DebugInfoMetadata.h

public:
   bool isSingleLocationExpression() const;
 
   /// If \p Expr is a non-variadic expression (i.e. one that does not contain
   /// DW_OP_LLVM_arg), returns \p Expr converted to variadic form by adding a
   /// leading [DW_OP_LLVM_arg, 0] to the expression; otherwise returns \p Expr.
   static const DIExpression *
   convertToVariadicExpression(const DIExpression *Expr);
 
+  /// If \p Expr is a valid single-location expression, i.e. it refers to only a
+  /// single debug operand at the start of the expression, then return that
+  /// expression in a non-variadic form by removing DW_OP_LLVM_arg from the
+  /// expression if it is present; otherwise returns std::nullopt.
+  static std::optional<const DIExpression *>
+  convertToNonVariadicExpression(const DIExpression *Expr);
+
   /// Inserts the elements of \p Expr into \p Ops modified to a canonical form,
   /// which uses DW_OP_LLVM_arg (i.e. is a variadic expression) and folds the
   /// implied derefence from the \p IsIndirect flag into the expression. This
   /// allows us to check equivalence between expressions with differing
   /// directness or variadicness.
   static void canonicalizeExpressionOps(SmallVectorImpl<uint64_t> &Ops,
                                         const DIExpression *Expr,
                                         bool IsIndirect);

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

if (auto *SP = dyn_cast<DISubprogram>(V->getScope())) {
     StringRef Name = SP->getName();
     if (!Name.empty())
       OS << Name << ":";
   }
   OS << V->getName();
   OS << " <- ";
 
   const DIExpression *Expr = MI->getDebugExpression();
-  if (Expr->getNumElements() && Expr->isSingleLocationExpression() &&
-      Expr->expr_op_begin()->getOp() == dwarf::DW_OP_LLVM_arg) {
-    SmallVector<uint64_t> Ops(
-        make_range(Expr->elements_begin() + 2, Expr->elements_end()));
-    Expr = DIExpression::get(Expr->getContext(), Ops);
-  }
+  // First convert this to a non-variadic expression if possible, to simplify
+  // the output.
+  if (auto NonVariadicExpr = DIExpression::convertToNonVariadicExpression(Expr))
+    Expr = *NonVariadicExpr;
+  // Then, output the possibly-simplified expression.
   if (Expr->getNumElements()) {
     OS << '[';
     ListSeparator LS;
     for (auto &Op : Expr->expr_ops()) {
       OS << LS << dwarf::OperationEncodingString(Op.getOp());
       for (unsigned I = 0; I < Op.getNumArgs(); ++I)
         OS << ' ' << Op.getArg(I);
     }

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.h

private:
   // Map of overlapping variable fragments.
   OverlapMap OverlapFragments;
   VarToFragments SeenFragments;
 
   /// Mapping of DBG_INSTR_REF instructions to their values, for those
   /// DBG_INSTR_REFs that call resolveDbgPHIs. These variable references solve
   /// a mini SSA problem caused by DBG_PHIs being cloned, this collection caches
   /// the result.
-  DenseMap<MachineInstr *, std::optional<ValueIDNum>> SeenDbgPHIs;
+  DenseMap<std::pair<MachineInstr *, unsigned>, std::optional<ValueIDNum>>
+      SeenDbgPHIs;
 
   DbgOpIDMap DbgOpStore;
 
   /// True if we need to examine call instructions for stack clobbers. We
   /// normally assume that they don't clobber SP, but stack probes on Windows
   /// do.
   bool AdjustsStackInCalls = false;
 
std::optional<SpillLocationNo> isRestoreInstruction(const MachineInstr &MI,
                                                       MachineFunction *MF,
                                                       unsigned &Reg);
 
   /// Given a spill instruction, extract the spill slot information, ensure it's
   /// tracked, and return the spill number.
   std::optional<SpillLocationNo>
   extractSpillBaseRegAndOffset(const MachineInstr &MI);
 
+  /// For an instruction reference given by \p InstNo and \p OpNo in instruction
+  /// \p MI returns the Value pointed to by that instruction reference if any
+  /// exists, otherwise returns None.
+  std::optional<ValueIDNum> getValueForInstrRef(unsigned InstNo, unsigned OpNo,
+                                                MachineInstr &MI,
+                                                const ValueTable *MLiveOuts,
+                                                const ValueTable *MLiveIns);
+
   /// Observe a single instruction while stepping through a block.
   void process(MachineInstr &MI, const ValueTable *MLiveOuts,
                const ValueTable *MLiveIns);
 
   /// Examines whether \p MI is a DBG_VALUE and notifies trackers.
   /// \returns true if MI was recognized and processed.
   bool transferDebugValue(const MachineInstr &MI);
 

llvm/lib/CodeGen/LiveDebugValues/InstrRefBasedImpl.cpp

public:
   bool recoverAsEntryValue(const DebugVariable &Var,
                            const DbgValueProperties &Prop,
                            const ValueIDNum &Num) {
     // Is this variable location a candidate to be an entry value. First,
     // should we be trying this at all?
     if (!ShouldEmitDebugEntryValues)
       return false;
 
+    const DIExpression *DIExpr = Prop.DIExpr;
+
     // We don't currently emit entry values for DBG_VALUE_LISTs.
-    if (Prop.IsVariadic)
+    if (Prop.IsVariadic) {
+      // If this debug value can be converted to be non-variadic, then do so;
+      // otherwise give up.
+      auto NonVariadicExpression =
+          DIExpression::convertToNonVariadicExpression(DIExpr);
+      if (!NonVariadicExpression)
         return false;
+      DIExpr = *NonVariadicExpression;
+    }
 
     // Is the variable appropriate for entry values (i.e., is a parameter).
-    if (!isEntryValueVariable(Var, Prop.DIExpr))
+    if (!isEntryValueVariable(Var, DIExpr))
       return false;
 
     // Is the value assigned to this variable still the entry value?
     if (!isEntryValueValue(Num))
       return false;
 
     // Emit a variable location using an entry value expression.
     DIExpression *NewExpr =
-        DIExpression::prepend(Prop.DIExpr, DIExpression::EntryValue);
+        DIExpression::prepend(DIExpr, DIExpression::EntryValue);
     Register Reg = MTracker->LocIdxToLocID[Num.getLoc()];
     MachineOperand MO = MachineOperand::CreateReg(Reg, false);
 
     PendingDbgValues.push_back(
-        emitMOLoc(MO, Var, {NewExpr, Prop.Indirect, Prop.IsVariadic}));
+        emitMOLoc(MO, Var, {NewExpr, Prop.Indirect, false}));
     return true;
   }
 
   /// Change a variable value after encountering a DBG_VALUE inside a block.
   void redefVar(const MachineInstr &MI) {
     DebugVariable Var(MI.getDebugVariable(), MI.getDebugExpression(),
                       MI.getDebugLoc()->getInlinedAt());
     DbgValueProperties Properties(MI);
void clobberMloc(LocIdx MLoc, ValueIDNum OldValue,
     // Examine all the variables based on this location.
     DenseSet<DebugVariable> NewMLocs;
     // If no new location has been found, every variable that depends on this
     // MLoc is dead, so end their existing MLoc->Var mappings as well.
     SmallVector<std::pair<LocIdx, DebugVariable>> LostMLocs;
     for (const auto &Var : ActiveMLocIt->second) {
       auto ActiveVLocIt = ActiveVLocs.find(Var);
       // Re-state the variable location: if there's no replacement then NewLoc
-      // is None and a $noreg DBG_VALUE will be created. Otherwise, a DBG_VALUE
-      // identifying the alternative location will be emitted.
+      // is std::nullopt and a $noreg DBG_VALUE will be created. Otherwise, a
+      // DBG_VALUE identifying the alternative location will be emitted.
       const DbgValueProperties &Properties = ActiveVLocIt->second.Properties;
 
       // Produce the new list of debug ops - an empty list if no new location
       // was found, or the existing list with the substitution MLoc -> NewLoc
       // otherwise.
       SmallVector<ResolvedDbgOp> DbgOps;
       if (NewLoc) {
         ResolvedDbgOp OldOp(MLoc);
bool InstrRefBasedLDV::transferDebugValue(const MachineInstr &MI) {
 
   // If performing final tracking of transfers, report this variable definition
   // to the TransferTracker too.
   if (TTracker)
     TTracker->redefVar(MI);
   return true;
 }
 
-bool InstrRefBasedLDV::transferDebugInstrRef(MachineInstr &MI,
-                                             const ValueTable *MLiveOuts,
-                                             const ValueTable *MLiveIns) {
-  if (!MI.isDebugRef())
-    return false;
-
-  // Only handle this instruction when we are building the variable value
-  // transfer function.
-  if (!VTracker && !TTracker)
-    return false;
-
-  unsigned InstNo = MI.getDebugOperand(0).getInstrRefInstrIndex();
-  unsigned OpNo = MI.getDebugOperand(0).getInstrRefOpIndex();
-
-  const DILocalVariable *Var = MI.getDebugVariable();
-  const DIExpression *Expr = MI.getDebugExpression();
-  const DILocation *DebugLoc = MI.getDebugLoc();
-  const DILocation *InlinedAt = DebugLoc->getInlinedAt();
-  assert(Var->isValidLocationForIntrinsic(DebugLoc) &&
-         "Expected inlined-at fields to agree");
-
-  DebugVariable V(Var, Expr, InlinedAt);
-
-  auto *Scope = LS.findLexicalScope(MI.getDebugLoc().get());
-  if (Scope == nullptr)
-    return true; // Handled by doing nothing. This variable is never in scope.
-
-  const MachineFunction &MF = *MI.getParent()->getParent();
-
+std::optional<ValueIDNum> InstrRefBasedLDV::getValueForInstrRef(
+    unsigned InstNo, unsigned OpNo, MachineInstr &MI,
+    const ValueTable *MLiveOuts, const ValueTable *MLiveIns) {
   // Various optimizations may have happened to the value during codegen,
   // recorded in the value substitution table. Apply any substitutions to
   // the instruction / operand number in this DBG_INSTR_REF, and collect
   // any subregister extractions performed during optimization.
+  const MachineFunction &MF = *MI.getParent()->getParent();
 
   // Create dummy substitution with Src set, for lookup.
   auto SoughtSub =
       MachineFunction::DebugSubstitution({InstNo, OpNo}, {0, 0}, 0);
 
   SmallVector<unsigned, 4> SeenSubregs;
   auto LowerBoundIt = llvm::lower_bound(MF.DebugValueSubstitutions, SoughtSub);
   while (LowerBoundIt != MF.DebugValueSubstitutions.end() &&
if (NewID && !MTracker->isSpill(L)) {
         }
       }
     } else {
       // If we can't handle subregisters, unset the new value.
       NewID = std::nullopt;
     }
   }
 
-  // We, we have a value number or std::nullopt. Tell the variable value tracker
-  // about it. The rest of this LiveDebugValues implementation acts exactly the
-  // same for DBG_INSTR_REFs as DBG_VALUEs (just, the former can refer to values
-  // that aren't immediately available).
-  DbgValueProperties Properties(Expr, false, true);
+  return NewID;
+}
+
+bool InstrRefBasedLDV::transferDebugInstrRef(MachineInstr &MI,
+                                             const ValueTable *MLiveOuts,
+                                             const ValueTable *MLiveIns) {
+  if (!MI.isDebugRef())
+    return false;
+
+  // Only handle this instruction when we are building the variable value
+  // transfer function.
+  if (!VTracker && !TTracker)
+    return false;
+
+  const DILocalVariable *Var = MI.getDebugVariable();
+  const DIExpression *Expr = MI.getDebugExpression();
+  const DILocation *DebugLoc = MI.getDebugLoc();
+  const DILocation *InlinedAt = DebugLoc->getInlinedAt();
+  assert(Var->isValidLocationForIntrinsic(DebugLoc) &&
+         "Expected inlined-at fields to agree");
+
+  DebugVariable V(Var, Expr, InlinedAt);
+
+  auto *Scope = LS.findLexicalScope(MI.getDebugLoc().get());
+  if (Scope == nullptr)
+    return true; // Handled by doing nothing. This variable is never in scope.
+
   SmallVector<DbgOpID> DbgOpIDs;
-  if (NewID)
+  for (const MachineOperand &MO : MI.debug_operands()) {
+    if (!MO.isDbgInstrRef()) {
+      assert(!MO.isReg() && "DBG_INSTR_REF should not contain registers");
+      DbgOpID ConstOpID = DbgOpStore.insert(DbgOp(MO));
+      DbgOpIDs.push_back(ConstOpID);
+      continue;
+    }
+
+    unsigned InstNo = MO.getInstrRefInstrIndex();
+    unsigned OpNo = MO.getInstrRefOpIndex();
+
+    // Default machine value number is <None> -- if no instruction defines
+    // the corresponding value, it must have been optimized out.
+    std::optional<ValueIDNum> NewID =
+        getValueForInstrRef(InstNo, OpNo, MI, MLiveOuts, MLiveIns);
+    // We have a value number or std::nullopt. If the latter, then kill the
+    // entire debug value.
+    if (NewID) {
       DbgOpIDs.push_back(DbgOpStore.insert(*NewID));
+    } else {
+      DbgOpIDs.clear();
+      break;
+    }
+  }
+
+  // We have a DbgOpID for every value or for none. Tell the variable value
+  // tracker about it. The rest of this LiveDebugValues implementation acts
+  // exactly the same for DBG_INSTR_REFs as DBG_VALUEs (just, the former can
+  // refer to values that aren't immediately available).
+  DbgValueProperties Properties(Expr, false, true);
   if (VTracker)
     VTracker->defVar(MI, Properties, DbgOpIDs);
 
   // If we're on the final pass through the function, decompose this INSTR_REF
   // into a plain DBG_VALUE.
   if (!TTracker)
     return true;
 
+  // Fetch the concrete DbgOps now, as we will need them later.
+  SmallVector<DbgOp> DbgOps;
+  for (DbgOpID OpID : DbgOpIDs) {
+    DbgOps.push_back(DbgOpStore.find(OpID));
+  }
+
   // Pick a location for the machine value number, if such a location exists.
   // (This information could be stored in TransferTracker to make it faster).
-  TransferTracker::LocationAndQuality FoundLoc;
+  SmallDenseMap<ValueIDNum, TransferTracker::LocationAndQuality> FoundLocs;
+  SmallVector<ValueIDNum> ValuesToFind;
+  // Initialized the preferred-location map with illegal locations, to be
+  // filled in later.
+  for (const DbgOp &Op : DbgOps) {
+    if (!Op.IsConst)
+      if (FoundLocs.insert({Op.ID, TransferTracker::LocationAndQuality()})
+              .second)
+        ValuesToFind.push_back(Op.ID);
+  }
+
   for (auto Location : MTracker->locations()) {
     LocIdx CurL = Location.Idx;
     ValueIDNum ID = MTracker->readMLoc(CurL);
-    if (NewID && ID == NewID) {
-      // If this is the first location with that value, pick it. Otherwise,
-      // consider whether it's a "longer term" location.
-      std::optional<TransferTracker::LocationQuality> ReplacementQuality =
-          TTracker->getLocQualityIfBetter(CurL, FoundLoc.getQuality());
-      if (ReplacementQuality) {
-        FoundLoc =
-            TransferTracker::LocationAndQuality(CurL, *ReplacementQuality);
-        if (FoundLoc.isBest())
+    auto ValueToFindIt = find(ValuesToFind, ID);
+    if (ValueToFindIt == ValuesToFind.end())
+      continue;
+    auto &Previous = FoundLocs.find(ID)->second;
+        TTracker->getLocQualityIfBetter(CurL, Previous.getQuality());
+      Previous = TransferTracker::LocationAndQuality(CurL, *ReplacementQuality);
+      if (Previous.isBest()) {
+        ValuesToFind.erase(ValueToFindIt);
+        if (ValuesToFind.empty())
           break;
       }
     }
   }
 
   SmallVector<ResolvedDbgOp> NewLocs;
-  if (!FoundLoc.isIllegal())
-    NewLocs.push_back(FoundLoc.getLoc());
+  for (const DbgOp &DbgOp : DbgOps) {
+    if (DbgOp.IsConst) {
+      NewLocs.push_back(DbgOp.MO);
+      continue;
+    }
+    LocIdx FoundLoc = FoundLocs.find(DbgOp.ID)->second.getLoc();
+    if (FoundLoc.isIllegal()) {
+      NewLocs.clear();
+      break;
+    }
+    NewLocs.push_back(FoundLoc);
+  }
   // Tell transfer tracker that the variable value has changed.
   TTracker->redefVar(MI, Properties, NewLocs);
 
-  // If there was a value with no location; but the value is defined in a
-  // later instruction in this block, this is a block-local use-before-def.
-  if (FoundLoc.isIllegal() && NewID && NewID->getBlock() == CurBB &&
-      NewID->getInst() > CurInst) {
-    SmallVector<DbgOp> UseBeforeDefLocs;
-    UseBeforeDefLocs.push_back(*NewID);
+  // If there were values with no location, but all such values are defined in
+  // later instructions in this block, this is a block-local use-before-def.
+  if (!DbgOps.empty() && NewLocs.empty()) {
+    bool IsValidUseBeforeDef = true;
+    uint64_t LastUseBeforeDef = 0;
+    for (auto ValueLoc : FoundLocs) {
+      ValueIDNum NewID = ValueLoc.first;
+      LocIdx FoundLoc = ValueLoc.second.getLoc();
+      if (!FoundLoc.isIllegal())
+        continue;
+      // If we have an value with no location that is not defined in this block,
+      // then it has no location in this block, leaving this value undefined.
+      if (NewID.getBlock() != CurBB || NewID.getInst() <= CurInst) {
+        IsValidUseBeforeDef = false;
+        break;
+      }
+      LastUseBeforeDef = std::max(LastUseBeforeDef, NewID.getInst());
+    }
+    if (IsValidUseBeforeDef) {
       TTracker->addUseBeforeDef(V, {MI.getDebugExpression(), false, true},
-                              UseBeforeDefLocs, NewID->getInst());
+                                DbgOps, LastUseBeforeDef);
+    }
   }
 
   // Produce a DBG_VALUE representing what this DBG_INSTR_REF meant.
   // This DBG_VALUE is potentially a $noreg / undefined location, if
   // FoundLoc is illegal.
   // (XXX -- could morph the DBG_INSTR_REF in the future).
   MachineInstr *DbgMI = MTracker->emitLoc(NewLocs, V, Properties);
 
std::optional<ValueIDNum> InstrRefBasedLDV::resolveDbgPHIs(
     MachineFunction &MF, const ValueTable *MLiveOuts,
     const ValueTable *MLiveIns, MachineInstr &Here, uint64_t InstrNum) {
   assert(MLiveOuts && MLiveIns &&
          "Tried to resolve DBG_PHI before location "
          "tables allocated?");
 
   // This function will be called twice per DBG_INSTR_REF, and might end up
   // computing lots of SSA information: memoize it.
-  auto SeenDbgPHIIt = SeenDbgPHIs.find(&Here);
+  auto SeenDbgPHIIt = SeenDbgPHIs.find(std::make_pair(&Here, InstrNum));
   if (SeenDbgPHIIt != SeenDbgPHIs.end())
     return SeenDbgPHIIt->second;
 
   std::optional<ValueIDNum> Result =
       resolveDbgPHIsImpl(MF, MLiveOuts, MLiveIns, Here, InstrNum);
-  SeenDbgPHIs.insert({&Here, Result});
+  SeenDbgPHIs.insert({std::make_pair(&Here, InstrNum), Result});
   return Result;
 }
 
 std::optional<ValueIDNum> InstrRefBasedLDV::resolveDbgPHIsImpl(
     MachineFunction &MF, const ValueTable *MLiveOuts,
     const ValueTable *MLiveIns, MachineInstr &Here, uint64_t InstrNum) {
   // Pick out records of DBG_PHI instructions that have been observed. If there
   // are none, then we cannot compute a value number.

llvm/lib/IR/DebugInfoMetadata.cpp

if (any_of(Expr->expr_ops(), [](auto ExprOp) {
     return Expr;
   SmallVector<uint64_t> NewOps;
   NewOps.reserve(Expr->getNumElements() + 2);
   NewOps.append({dwarf::DW_OP_LLVM_arg, 0});
   NewOps.append(Expr->elements_begin(), Expr->elements_end());
   return DIExpression::get(Expr->getContext(), NewOps);
 }
 
+std::optional<const DIExpression *>
+DIExpression::convertToNonVariadicExpression(const DIExpression *Expr) {
+  // Check for `isValid` covered by `isSingleLocationExpression`.
+  if (!Expr->isSingleLocationExpression())
+    return std::nullopt;
+
+  // An empty expression is already non-variadic.
+  if (!Expr->getNumElements())
+    return Expr;
+
+  auto ElementsBegin = Expr->elements_begin();
+  // If Expr does not have a leading DW_OP_LLVM_arg then we don't need to do
+  // anything.
+  if (*ElementsBegin != dwarf::DW_OP_LLVM_arg)
+    return Expr;
+
+  SmallVector<uint64_t> NonVariadicOps(
+      make_range(ElementsBegin + 2, Expr->elements_end()));
+  return DIExpression::get(Expr->getContext(), NonVariadicOps);
+}
+
 void DIExpression::canonicalizeExpressionOps(SmallVectorImpl<uint64_t> &Ops,
                                              const DIExpression *Expr,
                                              bool IsIndirect) {
   // If Expr is not already variadic, insert the implied `DW_OP_LLVM_arg 0`
   // to the existing expression ops.
   if (none_of(Expr->expr_ops(), [](auto ExprOp) {
         return ExprOp.getOp() == dwarf::DW_OP_LLVM_arg;
       }))

llvm/test/DebugInfo/MIR/InstrRef/livedebugvalues-transfer-variadic-instr-ref.mir

+# RUN: llc %s -o - -experimental-debug-variable-locations=true \
+# RUN:     -run-pass=livedebugvalues  | \
+# RUN: FileCheck %s --implicit-check-not=DBG_VALUE
+#
+## Check that LiveDebugValues can track and join DBG_INSTR_REFs that use
+## registers, stack slots, and constants in a single instruction.
+#
+# CHECK: ![[VAR_D:[0-9]+]] = !DILocalVariable(name: "d"
+
+# CHECK-LABEL: bb.0.entry
+#
+## Value at end of first block should use the registers given by DBG_PHI, and
+## fold the offset+deref from stack slot [$rsp+12] into the expression.
+# CHECK:       DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $ebp, $rsp, $ebx, 6
+
+# CHECK-LABEL: bb.1.while.body
+#
+## Prior DBG_VALUE_LIST should be live-in to this bb.1.
+# CHECK:       DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $ebp, $rsp, $ebx, 6
+#
+## When $ebp is spilled to a stack slot, a new debug value using the stack slot
+## should be inserted.
+# CHECK:       MOV32mr $rsp, {{.+}} $ebp
+# CHECK-NEXT:  DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_plus_uconst, 16, DW_OP_deref, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $rsp, $rsp, $ebx, 6
+#
+## Similarly produce a new debug value when $ebx is spilled.
+# CHECK:       MOV32mr $rsp, {{.+}} $ebx
+# CHECK-NEXT:  DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_plus_uconst, 16, DW_OP_deref, DW_OP_LLVM_arg, 2, DW_OP_plus_uconst, 20, DW_OP_deref, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $rsp, $rsp, $rsp, 6
+#
+## When the value in the stack slot is restored and then that slot is clobbered, the debug value should use the restored register.
+# CHECK:       MOV32mi $rsp
+# CHECK-NEXT:  DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_plus_uconst, 16, DW_OP_deref, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $rsp, $rsp, $ebx, 6
+#
+## Repeat for the next stack slot that gets restored and then clobbered.
+# CHECK:       MOV32mi $rsp
+# CHECK-NEXT:  DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $ebp, $rsp, $ebx, 6
+#
+## TODO: The debug value range should be terminated when the value in $ebx is clobbered.
+#
+## At the end of the block a new DBG_INSTR_REF begins a new range.
+# CHECK:       DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $ebp, $rsp, $ebx, 6
+
+# CHECK-LABEL: bb.2.while.end
+#
+## Finally, the two entry debug values should be joined, as they have identical
+## expressions, their respective stack slots and constants are identical, and
+## the remaining operands occupy the same registers out of each block.
+# CHECK:       DBG_VALUE_LIST ![[VAR_D]]
+# CHECK-SAME:  !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus_uconst, 12, DW_OP_deref, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value)
+# CHECK-SAME:  $ebp, $rsp, $ebx, 6
+
+
+--- |
+
+  ; ModuleID = 'test.ll'
+  source_filename = "test.cpp"
+  target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+  target triple = "x86_64-unknown-linux-gnu"
+  
+  ; Function Attrs: mustprogress uwtable
+  define dso_local noundef i32 @_Z3fooii(i32 noundef %a, i32 noundef %b) local_unnamed_addr !dbg !9 {
+  entry:
+    %call = tail call noundef i32 @_Z3bazv(), !dbg !18
+    call void @llvm.dbg.value(metadata !DIArgList(i32 %a, i32 %call, i32 %b), metadata !17, metadata !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_constu, 6, DW_OP_minus, DW_OP_stack_value)), !dbg !19
+    %cmp19 = icmp sgt i32 %b, %a, !dbg !20
+    br i1 %cmp19, label %while.body.preheader, label %while.end, !dbg !21
+  
+  while.body.preheader:                             ; preds = %entry
+    br label %while.body, !dbg !21
+  
+  while.body:                                       ; preds = %while.body.preheader, %while.body
+    %a.addr.021 = phi i32 [ %mul3, %while.body ], [ %a, %while.body.preheader ]
+    %b.addr.020 = phi i32 [ %add2, %while.body ], [ %b, %while.body.preheader ]
+    %call1 = tail call noundef i32 @_Z3bazv(), !dbg !22
+    tail call void asm sideeffect "", "~{rax},~{rbx},~{rcx},~{rdx},~{rsi},~{rdi},~{rbp},~{r8},~{r9},~{r10},~{r11},~{r12},~{r13},~{r14},~{r15},~{dirflag},~{fpsr},~{flags}"() #3, !dbg !24, !srcloc !25
+    %add2 = add i32 %b.addr.020, 2, !dbg !26
+    %mul3 = shl nsw i32 %a.addr.021, 1, !dbg !27
+    call void @llvm.dbg.value(metadata !DIArgList(i32 %mul3, i32 %call, i32 %add2), metadata !17, metadata !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_constu, 6, DW_OP_minus, DW_OP_stack_value)), !dbg !19
+    %cmp = icmp sgt i32 %add2, %mul3, !dbg !20
+    br i1 %cmp, label %while.body, label %while.end, !dbg !21, !llvm.loop !28
+  
+  while.end:                                        ; preds = %while.body, %entry
+    %b.addr.0.lcssa = phi i32 [ %b, %entry ], [ %add2, %while.body ]
+    %a.addr.0.lcssa = phi i32 [ %a, %entry ], [ %mul3, %while.body ]
+    %add7 = sub i32 %b.addr.0.lcssa, %call, !dbg !31
+    %sub8 = add i32 %add7, %a.addr.0.lcssa, !dbg !32
+    ret i32 %sub8, !dbg !33
+  }
+  
+  declare !dbg !34 noundef i32 @_Z3bazv() local_unnamed_addr
+  
+  ; Function Attrs: nocallback nofree nosync nounwind readnone speculatable willreturn
+  declare void @llvm.dbg.value(metadata, metadata, metadata)
+  
+  !llvm.dbg.cu = !{!0}
+  !llvm.module.flags = !{!2, !3, !4, !5, !6, !7}
+  !llvm.ident = !{!8}
+  
+  !0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, producer: "clang version 16.0.0", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, splitDebugInlining: false, nameTableKind: None)
+  !1 = !DIFile(filename: "test.cpp", directory: "/")
+  !2 = !{i32 7, !"Dwarf Version", i32 5}
+  !3 = !{i32 2, !"Debug Info Version", i32 3}
+  !4 = !{i32 1, !"wchar_size", i32 4}
+  !5 = !{i32 8, !"PIC Level", i32 2}
+  !6 = !{i32 7, !"PIE Level", i32 2}
+  !7 = !{i32 7, !"uwtable", i32 2}
+  !8 = !{!"clang version 16.0.0"}
+  !9 = distinct !DISubprogram(name: "foo", linkageName: "_Z3fooii", scope: !1, file: !1, line: 3, type: !10, scopeLine: 3, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0, retainedNodes: !13)
+  !10 = !DISubroutineType(types: !11)
+  !11 = !{!12, !12, !12}
+  !12 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+  !13 = !{!14, !15, !16, !17}
+  !14 = !DILocalVariable(name: "a", arg: 1, scope: !9, file: !1, line: 3, type: !12)
+  !15 = !DILocalVariable(name: "b", arg: 2, scope: !9, file: !1, line: 3, type: !12)
+  !16 = !DILocalVariable(name: "c", scope: !9, file: !1, line: 4, type: !12)
+  !17 = !DILocalVariable(name: "d", scope: !9, file: !1, line: 5, type: !12)
+  !18 = !DILocation(line: 4, column: 11, scope: !9)
+  !19 = !DILocation(line: 0, scope: !9)
+  !20 = !DILocation(line: 6, column: 12, scope: !9)
+  !21 = !DILocation(line: 6, column: 3, scope: !9)
+  !22 = !DILocation(line: 7, column: 5, scope: !23)
+  !23 = distinct !DILexicalBlock(scope: !9, file: !1, line: 6, column: 17)
+  !24 = !DILocation(line: 9, column: 5, scope: !23)
+  !25 = !{i64 129}
+  !26 = !DILocation(line: 8, column: 7, scope: !23)
+  !27 = !DILocation(line: 10, column: 7, scope: !23)
+  !28 = distinct !{!28, !21, !29, !30}
+  !29 = !DILocation(line: 12, column: 3, scope: !9)
+  !30 = !{!"llvm.loop.mustprogress"}
+  !31 = !DILocation(line: 13, column: 12, scope: !9)
+  !32 = !DILocation(line: 13, column: 16, scope: !9)
+  !33 = !DILocation(line: 13, column: 3, scope: !9)
+  !34 = !DISubprogram(name: "baz", linkageName: "_Z3bazv", scope: !1, file: !1, line: 1, type: !35, flags: DIFlagPrototyped, spFlags: DISPFlagOptimized, retainedNodes: !37)
+  !35 = !DISubroutineType(types: !36)
+  !36 = !{!12}
+  !37 = !{}
+
+...
+---
+name:            _Z3fooii
+alignment:       16
+tracksRegLiveness: true
+tracksDebugUserValues: true
+registers:       []
+liveins:
+  - { reg: '$edi', virtual-reg: '' }
+  - { reg: '$esi', virtual-reg: '' }
+frameInfo:
+  stackSize:       72
+  offsetAdjustment: -72
+  maxAlignment:    4
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  functionContext: ''
+  cvBytesOfCalleeSavedRegisters: 48
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+  - { id: 0, type: spill-slot, offset: -56, size: 8, alignment: 8, stack-id: default, 
+      callee-saved-register: '$rbx', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, type: spill-slot, offset: -48, size: 8, alignment: 16, stack-id: default, 
+      callee-saved-register: '$r12', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, type: spill-slot, offset: -40, size: 8, alignment: 8, stack-id: default, 
+      callee-saved-register: '$r13', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, type: spill-slot, offset: -32, size: 8, alignment: 16, stack-id: default, 
+      callee-saved-register: '$r14', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, type: spill-slot, offset: -24, size: 8, alignment: 8, stack-id: default, 
+      callee-saved-register: '$r15', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, type: spill-slot, offset: -16, size: 8, alignment: 16, stack-id: default, 
+      callee-saved-register: '$rbp', callee-saved-restored: true, debug-info-variable: '', 
+      debug-info-expression: '', debug-info-location: '' }
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -60, size: 4, alignment: 4, 
+      stack-id: default, callee-saved-register: '', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -64, size: 4, alignment: 4, 
+      stack-id: default, callee-saved-register: '', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -68, size: 4, alignment: 4, 
+      stack-id: default, callee-saved-register: '', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+callSites:       []
+debugValueSubstitutions: []
+constants:       []
+machineFunctionInfo: {}
+body:             |
+  bb.0.entry:
+    successors: %bb.1(0x78e38e39), %bb.3(0x071c71c7)
+    liveins: $edi, $esi, $rbp, $r15, $r14, $r13, $r12, $rbx
+  
+    frame-setup PUSH64r killed $rbp, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 16
+    frame-setup PUSH64r killed $r15, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 24
+    frame-setup PUSH64r killed $r14, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 32
+    frame-setup PUSH64r killed $r13, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 40
+    frame-setup PUSH64r killed $r12, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 48
+    frame-setup PUSH64r killed $rbx, implicit-def $rsp, implicit $rsp
+    frame-setup CFI_INSTRUCTION def_cfa_offset 56
+    $rsp = frame-setup SUB64ri8 $rsp, 24, implicit-def dead $eflags
+    frame-setup CFI_INSTRUCTION def_cfa_offset 80
+    CFI_INSTRUCTION offset $rbx, -56
+    CFI_INSTRUCTION offset $r12, -48
+    CFI_INSTRUCTION offset $r13, -40
+    CFI_INSTRUCTION offset $r14, -32
+    CFI_INSTRUCTION offset $r15, -24
+    CFI_INSTRUCTION offset $rbp, -16
+    DBG_PHI $esi, 5
+    DBG_PHI $edi, 3
+    $ebx = MOV32rr $esi
+    $ebp = MOV32rr $edi
+    CALL64pcrel32 target-flags(x86-plt) @_Z3bazv, csr_64, implicit $rsp, implicit $ssp, implicit-def $rsp, implicit-def $ssp, implicit-def $eax, debug-instr-number 4, debug-location !18
+    MOV32mr $rsp, 1, $noreg, 12, $noreg, $eax :: (store (s32) into %stack.2)
+    DBG_INSTR_REF !17, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value), dbg-instr-ref(3, 0), dbg-instr-ref(4, 6), dbg-instr-ref(5, 0), 6, debug-location !19
+    CMP32rr renamable $ebx, renamable $ebp, implicit-def $eflags, debug-location !20
+    JCC_1 %bb.3, 14, implicit $eflags, debug-location !21
+  
+  bb.1.while.body (align 16):
+    successors: %bb.1(0x78e38e39), %bb.3(0x071c71c7)
+    liveins: $ebp, $ebx
+  
+    MOV32mr $rsp, 1, $noreg, 16, $noreg, killed renamable $ebp :: (store (s32) into %stack.1)
+    MOV32mr $rsp, 1, $noreg, 20, $noreg, killed renamable $ebx :: (store (s32) into %stack.0)
+    CALL64pcrel32 target-flags(x86-plt) @_Z3bazv, csr_64, implicit $rsp, implicit $ssp, implicit-def $rsp, implicit-def $ssp, implicit-def dead $eax, debug-location !22
+    INLINEASM &"", 1 /* sideeffect attdialect */, 12 /* clobber */, implicit-def dead early-clobber $rax, 12 /* clobber */, implicit-def dead early-clobber $rbx, 12 /* clobber */, implicit-def dead early-clobber $rcx, 12 /* clobber */, implicit-def dead early-clobber $rdx, 12 /* clobber */, implicit-def dead early-clobber $rsi, 12 /* clobber */, implicit-def dead early-clobber $rdi, 12 /* clobber */, implicit-def dead early-clobber $rbp, 12 /* clobber */, implicit-def dead early-clobber $r8, 12 /* clobber */, implicit-def dead early-clobber $r9, 12 /* clobber */, implicit-def dead early-clobber $r10, 12 /* clobber */, implicit-def dead early-clobber $r11, 12 /* clobber */, implicit-def dead early-clobber $r12, 12 /* clobber */, implicit-def dead early-clobber $r13, 12 /* clobber */, implicit-def dead early-clobber $r14, 12 /* clobber */, implicit-def dead early-clobber $r15, 12 /* clobber */, implicit-def dead early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def dead early-clobber $eflags, !25, debug-location !24
+    renamable $ebp = MOV32rm $rsp, 1, $noreg, 16, $noreg :: (load (s32) from %stack.1)
+    renamable $ebx = MOV32rm $rsp, 1, $noreg, 20, $noreg :: (load (s32) from %stack.0)
+    MOV32mi $rsp, 1, $noreg, 20, $noreg, 0 :: (store (s32) into %stack.0)
+    MOV32mi $rsp, 1, $noreg, 16, $noreg, 0 :: (store (s32) into %stack.1)
+    renamable $ebx = ADD32ri8 killed renamable $ebx, 2, implicit-def dead $eflags, debug-instr-number 2, debug-location !26
+    renamable $ebp = nsw ADD32rr killed renamable $ebp, renamable $ebp, implicit-def dead $eflags, debug-instr-number 1, debug-location !27
+    DBG_INSTR_REF !17, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 2, DW_OP_mul, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_LLVM_arg, 3, DW_OP_minus, DW_OP_stack_value), dbg-instr-ref(1, 0), dbg-instr-ref(4, 6), dbg-instr-ref(2, 0), 6, debug-location !19
+    CMP32rr renamable $ebx, renamable $ebp, implicit-def $eflags, debug-location !20
+    JCC_1 %bb.1, 15, implicit $eflags, debug-location !21
+  
+  bb.3.while.end:
+    liveins: $ebp, $ebx
+  
+    renamable $ebx = SUB32rm killed renamable $ebx, $rsp, 1, $noreg, 12, $noreg, implicit-def dead $eflags, debug-location !31 :: (load (s32) from %stack.2)
+    renamable $ebx = ADD32rr killed renamable $ebx, killed renamable $ebp, implicit-def dead $eflags, debug-location !32
+    $eax = MOV32rr killed $ebx, debug-location !33
+    $rsp = frame-destroy ADD64ri8 $rsp, 24, implicit-def dead $eflags, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 56, debug-location !33
+    $rbx = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 48, debug-location !33
+    $r12 = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 40, debug-location !33
+    $r13 = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 32, debug-location !33
+    $r14 = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 24, debug-location !33
+    $r15 = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 16, debug-location !33
+    $rbp = frame-destroy POP64r implicit-def $rsp, implicit $rsp, debug-location !33
+    frame-destroy CFI_INSTRUCTION def_cfa_offset 8, debug-location !33
+    RET64 $eax, debug-location !33
+
+...

llvm/test/DebugInfo/MIR/InstrRef/survives-livedebugvars.mir

 # RUN: llc -start-after=phi-node-elimination -stop-after=virtregrewriter %s -mtriple=x86_64-unknown-unknown -o - -experimental-debug-variable-locations | FileCheck %s
 # RUN: llc -O0 -start-after=phi-node-elimination -stop-after=regallocfast %s -mtriple=x86_64-unknown-unknown -o - -experimental-debug-variable-locations | FileCheck %s --check-prefix=FASTREG
 #
 # Test that DBG_INSTR_REFs can pass through livedebugvariables to the end of
 # regalloc without problem. Program body copied from
 # livedebugvars-crossbb-interval.mir.
 #
 # CHECK-LABEL: bb.0:
-# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0), dbg-instr-ref(2, 0)
 # CHECK-NEXT:  JMP_1
 # CHECK-LABEL: bb.1:
-# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0), dbg-instr-ref(3, 0)
 # CHECK-NEXT:  JMP_1
 # CHECK-LABEL: bb.2:
-# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0), dbg-instr-ref(4, 0)
 # CHECK-NEXT:  CALL64pcrel32
 # CHECK-LABEL: bb.3:
-# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0)
+# CHECK:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0), dbg-instr-ref(5, 0)
 # CHECK-NEXT:  JMP_1
 #
 #
 # The fast register allocator puts some spills in -- these are no-ops as far
 # as the slot indexes are concerned. It doesn't matter which side of spills
 # the DBG_INSTR_REF lands on.
 #
 # FASTREG-LABEL: bb.0:
-# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(1, 0), dbg-instr-ref(2, 0)
 # FASTREG-DAG:   MOV64mr
 # FASTREG-DAG:   MOV32mr
 # FASTREG-NEXT:  JMP_1
 # FASTREG-LABEL: bb.1:
-# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0)
+# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(2, 0), dbg-instr-ref(3, 0)
 # FASTREG-NEXT:  JMP_1
 # FASTREG-LABEL: bb.2:
-# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0)
+# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(3, 0), dbg-instr-ref(4, 0)
 # FASTREG-NEXT:  CALL64pcrel32
 # FASTREG-LABEL: bb.3:
 # FASTREG-DAG:   MOV32rm
-# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(4, 0), dbg-instr-ref(5, 0)
 # FASTREG-DAG:   MOV32mr
 # FASTREG-NEXT:  JMP_1
 # FASTREG-LABEL: bb.4:
-# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(5, 0)
+# FASTREG:       DBG_INSTR_REF {{.+}}, dbg-instr-ref(5, 0)
+# FASTREG-DAG:   DBG_INSTR_REF {{.+}}, dbg-instr-ref(5, 0), dbg-instr-ref(1, 0)
 # FASTREG-NEXT:  RET64
 
 --- |
   ; ModuleID = 'tmp.ll'
   source_filename = "tmp.ll"
   target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 
   %struct.a = type { i32 }
--- |
   !12 = !{!13}
   !13 = !DIDerivedType(tag: DW_TAG_member, name: "c", scope: !1, file: !1, line: 2, baseType: !7, size: 32, align: 32)
   !14 = !DILocalVariable(name: "x", scope: !15, file: !1, line: 6, type: !7)
   !15 = distinct !DILexicalBlock(scope: !4, file: !1, line: 5, column: 22)
   !16 = !DILocation(line: 5, column: 19, scope: !4)
   !17 = !DILocation(line: 6, column: 14, scope: !15)
   !18 = !DILocation(line: 8, column: 2, scope: !15)
   !19 = !DILocation(line: 7, column: 2, scope: !15)
+  !29 = !DILocalVariable(name: "c", scope: !4, file: !1, line: 5, type: !10)
 
 ...
 ---
 name:            bar
 alignment:       16
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gr64 }

 body:             |
   bb.0:
     liveins: $rdi, $esi
 
     %4:gr32 = COPY $esi
     %2:gr64 = COPY $rdi
     %3:gr64 = COPY killed %2
     %5:gr32 = COPY killed %4
-    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(1, 0), debug-location !16
+    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(1, 0), debug-location !16
+    DBG_INSTR_REF !29, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), dbg-instr-ref(1, 0), dbg-instr-ref(2, 0), debug-location !16
     JMP_1 %bb.3
 
   bb.1:
-    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(2, 0), debug-location !16
+    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(2, 0), debug-location !16
+    DBG_INSTR_REF !29, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), dbg-instr-ref(2, 0), dbg-instr-ref(3, 0), debug-location !16
     JMP_1 %bb.4
 
   bb.2:
     ADJCALLSTACKDOWN64 0, 0, 0, implicit-def $rsp, implicit-def $eflags, implicit-def $ssp, implicit $rsp, implicit $ssp, debug-location !19
     $edi = COPY %6, debug-location !19
     $al = MOV8ri 0, debug-location !19
-    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(3, 0), debug-location !16
+    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(3, 0), debug-location !16
+    DBG_INSTR_REF !29, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), dbg-instr-ref(3, 0), dbg-instr-ref(4, 0), debug-location !16
     CALL64pcrel32 @foo, csr_64, implicit $rsp, implicit $ssp, implicit $al, implicit $edi, implicit-def $eax, debug-location !19
     ADJCALLSTACKUP64 0, 0, implicit-def $rsp, implicit-def $eflags, implicit-def $ssp, implicit $rsp, implicit $ssp, debug-location !19
     %7:gr32 = COPY $eax, debug-location !19
     JMP_1 %bb.1
 
   bb.3:
     %6:gr32 = MOV32rm %3, 1, $noreg, 0, $noreg, debug-location !17
-    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(4, 0), debug-location !16
+    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(4, 0), debug-location !16
+    DBG_INSTR_REF !29, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), dbg-instr-ref(4, 0), dbg-instr-ref(5, 0), debug-location !16
     JMP_1 %bb.2
 
   bb.4:
     $eax = COPY %5, debug-location !18
-    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(5, 0), debug-location !16
+    DBG_INSTR_REF !9, !DIExpression(DW_OP_LLVM_arg, 0), dbg-instr-ref(5, 0), debug-location !16
+    DBG_INSTR_REF !29, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus), dbg-instr-ref(5, 0), dbg-instr-ref(1, 0), debug-location !16
     RET64 implicit $eax, debug-location !18
 
 ...

llvm/unittests/IR/MetadataTest.cpp

EXPECT_VARIADIC_OPS_EQUAL(GET_EXPR(dwarf::DW_OP_plus_uconst, 14,
                                      dwarf::DW_OP_plus_uconst, 14,
                                      dwarf::DW_OP_LLVM_fragment, 32, 32));
 
 #undef EXPECT_CONVERT_IS_NOOP
 #undef EXPECT_VARIADIC_OPS_EQUAL
 #undef GET_EXPR
 }
 
+TEST_F(DIExpressionTest, convertToNonVariadicExpression) {
+#define EXPECT_CONVERT_IS_NOOP(TestExpr)                                       \
+  do {                                                                         \
+    std::optional<const DIExpression *> NonVariadic =                          \
+        DIExpression::convertToNonVariadicExpression(TestExpr);                \
+    EXPECT_TRUE(NonVariadic.has_value());                                      \
+    EXPECT_EQ(*NonVariadic, TestExpr);                                         \
+  } while (false)
+#define EXPECT_NON_VARIADIC_OPS_EQUAL(TestExpr, Expected)                      \
+  do {                                                                         \
+    std::optional<const DIExpression *> NonVariadic =                          \
+        DIExpression::convertToNonVariadicExpression(TestExpr);                \
+    EXPECT_TRUE(NonVariadic.has_value());                                      \
+    EXPECT_EQ(*NonVariadic, Expected);                                         \
+  } while (false)
+#define EXPECT_INVALID_CONVERSION(TestExpr)                                    \
+  do {                                                                         \
+    std::optional<const DIExpression *> NonVariadic =                          \
+        DIExpression::convertToNonVariadicExpression(TestExpr);                \
+    EXPECT_FALSE(NonVariadic.has_value());                                     \
+  } while (false)
+#define GET_EXPR(...) DIExpression::get(Context, {__VA_ARGS__})
+
+  // Expressions which are already non-variadic should be unaffected.
+  EXPECT_CONVERT_IS_NOOP(GET_EXPR());
+  EXPECT_CONVERT_IS_NOOP(GET_EXPR(dwarf::DW_OP_plus_uconst, 4));
+  EXPECT_CONVERT_IS_NOOP(
+      GET_EXPR(dwarf::DW_OP_plus_uconst, 4, dwarf::DW_OP_stack_value));
+  EXPECT_CONVERT_IS_NOOP(GET_EXPR(dwarf::DW_OP_plus_uconst, 6,
+                                  dwarf::DW_OP_stack_value,
+                                  dwarf::DW_OP_LLVM_fragment, 32, 32));
+  EXPECT_CONVERT_IS_NOOP(GET_EXPR(dwarf::DW_OP_plus_uconst, 14,
+                                  dwarf::DW_OP_LLVM_fragment, 32, 32));
+
+  // Variadic expressions with a single leading `LLVM_arg 0` and no other
+  // LLVM_args should have the leading arg removed.
+  EXPECT_NON_VARIADIC_OPS_EQUAL(GET_EXPR(dwarf::DW_OP_LLVM_arg, 0), GET_EXPR());
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_stack_value),
+      GET_EXPR(dwarf::DW_OP_stack_value));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_LLVM_fragment, 16, 32),
+      GET_EXPR(dwarf::DW_OP_LLVM_fragment, 16, 32));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_stack_value,
+               dwarf::DW_OP_LLVM_fragment, 24, 32),
+      GET_EXPR(dwarf::DW_OP_stack_value, dwarf::DW_OP_LLVM_fragment, 24, 32));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_plus_uconst, 4),
+      GET_EXPR(dwarf::DW_OP_plus_uconst, 4));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_plus_uconst, 4,
+               dwarf::DW_OP_stack_value),
+      GET_EXPR(dwarf::DW_OP_plus_uconst, 4, dwarf::DW_OP_stack_value));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_plus_uconst, 6,
+               dwarf::DW_OP_stack_value, dwarf::DW_OP_LLVM_fragment, 32, 32),
+      GET_EXPR(dwarf::DW_OP_plus_uconst, 6, dwarf::DW_OP_stack_value,
+               dwarf::DW_OP_LLVM_fragment, 32, 32));
+  EXPECT_NON_VARIADIC_OPS_EQUAL(GET_EXPR(dwarf::DW_OP_LLVM_arg, 0,
+                                         dwarf::DW_OP_plus_uconst, 14,
+                                         dwarf::DW_OP_LLVM_fragment, 32, 32),
+                                GET_EXPR(dwarf::DW_OP_plus_uconst, 14,
+                                         dwarf::DW_OP_LLVM_fragment, 32, 32));
+
+  // Variadic expressions that have any LLVM_args other than a leading
+  // `LLVM_arg 0` cannot be converted and so should return std::nullopt.
+  EXPECT_INVALID_CONVERSION(GET_EXPR(
+      dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_LLVM_arg, 1, dwarf::DW_OP_mul));
+  EXPECT_INVALID_CONVERSION(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_LLVM_arg, 1,
+               dwarf::DW_OP_plus, dwarf::DW_OP_stack_value));
+  EXPECT_INVALID_CONVERSION(
+      GET_EXPR(dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_LLVM_arg, 0,
+               dwarf::DW_OP_minus, dwarf::DW_OP_stack_value));
+  EXPECT_INVALID_CONVERSION(GET_EXPR(dwarf::DW_OP_constu, 5,
+                                     dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_div,
+                                     dwarf::DW_OP_stack_value));
+
+#undef EXPECT_CONVERT_IS_NOOP
+#undef EXPECT_NON_VARIADIC_OPS_EQUAL
+#undef EXPECT_INVALID_CONVERSION
+#undef GET_EXPR
+}
+
 TEST_F(DIExpressionTest, replaceArg) {
 #define EXPECT_REPLACE_ARG_EQ(Expr, OldArg, NewArg, ...)                       \
   do {                                                                         \
     uint64_t Elements[] = {__VA_ARGS__};                                       \
     ArrayRef<uint64_t> Expected = Elements;                                    \
     DIExpression *Expression = DIExpression::replaceArg(Expr, OldArg, NewArg); \
     EXPECT_EQ(Expression->getElements(), Expected);                            \
   } while (false)