[DebugInfo] Produce variadic DBG_INSTR_REFs from ISel

[StephenTozer]

Summary

This patch modifies SelectionDAG and FastISel to produce DBG_INSTR_REFs with variadic expressions, and produce DBG_INSTR_REFs for debug values with variadic location expressions. The former essentially means just prepending DW_OP_LLVM_arg, 0 to the existing expression. The latter is achieved in MachineFunction::finalizeDebugInstrRefs and InstrEmitter::EmitDbgInstrRef.

[jmorse]

Generally looks good -- needs tests though.

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

826–828

Haven't these all been excluded due to IsInvalidOp?

[jmorse]

The "arg zero" form of DIExpression is going to be a very common case -- is it uniqued, and would it be worth storing a copy somewhere and just using that?

[StephenTozer]

In D133929#3796075, @jmorse wrote:

The "arg zero" form of DIExpression is going to be a very common case -- is it uniqued, and would it be worth storing a copy somewhere and just using that?

DIExpressions should be uniqued based on their expression elements, and the get should just be a map lookup underneath, so I don't think there should be any issues with duplicate storage.

[StephenTozer]

Rebased, fixed some errors when emitting undef values and update some tests accordingly.

[StephenTozer]

Added some test coverage for SelectionDAG+FastISel emission of variadic instruction references.

[Orlando]

This LGTM with a few inline questions. The addition of DW_OP_arg, 0 to all INSTR_REF expressions does add some visual noise but I personally prefer this. Given how convoluted debug instruction handling can get, I like the idea of continuing in this direction and minimising the number of special cases.

llvm/lib/CodeGen/MachineFunction.cpp

1194

👍

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

699

Possibly worth expanding the comment to mention that these will have been emitted as INSTR_REF by the code above if instr ref is enabled.

849–850

Looks like this one slipped through the cracks - can this TODO be done? IMO eliminating duplicated code is especially important around these fiddly areas.

921

Is there a benefit to preserving the number of operands (rather than just using DBG_VALUE $noreg) - does it strip assertions about DW_OP_arg usage or something if we don't?

[StephenTozer]

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

921

The only important part about this is that it doesn't invalidate the DIExpression - even for an undef debug value, I believe there are still some points at which having a mismatched number of debug operands with the DIExpression's arg operands is a problem. This could be resolved by also changing the expression, making it empty while any fragment info or other expression elements that apply to undef values - which sounds like something that should be done for all explicitly killed debug values, but happy to start the ball rolling here.

[StephenTozer]

Address comments (extracted MO selection for const debug operands, expanded comments in EmitDbgValue), add a function for setting a DIExpression "undef".

[Orlando]

LGTM

[StephenTozer]

Rebased onto latest main, added unit tests for new convertToUndefExpression, and converted some existing tests to test for the new fully-undef debug values that SelectionDAG produces.

[Orlando]

Still LGTM, nice unittest.

llvm/lib/CodeGen/MachineFunction.cpp

 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/CodeGen/WasmEHFuncInfo.h"
 #include "llvm/CodeGen/WinEHFuncInfo.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/IR/Attributes.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Constant.h"
 #include "llvm/IR/DataLayout.h"
-#include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSlotTracker.h"

 }
 
 void MachineFunction::finalizeDebugInstrRefs() {
   auto *TII = getSubtarget().getInstrInfo();
 
   auto MakeUndefDbgValue = [&](MachineInstr &MI) {
     const MCInstrDesc &RefII = TII->get(TargetOpcode::DBG_VALUE_LIST);
     MI.setDesc(RefII);
-    MI.getDebugOperand(0).setReg(0);
+    MI.setDebugValueUndef();
   };
 
   DenseMap<Register, DebugInstrOperandPair> ArgDbgPHIs;
   for (auto &MBB : *this) {
     for (auto &MI : MBB) {
-      if (!MI.isDebugRef() || !MI.getDebugOperand(0).isReg())
+      if (!MI.isDebugRef())
         continue;
 
-      Register Reg = MI.getDebugOperand(0).getReg();
+      bool IsValidRef = true;
+
+      for (MachineOperand &MO : MI.debug_operands()) {
+        if (!MO.isReg())
+          continue;
+
+        Register Reg = MO.getReg();
 
         // Some vregs can be deleted as redundant in the meantime. Mark those
         // as DBG_VALUE $noreg. Additionally, some normal instructions are
         // quickly deleted, leaving dangling references to vregs with no def.
         if (Reg == 0 || !RegInfo->hasOneDef(Reg)) {
-        MakeUndefDbgValue(MI);
-        continue;
+          IsValidRef = false;
+          break;
         }
-      // Only convert Expr to variadic form when we're sure we're emitting a
-      // complete instruction reference.
-      MI.getDebugExpressionOp().setMetadata(
-        DIExpression::convertToVariadicExpression(MI.getDebugExpression()));
 
         assert(Reg.isVirtual());
         MachineInstr &DefMI = *RegInfo->def_instr_begin(Reg);
 
         // If we've found a copy-like instruction, follow it back to the
         // instruction that defines the source value, see salvageCopySSA docs
         // for why this is important.
         if (DefMI.isCopyLike() || TII->isCopyInstr(DefMI)) {
           auto Result = salvageCopySSA(DefMI, ArgDbgPHIs);
-        MI.getDebugOperand(0).ChangeToDbgInstrRef(Result.first, Result.second);
+          MO.ChangeToDbgInstrRef(Result.first, Result.second);
         } else {
           // Otherwise, identify the operand number that the VReg refers to.
           unsigned OperandIdx = 0;
-        for (const auto &MO : DefMI.operands()) {
-          if (MO.isReg() && MO.isDef() && MO.getReg() == Reg)
+          for (const auto &DefMO : DefMI.operands()) {
+            if (DefMO.isReg() && DefMO.isDef() && DefMO.getReg() == Reg)
               break;
             ++OperandIdx;
           }
           assert(OperandIdx < DefMI.getNumOperands());
 
           // Morph this instr ref to point at the given instruction and operand.
           unsigned ID = DefMI.getDebugInstrNum();
-        MI.getDebugOperand(0).ChangeToDbgInstrRef(ID, OperandIdx);
+          MO.ChangeToDbgInstrRef(ID, OperandIdx);
+        }
       }
+
+      if (!IsValidRef)
+        MakeUndefDbgValue(MI);
     }
   }
 }
 
 bool MachineFunction::useDebugInstrRef() const {
   // Disable instr-ref at -O0: it's very slow (in compile time). We can still
   // have optimized code inlined into this unoptimized code, however with
   // fewer and less aggressive optimizations happening, coverage and accuracy

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

case Intrinsic::dbg_declare: {
 
     if (Op) {
       assert(DI->getVariable()->isValidLocationForIntrinsic(MIMD.getDL()) &&
              "Expected inlined-at fields to agree");
       if (UseInstrRefDebugInfo && Op->isReg()) {
         // If using instruction referencing, produce this as a DBG_INSTR_REF,
         // to be later patched up by finalizeDebugInstrRefs. Tack a deref onto
         // the expression, we don't have an "indirect" flag in DBG_INSTR_REF.
-        auto *NewExpr =
-           DIExpression::prepend(DI->getExpression(), DIExpression::DerefBefore);
+        SmallVector<uint64_t, 3> Ops(
+            {dwarf::DW_OP_LLVM_arg, 0, dwarf::DW_OP_deref});
+        auto *NewExpr = DIExpression::prependOpcodes(DI->getExpression(), Ops);
         BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, MIMD.getDL(),
                 TII.get(TargetOpcode::DBG_INSTR_REF), /*IsIndirect*/ false, *Op,
                 DI->getVariable(), NewExpr);
       } else {
         // A dbg.declare describes the address of a source variable, so lower it
         // into an indirect DBG_VALUE.
         BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, MIMD.getDL(),
                 TII.get(TargetOpcode::DBG_VALUE), /*IsIndirect*/ true, *Op,
if (!V || isa<UndefValue>(V) || DI->hasArgList()) {
       } else {
         // If using instruction referencing, produce this as a DBG_INSTR_REF,
         // to be later patched up by finalizeDebugInstrRefs.
         SmallVector<MachineOperand, 1> MOs({MachineOperand::CreateReg(
             /* Reg */ Reg, /* isDef */ false, /* isImp */ false,
             /* isKill */ false, /* isDead */ false,
             /* isUndef */ false, /* isEarlyClobber */ false,
             /* SubReg */ 0, /* isDebug */ true)});
+        SmallVector<uint64_t, 2> Ops({dwarf::DW_OP_LLVM_arg, 0});
+        auto *NewExpr = DIExpression::prependOpcodes(DI->getExpression(), Ops);
         BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, MIMD.getDL(),
                 TII.get(TargetOpcode::DBG_INSTR_REF), /*IsIndirect*/ false, MOs,
-                DI->getVariable(), DI->getExpression());
+                DI->getVariable(), NewExpr);
       }
     } else {
       // We don't know how to handle other cases, so we drop.
       LLVM_DEBUG(dbgs() << "Dropping debug info for " << *DI << "\n");
     }
     return true;
   }
   case Intrinsic::dbg_label: {

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.h

public:
   /// if there is no variable location; alternately a half-formed DBG_INSTR_REF
   /// that refers to a virtual register and is corrected later in isel.
   MachineInstr *EmitDbgInstrRef(SDDbgValue *SD,
                                 DenseMap<SDValue, Register> &VRBaseMap);
 
   /// Emit a DBG_VALUE $noreg, indicating a variable has no location.
   MachineInstr *EmitDbgNoLocation(SDDbgValue *SD);
 
+  /// Emit a DBG_VALUE_LIST from the operands to SDDbgValue.
+  MachineInstr *EmitDbgValueList(SDDbgValue *SD,
+                                 DenseMap<SDValue, Register> &VRBaseMap);
+
   /// Emit a DBG_VALUE from the operands to SDDbgValue.
   MachineInstr *EmitDbgValueFromSingleOp(SDDbgValue *SD,
                                     DenseMap<SDValue, Register> &VRBaseMap);
 
   /// Generate machine instruction for a dbg_label node.
   MachineInstr *EmitDbgLabel(SDDbgLabel *SD);
 
   /// EmitNode - Generate machine code for a node and needed dependencies.

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

 }
 
 /// EmitDbgValue - Generate machine instruction for a dbg_value node.
 ///
 MachineInstr *
 InstrEmitter::EmitDbgValue(SDDbgValue *SD,
                            DenseMap<SDValue, Register> &VRBaseMap) {
   MDNode *Var = SD->getVariable();
-  MDNode *Expr = SD->getExpression();
   DebugLoc DL = SD->getDebugLoc();
   assert(cast<DILocalVariable>(Var)->isValidLocationForIntrinsic(DL) &&
          "Expected inlined-at fields to agree");
 
   SD->setIsEmitted();
 
   ArrayRef<SDDbgOperand> LocationOps = SD->getLocationOps();
   assert(!LocationOps.empty() && "dbg_value with no location operands?");
 
   if (SD->isInvalidated())
     return EmitDbgNoLocation(SD);
 
-  // Emit variadic dbg_value nodes as DBG_VALUE_LIST.
-  if (SD->isVariadic()) {
-    // DBG_VALUE_LIST := "DBG_VALUE_LIST" var, expression, loc (, loc)*
-    const MCInstrDesc &DbgValDesc = TII->get(TargetOpcode::DBG_VALUE_LIST);
-    // Build the DBG_VALUE_LIST instruction base.
-    auto MIB = BuildMI(*MF, DL, DbgValDesc);
-    MIB.addMetadata(Var);
-    MIB.addMetadata(Expr);
-    AddDbgValueLocationOps(MIB, DbgValDesc, LocationOps, VRBaseMap);
-    return &*MIB;
-  }
-
   // Attempt to produce a DBG_INSTR_REF if we've been asked to.
-  // We currently exclude the possibility of instruction references for
-  // variadic nodes; if at some point we enable them, this should be moved
-  // above the variadic block.
   if (EmitDebugInstrRefs)
     if (auto *InstrRef = EmitDbgInstrRef(SD, VRBaseMap))
       return InstrRef;
 
+  // Emit variadic dbg_value nodes as DBG_VALUE_LIST.
+  if (SD->isVariadic())
+    return EmitDbgValueList(SD, VRBaseMap);
+
   return EmitDbgValueFromSingleOp(SD, VRBaseMap);
 }
 
 void InstrEmitter::AddDbgValueLocationOps(
     MachineInstrBuilder &MIB, const MCInstrDesc &DbgValDesc,
     ArrayRef<SDDbgOperand> LocationOps,
     DenseMap<SDValue, Register> &VRBaseMap) {
   for (const SDDbgOperand &Op : LocationOps) {
for (const SDDbgOperand &Op : LocationOps) {
     } break;
     }
   }
 }
 
 MachineInstr *
 InstrEmitter::EmitDbgInstrRef(SDDbgValue *SD,
                               DenseMap<SDValue, Register> &VRBaseMap) {
-  assert(!SD->isVariadic());
-  SDDbgOperand DbgOperand = SD->getLocationOps()[0];
   MDNode *Var = SD->getVariable();
   const DIExpression *Expr = (DIExpression*)SD->getExpression();
   DebugLoc DL = SD->getDebugLoc();
   const MCInstrDesc &RefII = TII->get(TargetOpcode::DBG_INSTR_REF);
 
-  // Handle variable locations that don't actually depend on the instructions
-  // in the program: constants and stack locations.
-  if (DbgOperand.getKind() == SDDbgOperand::FRAMEIX ||
-      DbgOperand.getKind() == SDDbgOperand::CONST)
+  // Returns true if the given operand is not a legal debug operand for a
+  // DBG_INSTR_REF.
+  auto IsInvalidOp = [](SDDbgOperand DbgOp) {
+    return DbgOp.getKind() == SDDbgOperand::FRAMEIX;
+  };
+  // Returns true if the given operand is not itself an instruction reference
+  // but is a legal debug operand for a DBG_INSTR_REF.
+  auto IsNonInstrRefOp = [](SDDbgOperand DbgOp) {
+    return DbgOp.getKind() == SDDbgOperand::CONST;
+  };
+
+  // If this variable location does not depend on any instructions or contains
+  // any stack locations, produce it as a standard debug value instead.
+  if (any_of(SD->getLocationOps(), IsInvalidOp) ||
+      all_of(SD->getLocationOps(), IsNonInstrRefOp)) {
+    if (SD->isVariadic())
+      return EmitDbgValueList(SD, VRBaseMap);
     return EmitDbgValueFromSingleOp(SD, VRBaseMap);
+  }
 
   // Immediately fold any indirectness from the LLVM-IR intrinsic into the
   // expression:
-  if (SD->isIndirect()) {
-    std::vector<uint64_t> Elts = {dwarf::DW_OP_deref};
-    Expr = DIExpression::append(Expr, Elts);
-  }
+  if (SD->isIndirect())
+    Expr = DIExpression::append(Expr, dwarf::DW_OP_deref);
+  // If this is not already a variadic expression, it must be modified to become
+  // one.
+  if (!SD->isVariadic())
+    Expr = DIExpression::convertToVariadicExpression(Expr);
+
+  SmallVector<MachineOperand> MOs;
 
   // It may not be immediately possible to identify the MachineInstr that
   // defines a VReg, it can depend for example on the order blocks are
   // emitted in. When this happens, or when further analysis is needed later,
   // produce an instruction like this:
   //
-  //    DBG_INSTR_REF %0:gr64, 0, !123, !456
+  //    DBG_INSTR_REF !123, !456, %0:gr64
   //
   // i.e., point the instruction at the vreg, and patch it up later in
   // MachineFunction::finalizeDebugInstrRefs.
-  auto EmitHalfDoneInstrRef = [&](unsigned VReg) -> MachineInstr * {
-    SmallVector<MachineOperand, 1> MOs({MachineOperand::CreateReg(
+  auto AddVRegOp = [&](unsigned VReg) {
+    MOs.push_back(MachineOperand::CreateReg(
         /* Reg */ VReg, /* isDef */ false, /* isImp */ false,
         /* isKill */ false, /* isDead */ false,
         /* isUndef */ false, /* isEarlyClobber */ false,
-        /* SubReg */ 0, /* isDebug */ true)});
-    return BuildMI(*MF, DL, RefII, false, MOs, Var, Expr);
+        /* SubReg */ 0, /* isDebug */ true));
   };
+  unsigned OpCount = SD->getLocationOps().size();
+  for (unsigned OpIdx = 0; OpIdx < OpCount; ++OpIdx) {
+    SDDbgOperand DbgOperand = SD->getLocationOps()[OpIdx];
 
     // Try to find both the defined register and the instruction defining it.
     MachineInstr *DefMI = nullptr;
     unsigned VReg;
 
     if (DbgOperand.getKind() == SDDbgOperand::VREG) {
       VReg = DbgOperand.getVReg();
 
       // No definition means that block hasn't been emitted yet. Leave a vreg
       // reference to be fixed later.
-    if (!MRI->hasOneDef(VReg))
-      return EmitHalfDoneInstrRef(VReg);
+      if (!MRI->hasOneDef(VReg)) {
+        AddVRegOp(VReg);
+        continue;
+      }
 
       DefMI = &*MRI->def_instr_begin(VReg);
-  } else {
-    assert(DbgOperand.getKind() == SDDbgOperand::SDNODE);
+    } else if (DbgOperand.getKind() == SDDbgOperand::SDNODE) {
       // Look up the corresponding VReg for the given SDNode, if any.
       SDNode *Node = DbgOperand.getSDNode();
       SDValue Op = SDValue(Node, DbgOperand.getResNo());
       DenseMap<SDValue, Register>::iterator I = VRBaseMap.find(Op);
       // No VReg -> produce a DBG_VALUE $noreg instead.
-    if (I==VRBaseMap.end())
-      return EmitDbgNoLocation(SD);
+      if (I == VRBaseMap.end())
+        break;
 
       // Try to pick out a defining instruction at this point.
       VReg = getVR(Op, VRBaseMap);
 
       // Again, if there's no instruction defining the VReg right now, fix it up
       // later.
-    if (!MRI->hasOneDef(VReg))
-      return EmitHalfDoneInstrRef(VReg);
+      if (!MRI->hasOneDef(VReg)) {
+        AddVRegOp(VReg);
+        continue;
+      }
 
       DefMI = &*MRI->def_instr_begin(VReg);
+    } else {
+      assert(DbgOperand.getKind() == SDDbgOperand::CONST);
+      // TODO: Probably extract this out into a function shared with
+      // AddDbgValueLocationOps.
+      const Value *V = DbgOperand.getConst();
+      if (const ConstantInt *CI = dyn_cast<ConstantInt>(V)) {
+        if (CI->getBitWidth() > 64)
+          MOs.push_back(MachineOperand::CreateCImm(CI));
+        else
+          MOs.push_back(MachineOperand::CreateImm(CI->getSExtValue()));
+      } else if (const ConstantFP *CF = dyn_cast<ConstantFP>(V)) {
+        MOs.push_back(MachineOperand::CreateFPImm(CF));
+      } else if (isa<ConstantPointerNull>(V)) {
+        // Note: This assumes that all nullptr constants are zero-valued.
+        MOs.push_back(MachineOperand::CreateImm(0));
+      } else {
+        // Could be an Undef. In any case insert an Undef so we can see what we
+        // dropped.
+        AddVRegOp(0U);
+      }
+      continue;
     }
 
     // Avoid copy like instructions: they don't define values, only move them.
-  // Leave a virtual-register reference until it can be fixed up later, to find
-  // the underlying value definition.
-  if (DefMI->isCopyLike() || TII->isCopyInstr(*DefMI))
-    return EmitHalfDoneInstrRef(VReg);
+    // Leave a virtual-register reference until it can be fixed up later, to
+    // find the underlying value definition.
+    if (DefMI->isCopyLike() || TII->isCopyInstr(*DefMI)) {
+      AddVRegOp(VReg);
+      continue;
+    }
 
     // Find the operand number which defines the specified VReg.
     unsigned OperandIdx = 0;
     for (const auto &MO : DefMI->operands()) {
       if (MO.isReg() && MO.isDef() && MO.getReg() == VReg)
         break;
       ++OperandIdx;
     }
     assert(OperandIdx < DefMI->getNumOperands());
 
-  // Only convert Expr to variadic form when we're sure we're emitting a
-  // complete instruction reference.
-  if (!SD->isVariadic())
-    Expr = DIExpression::convertToVariadicExpression(Expr);
     // Make the DBG_INSTR_REF refer to that instruction, and that operand.
     unsigned InstrNum = DefMI->getDebugInstrNum();
-  SmallVector<MachineOperand, 1> MOs(
-      {MachineOperand::CreateDbgInstrRef(InstrNum, OperandIdx)});
+    MOs.push_back(MachineOperand::CreateDbgInstrRef(InstrNum, OperandIdx));
+  }
+
+  // If we haven't created a valid MachineOperand for every DbgOp, abort and
+  // produce an undef DBG_VALUE.
+  if (MOs.size() != OpCount)
+    return EmitDbgNoLocation(SD);
+
   return BuildMI(*MF, DL, RefII, false, MOs, Var, Expr);
 }
 
 MachineInstr *InstrEmitter::EmitDbgNoLocation(SDDbgValue *SD) {
   // An invalidated SDNode must generate an undef DBG_VALUE: although the
   // original value is no longer computed, earlier DBG_VALUEs live ranges
   // must not leak into later code.
   MDNode *Var = SD->getVariable();
   MDNode *Expr = SD->getExpression();
   DebugLoc DL = SD->getDebugLoc();
-  auto MIB = BuildMI(*MF, DL, TII->get(TargetOpcode::DBG_VALUE));
+  const MCInstrDesc& Desc = TII->get(SD->isVariadic()
+    ? TargetOpcode::DBG_VALUE_LIST
+    : TargetOpcode::DBG_VALUE);
+  auto MIB = BuildMI(*MF, DL, Desc);
+  if (!SD->isVariadic()) {
+    MIB.addReg(0U);
     MIB.addReg(0U);
+  }
+  MIB.addMetadata(Var);
+  MIB.addMetadata(Expr);
+  if (SD->isVariadic()) {
+    for (unsigned OpIdx = 0; OpIdx < SD->getLocationOps().size(); ++OpIdx)
       MIB.addReg(0U);
+  }
+  return &*MIB;
+}
+
+MachineInstr *
+InstrEmitter::EmitDbgValueList(SDDbgValue *SD,
+                               DenseMap<SDValue, Register> &VRBaseMap) {
+  MDNode *Var = SD->getVariable();
+  DIExpression *Expr = SD->getExpression();
+  DebugLoc DL = SD->getDebugLoc();
+  // DBG_VALUE_LIST := "DBG_VALUE_LIST" var, expression, loc (, loc)*
+  const MCInstrDesc &DbgValDesc = TII->get(TargetOpcode::DBG_VALUE_LIST);
+  // Build the DBG_VALUE_LIST instruction base.
+  auto MIB = BuildMI(*MF, DL, DbgValDesc);
   MIB.addMetadata(Var);
   MIB.addMetadata(Expr);
+  AddDbgValueLocationOps(MIB, DbgValDesc, SD->getLocationOps(), VRBaseMap);
   return &*MIB;
 }
 
 MachineInstr *
 InstrEmitter::EmitDbgValueFromSingleOp(SDDbgValue *SD,
                                        DenseMap<SDValue, Register> &VRBaseMap) {
   MDNode *Var = SD->getVariable();
   DIExpression *Expr = SD->getExpression();

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

if (Reg.isVirtual() && MF.useDebugInstrRef()) {
           /* isUndef */ false, /* isEarlyClobber */ false,
           /* SubReg */ 0, /* isDebug */ true)});
 
       auto *NewDIExpr = FragExpr;
       // We don't have an "Indirect" field in DBG_INSTR_REF, fold that into
       // the DIExpression.
       if (Indirect)
         NewDIExpr = DIExpression::prepend(FragExpr, DIExpression::DerefBefore);
+      SmallVector<uint64_t, 2> Ops({dwarf::DW_OP_LLVM_arg, 0});
+      NewDIExpr = DIExpression::prependOpcodes(NewDIExpr, Ops);
       return BuildMI(MF, DL, Inst, false, MOs, Variable, NewDIExpr);
     } else {
       // Create a completely standard DBG_VALUE.
       auto &Inst = TII->get(TargetOpcode::DBG_VALUE);
       return BuildMI(MF, DL, Inst, Indirect, Reg, Variable, FragExpr);
     }
   };
 

llvm/test/DebugInfo/X86/arg-dbg-value-list.ll

 ;; FIXME: We currently don't do any special bookkeeping for unused args used by
 ;; variadic dbg_values. When/if we support them, the DBG_VALUE_LIST should be
 ;; updated accordingly.
 
 ; RUN: llc %s -start-after=codegenprepare -stop-before=finalize-isel -o - | FileCheck %s
 
 ;; Check that unused argument values are handled the same way for variadic
 ;; dbg_values as non-variadics.
 
 ; CHECK: ![[A:[0-9]+]] = !DILocalVariable(name: "a",
 ; CHECK: ![[B:[0-9]+]] = !DILocalVariable(name: "b",
 ; CHECK: ![[C:[0-9]+]] = !DILocalVariable(name: "c",
 
 ; CHECK: DBG_VALUE $ecx, $noreg, ![[A]], !DIExpression(), debug-location
 ; CHECK: DBG_VALUE $edx, $noreg, ![[B]], !DIExpression(), debug-location
-; CHECK: DBG_VALUE $noreg, $noreg, ![[C]], !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value), debug-location
+; CHECK: DBG_VALUE_LIST ![[C]], !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value), $noreg, $noreg, debug-location
 
 target triple = "x86_64-pc-windows-msvc19.16.27034"
 define dso_local i32 @"?foo@@YAHHH@Z"(i32 %a, i32 %b) local_unnamed_addr !dbg !8 {
 entry:
   call void @llvm.dbg.value(metadata i32 %a, metadata !14, metadata !DIExpression()), !dbg !17
   call void @llvm.dbg.value(metadata i32 %b, metadata !15, metadata !DIExpression()), !dbg !17
   call void @llvm.dbg.value(metadata !DIArgList(i32 %a, i32 %b), metadata !16, metadata !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value)), !dbg !17
   ret i32 0

llvm/test/DebugInfo/X86/dbg-val-list-undef.ll

 ; RUN: llc %s -start-after=codegenprepare -stop-before=finalize-isel -o - | FileCheck %s
 
 ;; %y is unused and cannot (FIXME: currently) be salvaged. Ensure that the
 ;; variadic dbg_value using %y becomes undef.
 
 ; CHECK: ![[C:[0-9]+]] = !DILocalVariable(name: "c",
-; CHECK: DBG_VALUE $noreg, $noreg, ![[C]], !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value),  debug-location
+; CHECK: DBG_VALUE_LIST ![[C]], !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value), $noreg, $noreg, debug-location
 
 target triple = "x86_64-pc-windows-msvc19.16.27034"
 define dso_local i32 @"?foo@@YAHHH@Z"(i32 %a, i32 %b) local_unnamed_addr !dbg !8 {
 entry:
   %x = add i32 %a, %b
   %y = mul i32 %x, %b
   call void @llvm.dbg.value(metadata !DIArgList(i32 %x, i32 %y), metadata !16, metadata !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_plus, DW_OP_stack_value)), !dbg !17
   ret i32 %x

llvm/test/DebugInfo/X86/invalidated-dbg-value-is-undef.ll

 ; RUN: llc %s --stop-before=finalize-isel -o - | FileCheck %s --implicit-check-not=DBG_VALUE
 
 ;; Check that when a debug value is invalidated during Instruction Selection,
 ;; we produce an undef DBG_VALUE/DBG_VALUE_LIST.
 
 ; CHECK-LABEL: body:
 
-; CHECK: DBG_VALUE $noreg, $noreg, ![[VAR:[0-9]+]]
+; CHECK: DBG_VALUE_LIST ![[VAR:[0-9]+]], {{.+}}, $noreg, $noreg
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 @intel_pmu_enable_bts_config = external dso_local local_unnamed_addr global i32, align 4, !dbg !0
 
 define dso_local i32 @intel_pmu_enable_bts() local_unnamed_addr !dbg !16 {
 entry: