[Coroutines] Use dbg.declare for frame variables

[modocache]

Summary

https://gist.github.com/modocache/ed7c62f6e570766c0f39b35dad675c2f
is an example of a small C++ program that uses C++20 coroutines that
is difficult to debug, due to the loss of debug info for variables that
"spill" across coroutine suspension boundaries. This patch addresses
that issue by inserting 'llvm.dbg.declare' intrinsics that point the
debugger to the variables' location at an offset to the coroutine frame.

With this patch, I confirmed that running the 'frame variable' commands in
https://gist.github.com/modocache/ed7c62f6e570766c0f39b35dad675c2f at
the specified breakpoints results in the correct values being printed
for coroutine frame variables 'i' and 'j' when using an lldb built from
trunk, as well as with gdb 8.3 (lldb 9.0.1, however, could not print the
values). The added test case also verifies this improved behavior.

The existing coro-debug.ll test case is also modified to reflect the
locations at which Clang actually places calls to 'dbg.declare', and
additional checks are added to ensure this patch works as intended in that
example as well.

[modocache]

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

878

Beginner question of mine: is there a good way to reduce this debugging metadata to just the nodes that are necessary for my test? The 654 (!!) nodes here are the result of having run llvm-reduce on this test case, but I think many of these remaining nodes come from my having #include <experimental/coroutine> in my test program, and not actually necessary for the test case.

[dstenb]

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

878

Bugpoint can reduce metadata, but I have personally found that it is not the best at that. It might just be a PEBKAC problem though. I personally just try to minimize the number of metadata nodes that Clang emits by first creducing the source file, omitting information that is not needed (e.g. passing -gno-column-info and -fno-strict-aliasing depending on situation), and then hand reducing the IR file.

In this case it seems that we can remove all the import node references in !108 (I assume they are not needed for this test case), and then run opt -S on the file, to remove all unused nodes. That leaves around 120 metadata nodes. I don't know much about C++ metadata (especially coroutines), so not sure how much more can be removed, but perhaps that is a start?

[modocache]

Wow, great suggestion, @dstenb! That brought the number of nodes down to 119 -- a reduction of 82%! (Also, I ran clang-format on the include files to appease the pre-merge checkers -- although to be honest, I'm not sure I agree with the rationale here, because this diff didn't modify the include that clang-format is complaining about. But oh well.)

[modocache]

I played around a bit more and got it down to just 13 nodes. I think this is the bare minimum.

[modocache]

Remove Git commit hashes from the debug info metadata, just to slim it down a tiny bit more.

[aprantl]

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

878

Delta http://delta.tigris.org is always your friend. My favorite trick is taking all nodes that are lists of metadata (!1 = {!2, !3}) and making sure the elements are on a separate line followed by a run of delta. Other methods include removing (most) !dbg attachments on instructions and of course removing all unnecessary dbg.value intrinsics and their dependencies. You can also replace most "parent:"-kind fields with a link to the DIFile or CU.

[aprantl]

As a side note: Are you using LLDB as your debugger? We could probably use a couple of end-to-end tests in the LLDB testsuite for coroutines as well if you are interested in contributing one.

[aprantl]

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

29

we should also check that the !dbg location attachment is correct.

[modocache]

As a side note: Are you using LLDB as your debugger? We could probably use a couple of end-to-end tests in the LLDB testsuite for coroutines as well if you are interested in contributing one.

I would absolutely love to add some! I was hoping to supplement the tests here with something that would act as an end-to-end regression test of my specific C++ source file. I'll grep around for examples of how to do that, if there's a specific doc on how to add my first test please point it out to me :)

[aprantl]

In D75338#1898454, @modocache wrote:

As a side note: Are you using LLDB as your debugger? We could probably use a couple of end-to-end tests in the LLDB testsuite for coroutines as well if you are interested in contributing one.

I would absolutely love to add some! I was hoping to supplement the tests here with something that would act as an end-to-end regression test of my specific C++ source file. I'll grep around for examples of how to do that, if there's a specific doc on how to add my first test please point it out to me :)

A good starting point is the LLDB example test: https://github.com/llvm/llvm-project/tree/master/lldb/test/API/sample_test (This example shows *both* how to write an inline test and one with a separate python driver).

[modocache]

I tried adding an lldb test for this, but I encountered 39 failing tests when I ran ninja cxx check-lldb, and it made me wonder whether tests that use libc++ are currently broken?

Testing Time: 343.25s
********************
Failing Tests (39):
    lldb-api :: commands/expression/import-std-module/basic/TestImportStdModule.py
    lldb-api :: commands/expression/import-std-module/conflicts/TestStdModuleWithConflicts.py
    lldb-api :: commands/expression/import-std-module/deque-basic/TestDequeFromStdModule.py
    lldb-api :: commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py
    lldb-api :: commands/expression/import-std-module/forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule.py
    lldb-api :: commands/expression/import-std-module/forward_list/TestForwardListFromStdModule.py
    lldb-api :: commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py
    lldb-api :: commands/expression/import-std-module/list/TestListFromStdModule.py
    lldb-api :: commands/expression/import-std-module/no-std-module/TestMissingStdModule.py
    lldb-api :: commands/expression/import-std-module/queue/TestQueueFromStdModule.py
    lldb-api :: commands/expression/import-std-module/shared_ptr-dbg-info-content/TestSharedPtrDbgInfoContentFromStdModule.py
    lldb-api :: commands/expression/import-std-module/shared_ptr/TestSharedPtrFromStdModule.py
    lldb-api :: commands/expression/import-std-module/stack/TestStackFromStdModule.py
    lldb-api :: commands/expression/import-std-module/unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent.py
    lldb-api :: commands/expression/import-std-module/unique_ptr/TestUniquePtrFromStdModule.py
    lldb-api :: commands/expression/import-std-module/vector-bool/TestVectorBoolFromStdModule.py
    lldb-api :: commands/expression/import-std-module/vector-dbg-info-content/TestDbgInfoContentVectorFromStdModule.py
    lldb-api :: commands/expression/import-std-module/vector-of-vectors/TestVectorOfVectorsFromStdModule.py
    lldb-api :: commands/expression/import-std-module/vector/TestVectorFromStdModule.py
    lldb-api :: commands/expression/import-std-module/weak_ptr-dbg-info-content/TestDbgInfoContentWeakPtrFromStdModule.py
    lldb-api :: commands/expression/import-std-module/weak_ptr/TestWeakPtrFromStdModule.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/bitset/TestDataFormatterLibcxxBitset.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/forward_list/TestDataFormatterLibcxxForwardList.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/function/TestLibCxxFunction.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/iterator/TestDataFormatterLibccIterator.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/list/loop/TestDataFormatterLibcxxListLoop.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/map/TestDataFormatterLibccMap.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/multimap/TestDataFormatterLibccMultiMap.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/multiset/TestDataFormatterLibcxxMultiSet.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/optional/TestDataFormatterLibcxxOptional.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/queue/TestDataFormatterLibcxxQueue.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/string/TestDataFormatterLibcxxString.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/tuple/TestDataFormatterLibcxxTuple.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/unordered/TestDataFormatterUnordered.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/variant/TestDataFormatterLibcxxVariant.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/vbool/TestDataFormatterLibcxxVBool.py
    lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py
    lldb-api :: lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

  Expected Passes    : 1604
  Expected Failures  : 5
  Unsupported Tests  : 294
  Unexpected Failures: 39

This list of 39 tests lines up fairly well with ones that specify USE_LIBCPP := 1 in their Makefile, or @add_test_categories(["libc++"]) in their Python test file. They all fail with the same assertion that indicates no breakpoints were hit: AssertionError: 0 != 1 : Expected 1 thread to stop at breakpoint, 0 did. Unfortunately, I couldn't find a working example of a test that uses libc++.

Initially, using a freshly rm -rf and mkdir'ed build directory, I ran ninja check-lldb, and all builds and tests succeeded. Then, I tried to add a simple test case that included <cstdio> and <experimental/coroutine>, but this failed with fatal error: 'cstdio' file not found. So I'm currently in a bind: build the cxx target that my test would depend on, and my check-lldb test suite begins failing. Don't build it, and I can't run my test. Also, for what it's worth, I tried a configuration that used cmake -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;libcxx;libcxxabi;lld;lldb" and ninja check-all, and these same 39 tests also failed.

I checked the LLVM buildbots to see if any exercised a libc++ & lldb build. I found 3 active lldb buildbots:

• http://lab.llvm.org:8011/builders/lldb-x86_64-debian: It looks like the cmake-configure step of this bot uses -DLLVM_ENABLE_PROJECTS=lldb;llvm;clang;lld, so I think it doesn't build the libcxx project and so wouldn't encounter these failing tests.
• http://lab.llvm.org:8011/builders/lldb-aarch64-ubuntu: Same as above.
• http://lab.llvm.org:8011/builders/lldb-x64-windows-ninja: Same as above.

Let me know if this is PEBKAC, I'd love to add a coroutines regression test to the suite, but at the moment I can't figure out how to successfully run the tests.

[modocache]

Add checks for !dbg locations.

[jmorse]

Thanks for pushing through and finding out how to improve co-routine variable locations Brian; I have a few comments inline. Alas, I can't help with the lldb tests.

We do have Dexter too [0] for running integration tests, although I've no idea whether it'll play nice with coroutines.

[0] https://github.com/llvm/llvm-project/tree/master/debuginfo-tests/dexter

llvm/lib/Transforms/Coroutines/CoroFrame.cpp

748

This might be splitting hairs, but FindDbgAddrUses will also return dbg.addr instructions, and promoting those to be dbg.declares would mess with the validity of other variable locations. Could you std::remove_if DbgAddrIntrinsic's first?

(I don't think this would happen with C++, but there are other IR producers).

769

"We cannot replace"?

777–780

This will also erase dbg.value intrinsics, which I think will correspond to pointer variables that point(ed) at an alloca losing a location. Those dbg.values should be set to "undef" instead of being deleted, so that they terminate any earlier variable locations. I'd suggest using FindDbgAddrUses to get dbg.declares and delete those; then calling replaceDbgUsesWithUndef on the alloca. Or alternately, delete the dbg.declare's when they're migrated, and just call replaceDbgUsesWithUndef on the alloca.

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

34

These explicitly numbered metadata nodes (!dbg !13) should be replaced with pattern matching, as you've done a few lines down.

219

NB, we usually delete any un-necessary attributes.

llvm/test/Transforms/Coroutines/coro-debug.ll

29–30

Why are these dbg.declares moving down -- is it to check that they're moved into the entry block later on?

133

This might need to be CHECK-NEXT if you want to ensure the dbg.declare is in the entry.resume block, not just after the label.

[aprantl]

In D75338#1900254, @modocache wrote:

I tried adding an lldb test for this, but I encountered 39 failing tests when I ran ninja cxx check-lldb, and it made me wonder whether tests that use libc++ are currently broken?

They are not broken [1], but you may need to add libcxx and friends to your list of projects in your LLVM cmake invocation.

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/

[modocache]

They are not broken [1], but you may need to add libcxx and friends to your list of projects in your LLVM cmake invocation.

Thanks for the link to the buildbot! In my comment above, I mention:

Also, for what it's worth, I tried a configuration that used cmake -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;libcxx;libcxxabi;lld;lldb" and ninja check-all, and these same 39 tests also failed.

When you say "add libcxx and friends to your list of projects in your LLVM cmake invocation", do you mean -DLLVM_ENABLE_PROJECTS=, or something else? The primary difference I see between my own build invocation and the buildbot is that it passes -DLLDB_TEST_USER_ARGS=.... I'll try mimicking those to see if that works. But, if it's alright, I think I'd like to respond to @jmorse's review comments and commit this patch, and add an lldb regression test in a separate follow-up patch.

[modocache]

Thanks for the review, @jmorse! I responded to your comments: I delete only dbg.declare, replace other uses with undef, and I removed unnecessary attributes.

llvm/test/Transforms/Coroutines/coro-debug.ll

29–30

Right, sorry for the confusion. I mention in the commit message:

The existing coro-debug.ll test case is also modified to reflect the locations at which Clang actually places calls to 'dbg.declare', and additional checks are added to ensure this patch works as intended in that example as well.

This patch adds the CHECK below that the llvm.dbg.declare will correctly appear in the coroutine funclet f.resume, which makes it possible for debuggers to "find" the variable x when stopped in a resumed coroutine function.

But, the logic in the patch works on the assumption that the original llvm.dbg.declare describing the variable x appears not in the function's entry block (where Clang places it in a non-coroutine function), but instead in the basic block that represents the coroutine's resumption after its implicit "initial suspend" point (where Clang places it in a coroutine function, since Clang first codegen's the br instructions for the implicit suspend point, and then codegen's the function's body).

These dbg.declare don't need to be moved like this, but if they aren't, then the added CHECK below won't succeed -- the patch ensures that dbg.declare are migrated to coroutine funclets, but specifically only when dealing with IR in the form that Clang generates it.

[jmorse]

LGTM with the remaining inline comments addressed, cheers.

llvm/lib/Transforms/Coroutines/CoroFrame.cpp

748

I'd recommend putting findDbgDeclareUses in Local.h/Local.cpp, we already have utilities there for finding {all-dbg-users, debug-values, debug-addresses}, we may as well make this available to other developers.

781–784

This should now be redundant and removed, as replaceDbgUsesWithUndef should have undef'd all debug users.

llvm/test/Transforms/Coroutines/coro-debug.ll

29–30

Thanks for the explanation!

[modocache]

llvm/lib/Transforms/Coroutines/CoroFrame.cpp

748

Awesome! I'm very glad to do so, I just wasn't sure if having one user of the interface meant it shouldn't be in a shared header. But this sounds great to me, I'll try to add a unittest for it as well.

[modocache]

Thanks again for the review. I moved findDbgDeclareUses to Local.h (no unittest for now but I'll add one in a follow-up diff for both it and FindDbgAddrUses, which doesn't have one either) and I removed the redundant loop. I'll commit this now and add an lldb regression test in a follow-up patch.

llvm/lib/Transforms/Coroutines/CoroFrame.cpp

781–784

Oops! I'd missed this and accidentally left it in. Thanks for catching! :)

llvm/include/llvm/Transforms/Utils/Local.h

 
 namespace llvm {
 
 class AllocaInst;
 class AssumptionCache;
 class BasicBlock;
 class BranchInst;
 class CallInst;
+class DbgDeclareInst;
 class DbgVariableIntrinsic;
 class DbgValueInst;
 class DIBuilder;
 class Function;
 class Instruction;
 class LazyValueInfo;
 class LoadInst;
 class MDNode;

 void insertDebugValuesForPHIs(BasicBlock *BB,
                               SmallVectorImpl<PHINode *> &InsertedPHIs);
 
 /// Finds all intrinsics declaring local variables as living in the memory that
 /// 'V' points to. This may include a mix of dbg.declare and
 /// dbg.addr intrinsics.
 TinyPtrVector<DbgVariableIntrinsic *> FindDbgAddrUses(Value *V);
 
+/// Like \c FindDbgAddrUses, but only returns dbg.declare intrinsics, not
+/// dbg.addr.
+TinyPtrVector<DbgDeclareInst *> FindDbgDeclareUses(Value *V);
+
 /// Finds the llvm.dbg.value intrinsics describing a value.
 void findDbgValues(SmallVectorImpl<DbgValueInst *> &DbgValues, Value *V);
 
 /// Finds the debug info intrinsics describing a value.
 void findDbgUsers(SmallVectorImpl<DbgVariableIntrinsic *> &DbgInsts, Value *V);
 
 /// Replaces llvm.dbg.declare instruction when the address it
 /// describes is replaced with a new value. If Deref is true, an

llvm/lib/Transforms/Coroutines/CoroFrame.cpp

 //
 // TODO: pack values tightly using liveness info.
 //===----------------------------------------------------------------------===//
 
 #include "CoroInternal.h"
 #include "llvm/ADT/BitVector.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Analysis/PtrUseVisitor.h"
-#include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/IR/CFG.h"
+#include "llvm/IR/DIBuilder.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/InstIterator.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/circular_raw_ostream.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
+#include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/PromoteMemToReg.h"
+#include <algorithm>
 
 using namespace llvm;
 
 // The "coro-suspend-crossing" flag is very noisy. There is another debug type,
 // "coro-frame", which results in leaner debug spew.
 #define DEBUG_TYPE "coro-suspend-crossing"
 
 enum { SmallVectorThreshold = 32 };
if (auto *AI = dyn_cast<AllocaInst>(Orig)) {
         report_fatal_error("Coroutines cannot handle non static allocas yet");
       }
     }
 
     return Builder.CreateInBoundsGEP(FrameTy, FramePtr, Indices);
   };
 
   // Create a load instruction to reload the spilled value from the coroutine
-  // frame.
-  auto CreateReload = [&](Instruction *InsertBefore) {
+  // frame. Populates the Value pointer reference provided with the frame GEP.
+  auto CreateReload = [&](Instruction *InsertBefore, Value *&G) {
     assert(Index != InvalidFieldIndex && "accessing unassigned field number");
     Builder.SetInsertPoint(InsertBefore);
 
-    auto *G = GetFramePointer(Index, CurrentValue);
+    G = GetFramePointer(Index, CurrentValue);
     G->setName(CurrentValue->getName() + Twine(".reload.addr"));
 
     return isa<AllocaInst>(CurrentValue)
                ? G
                : Builder.CreateLoad(FrameTy->getElementType(Index), G,
                                     CurrentValue->getName() + Twine(".reload"));
   };
 
+  Value *GEP = nullptr, *CurrentGEP = nullptr;
   for (auto const &E : Spills) {
     // If we have not seen the value, generate a spill.
     if (CurrentValue != E.def()) {
       CurrentValue = E.def();
       CurrentBlock = nullptr;
       CurrentReload = nullptr;
 
       Index = E.fieldIndex();
if (CurrentValue != E.def()) {
             CurrentValue->getName() + Twine(".spill.addr"));
         Builder.CreateStore(CurrentValue, G);
       }
     }
 
     // If we have not seen the use block, generate a reload in it.
     if (CurrentBlock != E.userBlock()) {
       CurrentBlock = E.userBlock();
-      CurrentReload = CreateReload(&*CurrentBlock->getFirstInsertionPt());
+      CurrentReload = CreateReload(&*CurrentBlock->getFirstInsertionPt(), GEP);
     }
 
     // If we have a single edge PHINode, remove it and replace it with a reload
     // from the coroutine frame. (We already took care of multi edge PHINodes
     // by rewriting them in the rewritePHIs function).
     if (auto *PN = dyn_cast<PHINode>(E.user())) {
       assert(PN->getNumIncomingValues() == 1 && "unexpected number of incoming "
                                                 "values in the PHINode");
       PN->replaceAllUsesWith(CurrentReload);
       PN->eraseFromParent();
       continue;
     }
 
+    // If we have not seen this GEP instruction, migrate any dbg.declare from
+    // the alloca to it.
+    if (CurrentGEP != GEP) {
+      CurrentGEP = GEP;
+      TinyPtrVector<DbgDeclareInst *> DIs = FindDbgDeclareUses(CurrentValue);
+      if (!DIs.empty())
+        DIBuilder(*CurrentBlock->getParent()->getParent(),
+                  /*AllowUnresolved*/ false)
+            .insertDeclare(CurrentGEP, DIs.front()->getVariable(),
+                           DIs.front()->getExpression(),
+                           DIs.front()->getDebugLoc(), DIs.front());
+    }
+
     // Replace all uses of CurrentValue in the current instruction with reload.
     E.user()->replaceUsesOfWith(CurrentValue, CurrentReload);
   }
 
   BasicBlock *FramePtrBB = FramePtr->getParent();
 
   auto SpillBlock =
     FramePtrBB->splitBasicBlock(FramePtr->getNextNode(), "AllocaSpillBB");      
   SpillBlock->splitBasicBlock(&SpillBlock->front(), "PostSpill");
   Shape.AllocaSpillBlock = SpillBlock;
-  // If we found any allocas, replace all of their remaining uses with Geps.
-  // Note: we cannot do it indiscriminately as some of the uses may not be
-  // dominated by CoroBegin.
+  // If we found any alloca, replace all of their remaining uses with GEP
+  // instructions. Because new dbg.declare have been created for these alloca,
+  // we also delete the original dbg.declare and replace other uses with undef.
+  // Note: We cannot replace the alloca with GEP instructions indiscriminately,
+  // as some of the uses may not be dominated by CoroBegin.
   bool MightNeedToCopy = false;
   Builder.SetInsertPoint(&Shape.AllocaSpillBlock->front());
   SmallVector<Instruction *, 4> UsersToUpdate;
   for (auto &P : Allocas) {
     AllocaInst *const A = P.first;
+
+    for (auto *DI : FindDbgDeclareUses(A))
+      DI->eraseFromParent();
+    replaceDbgUsesWithUndef(A);
+
     UsersToUpdate.clear();
     for (User *U : A->users()) {
       auto *I = cast<Instruction>(U);
       if (DT.dominates(CB, I))
         UsersToUpdate.push_back(I);
       else
         MightNeedToCopy = true;
     }
     if (!UsersToUpdate.empty()) {
       auto *G = GetFramePointer(P.second, A);
       G->takeName(A);
       for (Instruction *I : UsersToUpdate)
static void eliminateSwiftError(Function &F, coro::Shape &Shape) {
   // promote them en masse.
   if (!AllocasToPromote.empty()) {
     DominatorTree DT(F);
     PromoteMemToReg(AllocasToPromote, DT);
   }
 }
 
 void coro::buildCoroutineFrame(Function &F, Shape &Shape) {
-  // Lower coro.dbg.declare to coro.dbg.value, since we are going to rewrite
-  // access to local variables.
-  LowerDbgDeclare(F);
-
   eliminateSwiftError(F, Shape);
 
   if (Shape.ABI == coro::ABI::Switch &&
       Shape.SwitchLowering.PromiseAlloca) {
     Shape.getSwitchCoroId()->clearPromise();
   }
 
   // Make sure that all coro.save, coro.suspend and the fallthrough coro.end

llvm/lib/Transforms/Utils/Local.cpp

for (User *U : MDV->users()) {
     if (auto *DII = dyn_cast<DbgVariableIntrinsic>(U))
       if (DII->isAddressOfVariable())
         Declares.push_back(DII);
   }
 
   return Declares;
 }
 
+TinyPtrVector<DbgDeclareInst *> llvm::FindDbgDeclareUses(Value *V) {
+  TinyPtrVector<DbgDeclareInst *> DDIs;
+  for (DbgVariableIntrinsic *DVI : FindDbgAddrUses(V))
+    if (auto *DDI = dyn_cast<DbgDeclareInst>(DVI))
+      DDIs.push_back(DDI);
+  return DDIs;
+}
+
 void llvm::findDbgValues(SmallVectorImpl<DbgValueInst *> &DbgValues, Value *V) {
   // This function is hot. Check whether the value has any metadata to avoid a
   // DenseMap lookup.
   if (!V->isUsedByMetadata())
     return;
   if (auto *L = LocalAsMetadata::getIfExists(V))
     if (auto *MDV = MetadataAsValue::getIfExists(V->getContext(), L))
       for (User *U : MDV->users())

llvm/test/Transforms/Coroutines/coro-debug-frame-variable.ll

+; RUN: opt < %s -O0 -enable-coroutines -S | FileCheck %s
+; RUN: opt < %s -passes='default<O0>' -enable-coroutines -S | FileCheck %s
+
+; Define a function 'f' that resembles the Clang frontend's output for the
+; following C++ coroutine:
+;
+;   void foo() {
+;     int i = 0;
+;     ++i;
+;     print(i);  // Prints '1'
+;
+;     co_await suspend_always();
+;
+;     int j = 0;
+;     ++i;
+;     print(i);  // Prints '2'
+;     ++j;
+;     print(j);  // Prints '1'
+;   }
+;
+; The CHECKs verify that dbg.declare intrinsics are created for the coroutine
+; funclet 'f.resume', and that they reference the address of the variables on
+; the coroutine frame. The debug locations for the original function 'f' are
+; static (!11 and !13), whereas the coroutine funclet will have its own new
+; ones with identical line and column numbers.
+;
+; CHECK-LABEL: define void @f() {
+; CHECK:       init.ready:
+; CHECK:         [[IGEP:%.+]] = getelementptr inbounds %f.Frame, %f.Frame* %FramePtr, i32 0, i32 4
+; CHECK:         call void @llvm.dbg.declare(metadata i32* [[IGEP]], metadata ![[IVAR:[0-9]+]], metadata !DIExpression()), !dbg ![[IDBGLOC:[0-9]+]]
+; CHECK:       await.ready:
+; CHECK:         [[JGEP:%.+]] = getelementptr inbounds %f.Frame, %f.Frame* %FramePtr, i32 0, i32 5
+; CHECK:         call void @llvm.dbg.declare(metadata i32* [[JGEP]], metadata ![[JVAR:[0-9]+]], metadata !DIExpression()), !dbg ![[JDBGLOC:[0-9]+]]
+;
+; CHECK-LABEL: define internal fastcc void @f.resume({{.*}}) {
+; CHECK:       init.ready:
+; CHECK:         [[IGEP_RESUME:%.+]] = getelementptr inbounds %f.Frame, %f.Frame* %FramePtr, i32 0, i32 4
+; CHECK:         call void @llvm.dbg.declare(metadata i32* [[IGEP_RESUME]], metadata ![[IVAR_RESUME:[0-9]+]], metadata !DIExpression()), !dbg ![[IDBGLOC_RESUME:[0-9]+]]
+; CHECK:       await.ready:
+; CHECK:         [[JGEP_RESUME:%.+]] = getelementptr inbounds %f.Frame, %f.Frame* %FramePtr, i32 0, i32 5
+; CHECK:         call void @llvm.dbg.declare(metadata i32* [[JGEP_RESUME]], metadata ![[JVAR_RESUME:[0-9]+]], metadata !DIExpression()), !dbg ![[JDBGLOC_RESUME:[0-9]+]]
+;
+; CHECK: ![[IVAR]] = !DILocalVariable(name: "i"
+; CHECK: ![[SCOPE:[0-9]+]] = distinct !DILexicalBlock(scope: !8, file: !1, line: 23, column: 12)
+; CHECK: ![[IDBGLOC]] = !DILocation(line: 24, column: 7, scope: ![[SCOPE]])
+; CHECK: ![[JVAR]] = !DILocalVariable(name: "j"
+; CHECK: ![[JDBGLOC]] = !DILocation(line: 32, column: 7, scope: ![[SCOPE]])
+; CHECK: ![[IVAR_RESUME]] = !DILocalVariable(name: "i"
+; CHECK: ![[RESUME_SCOPE:[0-9]+]] = distinct !DILexicalBlock(scope: !8, file: !1, line: 23, column: 12)
+; CHECK: ![[IDBGLOC_RESUME]] = !DILocation(line: 24, column: 7, scope: ![[RESUME_SCOPE]])
+; CHECK: ![[JVAR_RESUME]] = !DILocalVariable(name: "j"
+; CHECK: ![[JDBGLOC_RESUME]] = !DILocation(line: 32, column: 7, scope: ![[RESUME_SCOPE]])
+define void @f() {
+entry:
+  %__promise = alloca i8, align 8
+  %i = alloca i32, align 4
+  %j = alloca i32, align 4
+  %id = call token @llvm.coro.id(i32 16, i8* %__promise, i8* null, i8* null)
+  %alloc = call i1 @llvm.coro.alloc(token %id)
+  br i1 %alloc, label %coro.alloc, label %coro.init
+
+coro.alloc:                                       ; preds = %entry
+  %size = call i64 @llvm.coro.size.i64()
+  %memory = call i8* @new(i64 %size)
+  br label %coro.init
+
+coro.init:                                        ; preds = %coro.alloc, %entry
+  %phi.entry.alloc = phi i8* [ null, %entry ], [ %memory, %coro.alloc ]
+  %begin = call i8* @llvm.coro.begin(token %id, i8* %phi.entry.alloc)
+  %ready = call i1 @await_ready()
+  br i1 %ready, label %init.ready, label %init.suspend
+
+init.suspend:                                     ; preds = %coro.init
+  %save = call token @llvm.coro.save(i8* null)
+  call void @await_suspend()
+  %suspend = call i8 @llvm.coro.suspend(token %save, i1 false)
+  switch i8 %suspend, label %coro.ret [
+    i8 0, label %init.ready
+    i8 1, label %init.cleanup
+  ]
+
+init.cleanup:                                     ; preds = %init.suspend
+  br label %cleanup
+
+init.ready:                                       ; preds = %init.suspend, %coro.init
+  call void @await_resume()
+  call void @llvm.dbg.declare(metadata i32* %i, metadata !6, metadata !DIExpression()), !dbg !11
+  store i32 0, i32* %i, align 4
+  %i.init.ready.load = load i32, i32* %i, align 4
+  %i.init.ready.inc = add nsw i32 %i.init.ready.load, 1
+  store i32 %i.init.ready.inc, i32* %i, align 4
+  %i.init.ready.reload = load i32, i32* %i, align 4
+  call void @print(i32 %i.init.ready.reload)
+  %ready.again = call zeroext i1 @await_ready()
+  br i1 %ready.again, label %await.ready, label %await.suspend
+
+await.suspend:                                    ; preds = %init.ready
+  %save.again = call token @llvm.coro.save(i8* null)
+  %from.address = call i8* @from_address(i8* %begin)
+  call void @await_suspend()
+  %suspend.again = call i8 @llvm.coro.suspend(token %save.again, i1 false)
+  switch i8 %suspend.again, label %coro.ret [
+    i8 0, label %await.ready
+    i8 1, label %await.cleanup
+  ]
+
+await.cleanup:                                    ; preds = %await.suspend
+  br label %cleanup
+
+await.ready:                                      ; preds = %await.suspend, %init.ready
+  call void @await_resume()
+  call void @llvm.dbg.declare(metadata i32* %j, metadata !12, metadata !DIExpression()), !dbg !13
+  store i32 0, i32* %j, align 4
+  %i.await.ready.load = load i32, i32* %i, align 4
+  %i.await.ready.inc = add nsw i32 %i.await.ready.load, 1
+  store i32 %i.await.ready.inc, i32* %i, align 4
+  %j.await.ready.load = load i32, i32* %j, align 4
+  %j.await.ready.inc = add nsw i32 %j.await.ready.load, 1
+  store i32 %j.await.ready.inc, i32* %j, align 4
+  %i.await.ready.reload = load i32, i32* %i, align 4
+  call void @print(i32 %i.await.ready.reload)
+  %j.await.ready.reload = load i32, i32* %j, align 4
+  call void @print(i32 %j.await.ready.reload)
+  call void @return_void()
+  br label %coro.final
+
+coro.final:                                       ; preds = %await.ready
+  call void @final_suspend()
+  %coro.final.await_ready = call i1 @await_ready()
+  br i1 %coro.final.await_ready, label %final.ready, label %final.suspend
+
+final.suspend:                                    ; preds = %coro.final
+  %final.suspend.coro.save = call token @llvm.coro.save(i8* null)
+  %final.suspend.from_address = call i8* @from_address(i8* %begin)
+  call void @await_suspend()
+  %final.suspend.coro.suspend = call i8 @llvm.coro.suspend(token %final.suspend.coro.save, i1 true)
+  switch i8 %final.suspend.coro.suspend, label %coro.ret [
+    i8 0, label %final.ready
+    i8 1, label %final.cleanup
+  ]
+
+final.cleanup:                                    ; preds = %final.suspend
+  br label %cleanup
+
+final.ready:                                      ; preds = %final.suspend, %coro.final
+  call void @await_resume()
+  br label %cleanup
+
+cleanup:                                          ; preds = %final.ready, %final.cleanup, %await.cleanup, %init.cleanup
+  %cleanup.dest.slot.0 = phi i32 [ 0, %final.ready ], [ 2, %final.cleanup ], [ 2, %await.cleanup ], [ 2, %init.cleanup ]
+  %free.memory = call i8* @llvm.coro.free(token %id, i8* %begin)
+  %free = icmp ne i8* %free.memory, null
+  br i1 %free, label %coro.free, label %after.coro.free
+
+coro.free:                                        ; preds = %cleanup
+  call void @delete(i8* %free.memory)
+  br label %after.coro.free
+
+after.coro.free:                                  ; preds = %coro.free, %cleanup
+  switch i32 %cleanup.dest.slot.0, label %unreachable [
+    i32 0, label %cleanup.cont
+    i32 2, label %coro.ret
+  ]
+
+cleanup.cont:                                     ; preds = %after.coro.free
+  br label %coro.ret
+
+coro.ret:                                         ; preds = %cleanup.cont, %after.coro.free, %final.suspend, %await.suspend, %init.suspend
+  %end = call i1 @llvm.coro.end(i8* null, i1 false)
+  ret void
+
+unreachable:                                      ; preds = %after.coro.free
+  unreachable
+}
+
+declare void @llvm.dbg.declare(metadata, metadata, metadata)
+declare token @llvm.coro.id(i32, i8* readnone, i8* nocapture readonly, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i64 @llvm.coro.size.i64()
+declare token @llvm.coro.save(i8*)
+declare i8* @llvm.coro.begin(token, i8* writeonly)
+declare i8 @llvm.coro.suspend(token, i1)
+declare i8* @llvm.coro.free(token, i8* nocapture readonly)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare i8* @new(i64)
+declare void @delete(i8*)
+declare i1 @await_ready()
+declare void @await_suspend()
+declare void @await_resume()
+declare void @print(i32)
+declare i8* @from_address(i8*)
+declare void @return_void()
+declare void @final_suspend()
+
+!llvm.dbg.cu = !{!0}
+!llvm.linker.options = !{}
+!llvm.module.flags = !{!3, !4}
+!llvm.ident = !{!5}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, producer: "clang version 11.0.0", isOptimized: false, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, retainedTypes: !2, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "repro.cpp", directory: ".")
+!2 = !{}
+!3 = !{i32 7, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{!"clang version 11.0.0"}
+!6 = !DILocalVariable(name: "i", scope: !7, file: !1, line: 24, type: !10)
+!7 = distinct !DILexicalBlock(scope: !8, file: !1, line: 23, column: 12)
+!8 = distinct !DISubprogram(name: "foo", linkageName: "_Z3foov", scope: !1, file: !1, line: 23, type: !9, scopeLine: 23, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!9 = !DISubroutineType(types: !2)
+!10 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!11 = !DILocation(line: 24, column: 7, scope: !7)
+!12 = !DILocalVariable(name: "j", scope: !7, file: !1, line: 32, type: !10)
+!13 = !DILocation(line: 32, column: 7, scope: !7)

llvm/test/Transforms/Coroutines/coro-debug.ll

 ; Tests that debug information is sane after coro-split
 ; RUN: opt < %s -coro-split -S | FileCheck %s
 ; RUN: opt < %s -passes=coro-split -S | FileCheck %s
 
 source_filename = "simple-repro.c"
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Function Attrs: noinline nounwind
 define i8* @f(i32 %x) #0 !dbg !6 {
 entry:
   %x.addr = alloca i32, align 4
   %coro_hdl = alloca i8*, align 8
   store i32 %x, i32* %x.addr, align 4
-  call void @llvm.dbg.declare(metadata i32* %x.addr, metadata !12, metadata !13), !dbg !14
-  call void @llvm.dbg.declare(metadata i8** %coro_hdl, metadata !15, metadata !13), !dbg !16
   %0 = call token @llvm.coro.id(i32 0, i8* null, i8* bitcast (i8* (i32)* @f to i8*), i8* null), !dbg !16
   %1 = call i64 @llvm.coro.size.i64(), !dbg !16
   %call = call i8* @malloc(i64 %1), !dbg !16
   %2 = call i8* @llvm.coro.begin(token %0, i8* %call) #7, !dbg !16
   store i8* %2, i8** %coro_hdl, align 8, !dbg !16
   %3 = call i8 @llvm.coro.suspend(token none, i1 false), !dbg !17
   %conv = sext i8 %3 to i32, !dbg !17
   call void @coro.devirt.trigger(i8* null)
   switch i32 %conv, label %sw.default [
     i32 0, label %sw.bb
     i32 1, label %sw.bb1
   ], !dbg !17
 
 sw.bb:                                            ; preds = %entry
+  call void @llvm.dbg.declare(metadata i32* %x.addr, metadata !12, metadata !13), !dbg !14
+  call void @llvm.dbg.declare(metadata i8** %coro_hdl, metadata !15, metadata !13), !dbg !16
   br label %sw.epilog, !dbg !18
 
 sw.bb1:                                           ; preds = %entry
   br label %coro_Cleanup, !dbg !18
 
 sw.default:                                       ; preds = %entry
   br label %coro_Suspend, !dbg !18
 

 !20 = !DILocation(line: 59, column: 9, scope: !6)
 !21 = !DILocation(line: 59, column: 10, scope: !6)
 !22 = !DILocation(line: 59, column: 7, scope: !6)
 !23 = !DILocation(line: 59, column: 5, scope: !6)
 !24 = !DILocation(line: 62, column: 3, scope: !6)
 
 ; CHECK: define i8* @f(i32 %x) #0 !dbg ![[ORIG:[0-9]+]]
 ; CHECK: define internal fastcc void @f.resume(%f.Frame* noalias nonnull %FramePtr) #0 !dbg ![[RESUME:[0-9]+]]
+; CHECK: entry.resume:
+; CHECK-NEXT: call void @coro.devirt.trigger(i8* null)
+; CHECK-NEXT: call void @llvm.dbg.declare(metadata i32* %x.addr.reload.addr, metadata ![[RESUME_VAR:[0-9]+]]
 ; CHECK: define internal fastcc void @f.destroy(%f.Frame* noalias nonnull %FramePtr) #0 !dbg ![[DESTROY:[0-9]+]]
 ; CHECK: define internal fastcc void @f.cleanup(%f.Frame* noalias nonnull %FramePtr) #0 !dbg ![[CLEANUP:[0-9]+]]
 
 ; CHECK: ![[ORIG]] = distinct !DISubprogram(name: "f", linkageName: "flink"
-; CHECK: !DILocalVariable(name: "x", arg: 1, scope: ![[ORIG]]
 
 ; CHECK: ![[RESUME]] = distinct !DISubprogram(name: "f", linkageName: "flink"
-; CHECK: !DILocalVariable(name: "x", arg: 1, scope: ![[RESUME]]
+; CHECK: ![[RESUME_VAR]] = !DILocalVariable(name: "x", arg: 1, scope: ![[RESUME]]
 
 ; CHECK: ![[DESTROY]] = distinct !DISubprogram(name: "f", linkageName: "flink"
 
 ; CHECK: ![[CLEANUP]] = distinct !DISubprogram(name: "f", linkageName: "flink"