This is an archive of the discontinued LLVM Phabricator instance.

[lldb] Disable minimal import mode for RecordDecls that back FieldDecls
ClosedPublic

Authored by teemperor on May 23 2021, 3:27 PM.

Details

Summary

Clang adds a Decl in two phases to a DeclContext. First it adds it invisible
and then it makes it visible (which will add it to the lookup data structures).
It's important that we can't do lookups into the DeclContext we are currently
adding the Decl to during this process as once the Decl has been added, any
lookup will automatically build a new lookup map and add the added Decl to it.
The second step would then add the Decl a second time to the lookup which will
lead to weird errors later one. I made adding a Decl twice to a lookup an assertion
error in D84827.

In the first step Clang also does some computations on the added Decl if it's
for example a FieldDecl that is added to a RecordDecl.

One of these computations is checking if the FieldDecl is of a record type
and the record type has a deleted constexpr destructor which will delete
the constexpr destructor of the record that got the FieldDecl.

This can lead to a bug with the way we implement MinimalImport in LLDB
and the following code:

struct Outer {
  typedef int HookToOuter;
  struct NestedClass {
    HookToOuter RefToOuter;
  } NestedClassMember; // We are adding this.
};
  1. We just imported Outer minimally so far.
  2. We are now asked to add NestedClassMember as a FieldDecl.
  3. We import NestedClass minimally.
  4. We add NestedClassMember and clang does a lookup for a constexpr dtor in NestedClass. NestedClassMember hasn't been added to the lookup.
  5. The lookup into NestedClass will now load the members of NestedClass.
  6. We try to import the type of RefToOuter which will try to import the HookToOuter typedef.
  7. We import the typedef and while importing we check for conflicts in Outer via a lookup.
  8. The lookup into Outer will cause the invisible NestedClassMember to be added to the lookup.
  9. We continue normally until we get back to the addDecl call in step 2.
  10. We now add NestedClassMember to the lookup even though we already did that in step 8.

The fix here is disabling the minimal import for RecordTypes from FieldDecls. We actually already did this,
but so far we only force the definition of the type to be imported *after* we imported the FieldDecl. This
just moves that code *before* we import the FieldDecl so prevent the issue above.

Diff Detail

Event Timeline

teemperor created this revision.May 23 2021, 3:27 PM
teemperor requested review of this revision.May 23 2021, 3:27 PM
aprantl accepted this revision.May 24 2021, 9:03 AM
This revision is now accepted and ready to land.May 24 2021, 9:03 AM
shafik accepted this revision.May 24 2021, 2:05 PM

LGTM

clayborg added inline comments.May 24 2021, 2:16 PM
lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp
487–494

Is this comment needed anymore?

Do we want any safeguard on StoredDeclsList::prependDeclNoReplace(...)? Right now it can still cause duplicate NamedDecl pointers. I fix that tested helps prevent this:

/// Add a declaration to the list without checking if it replaces anything
/// unless the exact decl is already in the list.
void prependDeclNoReplace(NamedDecl *D) {
  // Check if the named declaration is already the only thing in this list.
  // If so it is we can just return as it is already in the front of the list.
  if (getAsDecl() == D)
    return;
  // Check if the named declaration is in the list. If so erase it so we can
  // ensure it is prepended to the front of the list.
  if (llvm::is_contained(getLookupResult(), D))
    erase(D);

  if (isNull()) {
    Data.setPointer(D);
    return;
  }
  ASTContext &C = D->getASTContext();
  DeclListNode *Node = C.AllocateDeclListNode(D);
  Node->Rest = Data.getPointer();
  Data.setPointer(Node);
}

The first two if statements are new additions that detect if the decl already exists. If it is the only one and already there, just return and if it is in the list, then erase it so we can add it back and prepend it so it ends up at the start of the list.

Do we want any safeguard on StoredDeclsList::prependDeclNoReplace(...)? Right now it can still cause duplicate NamedDecl pointers. I fix that tested helps prevent this:

/// Add a declaration to the list without checking if it replaces anything
/// unless the exact decl is already in the list.
void prependDeclNoReplace(NamedDecl *D) {
  // Check if the named declaration is already the only thing in this list.
  // If so it is we can just return as it is already in the front of the list.
  if (getAsDecl() == D)
    return;
  // Check if the named declaration is in the list. If so erase it so we can
  // ensure it is prepended to the front of the list.
  if (llvm::is_contained(getLookupResult(), D))
    erase(D);

  if (isNull()) {
    Data.setPointer(D);
    return;
  }
  ASTContext &C = D->getASTContext();
  DeclListNode *Node = C.AllocateDeclListNode(D);
  Node->Rest = Data.getPointer();
  Data.setPointer(Node);
}

The first two if statements are new additions that detect if the decl already exists. If it is the only one and already there, just return and if it is in the list, then erase it so we can add it back and prepend it so it ends up at the start of the list.

I originally also wanted to just make adding a Decl twice a no-op, but when I added an assert for this I found a few actually legitimate bugs so that's why I wanted to push for that approach in D84827. Also adding a Decl twice just seems kinda fishy in general. If we could get all the remaining bug fixes that D84827 depends on, we could land it (but finding reviewers is hard :) )

lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp
487–494

Jepp, it's just an explanation for why we don't do decls.add(copied_decl); here (which is what we are supposed to do).

(But I now see that that continue above can go now)

This revision was landed with ongoing or failed builds.May 25 2021, 3:09 AM
This revision was automatically updated to reflect the committed changes.

This commit seems to be causing an LLDB crash. I'm still working on gathering info and reducing it, but maybe the crash reason is obvious to you given this stack trace:

$ lldb -b -o "b ChromeMain" -o "r" -o "v" -o "p chrome_main_delegate" ~/src/chromium/src/out/Default/chrome
assert.h assertion failed at llvm-project/clang/lib/AST/ASTImporter.cpp:1874 in llvm::Error clang::ASTNodeImporter::ImportDeclContext(clang::DeclContext *, bool): ToDC == ToD->getLexicalDeclContext() && ToDC->containsDecl(ToD)
...
Stack dump:
0.      Program arguments: lldb -b -o "b ChromeMain" -o r -o v -o "p chrome_main_delegate" /home/rupprecht/src/chromium/src/out/Default/chrome
1.      HandleCommand(command = "p chrome_main_delegate")^@
2.      <lldb wrapper prefix>:45:51: current parser token ';'
3.      <lldb wrapper prefix>:44:1: parsing function body '$__lldb_expr'
4.      <lldb wrapper prefix>:44:1: in compound statement ('{}')
...
 #13 0x000055646a128c66 clang::ASTNodeImporter::ImportDeclContext(clang::DeclContext*, bool) llvm-project/clang/lib/AST/ASTImporter.cpp:1874:9
 #14 0x000055646a15cc0d clang::ASTImporter::ImportDefinition(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:9098:19
 #15 0x000055646789e47d lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:916:25
 #16 0x000055646a157235 clang::ASTImporter::Import(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8412:8
 #17 0x000055646789ac8a lldb_private::ClangASTImporter::CopyDecl(clang::ASTContext*, clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:79:8
 #18 0x00005564679c4205 lldb_private::ClangASTSource::CopyDecl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:1720:3
 #19 0x00005564679c415e lldb_private::ClangASTSource::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:492:5
 #20 0x00005564679a6b6a lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.h:223:7
 #21 0x000055646a30d9b2 clang::ExternalASTSource::FindExternalLexicalDecls(clang::DeclContext const*, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/clang/include/clang/AST/ExternalASTSource.h:186:3
 #22 0x000055646a30ae60 clang::DeclContext::LoadLexicalDeclsFromExternalStorage() const llvm-project/clang/lib/AST/DeclBase.cpp:1353:7
 #23 0x000055646a30bec2 clang::DeclContext::buildLookup() llvm-project/clang/lib/AST/DeclBase.cpp:1582:32
 #24 0x000055646a30bc4f clang::DeclContext::makeDeclVisibleInContextWithFlags(clang::NamedDecl*, bool, bool) llvm-project/clang/lib/AST/DeclBase.cpp:1859:5
 #25 0x000055646a30bd99 clang::DeclContext::addDeclInternal(clang::Decl*) llvm-project/clang/lib/AST/DeclBase.cpp:1559:1
 #26 0x000055646a12c1f0 clang::ASTNodeImporter::VisitTypedefNameDecl(clang::TypedefNameDecl*, bool) llvm-project/clang/lib/AST/ASTImporter.cpp:0:16
 #27 0x000055646a12c4cd clang::ASTNodeImporter::VisitTypedefDecl(clang::TypedefDecl*) llvm-project/clang/lib/AST/ASTImporter.cpp:2552:10
 #28 0x000055646a171e3a clang::declvisitor::Base<std::__u::add_pointer, clang::ASTNodeImporter, llvm::Expected<clang::Decl*> >::Visit(clang::Decl*) llvm-project/clang/include/clang/AST/DeclNodes.inc:335:1
 #29 0x000055646a15619d clang::ASTImporter::ImportImpl(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8240:19
 #30 0x000055646789e55c lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:0:23
 #31 0x000055646a157235 clang::ASTImporter::Import(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8412:8

The class definition for the expression being evaluated can be found here: https://source.chromium.org/chromium/chromium/src/+/master:chrome/app/chrome_main.cc;l=85;drc=83bf8a5bbe7f9af4f4531b65662cfeb0c232b583

(It crashes in the same-ish w/o assertions too; this is just the more helpful stack trace :) )

Sure I can take a look, but I don't see the immediate problem when looking at the backtrace.

I assume it takes some time to reduce the bug in Chrome? If you could get me the dump() output for the Decls D, (Decl*)ToDC, FromRD, ToD, (Decl*)ToD->getLexicalDeclContext() and whether the left or the right side of the && is true/false then maybe it becomes more obvious what's going wrong here.

This commit seems to be causing an LLDB crash. I'm still working on gathering info and reducing it, but maybe the crash reason is obvious to you given this stack trace:

$ lldb -b -o "b ChromeMain" -o "r" -o "v" -o "p chrome_main_delegate" ~/src/chromium/src/out/Default/chrome
assert.h assertion failed at llvm-project/clang/lib/AST/ASTImporter.cpp:1874 in llvm::Error clang::ASTNodeImporter::ImportDeclContext(clang::DeclContext *, bool): ToDC == ToD->getLexicalDeclContext() && ToDC->containsDecl(ToD)
...
Stack dump:
0.      Program arguments: lldb -b -o "b ChromeMain" -o r -o v -o "p chrome_main_delegate" /home/rupprecht/src/chromium/src/out/Default/chrome
1.      HandleCommand(command = "p chrome_main_delegate")^@
2.      <lldb wrapper prefix>:45:51: current parser token ';'
3.      <lldb wrapper prefix>:44:1: parsing function body '$__lldb_expr'
4.      <lldb wrapper prefix>:44:1: in compound statement ('{}')
...
 #13 0x000055646a128c66 clang::ASTNodeImporter::ImportDeclContext(clang::DeclContext*, bool) llvm-project/clang/lib/AST/ASTImporter.cpp:1874:9
 #14 0x000055646a15cc0d clang::ASTImporter::ImportDefinition(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:9098:19
 #15 0x000055646789e47d lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:916:25
 #16 0x000055646a157235 clang::ASTImporter::Import(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8412:8
 #17 0x000055646789ac8a lldb_private::ClangASTImporter::CopyDecl(clang::ASTContext*, clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:79:8
 #18 0x00005564679c4205 lldb_private::ClangASTSource::CopyDecl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:1720:3
 #19 0x00005564679c415e lldb_private::ClangASTSource::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.cpp:492:5
 #20 0x00005564679a6b6a lldb_private::ClangASTSource::ClangASTSourceProxy::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTSource.h:223:7
 #21 0x000055646a30d9b2 clang::ExternalASTSource::FindExternalLexicalDecls(clang::DeclContext const*, llvm::SmallVectorImpl<clang::Decl*>&) llvm-project/clang/include/clang/AST/ExternalASTSource.h:186:3
 #22 0x000055646a30ae60 clang::DeclContext::LoadLexicalDeclsFromExternalStorage() const llvm-project/clang/lib/AST/DeclBase.cpp:1353:7
 #23 0x000055646a30bec2 clang::DeclContext::buildLookup() llvm-project/clang/lib/AST/DeclBase.cpp:1582:32
 #24 0x000055646a30bc4f clang::DeclContext::makeDeclVisibleInContextWithFlags(clang::NamedDecl*, bool, bool) llvm-project/clang/lib/AST/DeclBase.cpp:1859:5
 #25 0x000055646a30bd99 clang::DeclContext::addDeclInternal(clang::Decl*) llvm-project/clang/lib/AST/DeclBase.cpp:1559:1
 #26 0x000055646a12c1f0 clang::ASTNodeImporter::VisitTypedefNameDecl(clang::TypedefNameDecl*, bool) llvm-project/clang/lib/AST/ASTImporter.cpp:0:16
 #27 0x000055646a12c4cd clang::ASTNodeImporter::VisitTypedefDecl(clang::TypedefDecl*) llvm-project/clang/lib/AST/ASTImporter.cpp:2552:10
 #28 0x000055646a171e3a clang::declvisitor::Base<std::__u::add_pointer, clang::ASTNodeImporter, llvm::Expected<clang::Decl*> >::Visit(clang::Decl*) llvm-project/clang/include/clang/AST/DeclNodes.inc:335:1
 #29 0x000055646a15619d clang::ASTImporter::ImportImpl(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8240:19
 #30 0x000055646789e55c lldb_private::ClangASTImporter::ASTImporterDelegate::ImportImpl(clang::Decl*) llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangASTImporter.cpp:0:23
 #31 0x000055646a157235 clang::ASTImporter::Import(clang::Decl*) llvm-project/clang/lib/AST/ASTImporter.cpp:8412:8

The class definition for the expression being evaluated can be found here: https://source.chromium.org/chromium/chromium/src/+/master:chrome/app/chrome_main.cc;l=85;drc=83bf8a5bbe7f9af4f4531b65662cfeb0c232b583

(It crashes in the same-ish w/o assertions too; this is just the more helpful stack trace :) )

Sure I can take a look, but I don't see the immediate problem when looking at the backtrace.

I assume it takes some time to reduce the bug in Chrome? If you could get me the dump() output for the Decls D, (Decl*)ToDC, FromRD, ToD, (Decl*)ToD->getLexicalDeclContext() and whether the left or the right side of the && is true/false then maybe it becomes more obvious what's going wrong here.

This is probably not going to be useful:

  • D->dump(): FieldDecl 0x169993f9fbd0 <<invalid sloc>> <invalid sloc> __r_ 'std::__compressed_pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >::__rep, std::allocator<char> >'
  • ((Decl*)ToDC)->dump() - prints:
a decl that inherits DeclContext isn't handled
UNREACHABLE executed at llvm-project/clang/lib/AST/DeclBase.cpp:928!
  • FromRD->dump() prints something waaaay to long to include here :)
ClassTemplateSpecializationDecl 0x169983fc2b98 <<invalid sloc>> <invalid sloc> class basic_string definition
|-DefinitionData standard_layout has_user_declared_ctor can_const_default_init
| |-DefaultConstructor exists non_trivial user_provided
| |-CopyConstructor non_trivial user_declared has_const_param needs_overload_resolution implicit_has_const_param
| |-MoveConstructor exists non_trivial user_declared 
| |-CopyAssignment non_trivial has_const_param user_declared needs_overload_resolution implicit_has_const_param
| |-MoveAssignment exists non_trivial user_declared
| `-Destructor non_trivial user_declared
|-private 'std::__basic_string_common<true>'
|-TemplateArgument type 'char'
| `-BuiltinType 0x1695c0305860 'char'
...
  • ((Decl*)ToD)->getLexicalDeclContext() errors: error: Execution was interrupted, reason: signal SIGSEGV: address access protected (fault address: 0x55e264000000). (ToD is nonnull: 0x00001699897a5e00)

I'm running out of time today & I'm off tomorrow, but I should be able to post something on Monday. I have a suspicion it's dependent on whatever flags chrome is building with, so I'll try to figure that out too.

Sure I can take a look, but I don't see the immediate problem when looking at the backtrace.

I assume it takes some time to reduce the bug in Chrome? If you could get me the dump() output for the Decls D, (Decl*)ToDC, FromRD, ToD, (Decl*)ToD->getLexicalDeclContext() and whether the left or the right side of the && is true/false then maybe it becomes more obvious what's going wrong here.

This is probably not going to be useful:

  • D->dump(): FieldDecl 0x169993f9fbd0 <<invalid sloc>> <invalid sloc> __r_ 'std::__compressed_pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >::__rep, std::allocator<char> >'
  • ((Decl*)ToDC)->dump() - prints:
a decl that inherits DeclContext isn't handled
UNREACHABLE executed at llvm-project/clang/lib/AST/DeclBase.cpp:928!

ToDC->getDeclKindName() is "ClassTemplateSpecialization". Does that explain the unreachable error message? Maybe it needs a special case somewhere?

  • FromRD->dump() prints something waaaay to long to include here :)
ClassTemplateSpecializationDecl 0x169983fc2b98 <<invalid sloc>> <invalid sloc> class basic_string definition
|-DefinitionData standard_layout has_user_declared_ctor can_const_default_init
| |-DefaultConstructor exists non_trivial user_provided
| |-CopyConstructor non_trivial user_declared has_const_param needs_overload_resolution implicit_has_const_param
| |-MoveConstructor exists non_trivial user_declared 
| |-CopyAssignment non_trivial has_const_param user_declared needs_overload_resolution implicit_has_const_param
| |-MoveAssignment exists non_trivial user_declared
| `-Destructor non_trivial user_declared
|-private 'std::__basic_string_common<true>'
|-TemplateArgument type 'char'
| `-BuiltinType 0x1695c0305860 'char'
...
  • ((Decl*)ToD)->getLexicalDeclContext() errors: error: Execution was interrupted, reason: signal SIGSEGV: address access protected (fault address: 0x55e264000000). (ToD is nonnull: 0x00001699897a5e00)

I'm running out of time today & I'm off tomorrow, but I should be able to post something on Monday. I have a suspicion it's dependent on whatever flags chrome is building with, so I'll try to figure that out too.

cmtice added a subscriber: cmtice.Jul 6 2021, 8:39 PM