[index] Return when DC is null in handleReference
ClosedPublic

Authored by MaskRay on Mon, Dec 25, 2:44 PM.

Details

Summary

DC may sometimes be NULL and getContainerInfo(DC, Container) will fail.

See https://github.com/jacobdufault/cquery/issues/192 for the full story.

https://github.com/nlohmann/json/blob/develop/test/src/unit-constructor1.cpp and many other files in that project's test/src/ direcotory cannot be indexed because

https://github.com/llvm-mirror/clang/blob/master/tools/libclang/CXIndexDataConsumer.cpp#L203

c++
    handleReference(ND, Loc, Cursor,
                    dyn_cast_or_null<NamedDecl>(ASTNode.Parent),
                    ASTNode.ContainerDC, ASTNode.OrigE, Kind);

dyn_cast_or_null<NamedDecl>(ASTNode.Parent) is somehow a null pointer and in https://github.com/llvm-mirror/clang/blob/master/tools/libclang/CXIndexDataConsumer.cpp#L935

c++
  ContainerInfo Container;
  getContainerInfo(DC, Container);

The null DC is casted ContInfo.cursor = getCursor(cast<Decl>(DC)); and SIGSEGV.

https://github.com/llvm-mirror/clang/blob/master/tools/libclang/CXIndexDataConsumer.cpp#L203

handleReference(ND, Loc, Cursor,
                dyn_cast_or_null<NamedDecl>(ASTNode.Parent),
                ASTNode.ContainerDC, ASTNode.OrigE, Kind);
#0  0x00007fa31b51a4b0 in clang::Decl::castFromDeclContext(clang::DeclContext const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#1  0x00007fa31b4a202a in clang::cxindex::CXIndexDataConsumer::handleReference(clang::NamedDecl const*, clang::SourceLocation, CXCursor, clang::NamedDecl const*, clang::DeclContext const*, clang::Expr const*, CXIdxEntityRefKind) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#2  0x00007fa31b4a1871 in clang::cxindex::CXIndexDataConsumer::handleDeclOccurence(clang::Decl const*, unsigned int, llvm::ArrayRef<clang::index::SymbolRelation>, clang::FileID, unsigned int, clang::index::IndexDataConsumer::ASTNodeInfo) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#3  0x00007fa31b77a9a5 in clang::index::IndexingContext::handleDeclOccurrence(clang::Decl const*, clang::SourceLocation, bool, clang::Decl const*, unsigned int, llvm::ArrayRef<clang::index::SymbolRelation>, clang::Expr const*, clang::Decl const*, clang::DeclContext const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#4  0x00007fa31b77acc6 in clang::index::IndexingContext::handleReference(clang::NamedDecl const*, clang::SourceLocation, clang::NamedDecl const*, clang::DeclContext const*, unsigned int, llvm::ArrayRef<clang::index::SymbolRelation>, clang::Expr const*, clang::Decl const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#5  0x00007fa31b7798e0 in (anonymous namespace)::IndexingDeclVisitor::handleTemplateArgumentLoc(clang::TemplateArgumentLoc const&, clang::NamedDecl const*, clang::DeclContext const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#6  0x00007fa31b7797f9 in (anonymous namespace)::IndexingDeclVisitor::VisitTemplateDecl(clang::TemplateDecl const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#7  0x00007fa31b7774d9 in clang::index::IndexingContext::indexDecl(clang::Decl const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#8  0x00007fa31b777709 in clang::declvisitor::Base<clang::declvisitor::make_const_ptr, (anonymous namespace)::IndexingDeclVisitor, bool>::Visit () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#9  0x00007fa31b7774d9 in clang::index::IndexingContext::indexDecl(clang::Decl const*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#10 0x00007fa31b6cf5ba in clang::ASTUnit::visitLocalTopLevelDecls(void*, bool (*)(void*, clang::Decl const*)) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#11 0x00007fa31b77241a in clang::index::indexASTUnit(clang::ASTUnit&, std::shared_ptr<clang::index::IndexDataConsumer>, clang::index::IndexingOptions) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#12 0x00007fa31b4b11ed in clang_indexTranslationUnit::$_1::operator()() const () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#13 0x00007fa31c804980 in llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#14 0x00007fa31c804a34 in RunSafelyOnThread_Dispatch(void*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#15 0x00007fa31c84bcba in ExecuteOnThread_Dispatch(void*) () from /home/maskray/Dev/Util/cquery/build/debug/bin/../lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5
#16 0x00007fa31ecbc494 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#17 0x00007fa31a4d3a8f in clone () from /lib/x86_64-linux-gnu/libc.so.6

Diff Detail

Repository
rL LLVM
MaskRay created this revision.Mon, Dec 25, 2:44 PM
MaskRay edited the summary of this revision. (Show Details)Mon, Dec 25, 2:48 PM
MaskRay added reviewers: akyrtzi, sammccall.
MaskRay edited the summary of this revision. (Show Details)Mon, Dec 25, 2:50 PM
MaskRay updated this revision to Diff 128148.Mon, Dec 25, 3:46 PM

DC -> Parent

MaskRay updated this revision to Diff 128149.Mon, Dec 25, 3:48 PM

DC -> Parent

MaskRay retitled this revision from [index] Return when DC is null in handleReference to [index] Return when Parent is null in handleReference.Mon, Dec 25, 3:48 PM
MaskRay edited the summary of this revision. (Show Details)
MaskRay updated this revision to Diff 128150.Mon, Dec 25, 3:53 PM

Sorry for changing this back and forth. But I do not have a powerful workstation and have to reverse engineer this.

MaskRay retitled this revision from [index] Return when Parent is null in handleReference to [index] Return when DC is null in handleReference.Mon, Dec 25, 4:05 PM
MaskRay edited the summary of this revision. (Show Details)

HighCommander4 narrowed it down https://github.com/jacobdufault/cquery/issues/219#issuecomment-354903152 to a much simpler reproduce:

c++
template <typename>
struct actor;

template <template <typename> class Actor = actor>
struct terminal;

This may trigger null pointer dereference of DC (because it calls cast). If the default template argument is removed, the crash goes away.

sammccall accepted this revision.Mon, Jan 8, 9:45 AM

Sorry, I was hoping @akyrtzi would see this.
I don't know enough about the code to know whether DC==null is an indicator of some other problem, but if this is crashing today then let's not block on that.

This revision is now accepted and ready to land.Mon, Jan 8, 9:45 AM

Could you add a test case for this change ? See examples in test/Index/Core.
Also for the test case code: template <template <typename> class Actor = actor>, is the actor reference in this code reported ?
See the test examples on how to print out and test how the source symbols are indexed.

@akyrtzi When I run c-index-test core -print-source-symbols -- a.cc on

template <typename>
struct actor;

template <template <typename> class Actor = actor>
struct terminal;

the issue disappears. It emerges only when clang_indexTranslationUnit is called with interactions of other things.

This revision was automatically updated to reflect the committed changes.

Ah, sorry I mislead you. To test this try using c-index-test -index-file /path/to/file, see other examples in test/Index, e.g. test/Index/index-file.cpp

MaskRay added a comment.EditedMon, Jan 8, 6:28 PM

Ah, sorry I mislead you. To test this try using c-index-test -index-file /path/to/file, see other examples in test/Index, e.g. test/Index/index-file.cpp

Thanks for the command. It is really helpful. I have re-run cquery with or without the !DC patch (on git-svn HEAD) and see no libclang crash. I guess the real issue was in somewhere else and in these weeks it has been fixed. The !DC check patch was useful then.

zsh
% cat a.cc
template <typename>
struct actor;

template <template <typename> class Actor = actor>
struct terminal;

# Arch Linux extra/clang 5.0.1-1
% c-index-test -index-file a.cc
[startedTranslationUnit]
[enteredMainFile]: a.cc
[indexEntityReference]: kind: c++-class-template | name: actor | USR: c:@ST>1#T@actor | lang: C++ | cursor: TypeRef=actor<type-parameter-0-0>:2:8 | loc: 2:8 | <parent>:: <<NULL>> | container: [TU] | refkind: direct
libclang: crash detected during indexing source file: {
  'source_filename' : '(null)'
  'command_line_args' : ['clang', 'a.cc'],
  'unsaved_files' : [],
  'options' : 1,
}
Failure (no details available)


# llvm+clang git-svn HEAD
% ~/Dev/llvm/release/bin/c-index-test -index-file a.cc
# no failure

% ~/Dev/llvm/release/bin/c-index-test -index-file test/src/unit-cbor.cpp -Isrc -Itest/thirdparty/catch -std=c++11 -resource-dir ~/Dev/llvm/release/lib/clang/7.0.0
# no failure