This is an archive of the discontinued LLVM Phabricator instance.

[clangd] Implement a function to lex the file to find candidate occurrences.
ClosedPublic

Authored by hokein on Oct 30 2019, 6:07 AM.

Details

Summary

This will be used for incoming cross-file rename (to detect index
staleness issue).

Diff Detail

Event Timeline

hokein created this revision.Oct 30 2019, 6:07 AM
Herald added a project: Restricted Project. · View Herald TranscriptOct 30 2019, 6:07 AM

Build result: pass - 59741 tests passed, 0 failed and 762 were skipped.
Log files: console-log.txt, CMakeCache.txt

ilya-biryukov added inline comments.Nov 4 2019, 8:30 AM
clang-tools-extra/clangd/SourceCode.cpp
769

Why do we have both raw_identifier and identifier?
We should run lexer in the raw mode, in which case we should always get raw_identifier.

clang-tools-extra/clangd/SourceCode.h
236

NIT: maybe use a shorter name for the paramter? Identifier conveys the same meaning.

clang-tools-extra/clangd/unittests/SourceCodeTests.cpp
690

Could you add a few more occurrences?
Maybe also with the weird things like multi-line tokens:

F\
o\
o
hokein updated this revision to Diff 227877.Nov 5 2019, 7:43 AM
hokein marked 4 inline comments as done.

address comments, and simplify the code.

clang-tools-extra/clangd/SourceCode.cpp
769

I just borrowed from collectIdentifiers. You are right, we only run a raw lexer, updated the code.

clang-tools-extra/clangd/unittests/SourceCodeTests.cpp
690

Done, note that multi-line identifier is not supported -- getRawIdentifier returns a literal name F\o\o which makes the check Tok.getRawIdentifier() == Identifier fail. I think it is fine to not support it.

Build result: pass - 59741 tests passed, 0 failed and 762 were skipped.
Log files: console-log.txt, CMakeCache.txt

ilya-biryukov accepted this revision.Nov 5 2019, 10:09 AM
ilya-biryukov added inline comments.
clang-tools-extra/clangd/SourceCode.cpp
766

NIT: use early exits to keep the code flat.

if (Tok.getKind() != ...)
  return;
auto Range = ...;
if (!Range)
  return;
...
clang-tools-extra/clangd/unittests/SourceCodeTests.cpp
690

Yeah, it's probably ok.
I'd still add this as a test-case to make sure we don't crash and fix the behavior.

Although agree it's not very important, this is super rare.

This revision is now accepted and ready to land.Nov 5 2019, 10:09 AM
hokein updated this revision to Diff 228011.Nov 6 2019, 12:53 AM
hokein marked an inline comment as done.

address comment.

This revision was automatically updated to reflect the committed changes.

Build result: fail - 59859 tests passed, 22 failed and 763 were skipped.

failed: libc++.std/thread/thread_mutex/thread_mutex_requirements/thread_mutex_requirements_mutex/thread_mutex_class/try_lock.pass.cpp
failed: lld.ELF/linkerscript/filename-spec.s
failed: Clang.Index/index-module-with-vfs.m
failed: Clang.Modules/double-quotes.m
failed: Clang.Modules/framework-public-includes-private.m
failed: Clang.VFS/external-names.c
failed: Clang.VFS/framework-import.m
failed: Clang.VFS/implicit-include.c
failed: Clang.VFS/include-mixed-real-and-virtual.c
failed: Clang.VFS/include-real-from-virtual.c
failed: Clang.VFS/include-virtual-from-real.c
failed: Clang.VFS/include.c
failed: Clang.VFS/incomplete-umbrella.m
failed: Clang.VFS/module-import.m
failed: Clang.VFS/module_missing_vfs.m
failed: Clang.VFS/real-path-found-first.m
failed: Clang.VFS/relative-path.c
failed: Clang.VFS/test_nonmodular.c
failed: Clang.VFS/umbrella-framework-import-skipnonexist.m
failed: Clang.VFS/vfsroot-include.c
failed: Clang.VFS/vfsroot-module.m
failed: Clang.VFS/vfsroot-with-overlay.c

Log files: console-log.txt, CMakeCache.txt