This is an archive of the discontinued LLVM Phabricator instance.

[clang] Make sure codecompletion is called for calls even when inside a token.
ClosedPublic

Authored by kadircet on Aug 21 2018, 7:57 AM.

Diff Detail

Repository
rL LLVM

Event Timeline

kadircet created this revision.Aug 21 2018, 7:57 AM
kadircet retitled this revision from [clangd] Make sure codecompletion is called for calls even when inside a token. to [clang] Make sure codecompletion is called for calls even when inside a token..Aug 21 2018, 7:59 AM

Maybe add tests? Lit tests for code completion (clang/test/CodeCompletion) should be a good fit

include/clang/Parse/Parser.h
2971 ↗(On Diff #161724)

Maybe put the field closer to the existing ones?

lib/Parse/ParseExpr.cpp
2820 ↗(On Diff #161724)

Why should we set this flag here? Can it be handled by Completer function that we pass into ParseExperssionList instead?

Sorry, missed the dependent revision adding tests to clangd at first. Having another test in sema would still be great, to make sure we guard against regressions if someone does not have clang-tools-extra checked out.

kadircet updated this revision to Diff 161966.Aug 22 2018, 8:16 AM
kadircet marked 2 inline comments as done.
  • Resolve discussions.
  • Add tests.
  • Fix a bug noticed by lit tests.

One major limitation of the current workaround is that IIRC clang can actually call code completion callbacks, including this method, multiple times (e.g. when completing inside a macro definition that gets expanded later in the file or during parser recovery).
The workaround will work at the first call, but won't trigger at the repeated calls. We could find a way around that by resetting the flag to false at the right points (probably at every top-level function call?) WDYT?

include/clang/Parse/Parser.h
217 ↗(On Diff #161966)

NIT: would go with something less harsh, like "workaround to make sure we only call CodeCompleteCall on the deepest call expression"

lib/Parse/ParseExpr.cpp
1661 ↗(On Diff #161966)

Shouldn't we ignore CodeCompleteCall if CalledOverloadCompletion is set to true?

1672 ↗(On Diff #161966)

What's the special handling in ObjC? Can we unify with our workaround?

1674 ↗(On Diff #161966)

Why don't we handle the CalledOverloadCompletion flag inside the Completer() itself?
We're currently special-casing the middle identifier and error case, why not have the same code path for CodeCompleteCall() calls at the start of the identifier?

kadircet updated this revision to Diff 163970.Sep 4 2018, 9:20 PM
  • Update tests.
  • Update comment.
  • Rebase.
kadircet marked 4 inline comments as done.Sep 4 2018, 9:23 PM
kadircet updated this revision to Diff 163971.Sep 4 2018, 9:25 PM
  • Update comments.
ilya-biryukov added inline comments.Sep 6 2018, 3:44 AM
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

Any idea why behavior changed in that case?

ilya-biryukov added inline comments.Sep 6 2018, 6:09 AM
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

Looked at it more thoroughly now...
So we're now showing both the signature help and the completion, right?
If that the case, LG, but can we include the rest of completion items from CHECK-N0?

Maybe also add a comment that the last item is from overload set completion, rather than the ordinary code completion? (To avoid confusion and make it clear why we need an extra check there)

kadircet updated this revision to Diff 164307.Sep 6 2018, 3:56 PM
  • Fix tests.
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

Yes, c-index-test binary collects results from both ProcessOverloadCandidate and ProcessCodeCompleteResults. Therefore they are merged. Adding the same set of results on the above is not enough for that particular case, due to the clever nature of CodeCompleteOverloadResult at https://github.com/llvm-mirror/clang/blob/master/lib/Sema/SemaCodeComplete.cpp#L4439.

It doesn't just try to tell you the overloads of that particular function but also tries to complete the current argument. Therefore the list simply gets huge by expansion of all the macros and other stuff.

But when looking into it I found out I was checking for wrong return values fixing that with the new diff.

ilya-biryukov added inline comments.Sep 7 2018, 4:28 AM
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

I don't think it's correct to call CodeCompleteExpression and CodeCompleteOrdinaryName in this context.
This completion assume we're at the start of the argument, which is not true anymore.
E.g. we can be producing weird results in the middle of expressions. Some examples from the top of my head:

func(var.^); // <-- (1) we add top-level completions in addition to members of bar
func(&^); // <-- (2) we provide incorrect ParamType

For (2) if ParamType is int*, we would incorrectly uprank items of type int* (should uprank items of type int instead).

I'll investigate a bit more to see if we can refactor the code to untangle signature help from code completion.

ilya-biryukov added inline comments.Sep 7 2018, 6:15 AM
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

D51782 removes completion-specific bits out of functions that produce signature help.
After it lands, it should be significantly easier to produce results for these cases.

Specifically, we will be able to only run signature help in the middle of the args, and signature help followed by a normal completion at the start of the args.

ilya-biryukov added inline comments.Sep 7 2018, 7:11 AM
test/Index/complete-block-property-assignment.m
71 ↗(On Diff #163971)

Landed D51782. This change should be easier now.
We can now call both signature help and code completion at the start of the arguments and only signature help in the middle of the arguments.

kadircet updated this revision to Diff 164477.Sep 7 2018, 11:36 AM
  • Rebase and use new signature help.

Currently the problem is, there are again some tests out there that rely on
CodeCompeleteOrdinaryName to be called even when getting overloads at an unknown
parameter type. These tests are:

Clang :: CodeCompletion/call.cpp
Clang :: Index/code-completion.cpp
Clang :: Index/complete-call.cpp
Clang :: Index/complete-functor-call.cpp
Clang :: Index/complete-optional-params.cpp
Clang :: Index/complete-pointer-and-reference-to-functions.cpp

You can run something like this to check for the output(change ../clangd_bugs):

ninja c-index-test && ./bin/c-index-test -code-completion-at=../clangd_bugs/tools/clang/test/Index/complete-call.cpp:53:12 ../clan
gd_bugs/tools/clang/test/Index/complete-call.cpp

As you can see current version generates only overloadcandidates, but tests
don't check them with a next loop, because they expect other completions, which
is not problematic but after checks for the items they also look for completion
context, which is not set as desired if CodeCompleteOrdinaryName or
CodeCompleteExpression is not called.

But for the first test case it is more complicated, it checks for code patterns
like dynamic_cast<EXPR>(EXPR); which is not generated if we don't call
CodeCompletionOrdinaryName or Expressions.

Currently the problem is, there are again some tests out there that rely on
CodeCompeleteOrdinaryName to be called even when getting overloads at an unknown
parameter type

CodeCompleteExpression works just fine there. Unknown parameter type should not block code completion.
The idea is that anywhere except the start of the argument expression, we need to call only signature help. At the start of the argument, we have to call both signature help and code completion.
All existing tests pass with the following diff:

--- a/lib/Parse/ParseExpr.cpp
+++ b/lib/Parse/ParseExpr.cpp
@@ -1659,12 +1659,19 @@ Parser::ParsePostfixExpressionSuffix(ExprResult LHS) {
 
       if (OpKind == tok::l_paren || !LHS.isInvalid()) {
         if (Tok.isNot(tok::r_paren)) {
-          if (ParseExpressionList(ArgExprs, CommaLocs, [&] {
-                QualType PreferredType = Actions.ProduceCallSignatureHelp(
-                    getCurScope(), LHS.get(), ArgExprs, PT.getOpenLocation());
-                Actions.CodeCompleteExpression(getCurScope(), PreferredType);
-              })) {
+          auto Completer = [&] {
+            QualType PreferredType = Actions.ProduceCallSignatureHelp(
+                getCurScope(), LHS.get(), ArgExprs, PT.getOpenLocation());
+            Actions.CodeCompleteExpression(getCurScope(), PreferredType);
+            CalledOverloadCompletion = true;
+          };
+          if (ParseExpressionList(ArgExprs, CommaLocs, Completer)) {
             (void)Actions.CorrectDelayedTyposInExpr(LHS);
+            if (PP.isCodeCompletionReached() && !CalledOverloadCompletion) {
+              CalledOverloadCompletion = true;
+              Actions.ProduceCallSignatureHelp(
+                  getCurScope(), LHS.get(), ArgExprs, PT.getOpenLocation());
+            }
             LHS = ExprError();
           } else if (LHS.isInvalid()) {
             for (auto &E : ArgExprs)

Please note there are other places where signature help methods (ProduceCallSignatureHelp and ProduceConstructorSignatureHelp) are called, we need to handle all of them. List of files that have those calls:

  • lib/Parse/ParseExprCXX.cpp
  • lib/Parse/ParseOpenMP.cpp
  • lib/Parse/ParseExpr.cpp
  • lib/Parse/ParseDecl.cpp
include/clang/Parse/Parser.h
219 ↗(On Diff #164477)

NIT: rename to CalledSignatureHelp, this shouldn't affect completion

kadircet updated this revision to Diff 164639.Sep 10 2018, 3:11 AM
kadircet marked an inline comment as done.
  • Change "inside parameter" checking and fix completion string printing for optional parameters.
  • Update tests.

Currently the problem is, there are again some tests out there that rely on
CodeCompeleteOrdinaryName to be called even when getting overloads at an unknown
parameter type

CodeCompleteExpression works just fine there. Unknown parameter type should not block code completion.
The idea is that anywhere except the start of the argument expression, we need to call only signature help. At the start of the argument, we have to call both signature help and code completion.

Yeah you are right, I thought I could check whether I am inside a parameter or not by looking at qual type, which was apparently wrong :D

But, unfortunately, there were still tests failing which was caused by printing of completion strings for overloaded methods with optional parameters. It was trying to access Text field, which is not defined for Optional parameters.
Since it didn't print optional parameters in the original behavior I kept it that way. Can send it as a different patch if need be.

kadircet marked 6 inline comments as done.Sep 10 2018, 3:13 AM
ilya-biryukov added inline comments.Sep 10 2018, 3:28 AM
include/clang/Parse/Parser.h
217 ↗(On Diff #164639)

s/ProduceSignaturehelp/ProduceSignatureHelp

lib/Parse/ParseDecl.cpp
2325 ↗(On Diff #164639)

Maybe call signature help here directly and remove param from lamdba?
Having a parameter in lambda adds both makes the lamdba body more complicated and adds extra state that we have to reason about.

Same for the other calls.

ilya-biryukov added a comment.EditedSep 10 2018, 3:46 AM
This comment has been deleted.
lib/Sema/CodeCompleteConsumer.cpp
622 ↗(On Diff #164639)

This change is really sneaky and unrelated to the rest of the patch. It should definitely be done separately from this patch.

Could you give more details on what fails exactly instead? Can we update the test instead?

kadircet updated this revision to Diff 164649.Sep 10 2018, 3:50 AM
kadircet marked 2 inline comments as done.
  • Resolve discussions.
ilya-biryukov added inline comments.Sep 10 2018, 3:56 AM
lib/Parse/ParseExpr.cpp
1663 ↗(On Diff #164649)

NIT: inline completer.

1663 ↗(On Diff #164649)

Maybe inline this into lambda body again?
Now that it's not called outside it, we don't need a variable anymore.

lib/Parse/ParseExprCXX.cpp
2827 ↗(On Diff #164649)

ActOnTypeName is called at a different point now, please move it back into the lambda.

2830 ↗(On Diff #164649)

Same here: maybe inline the lambda into the call to keep the changes minimal?

lib/Parse/ParseOpenMP.cpp
419 ↗(On Diff #164649)

Same here: maybe inline the lambda?

ilya-biryukov added inline comments.Sep 10 2018, 5:31 AM
lib/Sema/CodeCompleteConsumer.cpp
622 ↗(On Diff #164639)

As Kadir mentioned offline this actually LG, we shouldn't read C.Text for CK_Optional chunks.

Maybe leave a FIXME that we could actually print the CodeCompletionString from the optional chunk?

ilya-biryukov added inline comments.Sep 10 2018, 5:39 AM
lib/Parse/ParseExprCXX.cpp
2827 ↗(On Diff #164649)

As discussed offline, this actually LG.
It's weird that result of ActOnTypeName seems to be only needed for code completion and it's called even when completion is disabled.

However, this was the case before the patch as well

kadircet updated this revision to Diff 164659.Sep 10 2018, 5:47 AM
  • Inline completers.
ilya-biryukov accepted this revision.Sep 10 2018, 5:52 AM

Thanks! LGTM

lib/Parse/ParseExprCXX.cpp
1687 ↗(On Diff #164659)

NIT: inline this last completer.

This revision is now accepted and ready to land.Sep 10 2018, 5:52 AM
kadircet marked 7 inline comments as done.Sep 10 2018, 5:53 AM
kadircet updated this revision to Diff 164661.Sep 10 2018, 6:00 AM
kadircet marked an inline comment as done.
  • One last completer
This revision was automatically updated to reflect the committed changes.