This is an archive of the discontinued LLVM Phabricator instance.

[AST] Add a new TemplateName for templates found via a using declaration.
ClosedPublic

Authored by hokein on Apr 5 2022, 6:32 AM.

Details

Summary

This is template version of https://reviews.llvm.org/D114251.

This patch introduces a new template name kind (UsingTemplateName). The
UsingTemplateName stores the found using-shadow decl (and underlying
template can be retrieved from the using-shadow decl). With the new
template name, we can be able to find the using decl that a template
typeloc (e.g. TemplateSpecializationTypeLoc) found its underlying template,
which is useful for tooling use cases (include cleaner etc).

This patch merely focuses on adding the node to the AST.

Next steps:

  • support using-decl in qualified template name;
  • update the clangd and other tools to use this new node;
  • add ast matchers for matching different kinds of template names;

Diff Detail

Event Timeline

hokein created this revision.Apr 5 2022, 6:32 AM
Herald added a project: Restricted Project. · View Herald Transcript
hokein requested review of this revision.Apr 5 2022, 6:32 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptApr 5 2022, 6:32 AM

This looks pretty good!
The tests in clang are sadly indirect.
I think adding support to clangd's FindTarget would be a small change and would allow a fairly direct test, but maybe it will affect a bunch of existing tests or possibly have a blast radius. Up to you.

clang/include/clang/AST/ASTContext.h
2184 ↗(On Diff #420485)

nit: there's something of a canonical order in TemplateName::NameKind, I think it's useful to follow this as much as possible for consistency when navigating in these huge files.
This mostly means placing UsingTemplate code at the bottom of lists of template name cases.

clang/include/clang/AST/TemplateName.h
30

this list is alpha sorted, please preserve (and below)

415

docs on getUsingShadowDecl and getUnderlying

415

getFoundDecl() would be consistent with UsingType, and would describe the relation between this and the UsingShadowDecl, rather than echoing its type

416

maybe getUnderlyingTemplate? "underlying" acts as an adjective so this feels a little ungrammatical

417

nit: separate the public API from implementation details with a blank line at least

419

It seems really unfortunate that this is a TemplateDecl* instead of a TemplateName.

for:

template <int> class x;
namespace a { using ::x; }
a::x<0>;

we want a::x to include both a QualifiedTemplateName (modelling the a:: qualifier) and a UsingTemplateName (modelling the UsingShadowDecl that was found).

I'd guess we can change Template to be a TemplateName internally, and add getTemplate() while keeping the existing get[Template]Decl APIs on top of TemplateName::getAsTemplateDecl().
I suppose this could be a separate change (though it'd be nice to know whether it's going to work...)

Of course if that changed there's a natural followon to take the qualifier out of DependentTemplateName and turn it into a QualifiedTemplateName wrapping a DependentTemplateName. (Definitely out of scope for this patch, not sure if it's reasonable to expect you to do it at all)

clang/lib/AST/ASTContext.cpp
6175

you could also handle this in the QualifiedTemplate/Template case, since the decl is guaranteed to be present.

sammccall added inline comments.Apr 5 2022, 8:00 AM
clang/lib/AST/ItaniumMangle.cpp
2389

this seems pretty scary: we have to use a goto here rather than extract a function, because the target code also uses goto...

The nontrivial TemplateTemplateParmDecl case isn't possible here. So I suggest just calling mangleSourceNameWithAbiTags directly without trying to reuse logic.

clang/lib/AST/TemplateName.cpp
116

= getAsUsingTemplateName()

302

Why is this correct, rather than potentially printing the wrong sugar?

If the reasoning is that that the underlying TemplateName has the same name and can't have any unwanted sugar, that seems subtle and deserves a comment.

I think just printing the unqualified name directly would be clearer.

clang/lib/Sema/SemaDecl.cpp
507–511

nit: just Template(TD) or = TD?

clang/lib/Sema/SemaDeclCXX.cpp
11028

I don't think it can be valid to find the template name via a usingshadowdecl, because the deduction guide must be declared in the same scope as the template. (err_deduction_guide_wrong_scope).

Is this to prevent an unneccesary second diagnostic? (And if so, I wonder whether we should also include QualifiedTemplate, maybe as a separate change). It may deserve a comment

clang/lib/Tooling/CMakeLists.txt
62 ↗(On Diff #420485)

revert?

(though yes, this is annoying)

clang/test/AST/ast-dump-using-template.cpp
2

Maybe a comment here:
TemplateNames are not dumped, so the sugar here isn't obvious. However the "using" on the TemplateSpecializationTypes shows that the UsingTemplateName is present.

hokein updated this revision to Diff 420771.Apr 6 2022, 3:47 AM
hokein marked 12 inline comments as done.

address review comments;
rename methods;
fix templatename::print and add a test;
add a fixme for qualified template name;

hokein added a comment.Apr 6 2022, 3:49 AM

This looks pretty good!
The tests in clang are sadly indirect.

This is mostly due to the fact that TemplateName is not dumped. This patch extend the NodeDumper to print a using indicator, I think it is relatively obvious in the AST.

I think adding support to clangd's FindTarget would be a small change and would allow a fairly direct test, but maybe it will affect a bunch of existing tests or possibly have a blast radius. Up to you.

Adding support in clangd FindTarget is my next step, it would be a followup patch. I feel like the FindTarget tests are more related to the application logic of the new TemplateName, and are less direct compared with the tests in this patch.

clang/include/clang/AST/TemplateName.h
419

we want a::x to include both a QualifiedTemplateName (modelling the a:: qualifier) and a UsingTemplateName (modelling the UsingShadowDecl that was found).

I'd guess we can change Template to be a TemplateName internally, and add getTemplate() while keeping the existing get[Template]Decl APIs on top of TemplateName::getAsTemplateDecl().
I suppose this could be a separate change (though it'd be nice to know whether it's going to work...)

Agree, I think this is the right solution, but would require more work (a followup patch).

There are two options, 1) leave the qualified template name out, and support it in a followup patch, 2) workaround it -- we wrap the qualified template name with a using template name, though it doesn't quite match our mental model, I think it is probably ok. (I added a FIXME in SemaTemplate.cpp).

Let me know what you think, also happy to switch to 1).

clang/lib/AST/ASTContext.cpp
6175

yeah, but I think the current version is clearer.

clang/lib/AST/TemplateName.cpp
302

good point, you're right, this will print duplicated nested specifiers. Change it to print unqualified name only, and added a test.

clang/lib/Sema/SemaDeclCXX.cpp
11028

yeah, it prevents a bogus diagnostic introduced by this patch.

namespace N {
   template<typename T> struct NamedNS1 {}; // expected-note {{here}}
 }
 using N::NamedNS1;
 NamedNS1(int) -> NamedNS1<int>; // expected-error {{deduction guide must be declared in the same scope as template}}

With this patch without the change here, we emit an extra diagnostic deduced type 'NamedNS1<int>' of deduction guide is not written as a specialization of template 'NamedNS1'.

And this part of code might have another issue where it emits a wrong err_deduction_guide_defines_function for a correct code, (this is something I plan to take a look but not in this patch) .

clang/lib/Tooling/CMakeLists.txt
62 ↗(On Diff #420485)

oops, an accident change, reverted.

Thanks, I think this is just about ready to go.
Only substantive issue is I think we should address the QualifiedTemplate/UsingTemplate interaction in a more principled way.

clang/include/clang/AST/PropertiesBase.td
625

rename these properties too?

clang/include/clang/AST/TemplateName.h
419

I don't like 2), it adds code in order to introduce an incorrect "inside-out" AST in a surprising way that will lead to subtle bugs, and we'll need to undo this later. To me this seems probably worse than dropping one of the sugar nodes and never fixing it, which is already pretty bad.

I think we should do 1) instead - I think it's actually simpler to land the QualifiedTemplate-holds-TemplateName patch *first*, but either order seems fine.

clang/lib/Sema/SemaDeclCXX.cpp
11028

That makes sense, though it seems to me inconsistent to suppress the secondary diagnostic for UsingTemplate but emit it for QualifiedTemplate - it's equally bogus/valid in both cases.

Can we either suppress both, or suppress neither for now and suppress them in a subsequent patch?

hokein updated this revision to Diff 420805.Apr 6 2022, 5:04 AM
hokein marked an inline comment as done.

address remaining comments.

clang/include/clang/AST/TemplateName.h
419

ok, faire enough, switched to 1).

clang/lib/Sema/SemaDeclCXX.cpp
11028

ok, let's not do it for now, and add a FIXME for the newly-introduced diagnostic in the existing test.

sammccall added inline comments.Apr 6 2022, 6:12 AM
clang/include/clang/AST/TemplateName.h
408

(Per offline discussion)

The UnderlyingTN here seems like it would refer to the TemplateDecl pointed to by the UsingShadowDecl, with sugar from the RHS of the UsingDecl.

The only sugar possible is a namespace [alias] qualifier, so the only TemplateNames possible are Qualified and Ordinary. And the code isn't actually creating Qualified names here when it would be applicable.

So we should either:

  • change this to be a TemplateDecl* instead
  • start creating the Qualified names where appropriate

Arguments in favor of having the sugar qualified names:

  • it's a more complete AST
  • it's more consistent with sugar types

Arguments against the sugar qualified names

  • performance: it avoids adding a second level of indirection, extra storage etc
  • it avoids confusing callers into handling *all* TemplateName cases here
  • the sugar is still available: USD->getUsingDecl()->getQualifier()

I think we should not provide sugared names here, and change the type to TemplateDecl*

408

If we change the type to TemplateDecl, do we actually need to store it? Or can we just cast<TemplateDecl>(USD->getTargetDecl())?

hokein updated this revision to Diff 420976.Apr 6 2022, 12:41 PM

A revised iteration: store UsingShadowDecl directly in TemplateName::Storage,
UsingTemplateName class is no longer needed.

hokein updated this revision to Diff 420977.Apr 6 2022, 12:43 PM

some tweaks.

Storing the UsingShadowDecl directly is much nicer :-)

clang/include/clang/AST/TemplateName.h
235

nit: declaration *found* through

306

getAsUsingShadowDecl!

clang/lib/AST/ASTContext.cpp
6129

nit: no braces (other cases only use them where variables are declared)

clang/lib/AST/ASTImporter.cpp
9244

remove commented code

9251

This doesn't look right at all, you're returning an ordinary template name, not a using...

clang/lib/AST/TemplateName.cpp
80

assert(isa<TemplateDecl>(Using->getTargetDecl()))

(better to crash upfront than in getAsTemplateDecl() or so)

269

this is going to print the wrong qualifier: qualifier of the underlying rather than of the usingdecl

clang/lib/Sema/SemaTemplate.cpp
294

it would be really nice to have a test verifying that if we are ambiguous here and form an Overloaded TemplateName, then when it's later resolved it turns into a Using TemplateName where appropriate...

hokein updated this revision to Diff 421444.Apr 8 2022, 12:46 AM
hokein marked 5 inline comments as done.

address some comments, and cleanups.

hokein added inline comments.Apr 8 2022, 12:47 AM
clang/lib/AST/TemplateName.cpp
80

There is already a stricter assertion TD == Using->getUnderlyingDecl() in every construct side.

269

I think this is a good argument -- after the code namespace ns { using std::vector; }, what's the fully-qualified name of the vector UsingTemplateName within ns?

  1. ns::vector -- the qualified name of using-decl.
  2. or std::vector- the qualified name of the underlying template decl.

after a second thought, I'm leaning towards 2). AFAIK, the major place to print fully-qualified name is TemplateArgument::print (for template template parameters). 2) is providing more useful information for users to locate the actual template class. Imaging a case where we have a using long::long::long::abc in a global namespace, printing abc seems less useful.

This is also consistent with UsingType.

clang/lib/Sema/SemaTemplate.cpp
294

then when it's later resolved it turns into a Using TemplateName where appropriate...

clang doesn't seem to work like this -- it does form an Overloaded TemplateName during parsing, but a call to these overloaded function templates is a UnresolvedLookupExpr which stores all function candidates. After the template instantiation, the UnresolvedLookupExpr is resolved to a normal DeclRefExpr (that said there is no Using TemplateName being built).

The reason I added the isAmbiguous guard here was to ensure the Using->getUnderlyingDecl() == TD invariant, it is always true for correct cases. However, we pickup an *arbitrary* template for error-recovery (see line 231). Now I have figured out a better way to handle it (by storing the corresponding FoundUsingDecl) without needing this isAmbiguous guard. Removed.

hokein updated this revision to Diff 421856.Apr 11 2022, 2:44 AM

add a unittest for template name.

sammccall accepted this revision.Apr 11 2022, 3:20 AM
sammccall added inline comments.
clang/lib/AST/TemplateName.cpp
123

= getAsUsingShadowDecl

269

OK, that makes sense, but deserves a serious comment - it *looks* like a bug.

(I'd forgotten about the UsingType behavior. It has a 9-line comment!)

clang/lib/Sema/SemaDecl.cpp
1112–1116

FIXME here for qualified using case?

This revision is now accepted and ready to land.Apr 11 2022, 3:20 AM
hokein updated this revision to Diff 421891.Apr 11 2022, 5:54 AM
hokein marked 2 inline comments as done.

address remaining comments.

hokein edited the summary of this revision. (Show Details)Apr 11 2022, 5:57 AM
hokein updated this revision to Diff 422134.Apr 12 2022, 12:33 AM
hokein marked an inline comment as done.
hokein edited the summary of this revision. (Show Details)

Fix a use-after-free in the unittest, TemplateArgumentLoc is a local storage of
MatchCallback.

This revision was landed with ongoing or failed builds.Apr 12 2022, 1:48 AM
This revision was automatically updated to reflect the committed changes.
hokein reopened this revision.Apr 12 2022, 11:58 AM

reopening it for the reland version.

This revision is now accepted and ready to land.Apr 12 2022, 11:58 AM
hokein updated this revision to Diff 422310.Apr 12 2022, 11:58 AM

reland version: there is no freebit in the TemplateName's PointerUnion to store
5 different pointer types in 32-bit build, instead we change the TemplateDecl to
NamedDecl, which can be used to hold the TemplateDecl and UsingShadowDecl.

sammccall accepted this revision.Apr 13 2022, 4:36 AM

Nice fix! Just nits

clang/include/clang/AST/TemplateName.h
198

nit: I'd use Decl here instead of NamedDecl.

Sure, the lowest common type is NamedDecl, but that's not really important and we don't use any of the NamedDecl interface.

304

I think this second sentence is confusing rather than helpful to understanding to use this class (as opposed to its implementation).

The main way to get the underlying declaration is with getAsTemplateDecl(). From a public POV, a Using name has a using shadow decl *and* a template decl, the fact that we only store one of these is an implementation detail.

306

If (after followup patches) you have a qualified name Q wrapping a using name U, I'd expect Q.getAsUsingShadowDecl() to extract the pointer from U.storage.

I'd expect this based on the first sentence of the comment here, and also consistent with getAsTemplateDecl() which does this sort of unwrapping.

So no changes needed if you're OK with adding that unwrapping behavior in the next patch, otherwise I think we need to think a bit about how to set expectations with this API.

clang/lib/AST/TemplateName.cpp
90

nit: invert this so that UsingShadowDecl is the special case and TemplateDecl the general one?

112

and again here

256

I don't understand what "std::vector is much more common" there's no ground truth for us to observe.
Rather maybe "using declarations are used to import names more often than to export them, and using the original name is most useful in this case"?

hokein updated this revision to Diff 422586.Apr 13 2022, 11:48 AM
hokein marked 4 inline comments as done.

address comments.

This revision was landed with ongoing or failed builds.Apr 14 2022, 2:05 AM
This revision was automatically updated to reflect the committed changes.