This is an archive of the discontinued LLVM Phabricator instance.

Fix handling of overriden methods during ASTImport
ClosedPublic

Authored by shafik on Jan 18 2019, 2:07 PM.

Details

Summary

When importing classes we may add a CXXMethodDecl more than once to a CXXRecordDecl. This patch will fix the cases we currently know about and handle both the case where we are only dealing with declarations and when we have to merge a definition into an existing declaration.

Diff Detail

Repository
rC Clang

Event Timeline

shafik created this revision.Jan 18 2019, 2:07 PM

@martong the only issue is that I am seeing a regression on Analysis/ctu-main.cpp when I run check-clang. I am going to look into it but if you have any insights that would be helpful.

balazske added inline comments.Jan 21 2019, 12:20 AM
lib/AST/ASTImporter.cpp
3058

The cast<Decl> should not be needed here (and is not done at the other return places). The const_cast can be omitted too (FoundByLookup is not const now).

unittests/AST/ASTImporterTest.cpp
2242

I think names like BP and BFP could be better (P should stand for "Pattern" and always the last part of the name, so use BFP and BFIsDefP). And for storing B::F a BF can be better name than B.

2259

Use CodeWithoutDef.

2311

It is ensured only by FirstDeclMatcher that not the definition is found (this pattern may match the definition too) for BFP. It may be more accurate to include in the pattern that this is not a definition.

2324

For the out-of-class definition the hasParent(cxxRecordDecl(hasName("B"))) does not match? (Otherwise this count should be 1.)

2343

The previous tests use two TUs too so the name for this is not exact (here the code is different, maybe use ImportOverriddenMethodTwiceOutOfClassDefInSeparateCode?). This test does roughly the same as ImportOverriddenMethodTwiceDefinitionFirst but with the definition last and out-of-class version. (Name can be ImportOverriddenMethodTwiceOutOfClassDefLast too.) I am not sure why is the foo used here (B::F can be imported instead).

2365

hasName("B") is wrong here, should be "D"?

Hi Shafik, thank you for this patch! Generally it looks quite okay to me, but I have a few minor comments.

lib/AST/ASTImporter.cpp
3061

Could you please do this refactor and remove this sentence from the comment? (At line 3208 we repeat the same logic.)

unittests/AST/ASTImporterTest.cpp
2242

I agree. I think the contractions we use in ImportOverriddenMethodTwiceOutOfClassDef are more consistent.

2324

That does not match, because hasParent matches based on the lexical decl context. In the case of the out-of-class definition, the lexical decl context is the global namespace (the TranslationUnitDecl).

However, the semantical decl context of the out-of-class definition is the CXXRecordDecl with the name "B". But this relation is not symmetric. The definition is not the child (i.e. DeclContext::containsDecl is false) of the CXXRecordDecl rather the child of the TranslataionUnitDecl.

2384

Perhaps it would make sense to have expectations on DFP and on DFPIsDef too.

2392

Perhaps it would make sense to have expectations on D::f() {} too. Even better could be to have a lambda which takes a few Decls as parameters and does the expectations on those.

@martong the only issue is that I am seeing a regression on Analysis/ctu-main.cpp when I run check-clang. I am going to look into it but if you have any insights that would be helpful.

I am looking into it and will come back with what I have found.

I have found some other minor things in the tests.

unittests/AST/ASTImporterTest.cpp
2274

I think the parent should have hasName("D") here. Seems like a copy paste error.

2281

ImportedD seems to be unused. (And in the next test case too.)

2399

Could we check the redecl chain of D::f() too?

Shafik,
I have realized what's the problem with the ctu-main test. When we import the body and set the new body to the existing FunctionDecl then the parameters are still the old parameters so the new body does not refer to the formal parameters! This way the analyzer legally thinks that the parameter is not used inside the function because there is no DeclRef to that :(
This could be solved only if we merge *every* parts precisely, including the parameters, body, noexcept specifier, etc. But this would be a huge work.

Also, the test in ctu-main contains ODR violations. E.g, below fcl is first just a protoype, then it is defined in-class in the second TU.

// ctu-main.cpp
class mycls {
public:
  int fcl(int x);
  //...

// ctu-other.cpp
 class mycls {
 public:
  int fcl(int x) {
    return x + 5;
  }
  //...

In the second TU it should be defined out-of-class.

So my suggestion is to

  1. use out-of-class definition of functions in ctu-other.cpp. Since I tried it already i have the diff for the lit tests, attached.
  2. skip the case when there is a definition in the 'from' context and let it do the redecl chain.

For 2) I was thinking about this:

if (FoundByLookup) {
  if (auto *MD = dyn_cast<CXXMethodDecl>(FoundByLookup)) {
    if (D->getLexicalDeclContext() == D->getDeclContext()) {
      if (!D->doesThisDeclarationHaveABody())
        return cast<Decl>(const_cast<FunctionDecl *>(FoundByLookup));
      else {
        // Let's continue and build up the redecl chain in this case.
        // FIXME Merge the functions into one decl.
      }
    }
  }
}

Later, we must implement the case when we have to merge the definition into the prototype properly, but that should be definitely another patch.
Actually, this case comes up mostly with class template specializations , because different specializations may have a function prototype in one TU, but a definition for that in another TU (see e.g. the test MergeFieldDeclsOfClassTemplateSpecialization).

lib/AST/ASTImporter.cpp
3054

It is not trivial why we add this block to the code, so I think a comment would be really appreciated here.
I was thinking about something like this:

+  // We do not allow more than one in-class prototype of a function.  This is
+  // because AST clients like VTableBuilder asserts on this.  VTableBuilder
+  // assumes that only one function can override a function. Building a redecl
+  // chain would result there are more than one function which override the
+  // base function (even if they are part of the same redecl chain inside the
+  // derived class.)
martong added inline comments.Jan 22 2019, 8:45 AM
lib/AST/ASTImporter.cpp
3058

We must also register the decl into the map of imported decls as we do in all the other cases.

return Importer.MapImported(D, FoundByLookup);

So the problem is that there are references to ParmVarDecl from inside function body and at import of ParmVarDecl always a new one is created even if there is an already existing (in the existing function prototype)? Maybe it works in VisitParmVarDecl to search for already existing ParmVarDecl (not already imported) and return if found.

So the problem is that there are references to ParmVarDecl from inside function body and at import of ParmVarDecl always a new one is created even if there is an already existing (in the existing function prototype)?

Yes. During the import of the body we import a DeclRefExpr which refers to a ParmVarDecl but not the existing one, thus a new is created.

Maybe it works in VisitParmVarDecl to search for already existing ParmVarDecl (not already imported) and return if found.

Yes it would worth to try, indeed.

shafik marked 17 inline comments as done.Jan 23 2019, 10:21 PM

@martong I don't follow the discussion on VisitParmVarDecl an what seems like an alternate fix. Can you elaborate?

lib/AST/ASTImporter.cpp
3054

Just for clarification, from a C++ perspective, I would say "in-class declaration" rather than "in-class prototype" and therefore VTableBuilder assumes only one in class declaration and building a redecal chain would result in more than one in-class declaration being present.

Does that makes sense?

unittests/AST/ASTImporterTest.cpp
2343

I originally thought I did not need foo either but I could get it to import the definition of B::f() any other way.

I also saw similar behavior with clang-import-test where I need an expression that included something like foo() in order to obtain the out-of-line definition. It is not obvious to me why.

So in this case if I switch to using BFP instead of FooDef then this fails:

auto *ToBFOutOfClass = FirstDeclMatcher<CXXMethodDecl>().match(
  ToTU, cxxMethodDecl(hasName("f"), isDefinition()));
shafik updated this revision to Diff 183251.Jan 23 2019, 10:23 PM

Addressing comments on naming in the unit test and refactoring of duplicated code.

balazske added inline comments.Jan 24 2019, 12:04 AM
lib/AST/ASTImporter.cpp
2949

It would be better to return Error instead of Expected. The return value (Stmt*) is not used and it may be nullptr if the FromFD does not have a body.

2958

Code formatting is not OK in this function (and other places). (Indentation and spaces around "(" ")".)

unittests/AST/ASTImporterTest.cpp
2310

This line is too long. clang-format can be used to format the code automatically (or git-clang-format).

shafik updated this revision to Diff 183406.Jan 24 2019, 2:44 PM
shafik marked 3 inline comments as done.

Changes based on comments

  • Refactoring ImportFunctionDeclBody to return Error
  • Refactoring test/Analysis/ctu-main.cpp based on Gabor's patch
  • Removing merging of function body for later PR

@martong so I just tested your lit-tests.patch and it looks good!

lib/AST/ASTImporter.cpp
2958

Apologies, I forgot to run clang-format

martong accepted this revision.Jan 25 2019, 1:37 AM

Shafik, thanks for addressing the comments. This looks good to me now!

lib/AST/ASTImporter.cpp
3054

Yes, absolutely. Please change the comment as you suggest.

This revision is now accepted and ready to land.Jan 25 2019, 1:37 AM
balazske added inline comments.Jan 25 2019, 1:49 AM
unittests/AST/ASTImporterTest.cpp
2311

Looks mainly OK now. Only a small problem, I do not like names like BFPIsDefP here and later, in the previous test it is correctly named BFDefP (could be BFIsDefP too).

shafik updated this revision to Diff 183923.Jan 28 2019, 11:34 AM
shafik marked 2 inline comments as done.

Addressing comments on commenting and naming.

teemperor accepted this revision.Jan 28 2019, 11:52 AM

LGTM, thanks!

unittests/AST/ASTImporterTest.cpp
2270

I think this should be formatted in the same way as the snippet above.

2343

I'm actually wondering if we should do a full redecl chain check in the ASTImporter tests. E.g. iterating from the most recent decl and just make sure our redecl chain invariants are met (first one returns null and all getMostRecentDecl() calls returns the same decl). i assume usually this is implicitly done when we code gen or so, but this doesn't happen in the ASTImporter tests.

This revision was automatically updated to reflect the committed changes.