Page MenuHomePhabricator

Prevent devirtualization of calls to un-instantiated functions.
ClosedPublic

Authored by Sunil_Srivastava on Jul 6 2016, 10:45 AM.

Details

Summary

This review is for a fix for PR 27895, but it requires some discussion. The bugzilla and the email exchange with Richard Smith in http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160530/161107.html contain more details.

The basic problem: When the front end sees a call to a virtual function of a template class, it treats that as an odr use, therefore requiring the instantiation; except, for some reason, when it is an operator and used with an operator syntax. In those cases, if the call gets devirtualized later, it will generate a direct call to a function that has not been instantiated. Most of the time it will still work because someone else will instantiate it for some other reason, but if not, then we have converted a potentially dead code to a link time error of a missing definition.

As discussed in the email exchange, there are two alternative ways to fix the problem. Either will be sufficient, but we want to do both.

The first fix will be in the front end to force the instantiation on virtual calls that are potentially devirtualizable. For reasons described below I plan on submitting this in a later checkin.

The second fix, the matter of this review, is to avoid devirtualization under certain conditions. The conditions that I am choosing is to prevent devirtualization if the function could be implicitly instantiated but front end chose to not mark it for instantiation. I welcome any suggestions about improvement to this condition.

The reason I am planning on submitting them in this order, is that, once the first fix is checked in, there will be no way, known to me, to write a test for the second, devirtualization fix.

Now some discussion about the way I am testing the avoid-devirtualization condition: I have added a bit to FunctionDecl that I am setting whenever a function is put on the PendingInstantiations list. Then, at the time of devirtualization, if the target function isImplictlyInstantiable and does not have that bit set, I avoid the devirtualization. This change adds one bit to FunctionDecl, but the test is quite fast. However, it duplicates the information between this bit and the PendingInstantiationsList.

We have considered an alternate implementation which replaces the call (in CGClass.cpp) to
isMarkedForPendingInstantiation with

  • explcitly-test-whether-it-is-on-the-PendingInstations-list ||
  • has-already-been-instantiated.

This alternate implementation does not add anything to FunctionDecl, avoids information duplication, but the test for its presence in the list will be expensive; it will require a search in the list. It will be done only for devirtualizable calls though, which is not that common.

A related point that I am not sure about is: Are there circumstances where a function on the PendingInstantiations list can fails to be instantiated for reasons other than errors? If so then this checkin is not perfect, though it is still better than nothing.

I am open to suggestion as to which of these implementations makes more sense.

Once this change has been approved and submitted, I will open another review for the first fix, which will force the instantiation of functions that are target of potentially devirtualizable calls.

Diff Detail

Repository
rL LLVM

Event Timeline

Sunil_Srivastava retitled this revision from to Prevent devirtualization of calls to un-instantiated functions..
Sunil_Srivastava updated this object.
Sunil_Srivastava added a reviewer: rsmith.
Sunil_Srivastava added a subscriber: cfe-commits.
rsmith added inline comments.Jul 18 2016, 5:53 PM
include/clang/AST/Decl.h
1602 ↗(On Diff #62907)

Drop the MarkedFor here. This flag *is* the mark from the point of view of a user of the AST.

This flag also seems very similar to the Used flag on the Decl base class, so a documentation comment might help. Something like "Indicates that an instantiation of this function has been required but may not yet have been performed. If the function has already been instantiated, the value of this flag is unspecified."

... though it would be nicer if the value of the flag were always correct, even in the case where the function has already been instantiated.

1877–1878 ↗(On Diff #62907)

The fact that we have a separate list is an implementation detail of Sema; drop that from this comment.

lib/CodeGen/CGClass.cpp
2884–2886 ↗(On Diff #62907)

If the function is already defined (if we instantiated it already for some reason), we should be able to devirtualize calls to it.

lib/Sema/Sema.cpp
683–685 ↗(On Diff #62907)

Please add a test that we handle this properly if instantiation is triggered in a PCH.

Quuxplusone added inline comments.
test/CodeGen/no-devirt.cpp
16 ↗(On Diff #62907)

This is a really dumb question from the peanut gallery, but, could you explain why these two cases (lines 15 and 16) should differ? It really seems like both calls should be able to be devirtualized, because the compiler statically knows what they should call.

I think you mention the difference between lines 15 and 16 here:

except, for some reason, when it is an operator and used with an operator syntax

but you don't explain *why* the difference is desirable. Shouldn't we just fix that difference, then?

Is your first fix (

The first fix will be in the front end to force the instantiation on virtual calls that are potentially devirtualizable.

) basically "fix the difference between lines 15 and 16", or is it talking about something else entirely?

test/CodeGen/no-devirt.cpp
16 ↗(On Diff #62907)

AFAICS, The two cases (line 15 and 16) should not differ.

The first fix will "fix the difference between line 15 and 16". I have the change for that ready, but once we do that, there will be no way (known to me) of testing the second "fix". Hence the second "fix" is being proposed for commit before the first.

Quuxplusone added inline comments.Sep 8 2016, 11:36 AM
test/CodeGen/no-devirt.cpp
16 ↗(On Diff #62907)

Okay, makes sense to me now. (As long as the comment on line 15 gets updated in the "first fix". I believe you'll be touching this test case again anyway in the "first fix", because the expected output will change.)
Question answered. :)

Prazek added a subscriber: Prazek.Sep 10 2016, 11:16 PM
Prazek added inline comments.
lib/Sema/Sema.cpp
684 ↗(On Diff #62907)

Dry. Use auto

This is an update to address points raised by Richard Smith:

  1. The bit and the access functions have been renamed from MarkedForPendingInstantiation to InstantiationIsPending.
  2. It closely, though not entirely, tracks whether the function is on the PendingInstantiations list. More on this point below.
  3. The test explicitly allows devirtualization if Function->isDefined(). This is also needed by the change in point 2 above.
  4. The test has been enhanced to have PCH tests.

Now: Why the InstantiationIsPending bit is not precisely tracking the presence in the PendingInstantiations list?

Basically this is because I think that the call to Func2 in the test SHOULD get devirtualized. Func2 is not defined in this TU, an uncommon but possible situation. Given that, the compiler had no way to instantiate it, and it is a user error if Func2 does not get instantiated somewhere else. If Func2 does get instantiated somewhere else, then it is safe to devirtualize calls to it.

In contrast, operator[] is defined by the user, but for some reason (which will be removed by my next checkin, in situations we know of) the compiler has decided to not instantiate it (or rather, not decided to instantiate it, to be precise). In this case we do not want to devirtualize call to it, because the definition is not required to exist somewhere else. The whole motivation of this commit is to prevent such calls from devirtualization.

The instantiation loop removes items from the PendingInstantiations list and instantiates them, if the body is present. In the case of Func2, the body is not present, the function has already been removed from the list, yet it is not isDefined(). We need some way to distinguish this from the contrasting case of operator[], where the function was never put on the PendingInstantiations list. Hence in cases like Func2, I am not unsetting the InstantiationIsPending bit at the time of its removal from the list. Loosely speaking, we can say that the instantiation is indeed pending, though in some other TU.

Comments in Decls.h explain this behavior, though not in such details.

The expected behavior of the test will change by my next “first fix” commit, of course; in that the operator[] will get instantiated, and the call will be devirtualized.

I have read the patch, but I don't have enough knowledge about C++ rules and fronted to accept it. Richard?

lib/Sema/Sema.cpp
684 ↗(On Diff #62907)

auto *

rsmith accepted this revision.Feb 9 2017, 5:42 PM

Now: Why the InstantiationIsPending bit is not precisely tracking the presence in the PendingInstantiations list?

[...]

The instantiation loop removes items from the PendingInstantiations list and instantiates them, if the body is present. In the case of Func2, the body is not present, the function has already been removed from the list, yet it is not isDefined().

I see, so the only case in which the list and the flag differs is when we're doing the final instantiation step at the end of the TU? (In all other cases, we'd put the function back on the list for later instantiations to retry.) That seems fine.

This revision is now accepted and ready to land.Feb 9 2017, 5:42 PM
This revision was automatically updated to reflect the committed changes.