[Sema] Make sure the definition of a referenced virtual function is emitted when it is final

[ahatanak]

Summary

The test case I added used to fail because of a linker error.

Linkage failed because Sema::MarkDeclRefReferenced would prevent the virtual method definition from being emitted by setting OdrUse=false and then code-gen would devirtualized the virtual call because its class is marked final.

This patch fixes the bug.

rdar://problem/27455779

[vsk]

This comment has been deleted.

lib/Sema/SemaExpr.cpp

14680

Do you think it makes sense to eliminate all candidate virtual methods which can be devirtualized? If so, you could make "CanDevirtualizeMemberFunctionCall" a shared utility between Sema and CodeGen, and use it here. That function should give "the truth" about whether or not a call can be devirtualized.

14683

"MarkExprReferenced" has what looks like an incomplete version of "CanDevirtualizeMemberFunctionCall". Do you think there is an opportunity to share logic there as well?

test/CodeGenCXX/devirtualize-virtual-function-calls-final.cpp

271

Have you looked into why "ptr->operator()();" compiles? We are either missing a devirtualization opportunity, or we have inconsistent logic for setting MightBeOdrUse for member calls. Either way, I think this patch is the right vehicle to address the issue.

[ahatanak]

lib/Sema/SemaExpr.cpp

14680

I moved CanDevirtualizeMemberFunctionCall to DeclCXX.cpp and added overloaded functions of canDevirtualizeCall and used one of them here. This caused changes in the generated IR in two test cases (no-devirt.cpp and vtable-available-externally.cpp). Both changes look correct to me.

test/CodeGenCXX/devirtualize-virtual-function-calls-final.cpp

271

"ptr->operator()();" creates a MemberExpr, which MarkMemberReferenced handles. The difference between MarkMemberReferenced and MarkDeclRefReferenced is that the former sets OdrUse to false when the method is pure while the latter does so when the method is virtual. Prior to r174341, MarkDeclRefReferenced was setting OdrUse to false for pure methods too (see r174242).

[ahatanak]

Address review comments.

[vsk]

I suggest changing the API for 'canDevirtualizeCall' a bit but this is looking great overall. The code movement and test case changes look good.

include/clang/AST/DeclCXX.h

1902

Something like CXXMethodDecl *getDevirtualizedMethod(...) would fit in better with the current API. E.g, getCorrespondingMethodInClass is done this way (and returns nullptr when a method is not found).

lib/AST/DeclCXX.cpp

1693

Instead of templating canDevirtualizeCall, you could do something like const_cast<CXXMethodDecl *>(this)->getDevirtualizedMethod(...).

[ahatanak]

Define and use CXXMethodDecl::getDevirtualizedMethod, which returns the function that is called when a call is devirtualized.

[vsk]

Thanks, this looks great.

include/clang/AST/DeclCXX.h

bool isVirtual() const {
     // Member function is virtual if it is marked explicitly so, or if it is
     // declared in __interface -- then it is automatically pure virtual.
     if (CD->isVirtualAsWritten() || CD->isPure())
       return true;
 
     return (CD->begin_overridden_methods() != CD->end_overridden_methods());
   }
 
+  /// Checks whether a call to this method can be devirtualized.
+  ///
+  /// \param Base The object on which this virtual function is called.
+  /// \param IsAppleKext True if we are compiling for Apple kext.
+  bool canDevirtualizeCall(const Expr *Base, bool IsAppleKext) const;
+
+  /// Checks whether a call to this method can be devirtualized.
+  ///
+  /// \param Base The object on which this virtual function is called.
+  /// \param IsAppleKext True if we are compiling for Apple kext.
+  /// \param DM The called method if it's possible to devirtualize the call to
+  /// this method.
+  bool canDevirtualizeCall(const Expr *Base, bool IsAppleKext,
+                           CXXMethodDecl *&DM);
+
   /// \brief Determine whether this is a usual deallocation function
   /// (C++ [basic.stc.dynamic.deallocation]p2), which is an overloaded
   /// delete or delete[] operator with a particular signature.
   bool isUsualDeallocationFunction() const;
 
   /// \brief Determine whether this is a copy-assignment operator, regardless
   /// of whether it was declared implicitly or explicitly.
   bool isCopyAssignmentOperator() const;

include/clang/Sema/Sema.h

public:
   // should usually be true. This only needs to be set to false if the lack of
   // odr-use cannot be determined from the current context (for instance,
   // because the name denotes a virtual function and was written without an
   // explicit nested-name-specifier).
   void MarkAnyDeclReferenced(SourceLocation Loc, Decl *D, bool MightBeOdrUse);
   void MarkFunctionReferenced(SourceLocation Loc, FunctionDecl *Func,
                               bool MightBeOdrUse = true);
   void MarkVariableReferenced(SourceLocation Loc, VarDecl *Var);
-  void MarkDeclRefReferenced(DeclRefExpr *E);
+  void MarkDeclRefReferenced(DeclRefExpr *E, const Expr *Base = nullptr);
   void MarkMemberReferenced(MemberExpr *E);
 
   void UpdateMarkingForLValueToRValue(Expr *E);
   void CleanupVarDeclMarking();
 
   enum TryCaptureKind {
     TryCapture_Implicit, TryCapture_ExplicitByVal, TryCapture_ExplicitByRef
   };

lib/AST/DeclCXX.cpp

 }
 
 CXXMethodDecl *CXXMethodDecl::CreateDeserialized(ASTContext &C, unsigned ID) {
   return new (C, ID) CXXMethodDecl(CXXMethod, C, nullptr, SourceLocation(),
                                    DeclarationNameInfo(), QualType(), nullptr,
                                    SC_None, false, false, SourceLocation());
 }
 
+template<class T>
+static bool canDevirtualizeCall(const Expr *Base, T *MD, bool IsAppleKext,
+                                T *&DevirtualizedMethod) {
+  static_assert(
+      std::is_same<typename std::remove_const<T>::type, CXXMethodDecl>(),
+      "T must be CXXMethodDecl or const CXXMethodDecl");
+  assert(MD->isVirtual() && "this method is expected to be virtual");
+  DevirtualizedMethod = MD;
+
+  // When building with -fapple-kext, all calls must go through the vtable since
+  // the kernel linker can do runtime patching of vtables.
+  if (IsAppleKext)
+    return false;
+
+  // If the member function is marked 'final', we know that it can't be
+  // overridden and can therefore devirtualize it unless it's pure virtual.
+  if (MD->template hasAttr<FinalAttr>())
+    return !MD->isPure();
+
+  // If Base is unknown, we cannot devirtualize.
+  if (!Base)
+    return false;
+
+  // If the base expression (after skipping derived-to-base conversions) is a
+  // class prvalue, then we can devirtualize.
+  Base = Base->getBestDynamicClassTypeExpr();
+  if (Base->isRValue() && Base->getType()->isRecordType())
+    return true;
+
+  // If we don't even know what we would call, we can't devirtualize.
+  const CXXRecordDecl *BestDynamicDecl = Base->getBestDynamicClassType();
+  if (!BestDynamicDecl)
+    return false;
+
+  // There may be a method corresponding to MD in a derived class.
+  DevirtualizedMethod = MD->getCorrespondingMethodInClass(BestDynamicDecl);
+
+  // If that method is pure virtual, we can't devirtualize. If this code is
+  // reached, the result would be UB, not a direct call to the derived class
+  // function, and we can't assume the derived class function is defined.
+  if (DevirtualizedMethod->isPure())
+    return false;
+
+  // If that method is marked final, we can devirtualize it.
+  if (DevirtualizedMethod->template hasAttr<FinalAttr>())
+    return true;
+
+  // Similarly, if the class itself is marked 'final' it can't be overridden
+  // and we can therefore devirtualize the member function call.
+  if (BestDynamicDecl->hasAttr<FinalAttr>())
+    return true;
+
+  if (const DeclRefExpr *DRE = dyn_cast<DeclRefExpr>(Base)) {
+    if (const VarDecl *VD = dyn_cast<VarDecl>(DRE->getDecl())) {
+      // This is a record decl. We know the type and can devirtualize it.
+      return VD->getType()->isRecordType();
+    }
+
+    return false;
+  }
+
+  // We can devirtualize calls on an object accessed by a class member access
+  // expression, since by C++11 [basic.life]p6 we know that it can't refer to
+  // a derived class object constructed in the same location.
+  if (const MemberExpr *ME = dyn_cast<MemberExpr>(Base))
+    if (const ValueDecl *VD = dyn_cast<ValueDecl>(ME->getMemberDecl()))
+      return VD->getType()->isRecordType();
+
+  // Likewise for calls on an object accessed by a (non-reference) pointer to
+  // member access.
+  if (auto *BO = dyn_cast<BinaryOperator>(Base)) {
+    if (BO->isPtrMemOp()) {
+      auto *MPT = BO->getRHS()->getType()->castAs<MemberPointerType>();
+      if (MPT->getPointeeType()->isRecordType())
+        return true;
+    }
+  }
+
+  // We can't devirtualize the call.
+  return false;
+}
+
+bool CXXMethodDecl::canDevirtualizeCall(const Expr *Base,
+                                        bool IsAppleKext) const {
+  const CXXMethodDecl *DM;
+  return ::canDevirtualizeCall(Base, this, IsAppleKext, DM);
+}
+
+bool CXXMethodDecl::canDevirtualizeCall(const Expr *Base, bool IsAppleKext,
+                                        CXXMethodDecl *&DM) {
+  return ::canDevirtualizeCall(Base, this, IsAppleKext, DM);
+}
+
 bool CXXMethodDecl::isUsualDeallocationFunction() const {
   if (getOverloadedOperator() != OO_Delete &&
       getOverloadedOperator() != OO_Array_Delete)
     return false;
 
   // C++ [basic.stc.dynamic.deallocation]p2:
   //   A template instance is never a usual deallocation function,
   //   regardless of its signature.

lib/CodeGen/CGClass.cpp

llvm::Value *CodeGenFunction::EmitVTableTypeCheckedLoad(
   EmitCheck(std::make_pair(CheckResult, SanitizerKind::CFIVCall),
             SanitizerHandler::CFICheckFail, nullptr, nullptr);
 
   return Builder.CreateBitCast(
       Builder.CreateExtractValue(CheckedLoad, 0),
       cast<llvm::PointerType>(VTable->getType())->getElementType());
 }
 
-bool
-CodeGenFunction::CanDevirtualizeMemberFunctionCall(const Expr *Base,
-                                                   const CXXMethodDecl *MD) {
-  // When building with -fapple-kext, all calls must go through the vtable since
-  // the kernel linker can do runtime patching of vtables.
-  if (getLangOpts().AppleKext)
-    return false;
-
-  // If the member function is marked 'final', we know that it can't be
-  // overridden and can therefore devirtualize it unless it's pure virtual.
-  if (MD->hasAttr<FinalAttr>())
-    return !MD->isPure();
-
-  // If the base expression (after skipping derived-to-base conversions) is a
-  // class prvalue, then we can devirtualize.
-  Base = Base->getBestDynamicClassTypeExpr();
-  if (Base->isRValue() && Base->getType()->isRecordType())
-    return true;
-
-  // If we don't even know what we would call, we can't devirtualize.
-  const CXXRecordDecl *BestDynamicDecl = Base->getBestDynamicClassType();
-  if (!BestDynamicDecl)
-    return false;
-
-  // There may be a method corresponding to MD in a derived class.
-  const CXXMethodDecl *DevirtualizedMethod =
-      MD->getCorrespondingMethodInClass(BestDynamicDecl);
-
-  // If that method is pure virtual, we can't devirtualize. If this code is
-  // reached, the result would be UB, not a direct call to the derived class
-  // function, and we can't assume the derived class function is defined.
-  if (DevirtualizedMethod->isPure())
-    return false;
-
-  // If that method is marked final, we can devirtualize it.
-  if (DevirtualizedMethod->hasAttr<FinalAttr>())
-    return true;
-
-  // Similarly, if the class itself is marked 'final' it can't be overridden
-  // and we can therefore devirtualize the member function call.
-  if (BestDynamicDecl->hasAttr<FinalAttr>())
-    return true;
-
-  if (const DeclRefExpr *DRE = dyn_cast<DeclRefExpr>(Base)) {
-    if (const VarDecl *VD = dyn_cast<VarDecl>(DRE->getDecl())) {
-      // This is a record decl. We know the type and can devirtualize it.
-      return VD->getType()->isRecordType();
-    }
-
-    return false;
-  }
-
-  // We can devirtualize calls on an object accessed by a class member access
-  // expression, since by C++11 [basic.life]p6 we know that it can't refer to
-  // a derived class object constructed in the same location. However, we avoid
-  // devirtualizing a call to a template function that we could instantiate
-  // implicitly, but have not decided to do so. This is needed because if this
-  // function does not get instantiated, the devirtualization will create a
-  // direct call to a function whose body may not exist. In contrast, calls to
-  // template functions that are not defined in this TU are allowed to be 
-  // devirtualized under assumption that it is user responsibility to
-  // instantiate them in some other TU.
-  if (const MemberExpr *ME = dyn_cast<MemberExpr>(Base))
-    if (const ValueDecl *VD = dyn_cast<ValueDecl>(ME->getMemberDecl()))
-      return VD->getType()->isRecordType() && 
-             (MD->instantiationIsPending() || MD->isDefined() ||
-               !MD->isImplicitlyInstantiable());
-
-  // Likewise for calls on an object accessed by a (non-reference) pointer to
-  // member access.
-  if (auto *BO = dyn_cast<BinaryOperator>(Base)) {
-    if (BO->isPtrMemOp()) {
-      auto *MPT = BO->getRHS()->getType()->castAs<MemberPointerType>();
-      if (MPT->getPointeeType()->isRecordType())
-        return true;
-    }
-  }
-
-  // We can't devirtualize the call.
-  return false;
-}
-
 void CodeGenFunction::EmitForwardingCallToLambda(
                                       const CXXMethodDecl *callOperator,
                                       CallArgList &callArgs) {
   // Get the address of the call operator.
   const CGFunctionInfo &calleeFnInfo =
     CGM.getTypes().arrangeCXXMethodDeclaration(callOperator);
   llvm::Constant *calleePtr =
     CGM.GetAddrOfFunction(GlobalDecl(callOperator),

lib/CodeGen/CGExprCXX.cpp

RValue CodeGenFunction::EmitCXXMemberOrOperatorMemberCallExpr(
     bool HasQualifier, NestedNameSpecifier *Qualifier, bool IsArrow,
     const Expr *Base) {
   assert(isa<CXXMemberCallExpr>(CE) || isa<CXXOperatorCallExpr>(CE));
 
   // Compute the object pointer.
   bool CanUseVirtualCall = MD->isVirtual() && !HasQualifier;
 
   const CXXMethodDecl *DevirtualizedMethod = nullptr;
-  if (CanUseVirtualCall && CanDevirtualizeMemberFunctionCall(Base, MD)) {
+  if (CanUseVirtualCall &&
+      MD->canDevirtualizeCall(Base, getLangOpts().AppleKext)) {
     const CXXRecordDecl *BestDynamicDecl = Base->getBestDynamicClassType();
     DevirtualizedMethod = MD->getCorrespondingMethodInClass(BestDynamicDecl);
     assert(DevirtualizedMethod);
     const CXXRecordDecl *DevirtualizedClass = DevirtualizedMethod->getParent();
     const Expr *Inner = Base->ignoreParenBaseCasts();
     if (DevirtualizedMethod->getReturnType().getCanonicalType() !=
         MD->getReturnType().getCanonicalType())
       // If the return types are not the same, this might be a case where more

lib/CodeGen/CodeGenFunction.h

public:
   /// virtual function for virtual calls to members of RD. This is generally
   /// true when both vcall CFI and whole-program-vtables are enabled.
   bool ShouldEmitVTableTypeCheckedLoad(const CXXRecordDecl *RD);
 
   /// Emit a type checked load from the given vtable.
   llvm::Value *EmitVTableTypeCheckedLoad(const CXXRecordDecl *RD, llvm::Value *VTable,
                                          uint64_t VTableByteOffset);
 
-  /// CanDevirtualizeMemberFunctionCalls - Checks whether virtual calls on given
-  /// expr can be devirtualized.
-  bool CanDevirtualizeMemberFunctionCall(const Expr *Base,
-                                         const CXXMethodDecl *MD);
-
   /// EnterDtorCleanups - Enter the cleanups necessary to complete the
   /// given phase of destruction for a destructor.  The end result
   /// should call destructors on members and base classes in reverse
   /// order of their construction.
   void EnterDtorCleanups(const CXXDestructorDecl *Dtor, CXXDtorType Type);
 
   /// ShouldInstrumentFunction - Return true if the current function should be
   /// instrumented with __cyg_profile_func_* calls

lib/Sema/SemaExpr.cpp

static void MarkExprReferenced(Sema &SemaRef, SourceLocation Loc,
   CXXMethodDecl *MD = dyn_cast<CXXMethodDecl>(ME->getMemberDecl());
   if (!MD)
     return;
   // Only attempt to devirtualize if this is truly a virtual call.
   bool IsVirtualCall = MD->isVirtual() &&
                           ME->performsVirtualDispatch(SemaRef.getLangOpts());
   if (!IsVirtualCall)
     return;
-  const Expr *Base = ME->getBase();
-  const CXXRecordDecl *MostDerivedClassDecl = Base->getBestDynamicClassType();
-  if (!MostDerivedClassDecl)
-    return;
-  CXXMethodDecl *DM = MD->getCorrespondingMethodInClass(MostDerivedClassDecl);
-  if (!DM || DM->isPure())
-    return;
+
+  // If it's possible to devirtualize the call, mark the called function
+  // referenced.
+  CXXMethodDecl *DM;
+  if (MD->canDevirtualizeCall(ME->getBase(), SemaRef.getLangOpts().AppleKext,
+                              DM) &&
+      DM != MD)
     SemaRef.MarkAnyDeclReferenced(Loc, DM, MightBeOdrUse);
 } 
 
 /// \brief Perform reference-marking and odr-use handling for a DeclRefExpr.
-void Sema::MarkDeclRefReferenced(DeclRefExpr *E) {
+void Sema::MarkDeclRefReferenced(DeclRefExpr *E, const Expr *Base) {
   // TODO: update this with DR# once a defect report is filed.
   // C++11 defect. The address of a pure member should not be an ODR use, even
   // if it's a qualified reference.
   bool OdrUse = true;
-  if (CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(E->getDecl()))
-    if (Method->isVirtual())
+  if (const CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(E->getDecl()))
+    if (Method->isVirtual() &&
+        !Method->canDevirtualizeCall(Base, getLangOpts().AppleKext))
       OdrUse = false;
   MarkExprReferenced(*this, E->getLocation(), E->getDecl(), E, OdrUse);
 }
 
 /// \brief Perform reference-marking and odr-use handling for a MemberExpr.
 void Sema::MarkMemberReferenced(MemberExpr *E) {
   // C++11 [basic.def.odr]p2:
   //   A non-overloaded function whose name appears as a potentially-evaluated
   //   expression or a member of a set of candidate functions, if selected by
   //   overload resolution when referred to from a potentially-evaluated

lib/Sema/SemaOverload.cpp

static bool functionHasPassObjectSizeParams(const FunctionDecl *FD) {
   return llvm::any_of(FD->parameters(), [](const ParmVarDecl *P) {
     return P->hasAttr<PassObjectSizeAttr>();
   });
 }
 
 /// A convenience routine for creating a decayed reference to a function.
 static ExprResult
 CreateFunctionRefExpr(Sema &S, FunctionDecl *Fn, NamedDecl *FoundDecl,
-                      bool HadMultipleCandidates,
+                      const Expr *Base, bool HadMultipleCandidates,
                       SourceLocation Loc = SourceLocation(),
                       const DeclarationNameLoc &LocInfo = DeclarationNameLoc()){
   if (S.DiagnoseUseOfDecl(FoundDecl, Loc))
     return ExprError();
   // If FoundDecl is different from Fn (such as if one is a template
   // and the other a specialization), make sure DiagnoseUseOfDecl is
   // called on both.
   // FIXME: This would be more comprehensively addressed by modifying
   // DiagnoseUseOfDecl to accept both the FoundDecl and the decl
   // being used.
   if (FoundDecl != Fn && S.DiagnoseUseOfDecl(Fn, Loc))
     return ExprError();
   if (auto *FPT = Fn->getType()->getAs<FunctionProtoType>())
     S.ResolveExceptionSpec(Loc, FPT);
   DeclRefExpr *DRE = new (S.Context) DeclRefExpr(Fn, false, Fn->getType(),
                                                  VK_LValue, Loc, LocInfo);
   if (HadMultipleCandidates)
     DRE->setHadMultipleCandidates(true);
 
-  S.MarkDeclRefReferenced(DRE);
+  S.MarkDeclRefReferenced(DRE, Base);
   return S.ImpCastExprToType(DRE, S.Context.getPointerType(DRE->getType()),
                              CK_FunctionToPointerDecay);
 }
 
 static bool IsStandardConversion(Sema &S, Expr* From, QualType ToType,
                                  bool InOverloadResolution,
                                  StandardConversionSequence &SCS,
                                  bool CStyle,
Sema::CreateOverloadedUnaryOp(SourceLocation OpLoc, UnaryOperatorKind Opc,
   // Perform overload resolution.
   OverloadCandidateSet::iterator Best;
   switch (CandidateSet.BestViableFunction(*this, OpLoc, Best)) {
   case OR_Success: {
     // We found a built-in operator or an overloaded operator.
     FunctionDecl *FnDecl = Best->Function;
 
     if (FnDecl) {
+      Expr *Base = nullptr;
       // We matched an overloaded operator. Build a call to that
       // operator.
 
       // Convert the arguments.
       if (CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(FnDecl)) {
         CheckMemberOperatorAccess(OpLoc, Args[0], nullptr, Best->FoundDecl);
 
         ExprResult InputRes =
           PerformObjectArgumentInitialization(Input, /*Qualifier=*/nullptr,
                                               Best->FoundDecl, Method);
         if (InputRes.isInvalid())
           return ExprError();
-        Input = InputRes.get();
+        Base = Input = InputRes.get();
       } else {
         // Convert the arguments.
         ExprResult InputInit
           = PerformCopyInitialization(InitializedEntity::InitializeParameter(
                                                       Context,
                                                       FnDecl->getParamDecl(0)),
                                       SourceLocation(),
                                       Input);
         if (InputInit.isInvalid())
           return ExprError();
         Input = InputInit.get();
       }
 
       // Build the actual expression node.
       ExprResult FnExpr = CreateFunctionRefExpr(*this, FnDecl, Best->FoundDecl,
-                                                HadMultipleCandidates, OpLoc);
+                                                Base, HadMultipleCandidates,
+                                                OpLoc);
       if (FnExpr.isInvalid())
         return ExprError();
 
       // Determine the result type.
       QualType ResultTy = FnDecl->getReturnType();
       ExprValueKind VK = Expr::getValueKindForType(ResultTy);
       ResultTy = ResultTy.getNonLValueExprType(Context);
 
Sema::CreateOverloadedBinOp(SourceLocation OpLoc,
   // Perform overload resolution.
   OverloadCandidateSet::iterator Best;
   switch (CandidateSet.BestViableFunction(*this, OpLoc, Best)) {
     case OR_Success: {
       // We found a built-in operator or an overloaded operator.
       FunctionDecl *FnDecl = Best->Function;
 
       if (FnDecl) {
+        Expr *Base = nullptr;
         // We matched an overloaded operator. Build a call to that
         // operator.
 
         // Convert the arguments.
         if (CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(FnDecl)) {
           // Best->Access is only meaningful for class members.
           CheckMemberOperatorAccess(OpLoc, Args[0], Args[1], Best->FoundDecl);
 
           ExprResult Arg1 =
             PerformCopyInitialization(
               InitializedEntity::InitializeParameter(Context,
                                                      FnDecl->getParamDecl(0)),
               SourceLocation(), Args[1]);
           if (Arg1.isInvalid())
             return ExprError();
 
           ExprResult Arg0 =
             PerformObjectArgumentInitialization(Args[0], /*Qualifier=*/nullptr,
                                                 Best->FoundDecl, Method);
           if (Arg0.isInvalid())
             return ExprError();
-          Args[0] = Arg0.getAs<Expr>();
+          Base = Args[0] = Arg0.getAs<Expr>();
           Args[1] = RHS = Arg1.getAs<Expr>();
         } else {
           // Convert the arguments.
           ExprResult Arg0 = PerformCopyInitialization(
             InitializedEntity::InitializeParameter(Context,
                                                    FnDecl->getParamDecl(0)),
             SourceLocation(), Args[0]);
           if (Arg0.isInvalid())
             return ExprError();
 
           ExprResult Arg1 =
             PerformCopyInitialization(
               InitializedEntity::InitializeParameter(Context,
                                                      FnDecl->getParamDecl(1)),
               SourceLocation(), Args[1]);
           if (Arg1.isInvalid())
             return ExprError();
           Args[0] = LHS = Arg0.getAs<Expr>();
           Args[1] = RHS = Arg1.getAs<Expr>();
         }
 
         // Build the actual expression node.
         ExprResult FnExpr = CreateFunctionRefExpr(*this, FnDecl,
-                                                  Best->FoundDecl,
+                                                  Best->FoundDecl, Base,
                                                   HadMultipleCandidates, OpLoc);
         if (FnExpr.isInvalid())
           return ExprError();
 
         // Determine the result type.
         QualType ResultTy = FnDecl->getReturnType();
         ExprValueKind VK = Expr::getValueKindForType(ResultTy);
         ResultTy = ResultTy.getNonLValueExprType(Context);
case OR_Success: {
 
         Args[1] = InputInit.getAs<Expr>();
 
         // Build the actual expression node.
         DeclarationNameInfo OpLocInfo(OpName, LLoc);
         OpLocInfo.setCXXOperatorNameRange(SourceRange(LLoc, RLoc));
         ExprResult FnExpr = CreateFunctionRefExpr(*this, FnDecl,
                                                   Best->FoundDecl,
+                                                  Base,
                                                   HadMultipleCandidates,
                                                   OpLocInfo.getLoc(),
                                                   OpLocInfo.getInfo());
         if (FnExpr.isInvalid())
           return ExprError();
 
         // Determine the result type
         QualType ResultTy = FnDecl->getReturnType();
const FunctionProtoType *Proto =
     Method->getType()->getAs<FunctionProtoType>();
 
   unsigned NumParams = Proto->getNumParams();
 
   DeclarationNameInfo OpLocInfo(
                Context.DeclarationNames.getCXXOperatorName(OO_Call), LParenLoc);
   OpLocInfo.setCXXOperatorNameRange(SourceRange(LParenLoc, RParenLoc));
   ExprResult NewFn = CreateFunctionRefExpr(*this, Method, Best->FoundDecl,
-                                           HadMultipleCandidates,
+                                           Obj, HadMultipleCandidates,
                                            OpLocInfo.getLoc(),
                                            OpLocInfo.getInfo());
   if (NewFn.isInvalid())
     return true;
 
   // Build the full argument list for the method call (the implicit object
   // parameter is placed at the beginning of the list).
   SmallVector<Expr *, 8> MethodArgs(Args.size() + 1);
ExprResult BaseResult =
     PerformObjectArgumentInitialization(Base, /*Qualifier=*/nullptr,
                                         Best->FoundDecl, Method);
   if (BaseResult.isInvalid())
     return ExprError();
   Base = BaseResult.get();
 
   // Build the operator call.
   ExprResult FnExpr = CreateFunctionRefExpr(*this, Method, Best->FoundDecl,
-                                            HadMultipleCandidates, OpLoc);
+                                            Base, HadMultipleCandidates, OpLoc);
   if (FnExpr.isInvalid())
     return ExprError();
 
   QualType ResultTy = Method->getReturnType();
   ExprValueKind VK = Expr::getValueKindForType(ResultTy);
   ResultTy = ResultTy.getNonLValueExprType(Context);
   CXXOperatorCallExpr *TheCall =
     new (Context) CXXOperatorCallExpr(Context, OO_Arrow, FnExpr.get(),
ExprResult Sema::BuildLiteralOperatorCall(LookupResult &R,
   case OR_Ambiguous:
     Diag(R.getNameLoc(), diag::err_ovl_ambiguous_call) << R.getLookupName();
     CandidateSet.NoteCandidates(*this, OCD_ViableCandidates, Args);
     return ExprError();
   }
 
   FunctionDecl *FD = Best->Function;
   ExprResult Fn = CreateFunctionRefExpr(*this, FD, Best->FoundDecl,
-                                        HadMultipleCandidates,
+                                        nullptr, HadMultipleCandidates,
                                         SuffixInfo.getLoc(),
                                         SuffixInfo.getInfo());
   if (Fn.isInvalid())
     return true;
 
   // Check the argument types. This should almost always be a no-op, except
   // that array-to-pointer decay is applied to string literals.
   Expr *ConvArgs[2];

test/CodeGen/no-devirt.cpp

public:
   virtual T& operator [] (int idx);
   virtual T& func1 (int idx);
   virtual T& func2 (int idx);
   T ar[N+1];
 };
 struct Wrapper {
   TmplWithArray<bool, 10> data;
   bool indexIt(int a) {
-    if (a > 6) return data[a] ;      // Should not devirtualize
+    if (a > 6) return data[a] ;      // Should devirtualize
     if (a > 4) return data.func1(a); // Should devirtualize
     return data.func2(a);            // Should devirtualize
   }
 };
 
 #ifdef TMPL_DEF_IN_HEADER
 template <typename T, int N> T& TmplWithArray<T, N >::operator[](int idx) {
   return ar[idx];

 #endif // !TMPL_DEF_IN_HEADER
 extern Wrapper ew;
 bool stuff(int p)
 {
   return ew.indexIt(p);
 }
 #endif
 
-// CHECK-NOT: call {{.*}} @_ZN13TmplWithArrayIbLi10EEixEi
+// CHECK-DAG: call {{.*}} @_ZN13TmplWithArrayIbLi10EEixEi
 // CHECK-DAG: call {{.*}} @_ZN13TmplWithArrayIbLi10EE5func1Ei
 // CHECK-DAG: call {{.*}} @_ZN13TmplWithArrayIbLi10EE5func2Ei
 

test/CodeGenCXX/devirtualize-virtual-function-calls-final.cpp

namespace Test10 {
   };
 
   // CHECK-LABEL: define i32 @_ZN6Test101fEPNS_1BE
   int f(B *b) {
     // CHECK: call i32 @_ZN6Test101B1fEv
     return static_cast<A *>(b)->f();
   }
 }
+
+namespace Test11 {
+  // Check that the definition of Derived::operator() is emitted.
+
+  // CHECK-LABEL: define linkonce_odr void @_ZN6Test111SIiE4foo1Ev(
+  // CHECK: call void @_ZN6Test111SIiE7DerivedclEv(
+  // CHECK: call zeroext i1 @_ZN6Test111SIiE7DerivedeqERKNS_4BaseE(
+  // CHECK: call zeroext i1 @_ZN6Test111SIiE7DerivedntEv(
+  // CHECK: call dereferenceable(4) %"class.Test11::Base"* @_ZN6Test111SIiE7DerivedixEi(
+  // CHECK: define linkonce_odr void @_ZN6Test111SIiE7DerivedclEv(
+  // CHECK: define linkonce_odr zeroext i1 @_ZN6Test111SIiE7DerivedeqERKNS_4BaseE(
+  // CHECK: define linkonce_odr zeroext i1 @_ZN6Test111SIiE7DerivedntEv(
+  // CHECK: define linkonce_odr dereferenceable(4) %"class.Test11::Base"* @_ZN6Test111SIiE7DerivedixEi(
+  class Base {
+  public:
+    virtual void operator()() {}
+    virtual bool operator==(const Base &other) { return false; }
+    virtual bool operator!() { return false; }
+    virtual Base &operator[](int i) { return *this; }
+  };
+
+  template<class T>
+  struct S {
+    class Derived final : public Base {
+    public:
+      void operator()() override {}
+      bool operator==(const Base &other) override { return true; }
+      bool operator!() override { return true; }
+      Base &operator[](int i) override { return *this; }
+    };
+
+    Derived *ptr = nullptr, *ptr2 = nullptr;
+
+    void foo1() {
+      if (ptr && ptr2) {
+        // These calls get devirtualized. Linkage fails if the definitions of
+        // the called functions are not emitted.
+        (*ptr)();
+        (void)(*ptr == *ptr2);
+        (void)(!(*ptr));
+        (void)((*ptr)[1]);
+      }
+    }
+  };
+
+  void foo2() {
+    S<int> *s = new S<int>;
+    s->foo1();
+  }
+}

test/CodeGenCXX/vtable-available-externally.cpp

 namespace Test11 {
 struct D;
 // Can emit C's vtable available_externally.
 // CHECK-TEST11: @_ZTVN6Test111CE = available_externally unnamed_addr constant
 struct C {
   virtual D& operator=(const D&);
 };
 
-// Cannot emit D's vtable available_externally, because we cannot create
-// a reference to the inline virtual D::operator= function.
-// CHECK-TEST11: @_ZTVN6Test111DE = external unnamed_addr constant
+// Can emit D's vtable available_externally.
+// CHECK-TEST11: @_ZTVN6Test111DE = available_externally unnamed_addr constant
 struct D : C {
   virtual void key();
 };
 D f();
 
 void g(D& a) {
   C c;
   c = a;