[WPD] Add statistics

[tejohnson]

Summary

Add statistics to count overall devirtualized targets as well as the
various types of devirtualizations applied at callsites.

[mingmingl]

LGTM.

Only one question regarding whether the counter is intended for DevirtModule or DevirtIndex (or both)

llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp

1155

If I'm reading correctly, NumSingleImpl is incremented in DevirtModule::applySingleImplDevirt, but not for DevirtIndex::trySingleImplDevirt.

I'm wondering if this is intended for the purpose of this cl. If yes, maybe describe something like

The counter is incremented in regular LTO (or ThinLTO, etc)

[tejohnson]

In D123152#3430851, @luna wrote:

LGTM.

Only one question regarding whether the counter is intended for DevirtModule or DevirtIndex (or both)

Both, see inline reply below.

llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp

1155

Note that applySingleImplDevirt is called both for regular and thin LTO - for ThinLTO it is called during the backends, from importResolution(). This counter is keeping track of the number of actual single impl devirtualizations (i.e. callsites where that optimization is applied).

During the whole program portion (both regular and thin LTO) we keep track of the number of target functions that had devirtualizations in the NumDevirtTargets statistic.

[mingmingl]

In D123152#3430973, @tejohnson wrote:

In D123152#3430851, @luna wrote:

LGTM.

Only one question regarding whether the counter is intended for DevirtModule or DevirtIndex (or both)

Both, see inline reply below.

In D123152#3430973, @tejohnson wrote:

In D123152#3430851, @luna wrote:

LGTM.

Only one question regarding whether the counter is intended for DevirtModule or DevirtIndex (or both)

Both, see inline reply below.

llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp

1155

Note that applySingleImplDevirt is called both for regular and thin LTO - for ThinLTO it is called during the backends, from importResolution(). This counter is keeping track of the number of actual single impl devirtualizations (i.e. callsites where that optimization is applied).

During the whole program portion (both regular and thin LTO) we keep track of the number of target functions that had devirtualizations in the NumDevirtTargets statistic.

Ah I see your point. Basically DevirtIndex::run is called within runWholeProgramDevirtOnIndex in the thin-link step, and DevirtModule::run is called for thinLTO backend compile (after thin-link step).

Thanks for pointing it out!

llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp

 
 #include "llvm/Transforms/IPO/WholeProgramDevirt.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/DenseMapInfo.h"
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/BasicAliasAnalysis.h"
 #include "llvm/Analysis/OptimizationRemarkEmitter.h"
 #include "llvm/Analysis/TypeMetadataUtils.h"
 #include "llvm/Bitcode/BitcodeReader.h"
 #include "llvm/Bitcode/BitcodeWriter.h"

 #include <set>
 #include <string>
 
 using namespace llvm;
 using namespace wholeprogramdevirt;
 
 #define DEBUG_TYPE "wholeprogramdevirt"
 
+STATISTIC(NumDevirtTargets, "Number of whole program devirtualization targets");
+STATISTIC(NumSingleImpl, "Number of single implementation devirtualizations");
+STATISTIC(NumBranchFunnel, "Number of branch funnels");
+STATISTIC(NumUniformRetVal, "Number of uniform return value optimizations");
+STATISTIC(NumUniqueRetVal, "Number of unique return value optimizations");
+STATISTIC(NumVirtConstProp1Bit,
+          "Number of 1 bit virtual constant propagations");
+STATISTIC(NumVirtConstProp, "Number of virtual constant propagations");
+
 static cl::opt<PassSummaryAction> ClSummaryAction(
     "wholeprogramdevirt-summary-action",
     cl::desc("What to do with the summary when running this pass"),
     cl::values(clEnumValN(PassSummaryAction::None, "none", "Do nothing"),
                clEnumValN(PassSummaryAction::Import, "import",
                           "Import typeid resolutions from summary and globals"),
                clEnumValN(PassSummaryAction::Export, "export",
                           "Export typeid resolutions to summary and globals")),
void DevirtModule::applySingleImplDevirt(VTableSlotInfo &SlotInfo,
   auto Apply = [&](CallSiteInfo &CSInfo) {
     for (auto &&VCallSite : CSInfo.CallSites) {
       if (!OptimizedCalls.insert(&VCallSite.CB).second)
         continue;
 
       if (RemarksEnabled)
         VCallSite.emitRemark("single-impl",
                              TheFn->stripPointerCasts()->getName(), OREGetter);
+      NumSingleImpl++;
       auto &CB = VCallSite.CB;
       assert(!CB.getCalledFunction() && "devirtualizing direct call?");
       IRBuilder<> Builder(&CB);
       Value *Callee =
           Builder.CreateBitCast(TheFn, CB.getCalledOperand()->getType());
 
       // If trap checking is enabled, add support to compare the virtual
       // function pointer to the devirtualized target. In case of a mismatch,
bool DevirtModule::trySingleImplDevirt(
   // See if the program contains a single implementation of this virtual
   // function.
   Function *TheFn = TargetsForSlot[0].Fn;
   for (auto &&Target : TargetsForSlot)
     if (TheFn != Target.Fn)
       return false;
 
   // If so, update each call site to call that implementation directly.
-  if (RemarksEnabled)
+  if (RemarksEnabled || AreStatisticsEnabled())
     TargetsForSlot[0].WasDevirt = true;
 
   bool IsExported = false;
   applySingleImplDevirt(SlotInfo, TheFn, IsExported);
   if (!IsExported)
     return false;
 
   // If the only implementation has local linkage, we must promote to external
bool DevirtIndex::trySingleImplDevirt(MutableArrayRef<ValueInfo> TargetsForSlot,
 
   // If the summary list contains multiple summaries where at least one is
   // a local, give up, as we won't know which (possibly promoted) name to use.
   for (auto &S : TheFn.getSummaryList())
     if (GlobalValue::isLocalLinkage(S->linkage()) && Size > 1)
       return false;
 
   // Collect functions devirtualized at least for one call site for stats.
-  if (PrintSummaryDevirt)
+  if (PrintSummaryDevirt || AreStatisticsEnabled())
     DevirtTargets.insert(TheFn);
 
   auto &S = TheFn.getSummaryList()[0];
   bool IsExported = AddCalls(SlotInfo, TheFn);
   if (IsExported)
     ExportedGUIDs.insert(TheFn.getGUID());
 
   // Record in summary for use in devirtualization during the ThinLTO import
for (auto &&VCallSite : CSInfo.CallSites) {
       CallBase &CB = VCallSite.CB;
 
       // Jump tables are only profitable if the retpoline mitigation is enabled.
       Attribute FSAttr = CB.getCaller()->getFnAttribute("target-features");
       if (!FSAttr.isValid() ||
           !FSAttr.getValueAsString().contains("+retpoline"))
         continue;
 
+      NumBranchFunnel++;
       if (RemarksEnabled)
         VCallSite.emitRemark("branch-funnel",
                              JT->stripPointerCasts()->getName(), OREGetter);
 
       // Pass the address of the vtable in the nest register, which is r10 on
       // x86_64.
       std::vector<Type *> NewArgs;
       NewArgs.push_back(Int8PtrTy);
bool DevirtModule::tryEvaluateFunctionsWithArgs(
   return true;
 }
 
 void DevirtModule::applyUniformRetValOpt(CallSiteInfo &CSInfo, StringRef FnName,
                                          uint64_t TheRetVal) {
   for (auto Call : CSInfo.CallSites) {
     if (!OptimizedCalls.insert(&Call.CB).second)
       continue;
+    NumUniformRetVal++;
     Call.replaceAndErase(
         "uniform-ret-val", FnName, RemarksEnabled, OREGetter,
         ConstantInt::get(cast<IntegerType>(Call.CB.getType()), TheRetVal));
   }
   CSInfo.markDevirt();
 }
 
 bool DevirtModule::tryUniformRetValOpt(
     MutableArrayRef<VirtualCallTarget> TargetsForSlot, CallSiteInfo &CSInfo,
     WholeProgramDevirtResolution::ByArg *Res) {
   // Uniform return value optimization. If all functions return the same
   // constant, replace all calls with that constant.
   uint64_t TheRetVal = TargetsForSlot[0].RetVal;
   for (const VirtualCallTarget &Target : TargetsForSlot)
     if (Target.RetVal != TheRetVal)
       return false;
 
   if (CSInfo.isExported()) {
     Res->TheKind = WholeProgramDevirtResolution::ByArg::UniformRetVal;
     Res->Info = TheRetVal;
   }
 
   applyUniformRetValOpt(CSInfo, TargetsForSlot[0].Fn->getName(), TheRetVal);
-  if (RemarksEnabled)
+  if (RemarksEnabled || AreStatisticsEnabled())
     for (auto &&Target : TargetsForSlot)
       Target.WasDevirt = true;
   return true;
 }
 
 std::string DevirtModule::getGlobalName(VTableSlot Slot,
                                         ArrayRef<uint64_t> Args,
                                         StringRef Name) {
void DevirtModule::applyUniqueRetValOpt(CallSiteInfo &CSInfo, StringRef FnName,
   for (auto &&Call : CSInfo.CallSites) {
     if (!OptimizedCalls.insert(&Call.CB).second)
       continue;
     IRBuilder<> B(&Call.CB);
     Value *Cmp =
         B.CreateICmp(IsOne ? ICmpInst::ICMP_EQ : ICmpInst::ICMP_NE, Call.VTable,
                      B.CreateBitCast(UniqueMemberAddr, Call.VTable->getType()));
     Cmp = B.CreateZExt(Cmp, Call.CB.getType());
+    NumUniqueRetVal++;
     Call.replaceAndErase("unique-ret-val", FnName, RemarksEnabled, OREGetter,
                          Cmp);
   }
   CSInfo.markDevirt();
 }
 
 Constant *DevirtModule::getMemberAddr(const TypeMemberInfo *M) {
   Constant *C = ConstantExpr::getBitCast(M->Bits->GV, Int8PtrTy);
if (CSInfo.isExported()) {
       exportGlobal(Slot, Args, "unique_member", UniqueMemberAddr);
     }
 
     // Replace each call with the comparison.
     applyUniqueRetValOpt(CSInfo, TargetsForSlot[0].Fn->getName(), IsOne,
                          UniqueMemberAddr);
 
     // Update devirtualization statistics for targets.
-    if (RemarksEnabled)
+    if (RemarksEnabled || AreStatisticsEnabled())
       for (auto &&Target : TargetsForSlot)
         Target.WasDevirt = true;
 
     return true;
   };
 
   if (BitWidth == 1) {
     if (tryUniqueRetValOptFor(true))
for (auto Call : CSInfo.CallSites) {
     auto *RetType = cast<IntegerType>(Call.CB.getType());
     IRBuilder<> B(&Call.CB);
     Value *Addr =
         B.CreateGEP(Int8Ty, B.CreateBitCast(Call.VTable, Int8PtrTy), Byte);
     if (RetType->getBitWidth() == 1) {
       Value *Bits = B.CreateLoad(Int8Ty, Addr);
       Value *BitsAndBit = B.CreateAnd(Bits, Bit);
       auto IsBitSet = B.CreateICmpNE(BitsAndBit, ConstantInt::get(Int8Ty, 0));
+      NumVirtConstProp1Bit++;
       Call.replaceAndErase("virtual-const-prop-1-bit", FnName, RemarksEnabled,
                            OREGetter, IsBitSet);
     } else {
       Value *ValAddr = B.CreateBitCast(Addr, RetType->getPointerTo());
       Value *Val = B.CreateLoad(RetType, ValAddr);
+      NumVirtConstProp++;
       Call.replaceAndErase("virtual-const-prop", FnName, RemarksEnabled,
                            OREGetter, Val);
     }
   }
   CSInfo.markDevirt();
 }
 
 bool DevirtModule::tryVirtualConstProp(
for (auto &&CSByConstantArg : SlotInfo.ConstCSInfo) {
     uint64_t OffsetBit;
     if (TotalPaddingBefore <= TotalPaddingAfter)
       setBeforeReturnValues(TargetsForSlot, AllocBefore, BitWidth, OffsetByte,
                             OffsetBit);
     else
       setAfterReturnValues(TargetsForSlot, AllocAfter, BitWidth, OffsetByte,
                            OffsetBit);
 
-    if (RemarksEnabled)
+    if (RemarksEnabled || AreStatisticsEnabled())
       for (auto &&Target : TargetsForSlot)
         Target.WasDevirt = true;
 
 
     if (CSByConstantArg.second.isExported()) {
       ResByArg->TheKind = WholeProgramDevirtResolution::ByArg::VirtualConstProp;
       exportConstant(Slot, CSByConstantArg.first, "byte", OffsetByte,
                      ResByArg->Byte);
if (tryFindVirtualCallTargets(TargetsForSlot, TypeMemberInfos,
       if (!trySingleImplDevirt(ExportSummary, TargetsForSlot, S.second, Res)) {
         DidVirtualConstProp |=
             tryVirtualConstProp(TargetsForSlot, S.second, Res, S.first);
 
         tryICallBranchFunnel(TargetsForSlot, S.second, Res, S.first);
       }
 
       // Collect functions devirtualized at least for one call site for stats.
-      if (RemarksEnabled)
+      if (RemarksEnabled || AreStatisticsEnabled())
         for (const auto &T : TargetsForSlot)
           if (T.WasDevirt)
             DevirtTargets[std::string(T.Fn->getName())] = T.Fn;
     }
 
     // CFI-specific: if we are exporting and any llvm.type.checked.load
     // intrinsics were *not* devirtualized, we need to add the resulting
     // llvm.type.test intrinsics to the function summaries so that the
for (const auto &DT : DevirtTargets) {
 
       using namespace ore;
       OREGetter(F).emit(OptimizationRemark(DEBUG_TYPE, "Devirtualized", F)
                         << "devirtualized "
                         << NV("FunctionName", DT.first));
     }
   }
 
+  NumDevirtTargets += DevirtTargets.size();
+
   removeRedundantTypeTests();
 
   // Rebuild each global we touched as part of virtual constant propagation to
   // include the before and after bytes.
   if (DidVirtualConstProp)
     for (VTableBits &B : Bits)
       rebuildGlobal(B);
 
for (auto &S : CallSlots) {
     }
   }
 
   // Optionally have the thin link print message for each devirtualized
   // function.
   if (PrintSummaryDevirt)
     for (const auto &DT : DevirtTargets)
       errs() << "Devirtualized call to " << DT << "\n";
+
+  NumDevirtTargets += DevirtTargets.size();
 }

llvm/test/Transforms/WholeProgramDevirt/devirt-single-impl.ll

-; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt %s 2>&1 | FileCheck %s
+; -stats requires asserts
+; REQUIRES: asserts
+
+; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -stats %s 2>&1 | FileCheck %s
 
 target datalayout = "e-p:64:64"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; CHECK: remark: devirt-single.cc:30:32: single-impl: devirtualized a call to vf
 ; CHECK: remark: devirt-single.cc:13:0: devirtualized vf
 ; CHECK-NOT: devirtualized
 

 !1 = !DIFile(filename: "devirt-single.cc", directory: ".")
 !2 = !{i32 2, !"Dwarf Version", i32 4}
 !3 = !{i32 2, !"Debug Info Version", i32 3}
 !4 = !{!"clang version 4.0.0 (trunk 278098)"}
 !5 = distinct !DISubprogram(name: "call", linkageName: "_Z4callPv", scope: !1, file: !1, line: 29, isLocal: false, isDefinition: true, scopeLine: 9, flags: DIFlagPrototyped, isOptimized: false, unit: !0)
 !6 = !DILocation(line: 30, column: 32, scope: !5)
 !7 = distinct !DISubprogram(name: "vf", linkageName: "_ZN3vt12vfEv", scope: !1, file: !1, line: 13, isLocal: false, isDefinition: true, scopeLine: 13, flags: DIFlagPrototyped, isOptimized: false, unit: !0)
 !8 = !{i32 0, !"typeid"}
+
+; CHECK: 1 wholeprogramdevirt - Number of whole program devirtualization targets
+; CHECK: 1 wholeprogramdevirt - Number of single implementation devirtualizations

llvm/test/Transforms/WholeProgramDevirt/uniform-retval.ll

-; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility %s | FileCheck %s
+; -stats requires asserts
+; REQUIRES: asserts
+
+; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -stats %s 2>&1 | FileCheck %s
 
 target datalayout = "e-p:64:64"
 target triple = "x86_64-unknown-linux-gnu"
 
+; CHECK: remark: {{.*}} uniform-ret-val: devirtualized a call to vf1
+; CHECK: remark: {{.*}} devirtualized vf1
+; CHECK: remark: {{.*}} devirtualized vf2
+
 @vt1 = constant [1 x i8*] [i8* bitcast (i32 (i8*)* @vf1 to i8*)], !type !0
 @vt2 = constant [1 x i8*] [i8* bitcast (i32 (i8*)* @vf2 to i8*)], !type !0
 
 define i32 @vf1(i8* %this) readnone {
   ret i32 123
 }
 
 define i32 @vf2(i8* %this) readnone {
define i32 @call(i8* %obj) {
   ; CHECK: ret i32 123
   ret i32 %result
 }
 
 declare i1 @llvm.type.test(i8*, metadata)
 declare void @llvm.assume(i1)
 
 !0 = !{i32 0, !"typeid"}
+
+; CHECK: 2 wholeprogramdevirt - Number of whole program devirtualization targets
+; CHECK: 1 wholeprogramdevirt - Number of uniform return value optimizations

llvm/test/Transforms/WholeProgramDevirt/unique-retval.ll

-; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility %s | FileCheck %s
+; -stats requires asserts
+; REQUIRES: asserts
+
+; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -stats %s 2>&1 | FileCheck %s
 
 target datalayout = "e-p:64:64"
 target triple = "x86_64-unknown-linux-gnu"
 
+; CHECK: remark: {{.*}} unique-ret-val: devirtualized a call to vf0
+; CHECK: remark: {{.*}} unique-ret-val: devirtualized a call to vf0
+; CHECK: remark: {{.*}} devirtualized vf0
+; CHECK: remark: {{.*}} devirtualized vf1
+
 @vt1 = constant [1 x i8*] [i8* bitcast (i1 (i8*)* @vf0 to i8*)], !type !0
 @vt2 = constant [1 x i8*] [i8* bitcast (i1 (i8*)* @vf0 to i8*)], !type !0, !type !1
 @vt3 = constant [1 x i8*] [i8* bitcast (i1 (i8*)* @vf1 to i8*)], !type !0, !type !1
 @vt4 = constant [1 x i8*] [i8* bitcast (i1 (i8*)* @vf1 to i8*)], !type !1
 
 define i1 @vf0(i8* %this) readnone {
   ret i1 0
 }
define i32 @call2(i8* %obj) {
   ret i32 %result
 }
 
 declare i1 @llvm.type.test(i8*, metadata)
 declare void @llvm.assume(i1)
 
 !0 = !{i32 0, !"typeid1"}
 !1 = !{i32 0, !"typeid2"}
+
+; CHECK: 2 wholeprogramdevirt - Number of whole program devirtualization targets
+; CHECK: 2 wholeprogramdevirt - Number of unique return value optimizations

llvm/test/Transforms/WholeProgramDevirt/virtual-const-prop-check.ll

-; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt %s 2>&1 | FileCheck %s
+; -stats requires asserts
+; REQUIRES: asserts
+
+; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -stats %s 2>&1 | FileCheck %s
 ; Skipping vf0i1 is identical to setting public LTO visibility. We don't devirtualize vf0i1 and all other
 ; virtual call targets.
 ; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -wholeprogramdevirt-skip=vf0i1 %s 2>&1 | FileCheck %s --check-prefix=SKIP
 ; We have two set of call targets {vf0i1, vf1i1} and {vf1i32, vf2i32, vf3i32, vf4i32}.
 ; The command below prevents both of them from devirtualization.
 ; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -wholeprogramdevirt-skip=vf0i1,vf1i32 %s 2>&1 | FileCheck %s --check-prefix=SKIP-ALL
 ; Check wildcard
 ; RUN: opt -S -passes=wholeprogramdevirt -whole-program-visibility -pass-remarks=wholeprogramdevirt -wholeprogramdevirt-skip=vf?i1 %s 2>&1 | FileCheck %s --check-prefix=SKIP

 declare {i8*, i1} @llvm.type.checked.load(i8*, i32, metadata)
 declare void @llvm.assume(i1)
 declare void @__cxa_pure_virtual()
 
 ; CHECK: [[T8]] = !{i32 8, !"typeid"}
 ; CHECK: [[T0]] = !{i32 0, !"typeid"}
 
 !0 = !{i32 0, !"typeid"}
+
+; CHECK: 6 wholeprogramdevirt - Number of whole program devirtualization targets
+; CHECK: 1 wholeprogramdevirt - Number of virtual constant propagations
+; CHECK: 2 wholeprogramdevirt - Number of 1 bit virtual constant propagations