-
Notifications
You must be signed in to change notification settings - Fork 12.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ThinLTO] Move summary computation from BitcodeWriter to new pass
Summary: This is the first step in also serializing the index out to LLVM assembly. The per-module summary written to bitcode is moved out of the bitcode writer and to a new analysis pass (ModuleSummaryIndexWrapperPass). The pass itself uses a new builder class to compute index, and the builder class is used directly in places where we don't have a pass manager (e.g. llvm-as). Because we are computing summaries outside of the bitcode writer, we no longer can use value ids created by the bitcode writer's ValueEnumerator. This required changing the reference graph edge type to use a new ValueInfo class holding a union between a GUID (combined index) and Value* (permodule index). The Value* are converted to the appropriate value ID during bitcode writing. Also, this enables removal of the BitWriter library's dependence on the Analysis library that was previously required for the summary computation. Reviewers: joker.eph Subscribers: joker.eph, llvm-commits Differential Revision: http://reviews.llvm.org/D18763 llvm-svn: 265941
- llvmorg-20-init
- llvmorg-19.1.6
- llvmorg-19.1.5
- llvmorg-19.1.4
- llvmorg-19.1.3
- llvmorg-19.1.2
- llvmorg-19.1.1
- llvmorg-19.1.0
- llvmorg-19.1.0-rc4
- llvmorg-19.1.0-rc3
- llvmorg-19.1.0-rc2
- llvmorg-19.1.0-rc1
- llvmorg-19-init
- llvmorg-18.1.8
- llvmorg-18.1.7
- llvmorg-18.1.6
- llvmorg-18.1.5
- llvmorg-18.1.4
- llvmorg-18.1.3
- llvmorg-18.1.2
- llvmorg-18.1.1
- llvmorg-18.1.0
- llvmorg-18.1.0-rc4
- llvmorg-18.1.0-rc3
- llvmorg-18.1.0-rc2
- llvmorg-18.1.0-rc1
- llvmorg-18-init
- llvmorg-17.0.6
- llvmorg-17.0.5
- llvmorg-17.0.4
- llvmorg-17.0.3
- llvmorg-17.0.2
- llvmorg-17.0.1
- llvmorg-17.0.0
- llvmorg-17.0.0-rc4
- llvmorg-17.0.0-rc3
- llvmorg-17.0.0-rc2
- llvmorg-17.0.0-rc1
- llvmorg-17-init
- llvmorg-16.0.6
- llvmorg-16.0.5
- llvmorg-16.0.4
- llvmorg-16.0.3
- llvmorg-16.0.2
- llvmorg-16.0.1
- llvmorg-16.0.0
- llvmorg-16.0.0-rc4
- llvmorg-16.0.0-rc3
- llvmorg-16.0.0-rc2
- llvmorg-16.0.0-rc1
- llvmorg-16-init
- llvmorg-15.0.7
- llvmorg-15.0.6
- llvmorg-15.0.5
- llvmorg-15.0.4
- llvmorg-15.0.3
- llvmorg-15.0.2
- llvmorg-15.0.1
- llvmorg-15.0.0
- llvmorg-15.0.0-rc3
- llvmorg-15.0.0-rc2
- llvmorg-15.0.0-rc1
- llvmorg-15-init
- llvmorg-14.0.6
- llvmorg-14.0.5
- llvmorg-14.0.4
- llvmorg-14.0.3
- llvmorg-14.0.2
- llvmorg-14.0.1
- llvmorg-14.0.0
- llvmorg-14.0.0-rc4
- llvmorg-14.0.0-rc3
- llvmorg-14.0.0-rc2
- llvmorg-14.0.0-rc1
- llvmorg-14-init
- llvmorg-13.0.1
- llvmorg-13.0.1-rc3
- llvmorg-13.0.1-rc2
- llvmorg-13.0.1-rc1
- llvmorg-13.0.0
- llvmorg-13.0.0-rc4
- llvmorg-13.0.0-rc3
- llvmorg-13.0.0-rc2
- llvmorg-13.0.0-rc1
- llvmorg-13-init
- llvmorg-12.0.1
- llvmorg-12.0.1-rc4
- llvmorg-12.0.1-rc3
- llvmorg-12.0.1-rc2
- llvmorg-12.0.1-rc1
- llvmorg-12.0.0
- llvmorg-12.0.0-rc5
- llvmorg-12.0.0-rc4
- llvmorg-12.0.0-rc3
- llvmorg-12.0.0-rc2
- llvmorg-12.0.0-rc1
- llvmorg-12-init
- llvmorg-11.1.0
- llvmorg-11.1.0-rc3
- llvmorg-11.1.0-rc2
- llvmorg-11.1.0-rc1
- llvmorg-11.0.1
- llvmorg-11.0.1-rc2
- llvmorg-11.0.1-rc1
- llvmorg-11.0.0
- llvmorg-11.0.0-rc6
- llvmorg-11.0.0-rc5
- llvmorg-11.0.0-rc4
- llvmorg-11.0.0-rc3
- llvmorg-11.0.0-rc2
- llvmorg-11.0.0-rc1
- llvmorg-11-init
- llvmorg-10.0.1
- llvmorg-10.0.1-rc4
- llvmorg-10.0.1-rc3
- llvmorg-10.0.1-rc2
- llvmorg-10.0.1-rc1
- llvmorg-10.0.0
- llvmorg-10.0.0-rc6
- llvmorg-10.0.0-rc5
- llvmorg-10.0.0-rc4
- llvmorg-10.0.0-rc3
- llvmorg-10.0.0-rc2
- llvmorg-10.0.0-rc1
- llvmorg-10-init
- llvmorg-9.0.1
- llvmorg-9.0.1-rc3
- llvmorg-9.0.1-rc2
- llvmorg-9.0.1-rc1
- llvmorg-9.0.0
- llvmorg-9.0.0-rc6
- llvmorg-9.0.0-rc5
- llvmorg-9.0.0-rc4
- llvmorg-9.0.0-rc3
- llvmorg-9.0.0-rc2
- llvmorg-9.0.0-rc1
- llvmorg-8.0.1
- llvmorg-8.0.1-rc4
- llvmorg-8.0.1-rc3
- llvmorg-8.0.1-rc2
- llvmorg-8.0.1-rc1
- llvmorg-8.0.0
- llvmorg-8.0.0-rc5
- llvmorg-8.0.0-rc4
- llvmorg-8.0.0-rc3
- llvmorg-8.0.0-rc2
- llvmorg-8.0.0-rc1
- llvmorg-7.1.0
- llvmorg-7.1.0-rc1
- llvmorg-7.0.1
- llvmorg-7.0.1-rc3
- llvmorg-7.0.1-rc2
- llvmorg-7.0.1-rc1
- llvmorg-7.0.0
- llvmorg-7.0.0-rc3
- llvmorg-7.0.0-rc2
- llvmorg-7.0.0-rc1
- llvmorg-6.0.1
- llvmorg-6.0.1-rc3
- llvmorg-6.0.1-rc2
- llvmorg-6.0.1-rc1
- llvmorg-6.0.0
- llvmorg-6.0.0-rc3
- llvmorg-6.0.0-rc2
- llvmorg-6.0.0-rc1
- llvmorg-5.0.2
- llvmorg-5.0.2-rc2
- llvmorg-5.0.2-rc1
- llvmorg-5.0.1
- llvmorg-5.0.1-rc3
- llvmorg-5.0.1-rc2
- llvmorg-5.0.1-rc1
- llvmorg-5.0.0
- llvmorg-5.0.0-rc5
- llvmorg-5.0.0-rc4
- llvmorg-5.0.0-rc3
- llvmorg-5.0.0-rc2
- llvmorg-5.0.0-rc1
- llvmorg-4.0.1
- llvmorg-4.0.1-rc3
- llvmorg-4.0.1-rc2
- llvmorg-4.0.1-rc1
- llvmorg-4.0.0
- llvmorg-4.0.0-rc4
- llvmorg-4.0.0-rc3
- llvmorg-4.0.0-rc2
- llvmorg-4.0.0-rc1
- llvmorg-3.9.1
- llvmorg-3.9.1-rc3
- llvmorg-3.9.1-rc2
- llvmorg-3.9.1-rc1
- llvmorg-3.9.0
- llvmorg-3.9.0-rc3
- llvmorg-3.9.0-rc2
- llvmorg-3.9.0-rc1
1 parent
34ffcbb
commit 2d5487c
Showing
17 changed files
with
434 additions
and
183 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
//===- ModuleSummaryAnalysis.h - Module summary index builder ---*- C++ -*-===// | ||
// | ||
// The LLVM Compiler Infrastructure | ||
// | ||
// This file is distributed under the University of Illinois Open Source | ||
// License. See LICENSE.TXT for details. | ||
// | ||
//===----------------------------------------------------------------------===// | ||
/// \file | ||
/// This is the interface to build a ModuleSummaryIndex for a module. | ||
/// | ||
//===----------------------------------------------------------------------===// | ||
|
||
#ifndef LLVM_ANALYSIS_MODULESUMMARYANALYSIS_H | ||
#define LLVM_ANALYSIS_MODULESUMMARYANALYSIS_H | ||
|
||
#include "llvm/ADT/STLExtras.h" | ||
#include "llvm/IR/ModuleSummaryIndex.h" | ||
#include "llvm/Pass.h" | ||
|
||
namespace llvm { | ||
|
||
class BlockFrequencyInfo; | ||
|
||
/// Class to build a module summary index for the given Module, possibly from | ||
/// a Pass. | ||
class ModuleSummaryIndexBuilder { | ||
/// The index being built | ||
std::unique_ptr<ModuleSummaryIndex> Index; | ||
/// The module for which we are building an index | ||
const Module *M; | ||
|
||
public: | ||
/// Default constructor | ||
ModuleSummaryIndexBuilder() = default; | ||
|
||
/// Constructor that builds an index for the given Module. An optional | ||
/// callback can be supplied to obtain the frequency info for a function. | ||
ModuleSummaryIndexBuilder( | ||
const Module *M, | ||
std::function<BlockFrequencyInfo *(const Function &F)> Ftor = nullptr); | ||
|
||
/// Get a reference to the index owned by builder | ||
ModuleSummaryIndex &getIndex() const { return *Index; } | ||
|
||
/// Take ownership of the built index | ||
std::unique_ptr<ModuleSummaryIndex> takeIndex() { return std::move(Index); } | ||
|
||
private: | ||
/// Compute info for given function with optional frequency information | ||
void computeFunctionInfo(const Function &F, | ||
BlockFrequencyInfo *BFI = nullptr); | ||
|
||
/// Compute info for given variable with optional frequency information | ||
void computeVariableInfo(const GlobalVariable &V); | ||
}; | ||
|
||
/// Legacy wrapper pass to provide the ModuleSummaryIndex object. | ||
class ModuleSummaryIndexWrapperPass : public ModulePass { | ||
std::unique_ptr<ModuleSummaryIndexBuilder> IndexBuilder; | ||
|
||
public: | ||
static char ID; | ||
|
||
ModuleSummaryIndexWrapperPass(); | ||
|
||
/// Get the index built by pass | ||
ModuleSummaryIndex &getIndex() { return IndexBuilder->getIndex(); } | ||
const ModuleSummaryIndex &getIndex() const { | ||
return IndexBuilder->getIndex(); | ||
} | ||
|
||
bool runOnModule(Module &M) override; | ||
bool doFinalization(Module &M) override; | ||
void getAnalysisUsage(AnalysisUsage &AU) const override; | ||
}; | ||
|
||
//===--------------------------------------------------------------------===// | ||
// | ||
// createModuleSummaryIndexWrapperPass - This pass builds a ModuleSummaryIndex | ||
// object for the module, to be written to bitcode or LLVM assembly. | ||
// | ||
ModulePass *createModuleSummaryIndexWrapperPass(); | ||
} | ||
|
||
#endif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,186 @@ | ||
//===- ModuleSummaryAnalysis.cpp - Module summary index builder -----------===// | ||
// | ||
// The LLVM Compiler Infrastructure | ||
// | ||
// This file is distributed under the University of Illinois Open Source | ||
// License. See LICENSE.TXT for details. | ||
// | ||
//===----------------------------------------------------------------------===// | ||
// | ||
// This pass builds a ModuleSummaryIndex object for the module, to be written | ||
// to bitcode or LLVM assembly. | ||
// | ||
//===----------------------------------------------------------------------===// | ||
|
||
#include "llvm/Analysis/ModuleSummaryAnalysis.h" | ||
#include "llvm/Analysis/BlockFrequencyInfo.h" | ||
#include "llvm/Analysis/BlockFrequencyInfoImpl.h" | ||
#include "llvm/Analysis/BranchProbabilityInfo.h" | ||
#include "llvm/Analysis/LoopInfo.h" | ||
#include "llvm/IR/CallSite.h" | ||
#include "llvm/IR/Dominators.h" | ||
#include "llvm/IR/IntrinsicInst.h" | ||
#include "llvm/IR/ValueSymbolTable.h" | ||
#include "llvm/Pass.h" | ||
using namespace llvm; | ||
|
||
#define DEBUG_TYPE "module-summary-analysis" | ||
|
||
// Walk through the operands of a given User via worklist iteration and populate | ||
// the set of GlobalValue references encountered. Invoked either on an | ||
// Instruction or a GlobalVariable (which walks its initializer). | ||
static void findRefEdges(const User *CurUser, DenseSet<const Value *> &RefEdges, | ||
SmallPtrSet<const User *, 8> &Visited) { | ||
SmallVector<const User *, 32> Worklist; | ||
Worklist.push_back(CurUser); | ||
|
||
while (!Worklist.empty()) { | ||
const User *U = Worklist.pop_back_val(); | ||
|
||
if (!Visited.insert(U).second) | ||
continue; | ||
|
||
ImmutableCallSite CS(U); | ||
|
||
for (const auto &OI : U->operands()) { | ||
const User *Operand = dyn_cast<User>(OI); | ||
if (!Operand) | ||
continue; | ||
if (isa<BlockAddress>(Operand)) | ||
continue; | ||
if (isa<GlobalValue>(Operand)) { | ||
// We have a reference to a global value. This should be added to | ||
// the reference set unless it is a callee. Callees are handled | ||
// specially by WriteFunction and are added to a separate list. | ||
if (!(CS && CS.isCallee(&OI))) | ||
RefEdges.insert(Operand); | ||
continue; | ||
} | ||
Worklist.push_back(Operand); | ||
} | ||
} | ||
} | ||
|
||
void ModuleSummaryIndexBuilder::computeFunctionInfo(const Function &F, | ||
BlockFrequencyInfo *BFI) { | ||
// Summary not currently supported for anonymous functions, they must | ||
// be renamed. | ||
if (!F.hasName()) | ||
return; | ||
|
||
unsigned NumInsts = 0; | ||
// Map from callee ValueId to profile count. Used to accumulate profile | ||
// counts for all static calls to a given callee. | ||
DenseMap<const Value *, CalleeInfo> CallGraphEdges; | ||
DenseSet<const Value *> RefEdges; | ||
|
||
SmallPtrSet<const User *, 8> Visited; | ||
for (Function::const_iterator BB = F.begin(), E = F.end(); BB != E; ++BB) | ||
for (BasicBlock::const_iterator I = BB->begin(), E = BB->end(); I != E; | ||
++I) { | ||
if (!isa<DbgInfoIntrinsic>(I)) | ||
++NumInsts; | ||
|
||
if (auto CS = ImmutableCallSite(&*I)) { | ||
auto *CalledFunction = CS.getCalledFunction(); | ||
if (CalledFunction && CalledFunction->hasName() && | ||
!CalledFunction->isIntrinsic()) { | ||
auto ScaledCount = BFI ? BFI->getBlockProfileCount(&*BB) : None; | ||
auto *CalleeId = | ||
M->getValueSymbolTable().lookup(CalledFunction->getName()); | ||
CallGraphEdges[CalleeId] += | ||
(ScaledCount ? ScaledCount.getValue() : 0); | ||
} | ||
} | ||
findRefEdges(&*I, RefEdges, Visited); | ||
} | ||
|
||
std::unique_ptr<FunctionSummary> FuncSummary = | ||
llvm::make_unique<FunctionSummary>(F.getLinkage(), NumInsts); | ||
FuncSummary->addCallGraphEdges(CallGraphEdges); | ||
FuncSummary->addRefEdges(RefEdges); | ||
std::unique_ptr<GlobalValueInfo> GVInfo = | ||
llvm::make_unique<GlobalValueInfo>(0, std::move(FuncSummary)); | ||
Index->addGlobalValueInfo(F.getName(), std::move(GVInfo)); | ||
} | ||
|
||
void ModuleSummaryIndexBuilder::computeVariableInfo(const GlobalVariable &V) { | ||
DenseSet<const Value *> RefEdges; | ||
SmallPtrSet<const User *, 8> Visited; | ||
findRefEdges(&V, RefEdges, Visited); | ||
std::unique_ptr<GlobalVarSummary> GVarSummary = | ||
llvm::make_unique<GlobalVarSummary>(V.getLinkage()); | ||
GVarSummary->addRefEdges(RefEdges); | ||
std::unique_ptr<GlobalValueInfo> GVInfo = | ||
llvm::make_unique<GlobalValueInfo>(0, std::move(GVarSummary)); | ||
Index->addGlobalValueInfo(V.getName(), std::move(GVInfo)); | ||
} | ||
|
||
ModuleSummaryIndexBuilder::ModuleSummaryIndexBuilder( | ||
const Module *M, | ||
std::function<BlockFrequencyInfo *(const Function &F)> Ftor) | ||
: Index(llvm::make_unique<ModuleSummaryIndex>()), M(M) { | ||
// Compute summaries for all functions defined in module, and save in the | ||
// index. | ||
for (auto &F : *M) { | ||
if (F.isDeclaration()) | ||
continue; | ||
|
||
BlockFrequencyInfo *BFI = nullptr; | ||
std::unique_ptr<BlockFrequencyInfo> BFIPtr; | ||
if (Ftor) | ||
BFI = Ftor(F); | ||
else if (F.getEntryCount().hasValue()) { | ||
LoopInfo LI{DominatorTree(const_cast<Function &>(F))}; | ||
BranchProbabilityInfo BPI{F, LI}; | ||
BFIPtr = llvm::make_unique<BlockFrequencyInfo>(F, BPI, LI); | ||
BFI = BFIPtr.get(); | ||
} | ||
|
||
computeFunctionInfo(F, BFI); | ||
} | ||
|
||
// Compute summaries for all variables defined in module, and save in the | ||
// index. | ||
for (const GlobalVariable &G : M->globals()) { | ||
if (G.isDeclaration()) | ||
continue; | ||
computeVariableInfo(G); | ||
} | ||
} | ||
|
||
char ModuleSummaryIndexWrapperPass::ID = 0; | ||
INITIALIZE_PASS_BEGIN(ModuleSummaryIndexWrapperPass, "module-summary-analysis", | ||
"Module Summary Analysis", false, true) | ||
INITIALIZE_PASS_DEPENDENCY(BlockFrequencyInfoWrapperPass) | ||
INITIALIZE_PASS_END(ModuleSummaryIndexWrapperPass, "module-summary-analysis", | ||
"Module Summary Analysis", false, true) | ||
|
||
ModulePass *llvm::createModuleSummaryIndexWrapperPass() { | ||
return new ModuleSummaryIndexWrapperPass(); | ||
} | ||
|
||
ModuleSummaryIndexWrapperPass::ModuleSummaryIndexWrapperPass() | ||
: ModulePass(ID) { | ||
initializeModuleSummaryIndexWrapperPassPass(*PassRegistry::getPassRegistry()); | ||
} | ||
|
||
bool ModuleSummaryIndexWrapperPass::runOnModule(Module &M) { | ||
IndexBuilder = llvm::make_unique<ModuleSummaryIndexBuilder>( | ||
&M, [this](const Function &F) { | ||
return &(this->getAnalysis<BlockFrequencyInfoWrapperPass>( | ||
*const_cast<Function *>(&F)) | ||
.getBFI()); | ||
}); | ||
return false; | ||
} | ||
|
||
bool ModuleSummaryIndexWrapperPass::doFinalization(Module &M) { | ||
IndexBuilder.reset(); | ||
return false; | ||
} | ||
|
||
void ModuleSummaryIndexWrapperPass::getAnalysisUsage(AnalysisUsage &AU) const { | ||
AU.setPreservesAll(); | ||
AU.addRequired<BlockFrequencyInfoWrapperPass>(); | ||
} |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
set(LLVM_LINK_COMPONENTS | ||
Analysis | ||
AsmParser | ||
BitWriter | ||
Core | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters