Addressing Wenlei's feeback: add an iterator API for SampleContextTracker and other refactors.

changed all offset usage to preferred address based virtual address

Addressing Wenlei's feedback

add a valid state check for the trie conversion.

Updating D125246: [CSSPGO][llvm-profgen] Reimplement CS profile generator using context trie

rebase

rebase

addressing Wenlei's feedback

Updating D127031: [CSSPGO][llvm-profgen] Reimplement SampleContextTracker using context trie

rebase, memory leak issue is addressed in the https://reviews.llvm.org/D125246

rebase

Put all FSamples in a std::list<FunctionSamples> to avoid memory new and delete.

move memory free code inside llvm-profgen

free memory allocated by llvm-profgen

Updating D127031: [CSSPGO][llvm-profgen] Reimplement SampleContextTracker using context trie

addressing feedback from Hongtao

Updating D127031: [CSSPGO][llvm-profgen] Reimplement SampleContextTracker using context trie

Make a union type for fullcontext(array) and context trie node.

added more commments.

Updating D127026: [CSSPGO][llvm-profgen] Reimplement computeSummaryAndThreshold using context trie

changed to use virtual function for collectProfiledFunctions.

Addressing reviewers' feedback. Also added the build time and memory usage number in the summary.

Updating D127026: [CSSPGO][llvm-profgen] Reimplement computeSummaryAndThreshold using context trie

Updating D127026: [CSSPGO][llvm-profgen] Reimplement computeSummaryAndThreshold using context trie

LGTM.

Updating D127026: [CSSPGO][llvm-profgen] Reimplement computeSummaryAndThreshold using context trie

use std::move to avoid memory copy
remove the "memory free" code, will add in a separate patch.

Updating D127031: [CSSPGO][llvm-profgen] Reimplement SampleContextTracker using context trie

rebase
using context tracker directly

Addressing feedback.

upper_bound to lower_bound

addressing Wenlei's feedback

Updating D125255: [llvm-profgen] Support a threshold to control hiding warning summaries

IsCritical -> AlwaysEmit

In D125246#3501359, @hoy wrote:

Our experiment on one internal service showed that the context key's memory can be reduced from 80GB to 300MB.

This is awesome! Thanks for the work!

How about running time?

In D125255#3502506, @wenlei wrote:

In D125255#3502490, @wlei wrote:

In D125255#3502398, @wenlei wrote:

add a parameter to emitWarningSummary to customize value for critical warning, default value increase to 20%.

Do we even need a threshold for emitting critical ones? If it's critical, we should always emit them?

Sorry for the confusing, " customize value for critical warning" is wrong, I meant the parameter is used to treat warnings of different importance differently. For critical warning, the threshold is 0, then it's always emitted. For non-critical ones, developer can still set a threshold or use the default one, then emit it if it's above it.

Ok, maybe a verbose mode is enough for enabling all warnings. I can see that a threshold for non-critical warnings could be potentially useful, but also wondering if we could just keep things very simple here. The goal is not to give user a lot of control on what to print out, but to make sure 1) the default warning printing is meaningful to average users, 2) advanced users have a way to print everything.

Updating D125255: [llvm-profgen] Support a threshold to control hiding warning summaries

In D125255#3502398, @wenlei wrote:

add a parameter to emitWarningSummary to customize value for critical warning, default value increase to 20%.

Do we even need a threshold for emitting critical ones? If it's critical, we should always emit them?

In D125255#3502157, @wenlei wrote:

I'm not sure if a universal threshold for filtering warning is a good way to deal with this. Maybe look at this from a different angle -- for some warnings, any instance is bad, for others, we can tolerate them. Hence, an alternative: for critical ones, we should always print out any non-zero value; for others, print them only under a "verbose" mode?

Updating D125255: [llvm-profgen] Support a threshold to control hiding warning summaries