Adjust the hotness threshold from 99.9% to 99%.

[danielcdh]

Summary

We originally set the hotness threshold as 99.9% to be consistent with gcc FDO. But because the inline heuristic is different between 2 compilers: llvm uses bottom-up algorithm while gcc uses priority based. The LLVM algorithm tends to inline too much early that prevents hot callsites from further inlined into its caller. Due to this restriction, we think it is reasonable to lower the hotness threshold to give priority to those that are really hot. Our experiments show that this change would improve performance on large applications. Note that the inline heuristic has great room for further tuning. Once the inline heuristics are refined, we could adjust this threshold to allow inlining for less hot callsites.

[tejohnson]

LGTM

[davidxl]

Bottom up ininling can also create lots of cold inline instances. Other than the effect of blocking hotter callers from being inlined, current Machine Block Layout also has problems forming long hot traces leaving holes in code layout.

I wonder if another way to fix this is better: 1) compute 99.9% working set size 2) if it is too large compared with the working set threshold, drop the hot cutoff.

lib/Analysis/ProfileSummaryInfo.cpp

 // considered hot/cold. These two parameters are percentile values (multiplied
 // by 10000). If the counts are sorted in descending order, the minimum count to
 // reach ProfileSummaryCutoffHot gives the threshold to determine a hot count.
 // Similarly, the minimum count to reach ProfileSummaryCutoffCold gives the
 // threshold for determining cold count (everything <= this threshold is
 // considered cold).
 
 static cl::opt<int> ProfileSummaryCutoffHot(
-    "profile-summary-cutoff-hot", cl::Hidden, cl::init(999000), cl::ZeroOrMore,
+    "profile-summary-cutoff-hot", cl::Hidden, cl::init(990000), cl::ZeroOrMore,
     cl::desc("A count is hot if it exceeds the minimum count to"
              " reach this percentile of total counts."));
 
 static cl::opt<int> ProfileSummaryCutoffCold(
     "profile-summary-cutoff-cold", cl::Hidden, cl::init(999999), cl::ZeroOrMore,
     cl::desc("A count is cold if it is below the minimum count"
              " to reach this percentile of total counts."));