This is an archive of the discontinued LLVM Phabricator instance.

[PGO] Don't include full file path in static function profile counter names
ClosedPublic

Authored by jakev on Jul 5 2016, 6:35 PM.

Details

Summary

Currently, the full build path gets included in the name of static function profile counters. When the training build and the profile guided build are in different directories, we end up just dropping the profile data and get a warning about unprofiled functions. Using only the filename in the counter prevents this problem, but seems good only as a short term solution.

Diff Detail

Repository
rL LLVM

Event Timeline

jakev updated this revision to Diff 62568.Jul 5 2016, 6:35 PM
jakev retitled this revision from to [PGO] Don't include full file path in static function profile counter names.
jakev updated this object.
jakev added reviewers: davidxl, xur, silvas.
jakev set the repository for this revision to rL LLVM.
davidxl edited edge metadata.Jul 8 2016, 4:12 PM

I think this should be controlled by an internal option -- I think default can be stripping the path as done in this patch: it is unlikely that two static functions have source files with the same name and the same cfg hash.

silvas edited edge metadata.Jul 8 2016, 5:36 PM

For posterity, I want to expand a bit on Jake's explanation:

When the training build and the profile guided build are in different directories, we end up just dropping the profile data and get a warning about unprofiled functions.

To expand on Jake's explanation of this issue, the issue we ran into when building one of our games is actually slightly more complicated than he described. The main complication is that the build generates source files into the build directory. Thus, different build directories (as one might use for separate prof-gen and prof-use builds) cause issues.

Concretely:
During the build, SOURCE_DIR/foo/bar/baz.cpp files are processed by a special tool into BUILD_DIR/foo/bar/baz.hashed.cpp before being passed to the compiler (essentially, the tool replaces certain marked strings with an integer hash code of the string).

During a PGO build, the following happens:

  1. The prof-gen build is done into PGO_TRAIN_BUILD_DIR. Thus, counters for *all* static functions end up with a name like PGO_TRAIN_BUILD_DIR/foo/bar/baz.hashed.cpp:funcname.
  2. The prof-use build is done into PGO_BUILD_DIR. Thus, the counter name that is expected is PGO_BUILD_DIR/foo/bar/baz.hashed.cpp:funcname. Thus, the counter is not found. This affects *all* static functions.

Pre-inline mitigates the issue significantly (since static functions are likely to be inlined), but we still saw it occur in practice.

jakev updated this revision to Diff 63371.Jul 8 2016, 6:58 PM
jakev edited edge metadata.

Added a flag for controlling the behavior of this. Keeping the name short but informative makes the flag name somewhat weird but just let me know if you have some preference on the naming.

davidxl accepted this revision.Jul 9 2016, 2:37 PM
davidxl edited edge metadata.

lgtm

This revision is now accepted and ready to land.Jul 9 2016, 2:37 PM
jakev added a comment.Jul 11 2016, 7:35 PM

Sean or David, could one of you please commit this for me, I don't have commit access. Thanks!

This revision was automatically updated to reflect the committed changes.

Done: r275193.

David