Page MenuHomePhabricator

[clang][LTO] Pass-through remarks options and set auto hotness threshold
AbandonedPublic

Authored by weiwang on Jul 21 2020, 9:13 AM.

Details

Summary

Expand remarks hotness threshold option -fdiagnostics-hotness-threshold in clang
driver command-line to both filtering remarks by hotness threshold from profile
summary and automatic remarks options pass-through to linker.

Remarks hotness filtering relies on several driver options. Table below lists
how different options are correlated and affect final remarks outputs:

profilehotnessthresholdremarks printed
NoNoNoAll
NoNoYesNone
NoYesNoAll
NoYesYesNone
YesNoNoAll
YesNoYesNone
YesYesNoAll
YesYesYes>=threshold

The new argument value -fdiagnostics-hotness-threshold=auto indicates threshold
will be synced with hotness threshold from profile summary during compilation.
In addition, when the following conditions are met, remarks related options are
passed into linker as well:

  1. LTO is enabled;
  2. Single arch target is specified;
  3. The linker is lld;

The "auto" threshold relies on the availability of profile summary. In case of
missing such information, no remarks will be generated.

This gives novice user a convenient way to collect and filter remarks throughout
a typical toolchain invocation with sample profile and LTO using single switch
from the clang driver.

A typical use of this option from clang command-line:

  • Using -Rpass* options to print remarks to screen:

clang -fuse-ld=lld -flto=thin

-fprofile-sample-use=foo_sample.txt
-Rpass=inline
-Rpass-missed=inline -Rpass-analysis=inline
-fdiagnostics-show-hotness -fdiagnostics-hotness-threshold=auto
-o foo foo.cpp

Remarks will be dumped to screen from both pre-lto and lto compilation.

  • Using serialized remarks options:

clang -fuse-ld=lld -flto=thin

-fprofile-sample-use=foo_sample.txt -fsave-optimization-record
-fdiagnostics-show-hotness -fdiagnostics-hotness-threshold=auto
-o foo foo.cpp

This will produce multiple yaml files containing optimization remarks:

  • foo.opt.yaml : remarks from pre-lto
  • foo.opt.ld.yaml.thin.1.yaml: remark during lto

Both types of options can be used together.

Given its restricted use cases, this shouldn't be viewed as a complete
replacement of current remarks option handling for the linker.

In order to make the option consistent across various tools, the support for the
new 'auto' argument is also available in the following tools:

  • lld: -opt-remarks-with-hotness=auto
  • llvm-lto: -lto-pass-remarks-hotness-threshold=auto
  • opt/llc: --pass-remarks-hotness-threshold=auto

Diff Detail

Unit TestsFailed

TimeTest
150 mslinux > Polly.ScopInfo::Unknown Unit Message ("")
Script: -- : 'RUN: at line 1'; opt -polly-process-unprofitable -polly-remarks-minimal -polly-use-llvm-names -polly-import-jscop-dir=/mnt/disks/ssd0/agent/llvm-project/polly/test/ScopInfo -polly-codegen-verify -basic-aa -scoped-noalias -tbaa -polly-scops -analyze < /mnt/disks/ssd0/agent/llvm-project/polly/test/ScopInfo/memcpy-raw-source.ll

Event Timeline

weiwang created this revision.Jul 21 2020, 9:13 AM
Herald added a reviewer: MaskRay. · View Herald Transcript
Herald added projects: Restricted Project, Restricted Project. · View Herald Transcript
wenlei added a subscriber: wenlei.Jul 21 2020, 9:22 AM

I put everything together in a single diff so that it is easier to get the whole idea. The change itself is too big to go in as a single diff, and I'd like to get inputs on how to split it.

weiwang updated this revision to Diff 281077.Jul 27 2020, 4:13 PM

Fix msvc build failure

hoyFB added a subscriber: hoyFB.Jul 27 2020, 5:25 PM

still waiting for inputs. Thanks.

weiwang abandoned this revision.Aug 11 2020, 10:56 PM

Diff has been split into 3 smaller ones:

  1. [lld] Enable remarks hotness filtering in lld: https://reviews.llvm.org/D85809
  2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
  3. [remarks] Optimization remarks hotness filtering from profile summary: https://reviews.llvm.org/D85808