[Clang] Add options -fprofile-filter-files and -fprofile-exclude-files to filter the files to instrument with gcov

[calixte]

Summary

These options are taking regex separated by colons to filter files.

• if both are empty then all files are instrumented
• if -fprofile-filter-files is empty then all the filenames matching any of the regex from exclude are not instrumented
• if -fprofile-exclude-files is empty then all the filenames matching any of the regex from filter are instrumented
• if both aren't empty then all the filenames which match any of the regex in filter and which don't match all the regex in filter are instrumented
• this patch is a follow-up of https://reviews.llvm.org/D52033

[vsk]

Please document the filter behavior (see docs/UsersManual.rst).

include/clang/Driver/CC1Options.td

236 ↗

(On Diff #165279)

Have you checked whether gcc supports similar options? If so, it would be great if we could match their name & behavior.

[calixte]

include/clang/Driver/CC1Options.td

236 ↗

(On Diff #165279)

The only one I found -finstrument-functions-exclude-file-list (https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html).
But no regex and no way to include one file only.
I took the names from gcovr: https://manpages.debian.org/jessie/gcovr/gcovr.1.en.html

[marco-c]

include/clang/Driver/CC1Options.td

236 ↗

(On Diff #165279)

We could file a bug in GCC's Bugzilla and agree with them about the options.

[vsk]

test/CodeGen/code-coverage-filter.c

5

Could you use more descriptive check prefixes here, like 'NO-HEADERS'? Also, if it's possible to use fewer '\' escape tokens by wrapping the regex list in single-quotes, that would make things easier to read.

33

1. This test will break on bots which use 16-bit ints. For simplicity, consider sticking to void and getting rid of unnecessary control flow.
2. Is there any need to check '#0'?

[vsk]

include/clang/Driver/CC1Options.td

236 ↗

(On Diff #165279)

+ 1, I think it's a great idea to loop in some gcc developers.

[calixte]

Fix tests

[calixte]

I reported a bug for gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87442
@vsk I'd like to add documentation in Docs/UsersManual.rst, but I've no idea on what's a good place for this (I look for option -coverage-no-function-names-in-data, but I didn't get anything). So could you give a me a clue please ?

[calixte]

@vsk, gcc guys are ok for -fprofile-filter-files and -fprofile-exclude-files, are you ok with that ?
Should these options prefixed by -Xclang or not ?

[vsk]

In D52034#1268277, @calixte wrote:

@vsk, gcc guys are ok for -fprofile-filter-files and -fprofile-exclude-files, are you ok with that ?

That sounds fine to me.

Should these options prefixed by -Xclang or not ?

I don't think they should be. These options should be easy to surface to users.

[vsk]

In D52034#1246379, @calixte wrote:

I reported a bug for gcc: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87442

Thank you!

@vsk I'd like to add documentation in Docs/UsersManual.rst, but I've no idea on what's a good place for this (I look for option -coverage-no-function-names-in-data, but I didn't get anything). So could you give a me a clue please ?

That's the right file to edit. Please create a section for gcov-based profiling. I won't ask that you describe the entire gcov pipeline, but it would really help to have a description of the driver flags you're adding & some examples of how to use them.

[calixte]

Change options names to -fprofile-exclude-files & -fprofile-filter-files and add some doc in UserManual.

[calixte]

Fix plural form of regex and just use ';' as separator.

[calixte]

Update doc.

[sylvestre.ledru]

Can be done in a new commit but I think this should be mentioned in the clang release notes.

[calixte]

Update ReleaseNotes

[calixte]

Forgot the ellipsis in the release notes.

docs/UsersManual.rst

.. note::
   ``-fexperimental-new-pass-manager``.
 
 .. note::
 
   Profile data remapping is currently only supported for C++ mangled names
   following the Itanium C++ ABI mangling scheme. This covers all C++ targets
   supported by Clang other than Windows.
 
+GCOV-based Profiling
+--------------------
+
+GCOV is a test coverage program, it helps to know how often a line of code
+is executed. When instrumenting the code with ``--coverage`` option, some
+counters are added for each edge linking basic blocks.
+
+At compile time, some gcno files are generated containing informations about
+blocks and edges between them. At runtime the counters are incremented and at
+exit the counters are dumped in gcda files.
+
+The tool ``llvm-cov gcov`` will get gcno, gcda and source files to generate
+a report ``.c.gcov``.
+
+.. option:: -fprofile-filter-files=[regexes]
+
+  Define a list of regexes separated by a semi-colon.
+  If a file name matches any of the regexes then the file is instrumented.
+
+   .. code-block:: console
+
+     $ clang --coverage -fprofile-filter-files=".*\.c$" foo.c
+
+  This will instrument only files finishing with .c and so no instrumentation
+  in .h files for example.
+
+.. option:: -fprofile-exclude-files=[regexes]
+
+  Define a list of regexes separated by a semi-colon.
+  If a file name doesn't matches all the regexes then the file is instrumented.
+
+  .. code-block:: console
+
+     $ clang --coverage -fprofile-exclude-files="^/usr/include/.*$" foo.c
+
+  This will instrument all the files except the ones in /usr/include.
+
+If both options are used then a file is instrumented if its name matches any
+of the regexes from ``-fprofile-filter-list`` and doesn't match all the regexes
+from ``-fprofile-exclude-list``.
+
+.. code-block:: console
+
+   $ clang --coverage -fprofile-exclude-files="^/usr/include/.*$" \
+           -fprofile-filter-files="^/usr/.*$"
+          
+In that case ``/usr/foo/oof.h`` is instrumented since it matches filter regex and
+doesn't match exclude regex, but ``/usr/include/foo.h`` doesn't since it matches
+exclude regex.
+
 Controlling Debug Information
 -----------------------------
 
 Controlling Size of Debug Information
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Debug info kind generated by Clang can be set by one of the flags listed
 below. If multiple flags are present, the last one is used.

include/clang/Driver/Options.td

 def fno_profile_generate : Flag<["-"], "fno-profile-generate">,
     Group<f_Group>, Flags<[DriverOption]>,
     HelpText<"Disable generation of profile instrumentation.">;
 def fno_profile_instr_use : Flag<["-"], "fno-profile-instr-use">,
     Group<f_Group>, Flags<[DriverOption]>,
     HelpText<"Disable using instrumentation data for profile-guided optimization">;
 def fno_profile_use : Flag<["-"], "fno-profile-use">,
     Alias<fno_profile_instr_use>;
+def fprofile_filter_files_EQ : Joined<["-"], "fprofile-filter-files=">,
+    Group<f_Group>, Flags<[CC1Option, CoreOption]>,
+    HelpText<"Instrument only functions from files where names match any regex separated by a semi-colon">;
+def fprofile_exclude_files_EQ : Joined<["-"], "fprofile-exclude-files=">,
+    Group<f_Group>, Flags<[CC1Option, CoreOption]>,
+    HelpText<"Instrument only functions from files where names don't match all the regexes separated by a semi-colon">;
 
 def faddrsig : Flag<["-"], "faddrsig">, Group<f_Group>, Flags<[CoreOption, CC1Option]>,
   HelpText<"Emit an address-significance table">;
 def fno_addrsig : Flag<["-"], "fno-addrsig">, Group<f_Group>, Flags<[CoreOption]>,
   HelpText<"Don't emit an address-significance table">;
 def fblocks : Flag<["-"], "fblocks">, Group<f_Group>, Flags<[CoreOption, CC1Option]>,
   HelpText<"Enable the 'blocks' language feature">;
 def fbootclasspath_EQ : Joined<["-"], "fbootclasspath=">, Group<f_Group>;

include/clang/Frontend/CodeGenOptions.h

public:
   /// The filename with path we use for coverage data files. The runtime
   /// allows further manipulation with the GCOV_PREFIX and GCOV_PREFIX_STRIP
   /// environment variables.
   std::string CoverageDataFile;
 
   /// The filename with path we use for coverage notes files.
   std::string CoverageNotesFile;
 
+  /// Regexes separated by a semi-colon to filter the files to instrument.
+  std::string ProfileFilterFiles;
+
+  /// Regexes separated by a semi-colon to filter the files to not instrument.
+  std::string ProfileExcludeFiles;
+
   /// The version string to put into coverage files.
   char CoverageVersion[4];
 
   /// Enable additional debugging information.
   std::string DebugPass;
 
   /// The string to embed in debug information as the current working directory.
   std::string DebugCompilationDir;

lib/CodeGen/BackendUtil.cpp

static Optional<GCOVOptions> getGCOVOptions(const CodeGenOptions &CodeGenOpts) {
   // LLVM's -default-gcov-version flag is set to something invalid.
   GCOVOptions Options;
   Options.EmitNotes = CodeGenOpts.EmitGcovNotes;
   Options.EmitData = CodeGenOpts.EmitGcovArcs;
   llvm::copy(CodeGenOpts.CoverageVersion, std::begin(Options.Version));
   Options.UseCfgChecksum = CodeGenOpts.CoverageExtraChecksum;
   Options.NoRedZone = CodeGenOpts.DisableRedZone;
   Options.FunctionNamesInData = !CodeGenOpts.CoverageNoFunctionNamesInData;
+  Options.Filter = CodeGenOpts.ProfileFilterFiles;
+  Options.Exclude = CodeGenOpts.ProfileExcludeFiles;
   Options.ExitBlockBeforeBody = CodeGenOpts.CoverageExitBlockBeforeBody;
   return Options;
 }
 
 void EmitAssemblyHelper::CreatePasses(legacy::PassManager &MPM,
                                       legacy::FunctionPassManager &FPM) {
   // Handle disabling of all LLVM passes, where we want to preserve the
   // internal module before any optimization.

lib/Driver/ToolChains/Clang.cpp

if (Args.hasFlag(options::OPT_fcoverage_mapping,
     if (!ProfileGenerateArg)
       D.Diag(clang::diag::err_drv_argument_only_allowed_with)
           << "-fcoverage-mapping"
           << "-fprofile-instr-generate";
 
     CmdArgs.push_back("-fcoverage-mapping");
   }
 
+  if (Args.hasArg(options::OPT_fprofile_exclude_files_EQ)) {
+    auto *Arg = Args.getLastArg(options::OPT_fprofile_exclude_files_EQ);
+    if (!Args.hasArg(options::OPT_coverage))
+      D.Diag(clang::diag::err_drv_argument_only_allowed_with)
+          << "-fprofile-exclude-files="
+          << "--coverage";
+
+    StringRef v = Arg->getValue();
+    CmdArgs.push_back(
+        Args.MakeArgString(Twine("-fprofile-exclude-files=" + v)));
+  }
+
+  if (Args.hasArg(options::OPT_fprofile_filter_files_EQ)) {
+    auto *Arg = Args.getLastArg(options::OPT_fprofile_exclude_files_EQ);
+    if (!Args.hasArg(options::OPT_coverage))
+      D.Diag(clang::diag::err_drv_argument_only_allowed_with)
+          << "-fprofile-filter-files="
+          << "--coverage";
+
+    StringRef v = Arg->getValue();
+    CmdArgs.push_back(Args.MakeArgString(Twine("-fprofile-filter-files=" + v)));
+  }
+
   if (C.getArgs().hasArg(options::OPT_c) ||
       C.getArgs().hasArg(options::OPT_S)) {
     if (Output.isFilename()) {
       CmdArgs.push_back("-coverage-notes-file");
       SmallString<128> OutputFilename;
       if (Arg *FinalOutput = C.getArgs().getLastArg(options::OPT_o))
         OutputFilename = FinalOutput->getValue();
       else

lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   Opts.EmitGcovArcs = Args.hasArg(OPT_femit_coverage_data);
   Opts.EmitGcovNotes = Args.hasArg(OPT_femit_coverage_notes);
   if (Opts.EmitGcovArcs || Opts.EmitGcovNotes) {
     Opts.CoverageDataFile = Args.getLastArgValue(OPT_coverage_data_file);
     Opts.CoverageNotesFile = Args.getLastArgValue(OPT_coverage_notes_file);
     Opts.CoverageExtraChecksum = Args.hasArg(OPT_coverage_cfg_checksum);
     Opts.CoverageNoFunctionNamesInData =
         Args.hasArg(OPT_coverage_no_function_names_in_data);
+    Opts.ProfileFilterFiles =
+        Args.getLastArgValue(OPT_fprofile_filter_files_EQ);
+    Opts.ProfileExcludeFiles =
+        Args.getLastArgValue(OPT_fprofile_exclude_files_EQ);
     Opts.CoverageExitBlockBeforeBody =
         Args.hasArg(OPT_coverage_exit_block_before_body);
     if (Args.hasArg(OPT_coverage_version_EQ)) {
       StringRef CoverageVersion = Args.getLastArgValue(OPT_coverage_version_EQ);
       if (CoverageVersion.size() != 4) {
         Diags.Report(diag::err_drv_invalid_value)
             << Args.getLastArg(OPT_coverage_version_EQ)->getAsString(Args)
             << CoverageVersion;

test/CodeGen/Inputs/code-coverage-filter1.h

+void test1() {}

test/CodeGen/Inputs/code-coverage-filter2.h

+void test2() {}

test/CodeGen/code-coverage-filter.c

+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes %s -o - \
+// RUN:    | FileCheck -check-prefix=ALL %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-exclude-files=".*\.h$" %s -o - \
+// RUN:    | FileCheck -check-prefix=NO-HEADER %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-filter-files=".*\.c$" %s -o - \
+// RUN:    | FileCheck -check-prefix=NO-HEADER %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-filter-files=".*\.c$;.*1\.h$" %s -o - \
+// RUN:    | FileCheck -check-prefix=NO-HEADER2 %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-exclude-files=".*2\.h$;.*1\.h$" %s -o - \
+// RUN:    | FileCheck -check-prefix=JUST-C %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-exclude-files=".*code\-coverage\-filter\.c$" %s -o - \
+// RUN:    | FileCheck -check-prefix=HEADER %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-filter-files=".*\.c$" -fprofile-exclude-files=".*\.c$" %s -o - \
+// RUN:    | FileCheck -check-prefix=NONE %s
+// RUN: %clang_cc1 -emit-llvm -femit-coverage-data -femit-coverage-notes -fprofile-filter-files=".*\.c$" -fprofile-exclude-files=".*\.h$" %s -o - \
+// RUN:    | FileCheck -check-prefix=JUST-C %s
+
+#include "Inputs/code-coverage-filter1.h"
+#include "Inputs/code-coverage-filter2.h"
+
+void test() {
+  test1();
+  test2();
+}
+
+// ALL: define void @test1() #0 {{.*}}
+// ALL: {{.*}}__llvm_gcov_ctr{{.*}}
+// ALL: ret void
+// ALL: define void @test2() #0 {{.*}}
+// ALL: {{.*}}__llvm_gcov_ctr{{.*}}
+// ALL: ret void
+// ALL: define void @test() #0 {{.*}}
+// ALL: {{.*}}__llvm_gcov_ctr{{.*}}
+// ALL: ret void
+
+// NO-HEADER: define void @test1() #0 {{.*}}
+// NO-HEADER-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER: ret void
+// NO-HEADER: define void @test2() #0 {{.*}}
+// NO-HEADER-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER: ret void
+// NO-HEADER: define void @test() #0 {{.*}}
+// NO-HEADER: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER: ret void
+
+// NO-HEADER2: define void @test1() #0 {{.*}}
+// NO-HEADER2: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER2: ret void
+// NO-HEADER2: define void @test2() #0 {{.*}}
+// NO-HEADER2-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER2: ret void
+// NO-HEADER2: define void @test() #0 {{.*}}
+// NO-HEADER2: {{.*}}__llvm_gcov_ctr{{.*}}
+// NO-HEADER2: ret void
+
+// JUST-C: define void @test1() #0 {{.*}}
+// JUST-C-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// JUST-C: ret void
+// JUST-C: define void @test2() #0 {{.*}}
+// JUST-C-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// JUST-C: ret void
+// JUST-C: define void @test() #0 {{.*}}
+// JUST-C: {{.*}}__llvm_gcov_ctr{{.*}}
+// JUST-C: ret void
+
+// HEADER: define void @test1() #0 {{.*}}
+// HEADER: {{.*}}__llvm_gcov_ctr{{.*}}
+// HEADER: ret void
+// HEADER: define void @test2() #0 {{.*}}
+// HEADER: {{.*}}__llvm_gcov_ctr{{.*}}
+// HEADER: ret void
+// HEADER: define void @test() #0 {{.*}}
+// HEADER-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// HEADER: ret void
+
+// NONE: define void @test1() #0 {{.*}}
+// NONE-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NONE: ret void
+// NONE: define void @test2() #0 {{.*}}
+// NONE-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NONE: ret void
+// NONE: define void @test() #0 {{.*}}
+// NONE-NOT: {{.*}}__llvm_gcov_ctr{{.*}}
+// NONE: ret void