[Profile] document %h and %m

[davidxl]

Summary

Added documentation for %h and %m specifiers. %<n>m specifier which specifies the number of copies is not documented yet (treated as internal for now)

[davidxl]

Fixed a typo

[silvas]

LGTM with some small wording nits.

We may want to extend this to mention number modifier to %m (e.g. %4m). Perhaps it is better to leave that to more advanced documentation -- your experiments showed that even just 1 merge pool is quite scalable IIRC.

docs/UsersManual.rst

1500 ↗

(On Diff #64759)

I would suggest saying Additionally, multiple... instead of just Multiple to highlight that this is an additional behavior that is caused by using %m.

1501 ↗

(On Diff #64759)

You probably want to say "raw profiles" instead of just "profile" to clarify that this is a different sense of "merge" than llvm-profdata merge ... (or otherwise clarify).

[vsk]

In D22593#490490, @silvas wrote:

LGTM with some small wording nits.

We may want to extend this to mention number modifier to %m (e.g. %4m). Perhaps it is better to leave that to more advanced documentation -- your experiments showed that even just 1 merge pool is quite scalable IIRC.

Actually, I think this is the right place to document the numeric modifier to %m. LGTM with Sean's nits.

[davidxl]

docs/UsersManual.rst

1500 ↗

(On Diff #64759)

Done.

1501 ↗

(On Diff #64759)

Done. I also added another sentence clarifying it.

[davidxl]

Addressed review comments.

I still think %4m etc is an advanced feature that needs more explanation. We can delay that to a later time.

[silvas]

LGTM (also, another small suggestion).

docs/UsersManual.rst

1502 ↗

(On Diff #64793)

"running on the same or different hosts" should probably just be "that share a file system" to clarify that we depend on the filesystem guarantees and aren't doing anything else special.

cfe/trunk/docs/UsersManual.rst

1. Build an instrumented version of the code by compiling and linking with the
    ``-fprofile-instr-generate`` option.
 
    .. code-block:: console
 
      $ clang++ -O2 -fprofile-instr-generate code.cc -o code
 
 2. Run the instrumented executable with inputs that reflect the typical usage.
    By default, the profile data will be written to a ``default.profraw`` file
-   in the current directory. You can override that default by setting the
-   ``LLVM_PROFILE_FILE`` environment variable to specify an alternate file.
+   in the current directory. You can override that default by using option
+   ``-fprofile-instr-generate=`` or by setting the ``LLVM_PROFILE_FILE`` 
+   environment variable to specify an alternate file. If non-default file name
+   is specified by both the environment variable and the command line option,
+   the environment variable takes precedence. The file name pattern specified
+   can include different modifiers: ``%p``, ``%h``, and ``%m``.
+
    Any instance of ``%p`` in that file name will be replaced by the process
    ID, so that you can easily distinguish the profile output from multiple
    runs.
 
    .. code-block:: console
 
      $ LLVM_PROFILE_FILE="code-%p.profraw" ./code
 
+   The modifier ``%h`` can be used in scenarios where the same instrumented
+   binary is run in multiple different host machines dumping profile data
+   to a shared network based storage. The ``%h`` specifier will be substituted
+   with the hostname so that profiles collected from different hosts do not
+   clobber each other.
+
+   While the use of ``%p`` specifier can reduce the likelihood for the profiles
+   dumped from different processes to clobber each other, such clobbering can still
+   happen because of the ``pid`` re-use by the OS. Another side-effect of using
+   ``%p`` is that the storage requirement for raw profile data files is greatly
+   increased.  To avoid issues like this, the ``%m`` specifier can used in the profile
+   name.  When this specifier is used, the profiler runtime will substitute ``%m``
+   with a unique integer identifier associated with the instrumented binary. Additionally,
+   multiple raw profiles dumped from different processes that share a file system (can be
+   on different hosts) will be automatically merged by the profiler runtime during the
+   dumping. If the program links in multiple instrumented shared libraries, each library
+   will dump the profile data into its own profile data file (with its unique integer
+   id embedded in the profile name). Note that the merging enabled by ``%m`` is for raw
+   profile data generated by profiler runtime. The resulting merged "raw" profile data
+   file still needs to be converted to a different format expected by the compiler (
+   see step 3 below).
+
+   .. code-block:: console
+
+     $ LLVM_PROFILE_FILE="code-%m.profraw" ./code
+
+
 3. Combine profiles from multiple runs and convert the "raw" profile format to
    the input expected by clang. Use the ``merge`` command of the
    ``llvm-profdata`` tool to do this.
 
    .. code-block:: console
 
      $ llvm-profdata merge -output=code.profdata code-*.profraw