[profile] Add a mode to continuously sync counter updates to a file

[vsk]

Summary

Add support for continuously syncing profile counter updates to a file.

The motivation for this is that programs do not always exit cleanly. On
iOS, for example, programs are usually killed via a signal from the OS.
Running atexit() handlers after catching a signal is unreliable, so some
method for progressively writing out profile data is necessary.

The approach taken here is to mmap() the __llvm_prf_cnts section onto
a raw profile. To do this, the linker must page-align the counter and
data sections, and the runtime must ensure that counters are mapped to a
page-aligned offset within a raw profile.

Continuous mode is (for the moment) incompatible with the online merging
mode. This limitation is lifted in https://reviews.llvm.org/D69586.

Continuous mode is also (for the moment) incompatible with value
profiling, as I'm not sure whether there is interest in this and the
implementation may be tricky.

As I have not been able to test extensively on non-Darwin platforms,
only Darwin support is included for the moment. However, continuous mode
may "just work" without modification on Linux and some UNIX-likes. AIUI
the default value for the GNU linker's --section-alignment flag is set
to the page size on many systems. This appears to be true for LLD as
well, as its no_nmagic option is on by default. Continuous mode will
not "just work" on Fuchsia or Windows, as it's not possible to mmap() a
section on these platforms. There is a proposal to add a layer of
indirection to the profile instrumentation to support these platforms.

rdar://54210980

[sajjadm]

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

compiler-rt/lib/profile/InstrProfiling.h

179

I'm afraid I don't follow, could you explain in more detail why changing the filename is a problem?

[vsk]

In D68351#1691883, @sajjadm wrote:

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

This sounds tricky to me, as the initialization requires some work to be done in each copy of the profile runtime linked into the process.

Could you clarify whether you're referring to processes which use the profiling runtime's static initializer, or not? My understanding is that for processes which do use the runtime's static initializer, it's always desirable to set up the mmap'd file in that initializer. If that's not true, I'd appreciate a counterexample. For processes that don't use the runtime's static initializer, the situation (at Apple, at least) is typically that it's a kernel extension or bare-metal piece of firmware, and the continuous mode isn't really applicable anyway.

compiler-rt/lib/profile/InstrProfiling.h

179

The problem is that there are N copies of __llvm_profile_set_filename in a process, one for each instrumented image (1 executable + (N-1) DSOs). When one copy of __llvm_profile_set_filename is called, it can change the filename visible to all images because of weak symbol coalescing. However, in continuous mode, that is not sufficient. What's needed is to remap the __llvm_prf_cnts -> FILE mappings in each image. And a call to __llvm_profile_set_filename cannot achieve that. At least, not unless each image publishes its section mappings -- imho that level of complexity would be excessive.

[sajjadm]

In D68351#1691915, @vsk wrote:

In D68351#1691883, @sajjadm wrote:

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

This sounds tricky to me, as the initialization requires some work to be done in each copy of the profile runtime linked into the process.

Could you clarify whether you're referring to processes which use the profiling runtime's static initializer, or not? My understanding is that for processes which do use the runtime's static initializer, it's always desirable to set up the mmap'd file in that initializer. If that's not true, I'd appreciate a counterexample. For processes that don't use the runtime's static initializer, the situation (at Apple, at least) is typically that it's a kernel extension or bare-metal piece of firmware, and the continuous mode isn't really applicable anyway.

We do use the static initializer, but in our use case the process doesn't necessarily know where to write coverage data until some other application-specific code has run. Ideally, I would like to be able to have the instrumented process receive a shared memory buffer from another process, do continuous profile sync into that memory region, and not care whether it is file-backed or anonymous mapping. So it would be great if application code could give the runtime some sort of handle to that object [1] and tell the runtime to use that space for profiling.

[1] This could be just a pointer to a buffer that the application has already mmap'd, or it could be a file descriptor that the runtime is responsible for calling mmap on. Not sure which one would make more sense.

[vsk]

In D68351#1691934, @sajjadm wrote:

In D68351#1691915, @vsk wrote:

In D68351#1691883, @sajjadm wrote:

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

This sounds tricky to me, as the initialization requires some work to be done in each copy of the profile runtime linked into the process.

Could you clarify whether you're referring to processes which use the profiling runtime's static initializer, or not? My understanding is that for processes which do use the runtime's static initializer, it's always desirable to set up the mmap'd file in that initializer. If that's not true, I'd appreciate a counterexample. For processes that don't use the runtime's static initializer, the situation (at Apple, at least) is typically that it's a kernel extension or bare-metal piece of firmware, and the continuous mode isn't really applicable anyway.

We do use the static initializer, but in our use case the process doesn't necessarily know where to write coverage data until some other application-specific code has run.

I see, thanks for explaining.

Ideally, I would like to be able to have the instrumented process receive a shared memory buffer from another process, do continuous profile sync into that memory region, and not care whether it is file-backed or anonymous mapping. So it would be great if application code could give the runtime some sort of handle to that object [1] and tell the runtime to use that space for profiling.

I see. Is the lack of support for value profiling a concern, or would this just be for code coverage?

Regardless, I think it's possible to achieve this. If __llvm_profile_set_file_object is called in every copy of the profile runtime in a process, only minor changes to this patch would be required [1]. Judging by how setProfileFile is implemented, I suspect this is the case. If not, a much more invasive change would be needed.

[1] You could do this by recognizing a "%dc" filename pattern for "delayed continuous sync" mode (or some equivalent). The static initializer would detect this and defer the mmap() initialization. Later, when __llvm_profile_set_file_object is called, continuous mode can be initialized.

[1] This could be just a pointer to a buffer that the application has already mmap'd, or it could be a file descriptor that the runtime is responsible for calling mmap on. Not sure which one would make more sense.

I'm not sure it's possible to mmap() the contents of a section using MAP_FIXED over another non file-backed VM region. You may need to experiment with this.

[sajjadm]

In D68351#1691969, @vsk wrote:

In D68351#1691934, @sajjadm wrote:

In D68351#1691915, @vsk wrote:

In D68351#1691883, @sajjadm wrote:

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

This sounds tricky to me, as the initialization requires some work to be done in each copy of the profile runtime linked into the process.

Could you clarify whether you're referring to processes which use the profiling runtime's static initializer, or not? My understanding is that for processes which do use the runtime's static initializer, it's always desirable to set up the mmap'd file in that initializer. If that's not true, I'd appreciate a counterexample. For processes that don't use the runtime's static initializer, the situation (at Apple, at least) is typically that it's a kernel extension or bare-metal piece of firmware, and the continuous mode isn't really applicable anyway.

We do use the static initializer, but in our use case the process doesn't necessarily know where to write coverage data until some other application-specific code has run.

I see, thanks for explaining.

Ideally, I would like to be able to have the instrumented process receive a shared memory buffer from another process, do continuous profile sync into that memory region, and not care whether it is file-backed or anonymous mapping. So it would be great if application code could give the runtime some sort of handle to that object [1] and tell the runtime to use that space for profiling.

I see. Is the lack of support for value profiling a concern, or would this just be for code coverage?

This would just be for code coverage.

Regardless, I think it's possible to achieve this. If __llvm_profile_set_file_object is called in every copy of the profile runtime in a process, only minor changes to this patch would be required [1]. Judging by how setProfileFile is implemented, I suspect this is the case. If not, a much more invasive change would be needed.

edit: deleted question that was answered in an inline comment

[1] You could do this by recognizing a "%dc" filename pattern for "delayed continuous sync" mode (or some equivalent). The static initializer would detect this and defer the mmap() initialization. Later, when __llvm_profile_set_file_object is called, continuous mode can be initialized.

[1] This could be just a pointer to a buffer that the application has already mmap'd, or it could be a file descriptor that the runtime is responsible for calling mmap on. Not sure which one would make more sense.

I'm not sure it's possible to mmap() the contents of a section using MAP_FIXED over another non file-backed VM region. You may need to experiment with this.

Good point, I'll look into it.

it seems like this approach can be extended down the road without much effort, so that's great. Thanks for humoring my questions. :)

[vsk]

In D68351#1692003, @sajjadm wrote:

In D68351#1691969, @vsk wrote:

In D68351#1691934, @sajjadm wrote:

In D68351#1691915, @vsk wrote:

In D68351#1691883, @sajjadm wrote:

Not sure it belongs in this CL specifically, but would it be possible to later add a mode that defers the initialization of the mmap'd file until specifically requested by the process?

This sounds tricky to me, as the initialization requires some work to be done in each copy of the profile runtime linked into the process.

Could you clarify whether you're referring to processes which use the profiling runtime's static initializer, or not? My understanding is that for processes which do use the runtime's static initializer, it's always desirable to set up the mmap'd file in that initializer. If that's not true, I'd appreciate a counterexample. For processes that don't use the runtime's static initializer, the situation (at Apple, at least) is typically that it's a kernel extension or bare-metal piece of firmware, and the continuous mode isn't really applicable anyway.

We do use the static initializer, but in our use case the process doesn't necessarily know where to write coverage data until some other application-specific code has run.

I see, thanks for explaining.

Ideally, I would like to be able to have the instrumented process receive a shared memory buffer from another process, do continuous profile sync into that memory region, and not care whether it is file-backed or anonymous mapping. So it would be great if application code could give the runtime some sort of handle to that object [1] and tell the runtime to use that space for profiling.

I see. Is the lack of support for value profiling a concern, or would this just be for code coverage?

This would just be for code coverage.

Regardless, I think it's possible to achieve this. If __llvm_profile_set_file_object is called in every copy of the profile runtime in a process, only minor changes to this patch would be required [1]. Judging by how setProfileFile is implemented, I suspect this is the case. If not, a much more invasive change would be needed.

How does one get multiple copies of the runtime in the same process? Does that happen when you load a shared object file that was built with instrumentation?

Yes, exactly.

I think in our use case all our instrumented code is statically linked together, so that might not be a problem for me.

Oh, that makes sense then. There would just need to be a comment near __llvm_profile_set_file_object to explain that it only works on a per-image basis (i.e. that clients with instrumented DSOs should exercise caution with it).

[1] You could do this by recognizing a "%dc" filename pattern for "delayed continuous sync" mode (or some equivalent). The static initializer would detect this and defer the mmap() initialization. Later, when __llvm_profile_set_file_object is called, continuous mode can be initialized.

[1] This could be just a pointer to a buffer that the application has already mmap'd, or it could be a file descriptor that the runtime is responsible for calling mmap on. Not sure which one would make more sense.

I'm not sure it's possible to mmap() the contents of a section using MAP_FIXED over another non file-backed VM region. You may need to experiment with this.

Good point, I'll look into it.

it seems like this approach can be extended down the road without much effort, so that's great. Thanks for humoring my questions. :)

I think so, no worries!

[ejvaughan]

compiler-rt/lib/profile/InstrProfilingFile.c

406

Does FilenameBuf need to be free'd, or is it stack allocated?

[vsk]

compiler-rt/lib/profile/InstrProfilingFile.c

406

It's backed by a call to __builtin_alloca, which should just be a simple stack allocation.

[Dor1s]

@vsk Vedant, sorry I'm a bit swamped right now and may not be able to review this promptly. Please let me know If my feedback is important here, I'll try to make up some time in that case. Sorry!

[vsk]

In D68351#1692444, @Dor1s wrote:

@vsk Vedant, sorry I'm a bit swamped right now and may not be able to review this promptly. Please let me know If my feedback is important here, I'll try to make up some time in that case. Sorry!

No problem, I understand. Any feedback is appreciated.

[davidxl]

+petr who has similar needs for Fuchia platform

[phosek]

In D68351#1693307, @davidxl wrote:

+petr who has similar needs for Fuchia platform

Thank you for including me, we have exactly the same use case in Fuchsia and this is the solution I've initially considered and been experimenting with locally based on the discussion with @davidxl. However, after internal discussion we've decided against this solution for two reasons:

1. This requires the ability to mmap the output file over the (portion of) binary which is something we don't allow in Fuchsia for security reasons; once all modules have been mapped in by the dynamic linker, we don't allow any further changes to their mapping and using this solution would require special dynamic linker which is possible but highly undesirable.
2. It introduces bloat to both the binary and the output file. It also complicates the runtime implementation due to alignment and padding requirements.

The alternative solution we've came up and that I've now been implementing is to change the instrumentation to allow extra level of indirection. Concretely, the instrumentation conceptually does the following: c = __profc_foo[idx]; c++; __profc_foo[idx] = c. We'd like to change this c = *(&__profc[idx] + *bias); c++; *(&__profc[idx] + *bias) = c where bias is a global variable set by the runtime to be the offset between the __llvm_prf_cnts section and the corresponding location in the file that's mmapped in. Initially, that offset would be 0 which would result in exactly the same behavior as today, but the runtime can mmap the output file into address space and then change the offset to make the counters be continuously updated.

The advantage of this solution is that there are no changes needed to the profile format. It also doesn't require mmapping the output file over the binary, the output file can be mmapped anywhere in the address space. The disadvantage is extra overhead since instrumentation is going to be slightly more complicated, although we don't have any numbers yet to quantify how much slower it's going to be. The implementation should be fairly minimal and my tentative plan was to gate it on compiler switch, so it wouldn't affect existing in any way (modulo introducing one new variable in the runtime to hold the bias). I'm hoping to have the prototype ready and uploaded for review within the next few days. What's your opinion on this idea? Would this be something that you'd be interested in as an alternative approach?

[vsk]

In D68351#1693693, @phosek wrote:

In D68351#1693307, @davidxl wrote:

+petr who has similar needs for Fuchia platform

Thank you for including me, we have exactly the same use case in Fuchsia and this is the solution I've initially considered and been experimenting with locally based on the discussion with @davidxl. However, after internal discussion we've decided against this solution for two reasons:

1. This requires the ability to mmap the output file over the (portion of) binary which is something we don't allow in Fuchsia for security reasons; once all modules have been mapped in by the dynamic linker, we don't allow any further changes to their mapping and using this solution would require special dynamic linker which is possible but highly undesirable.
2. It introduces bloat to both the binary and the output file. It also complicates the runtime implementation due to alignment and padding requirements.

The alternative solution we've came up and that I've now been implementing is to change the instrumentation to allow extra level of indirection. Concretely, the instrumentation conceptually does the following: c = __profc_foo[idx]; c++; __profc_foo[idx] = c. We'd like to change this c = *(&__profc[idx] + *bias); c++; *(&__profc[idx] + *bias) = c where bias is a global variable set by the runtime to be the offset between the __llvm_prf_cnts section and the corresponding location in the file that's mmapped in. Initially, that offset would be 0 which would result in exactly the same behavior as today, but the runtime can mmap the output file into address space and then change the offset to make the counters be continuously updated.

The advantage of this solution is that there are no changes needed to the profile format. It also doesn't require mmapping the output file over the binary, the output file can be mmapped anywhere in the address space. The disadvantage is extra overhead since instrumentation is going to be slightly more complicated, although we don't have any numbers yet to quantify how much slower it's going to be. The implementation should be fairly minimal and my tentative plan was to gate it on compiler switch, so it wouldn't affect existing in any way (modulo introducing one new variable in the runtime to hold the bias). I'm hoping to have the prototype ready and uploaded for review within the next few days. What's your opinion on this idea? Would this be something that you'd be interested in as an alternative approach?

@phosek thank you for sharing this alternative. I hope you don't mind my calling this the bias method, after the term in the proposed instrumentation.

The TLDR is that I don't think the bias method is a good fit for Linux/Darwin. Imho, this method won't reduce the complexity of continuous mode, and its instrumentation overhead is likely to outweigh the savings from reduced section alignment. Let me try and justify these claims :).

First and most critically, note that (on Linux & Darwin, at least), mmap() requires that the file offset argument be page-aligned. So, I don't believe there's a way to avoid changing the raw profile format, or to avoid the concomitant complexity necessary to calculate padding bytes in the runtime. I don't see how the bias method solves this problem: this seems to be a fundamental limitation of mmap().

Second, note that the size overhead of counter increment instrumentation dwarfs the alignment overhead for __llvm_prf_{cnts,data} for all but the smallest of programs. Assuming 16K pages, in the worst case, the section alignment costs 32K per image. Assuming an increment can be encoded in 7 bytes as incq l___profc_main(%rip), this section alignment cost is equivalent to ~4700 increments. For comparison, this is roughly the number of counter increments in FileCheck.cpp.o. If the size overhead of counter instrumentation were to double, as I believe it would with the bias method, it would rapidly erase the memory savings from eliminating the section alignment requirement.

Third, I'm not sure I understand the security constraints in Fuchsia that make it undesirable to map the contents of a section onto a file. Is the concern that an attacker process may update the file, thereby changing the in-memory contents of the section? I'll note that on Darwin, at least, the kernel does not permit this kind of sharing. If an attacker process modifies a file on disk that has the in-memory contents of a section MAP_SHARED over it, the in-memory contents of the mapped section are not changed (I verified this by making a small modification to darwin-proof-of-concept.c). If there is some security aspect to the problem I'm missing here, could you please elaborate on it? I'm hoping that the bias method will not be necessary to support continuous mode on Fuchsia, but it seems this depends on the answer to the security question. Either way, istm that the high level approach taken in this patch as-written is a better fit for Linux/Darwin, and moreover probably provides support that can be used to implement the bias method.

I appreciate your feedback and think we can definitely work together to build some kind of common solution!

[vsk]

Updates:

• Rebase.
• I've added a caveat about the need for a different kind of instrumentation to support continuous mode on Fuchsia/Windows. Based on Petr's experiment, it looks like the current instrumentation is a better fit for Darwin.
• I've contacted the Darwin Runtime team about the possibility of supporting continuous mode + on-line merging mode, using a shared memory object that's mmap'd to disk (mmap() alone appears to be insufficient). If this is feasible, I would prefer to revisit it as a follow-up.

PTAL, thanks.

[vsk]

David was correct (& the Darwin team confirmed this) -- there is no reason for continuous sync mode to be incompatible with online profile merging. That limitation is lifted in D69586.

[davidxl]

clang/lib/Driver/ToolChains/Darwin.cpp

1119

The host tool may not know the page size of the target. Also why making the alignment 16K instead of 4K?

1155

The cost of alignment is always paid even when %c is not specified?

compiler-rt/lib/profile/InstrProfilingFile.c

104

emit warning messages?

compiler-rt/test/profile/ContinuousSyncMode/darwin-proof-of-concept.c

108

What does this loop verify?

[vsk]

• Add a warning in __llvm_profile_set_file_object, some clarifying comments and rebase.

clang/lib/Driver/ToolChains/Darwin.cpp

1119

The maximum supported page size for iPhone 6+ is 16K, and 4K for Macs. I thought that the simplicity of using a common constant outweighed the memory savings from reduced alignment, but am happy to make this tighter if you'd prefer.

1155

Yes, this is done because %c is expected to become the default in Xcode, and to remove the need for recompilation when switching %c on/off.

compiler-rt/test/profile/ContinuousSyncMode/darwin-proof-of-concept.c

108

I've added some clarifying comments. This is to verify that the data written after the counters is equal to the "data[]" array, i.e. {1, 2, 3}.

[davidxl]

lgtm

[dmajor]

compiler-rt/lib/profile/InstrProfilingFile.c

377

@vsk Doesn't this unconditionally break profiling on Fuschia/Windows? Should there at least be a check for __llvm_profile_is_continuous_mode_enabled() before erroring out? I'm currently testing my project's Windows build with trunk, and it's failing here even though we didn't enable continuous mode.

[vsk]

compiler-rt/lib/profile/InstrProfilingFile.c

377

Yes, sorry about the breakage, this should be addressed by 1aacf58819a27f428a46ce839a0ee797af03c1fd.

clang/docs/SourceBasedCodeCoverage.rst

 * "%Nm" expands out to the instrumented binary's signature. When this pattern
   is specified, the runtime creates a pool of N raw profiles which are used for
   on-line profile merging. The runtime takes care of selecting a raw profile
   from the pool, locking it, and updating it before the program exits.  If N is
   not specified (i.e the pattern is "%m"), it's assumed that ``N = 1``. N must
   be between 1 and 9. The merge pool specifier can only occur once per filename
   pattern.
 
+* "%c" expands out to nothing, but enables a mode in which profile counter
+  updates are continuously synced to a file. This means that if the
+  instrumented program crashes, or is killed by a signal, perfect coverage
+  information can still be recovered. Continuous mode is not yet compatible with
+  the "%Nm" merging mode described above, does not support value profiling for
+  PGO, and is only supported on Darwin. Support for Linux may be mostly
+  complete but requires testing, and support for Fuchsia/Windows may require
+  more extensive changes: please get involved if you are interested in porting
+  this feature.
+
 .. code-block:: console
 
     # Step 2: Run the program.
     % LLVM_PROFILE_FILE="foo.profraw" ./foo
 
 Creating coverage reports
 =========================
 

clang/lib/Driver/ToolChains/Darwin.cpp

 #include "clang/Config/config.h"
 #include "clang/Driver/Compilation.h"
 #include "clang/Driver/Driver.h"
 #include "clang/Driver/DriverDiagnostic.h"
 #include "clang/Driver/Options.h"
 #include "clang/Driver/SanitizerArgs.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Option/ArgList.h"
+#include "llvm/ProfileData/InstrProf.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/TargetParser.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include <cstdlib> // ::getenv
 
 using namespace clang::driver;
 using namespace clang::driver::tools;

 }
 
 /// Add an export directive for \p Symbol to the link command.
 static void addExportedSymbol(ArgStringList &CmdArgs, const char *Symbol) {
   CmdArgs.push_back("-exported_symbol");
   CmdArgs.push_back(Symbol);
 }
 
+/// Add a sectalign directive for \p Segment and \p Section to the maximum
+/// expected page size for Darwin.
+///
+/// On iPhone 6+ the max supported page size is 16K. On macOS, the max is 4K.
+/// Use a common alignment constant (16K) for now, and reduce the alignment on
+/// macOS if it proves important.
+static void addSectalignToPage(const ArgList &Args, ArgStringList &CmdArgs,
+                               StringRef Segment, StringRef Section) {
+  for (const char *A : {"-sectalign", Args.MakeArgString(Segment),
+                        Args.MakeArgString(Section), "0x4000"})
+    CmdArgs.push_back(A);
+}
+
 void Darwin::addProfileRTLibs(const ArgList &Args,
                               ArgStringList &CmdArgs) const {
   if (!needsProfileRT(Args)) return;
 
   AddLinkRuntimeLib(Args, CmdArgs, "profile",
                     RuntimeLinkOptions(RLO_AlwaysLink | RLO_FirstLink));
 
+  bool ForGCOV = needsGCovInstrumentation(Args);
+
   // If we have a symbol export directive and we're linking in the profile
   // runtime, automatically export symbols necessary to implement some of the
   // runtime's functionality.
   if (hasExportSymbolDirective(Args)) {
-    if (needsGCovInstrumentation(Args)) {
+    if (ForGCOV) {
       addExportedSymbol(CmdArgs, "___gcov_flush");
       addExportedSymbol(CmdArgs, "_flush_fn_list");
       addExportedSymbol(CmdArgs, "_writeout_fn_list");
     } else {
       addExportedSymbol(CmdArgs, "___llvm_profile_filename");
       addExportedSymbol(CmdArgs, "___llvm_profile_raw_version");
     }
     addExportedSymbol(CmdArgs, "_lprofDirMode");
   }
+
+  // Align __llvm_prf_{cnts,data} sections to the maximum expected page
+  // alignment. This allows profile counters to be mmap()'d to disk. Note that
+  // it's not enough to just page-align __llvm_prf_cnts: the following section
+  // must also be page-aligned so that its data is not clobbered by mmap().
+  //
+  // The section alignment is only needed when continuous profile sync is
+  // enabled, but this is expected to be the default in Xcode. Specifying the
+  // extra alignment also allows the same binary to be used with/without sync
+  // enabled.
+  if (!ForGCOV) {
+    for (auto IPSK : {llvm::IPSK_cnts, llvm::IPSK_data}) {
+      addSectalignToPage(
+          Args, CmdArgs, "__DATA",
+          llvm::getInstrProfSectionName(IPSK, llvm::Triple::MachO,
+                                        /*AddSegmentInfo=*/false));
+    }
+  }
 }
 
 void DarwinClang::AddLinkSanitizerLibArgs(const ArgList &Args,
                                           ArgStringList &CmdArgs,
                                           StringRef Sanitizer,
                                           bool Shared) const {
   auto RLO = RuntimeLinkOptions(RLO_AlwaysLink | (Shared ? RLO_AddRPath : 0U));
   AddLinkRuntimeLib(Args, CmdArgs, Sanitizer, RLO, Shared);

clang/test/Driver/darwin-ld.c

 // RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=LINK_PROFILE_FIRST %s < %t.log
 // RUN: %clang -target i386-apple-darwin9 -fprofile-instr-generate -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=LINK_PROFILE_FIRST %s < %t.log
 // RUN: %clang -target arm64-apple-ios5.0 -miphoneos-version-min=5.0 -fprofile-instr-generate -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=LINK_PROFILE_FIRST %s < %t.log
 // LINK_PROFILE_FIRST: {{ld(.exe)?"}} "{{[^"]+}}libclang_rt.profile_{{[a-z]+}}.a"
 
+// RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -### %t.o 2> %t.log
+// RUN: FileCheck -check-prefix=PROFILE_SECTALIGN %s < %t.log
+// RUN: %clang -target arm64-apple-ios12 -fprofile-instr-generate -### %t.o 2> %t.log
+// RUN: FileCheck -check-prefix=PROFILE_SECTALIGN %s < %t.log
+// PROFILE_SECTALIGN: "-sectalign" "__DATA" "__llvm_prf_cnts" "0x4000" "-sectalign" "__DATA" "__llvm_prf_data" "0x4000"
+
 // RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -exported_symbols_list /dev/null -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=PROFILE_EXPORT %s < %t.log
 // RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -Wl,-exported_symbols_list,/dev/null -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=PROFILE_EXPORT %s < %t.log
 // RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -Wl,-exported_symbol,foo -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=PROFILE_EXPORT %s < %t.log
 // RUN: %clang -target x86_64-apple-darwin12 -fprofile-instr-generate -Xlinker -exported_symbol -Xlinker foo -### %t.o 2> %t.log
 // RUN: FileCheck -check-prefix=PROFILE_EXPORT %s < %t.log

compiler-rt/lib/profile/InstrProfData.inc

 #ifndef INSTR_PROF_RAW_HEADER
 #define INSTR_PROF_RAW_HEADER(Type, Name, Initializer)
 #else
 #define INSTR_PROF_DATA_DEFINED
 #endif
 INSTR_PROF_RAW_HEADER(uint64_t, Magic, __llvm_profile_get_magic())
 INSTR_PROF_RAW_HEADER(uint64_t, Version, __llvm_profile_get_version())
 INSTR_PROF_RAW_HEADER(uint64_t, DataSize, DataSize)
+INSTR_PROF_RAW_HEADER(uint64_t, PaddingBytesBeforeCounters, PaddingBytesBeforeCounters)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersSize, CountersSize)
+INSTR_PROF_RAW_HEADER(uint64_t, PaddingBytesAfterCounters, PaddingBytesAfterCounters)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesSize,  NamesSize)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersDelta, (uintptr_t)CountersBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesDelta, (uintptr_t)NamesBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, ValueKindLast, IPVK_Last)
 #undef INSTR_PROF_RAW_HEADER
 /* INSTR_PROF_RAW_HEADER  end */
 
 /* VALUE_PROF_FUNC_PARAM start */

 #define INSTR_PROF_RAW_MAGIC_64 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'r' << 8 | (uint64_t)129
 #define INSTR_PROF_RAW_MAGIC_32 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'R' << 8 | (uint64_t)129
 
 /* Raw profile format version (start from 1). */
-#define INSTR_PROF_RAW_VERSION 4
+#define INSTR_PROF_RAW_VERSION 5
 /* Indexed profile format version (start from 1). */
 #define INSTR_PROF_INDEX_VERSION 5
 /* Coverage mapping format vresion (start from 0). */
 #define INSTR_PROF_COVMAP_VERSION 2
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentaiton

compiler-rt/lib/profile/InstrProfiling.h

 
 typedef struct ValueProfNode * PtrToNodeT;
 typedef struct ValueProfNode {
 #define INSTR_PROF_VALUE_NODE(Type, LLVMType, Name, Initializer) Type Name;
 #include "InstrProfData.inc"
 } ValueProfNode;
 
 /*!
+ * \brief Return 1 if profile counters are continuously synced to the raw
+ * profile via an mmap(). This is in contrast to the default mode, in which
+ * the raw profile is written out at program exit time.
+ */
+int __llvm_profile_is_continuous_mode_enabled(void);
+
+/*!
+ * \brief Enable continuous mode.
+ *
+ * See \ref __llvm_profile_is_continuous_mode_enabled. The behavior is undefined
+ * if continuous mode is already enabled, or if it cannot be enable due to
+ * conflicting options.
+ */
+void __llvm_profile_enable_continuous_mode(void);
+
+/*!
  * \brief Get number of bytes necessary to pad the argument to eight
  * byte boundary.
  */
 uint8_t __llvm_profile_get_num_padding_bytes(uint64_t SizeInBytes);
 
 /*!
  * \brief Get required size for profile buffer.
  */

  * \a __llvm_profile_write_file().
  *
  * \c Name is not copied, so it must remain valid.  Passing NULL resets the
  * filename logic to the default behaviour.
  *
  * Note: There may be multiple copies of the profile runtime (one for each
  * instrumented image/DSO). This API only modifies the filename within the
  * copy of the runtime available to the calling image.
+ *
+ * Warning: This is a no-op if continuous mode (\ref
+ * __llvm_profile_is_continuous_mode_enabled) is on. The reason for this is
+ * that in continuous mode, profile counters are mmap()'d to the profile at
+ * program initialization time. Support for transferring the mmap'd profile
+ * counts to a new file has not been implemented.
  */
 void __llvm_profile_set_filename(const char *Name);
 
 /*!
  * \brief Set the FILE object for writing instrumentation data.
  *
  * Sets the FILE object to be used for subsequent calls to
  * \a __llvm_profile_write_file(). The profile file name set by environment
  * variable, command-line option, or calls to \a  __llvm_profile_set_filename
  * will be ignored.
  *
  * \c File will not be closed after a call to \a __llvm_profile_write_file() but
  * it may be flushed. Passing NULL restores default behavior.
  *
  * If \c EnableMerge is nonzero, the runtime will always merge profiling data
  * with the contents of the profiling file. If EnableMerge is zero, the runtime
  * may still merge the data if it would have merged for another reason (for
  * example, because of a %m specifier in the file name).
  *
  * Note: There may be multiple copies of the profile runtime (one for each
  * instrumented image/DSO). This API only modifies the file object within the
  * copy of the runtime available to the calling image.
+ *
+ * Warning: This is a no-op if continuous mode (\ref
+ * __llvm_profile_is_continuous_mode_enabled) is on. The reason for this is
+ * that in continuous mode, profile counters are mmap()'d to the profile at
+ * program initialization time. Support for transferring the mmap'd profile
+ * counts to a new file has not been implemented.
  */
 void __llvm_profile_set_file_object(FILE *File, int EnableMerge);
 
 /*! \brief Register to write instrumentation data to file at exit. */
 int __llvm_profile_register_write_file_atexit(void);
 
 /*! \brief Initialize file handling. */
 void __llvm_profile_initialize_file(void);

 
 /*! \brief Get the version of the file format. */
 uint64_t __llvm_profile_get_version(void);
 
 /*! \brief Get the number of entries in the profile data section. */
 uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
                                       const __llvm_profile_data *End);
 
+/* ! \brief Given the sizes of the data and counter information, return the
+ * number of padding bytes before and after the counters, and after the names,
+ * in the raw profile.
+ *
+ * Note: In this context, "size" means "number of entries", i.e. the first two
+ * arguments must be the result of __llvm_profile_get_data_size() and of
+ * (__llvm_profile_end_counters() - __llvm_profile_begin_counters()) resp.
+ *
+ * Note: When mmap() mode is disabled, no padding bytes before/after counters
+ * are needed. However, in mmap() mode, the counter section in the raw profile
+ * must be page-aligned: this API computes the number of padding bytes
+ * needed to achieve that.
+ */
+void __llvm_profile_get_padding_sizes_for_counters(
+    uint64_t DataSize, uint64_t CountersSize, uint64_t NamesSize,
+    uint64_t *PaddingBytesBeforeCounters, uint64_t *PaddingBytesAfterCounters,
+    uint64_t *PaddingBytesAfterNames);
+
 /*!
  * \brief Set the flag that profile data has been dumped to the file.
  * This is useful for users to disable dumping profile data to the file for
  * certain processes in case the processes don't have permission to write to
  * the disks, and trying to do so would result in side effects such as crashes.
  */
 void __llvm_profile_set_dumped();
 

compiler-rt/lib/profile/InstrProfilingBuffer.c

 /*===- InstrProfilingBuffer.c - Write instrumentation to a memory buffer --===*\
 |*
 |* Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 |* See https://llvm.org/LICENSE.txt for license information.
 |* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 |*
 \*===----------------------------------------------------------------------===*/
 
 #include "InstrProfiling.h"
 #include "InstrProfilingInternal.h"
+#include "InstrProfilingPort.h"
+
+/* When continuous mode is enabled (%c), this parameter is set to 1. This is
+ * incompatible with the in-process merging mode. Lifting this restriction
+ * may be complicated, as merging mode requires a lock on the profile, and
+ * mmap() mode would require that lock to be held for the entire process
+ * lifetime.
+ *
+ * This parameter is defined here in InstrProfilingBuffer.o, instead of in
+ * InstrProfilingFile.o, to sequester all libc-dependent code in
+ * InstrProfilingFile.o. The test `instrprof-without-libc` will break if this
+ * layering is violated. */
+static int ContinuouslySyncProfile = 0;
+
+COMPILER_RT_VISIBILITY int __llvm_profile_is_continuous_mode_enabled(void) {
+  return ContinuouslySyncProfile;
+}
+
+COMPILER_RT_VISIBILITY void __llvm_profile_enable_continuous_mode(void) {
+  ContinuouslySyncProfile = 1;
+}
 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_size_for_buffer(void) {
   const __llvm_profile_data *DataBegin = __llvm_profile_begin_data();
   const __llvm_profile_data *DataEnd = __llvm_profile_end_data();
   const uint64_t *CountersBegin = __llvm_profile_begin_counters();
   const uint64_t *CountersEnd = __llvm_profile_end_counters();
   const char *NamesBegin = __llvm_profile_begin_names();
   const char *NamesEnd = __llvm_profile_end_names();
 
   return __llvm_profile_get_size_for_buffer_internal(
       DataBegin, DataEnd, CountersBegin, CountersEnd, NamesBegin, NamesEnd);
 }
 
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
                                       const __llvm_profile_data *End) {
   intptr_t BeginI = (intptr_t)Begin, EndI = (intptr_t)End;
   return ((EndI + sizeof(__llvm_profile_data) - 1) - BeginI) /
          sizeof(__llvm_profile_data);
 }
 
+/// Calculate the number of padding bytes needed to add to \p Offset in order
+/// for (\p Offset + Padding) to be page-aligned.
+static uint64_t calculateBytesNeededToPageAlign(uint64_t Offset,
+                                                unsigned PageSize) {
+  uint64_t OffsetModPage = Offset % PageSize;
+  if (OffsetModPage > 0)
+    return PageSize - OffsetModPage;
+  return 0;
+}
+
+COMPILER_RT_VISIBILITY
+void __llvm_profile_get_padding_sizes_for_counters(
+    uint64_t DataSize, uint64_t CountersSize, uint64_t NamesSize,
+    uint64_t *PaddingBytesBeforeCounters, uint64_t *PaddingBytesAfterCounters,
+    uint64_t *PaddingBytesAfterNames) {
+  if (!__llvm_profile_is_continuous_mode_enabled()) {
+    *PaddingBytesBeforeCounters = 0;
+    *PaddingBytesAfterCounters = 0;
+    *PaddingBytesAfterNames = __llvm_profile_get_num_padding_bytes(NamesSize);
+    return;
+  }
+
+  // In continuous mode, the file offsets for headers and for the start of
+  // counter sections need to be page-aligned.
+  unsigned PageSize = getpagesize();
+  uint64_t DataSizeInBytes = DataSize * sizeof(__llvm_profile_data);
+  uint64_t CountersSizeInBytes = CountersSize * sizeof(uint64_t);
+  *PaddingBytesBeforeCounters = calculateBytesNeededToPageAlign(
+      sizeof(__llvm_profile_header) + DataSizeInBytes, PageSize);
+  *PaddingBytesAfterCounters =
+      calculateBytesNeededToPageAlign(CountersSizeInBytes, PageSize);
+  *PaddingBytesAfterNames =
+      calculateBytesNeededToPageAlign(NamesSize, PageSize);
+}
+
 COMPILER_RT_VISIBILITY
 uint64_t __llvm_profile_get_size_for_buffer_internal(
     const __llvm_profile_data *DataBegin, const __llvm_profile_data *DataEnd,
     const uint64_t *CountersBegin, const uint64_t *CountersEnd,
     const char *NamesBegin, const char *NamesEnd) {
   /* Match logic in __llvm_profile_write_buffer(). */
   const uint64_t NamesSize = (NamesEnd - NamesBegin) * sizeof(char);
-  const uint8_t Padding = __llvm_profile_get_num_padding_bytes(NamesSize);
+  uint64_t DataSize = __llvm_profile_get_data_size(DataBegin, DataEnd);
+  uint64_t CountersSize = CountersEnd - CountersBegin;
+
+  /* Determine how much padding is needed before/after the counters and after
+   * the names. */
+  uint64_t PaddingBytesBeforeCounters, PaddingBytesAfterCounters,
+      PaddingBytesAfterNames;
+  __llvm_profile_get_padding_sizes_for_counters(
+      DataSize, CountersSize, NamesSize, &PaddingBytesBeforeCounters,
+      &PaddingBytesAfterCounters, &PaddingBytesAfterNames);
+
   return sizeof(__llvm_profile_header) +
-         (__llvm_profile_get_data_size(DataBegin, DataEnd) *
-          sizeof(__llvm_profile_data)) +
-         (CountersEnd - CountersBegin) * sizeof(uint64_t) + NamesSize + Padding;
+         (DataSize * sizeof(__llvm_profile_data)) + PaddingBytesBeforeCounters +
+         (CountersSize * sizeof(uint64_t)) + PaddingBytesAfterCounters +
+         NamesSize + PaddingBytesAfterNames;
 }
 
 COMPILER_RT_VISIBILITY
 void initBufferWriter(ProfDataWriter *BufferWriter, char *Buffer) {
   BufferWriter->Write = lprofBufferWriter;
   BufferWriter->WriterCtx = Buffer;
 }
 

compiler-rt/lib/profile/InstrProfilingFile.c

 #include <unistd.h>
 #if defined(__linux__)
 #include <sys/types.h>
 #endif
 #endif
 
 #include "InstrProfiling.h"
 #include "InstrProfilingInternal.h"
+#include "InstrProfilingPort.h"
 #include "InstrProfilingUtil.h"
 
 /* From where is profile name specified.
  * The order the enumerators define their
  * precedence. Re-order them may lead to
  * runtime behavior change. */
 typedef enum ProfileNameSpecifier {
   PNS_unknown = 0,

 }
 
 static FILE *ProfileFile = NULL;
 static FILE *getProfileFile() { return ProfileFile; }
 static void setProfileFile(FILE *File) { ProfileFile = File; }
 
 COMPILER_RT_VISIBILITY void __llvm_profile_set_file_object(FILE *File,
                                                            int EnableMerge) {
+  if (__llvm_profile_is_continuous_mode_enabled()) {
+    PROF_WARN("__llvm_profile_set_file_object(fd=%d) not supported, because "
+              "continuous sync mode (%%c) is enabled",
+              fileno(File));
+    return;
+  }
   setProfileFile(File);
   setProfileMergeRequested(EnableMerge);
 }
 
 static int getCurFilenameLength();
 static const char *getCurFilename(char *FilenameBuf, int ForceUseBuf);
 static unsigned doMerging() {
   return lprofCurFilename.MergePoolSize || isProfileMergeRequested();
static void truncateCurrentFile(void) {
   if (!Filename)
     return;
 
   /* By pass file truncation to allow online raw profile
    * merging. */
   if (lprofCurFilename.MergePoolSize)
     return;
 
+  /* Only create the profile directory and truncate an existing profile once.
+   * In continuous mode, this is necessary, as the profile is written-to by the
+   * runtime initializer. */
+  const char *lprofInitOnceEnv = "__LLVM_PROFILE_RT_INIT_ONCE";
+  int initialized = getenv(lprofInitOnceEnv) != NULL;
+  if (initialized)
+    return;
+  setenv(lprofInitOnceEnv, lprofInitOnceEnv, 1);
+
   createProfileDir(Filename);
 
   /* Truncate the file.  Later we'll reopen and append. */
   File = fopen(Filename, "w");
   if (!File)
     return;
   fclose(File);
 }
 
+static void initializeProfileForContinuousMode(void) {
+#if defined(__Fuchsia__) || defined(_WIN32)
+  PROF_ERR("%s\n", "Continuous mode not yet supported on Fuchsia or Windows.");
+#else // defined(__Fuchsia__) || defined(_WIN32)
+  if (!__llvm_profile_is_continuous_mode_enabled())
+    return;
+
+  /* Get the sizes of various profile data sections. Taken from
+   * __llvm_profile_get_size_for_buffer(). */
+  const __llvm_profile_data *DataBegin = __llvm_profile_begin_data();
+  const __llvm_profile_data *DataEnd = __llvm_profile_end_data();
+  const uint64_t *CountersBegin = __llvm_profile_begin_counters();
+  const uint64_t *CountersEnd = __llvm_profile_end_counters();
+  const char *NamesBegin = __llvm_profile_begin_names();
+  const char *NamesEnd = __llvm_profile_end_names();
+  const uint64_t NamesSize = (NamesEnd - NamesBegin) * sizeof(char);
+  uint64_t DataSize = __llvm_profile_get_data_size(DataBegin, DataEnd);
+  uint64_t CountersSize = CountersEnd - CountersBegin;
+
+  /* Check that the counter and data sections in this image are page-aligned. */
+  unsigned PageSize = getpagesize();
+  if ((intptr_t)CountersBegin % PageSize != 0) {
+    PROF_ERR("Counters section not page-aligned (start = %p, pagesz = %u).\n",
+             CountersBegin, PageSize);
+    return;
+  }
+  if ((intptr_t)DataBegin % PageSize != 0) {
+    PROF_ERR("Data section not page-aligned (start = %p, pagesz = %u).\n",
+             DataBegin, PageSize);
+    return;
+  }
+
+  /* Open the raw profile in append mode. */
+  int Length = getCurFilenameLength();
+  char *FilenameBuf = (char *)COMPILER_RT_ALLOCA(Length + 1);
+  const char *Filename = getCurFilename(FilenameBuf, 0);
+  if (!Filename)
+    return;
+  FILE *File = fopen(Filename, "a+b");
+  if (!File)
+    return;
+
+  int Fileno = fileno(File);
+
+  /* Check that the offset within the file is page-aligned. */
+  off_t CurrentFileOffset = ftello(File);
+  off_t OffsetModPage = CurrentFileOffset % PageSize;
+  if (OffsetModPage != 0) {
+    PROF_ERR("Continuous counter sync mode is enabled, but raw profile is not"
+             "page-aligned. CurrentFileOffset = %lld, pagesz = %u.\n",
+             CurrentFileOffset, PageSize);
+    return;
+  }
+
+  /* Determine how much padding is needed before/after the counters and after
+   * the names. */
+  uint64_t PaddingBytesBeforeCounters, PaddingBytesAfterCounters,
+      PaddingBytesAfterNames;
+  __llvm_profile_get_padding_sizes_for_counters(
+      DataSize, CountersSize, NamesSize, &PaddingBytesBeforeCounters,
+      &PaddingBytesAfterCounters, &PaddingBytesAfterNames);
+
+  uint64_t PageAlignedCountersLength =
+      (CountersSize * sizeof(uint64_t)) + PaddingBytesAfterCounters;
+  uint64_t FileOffsetToCounters =
+      CurrentFileOffset + sizeof(__llvm_profile_header) +
+      (DataSize * sizeof(__llvm_profile_data)) + PaddingBytesBeforeCounters;
+
+  /* Write the partial profile. This grows the file to a point where the mmap()
+   * can succeed. Leak the file handle, as the file should stay open. */
+  setProfileFile(File);
+  int rc = writeFile(Filename);
+  if (rc)
+    PROF_ERR("Failed to write file \"%s\": %s\n", Filename, strerror(errno));
+  setProfileFile(NULL);
+
+  uint64_t *CounterMmap = (uint64_t *)mmap(
+      (void *)CountersBegin, PageAlignedCountersLength, PROT_READ | PROT_WRITE,
+      MAP_FIXED | MAP_SHARED, Fileno, FileOffsetToCounters);
+  if (CounterMmap != CountersBegin) {
+    PROF_ERR(
+        "Continuous counter sync mode is enabled, but mmap() failed (%s).\n"
+        "  - CountersBegin: %p\n"
+        "  - PageAlignedCountersLength: %llu\n"
+        "  - Fileno: %d\n"
+        "  - FileOffsetToCounters: %llu\n",
+        strerror(errno), CountersBegin, PageAlignedCountersLength, Fileno,
+        FileOffsetToCounters);
+    return;
+  }
+#endif // defined(__Fuchsia__) || defined(_WIN32)
+}
+
 static const char *DefaultProfileName = "default.profraw";
 static void resetFilenameToDefault(void) {
   if (lprofCurFilename.FilenamePat && lprofCurFilename.OwnsFilenamePat) {
     free((void *)lprofCurFilename.FilenamePat);
   }
   memset(&lprofCurFilename, 0, sizeof(lprofCurFilename));
   lprofCurFilename.FilenamePat = DefaultProfileName;
   lprofCurFilename.PNS = PNS_default;
if (FilenamePat[I] == '%') {
       } else if (FilenamePat[I] == 'h') {
         if (!NumHosts++)
           if (COMPILER_RT_GETHOSTNAME(Hostname, COMPILER_RT_MAX_HOSTLEN)) {
             PROF_WARN("Unable to get hostname for filename pattern %s. Using "
                       "the default name.",
                       FilenamePat);
             return -1;
           }
+      } else if (FilenamePat[I] == 'c') {
+        if (__llvm_profile_is_continuous_mode_enabled()) {
+          PROF_WARN("%%c specifier can only be specified once in %s.\n",
+                    FilenamePat);
+          return -1;
+        }
+        if (MergingEnabled) {
+          PROF_WARN("%%c specifier can not be used with profile merging (%%m) "
+                    "in %s.\n",
+                    FilenamePat);
+          return -1;
+        }
+
+        __llvm_profile_enable_continuous_mode();
+        I++; /* advance to 'c' */
       } else if (containsMergeSpecifier(FilenamePat, I)) {
         if (MergingEnabled) {
           PROF_WARN("%%m specifier can only be specified once in %s.\n",
                     FilenamePat);
           return -1;
         }
+        if (__llvm_profile_is_continuous_mode_enabled()) {
+          PROF_WARN("%%c specifier can not be used with profile merging (%%m) "
+                    "in %s.\n",
+                    FilenamePat);
+          return -1;
+        }
         MergingEnabled = 1;
         if (FilenamePat[I] == 'm')
           lprofCurFilename.MergePoolSize = 1;
         else {
           lprofCurFilename.MergePoolSize = FilenamePat[I] - '0';
           I++; /* advance to 'm' */
         }
       }
     }
 
   lprofCurFilename.NumPids = NumPids;
   lprofCurFilename.NumHosts = NumHosts;
   return 0;
 }
 
 static void parseAndSetFilename(const char *FilenamePat,
                                 ProfileNameSpecifier PNS,
                                 unsigned CopyFilenamePat) {
 
   const char *OldFilenamePat = lprofCurFilename.FilenamePat;
   ProfileNameSpecifier OldPNS = lprofCurFilename.PNS;
 
+  /* The old profile name specifier takes precedence over the old one. */
   if (PNS < OldPNS)
     return;
 
   if (!FilenamePat)
     FilenamePat = DefaultProfileName;
 
   if (OldFilenamePat && !strcmp(OldFilenamePat, FilenamePat)) {
     lprofCurFilename.PNS = PNS;
static void parseAndSetFilename(const char *FilenamePat,
   } else {
     if (getenv("LLVM_PROFILE_VERBOSE"))
       PROF_NOTE("Override old profile path \"%s\" via %s to \"%s\" via %s.\n",
                 OldFilenamePat, getPNSStr(OldPNS), lprofCurFilename.FilenamePat,
                 getPNSStr(PNS));
   }
 
   truncateCurrentFile();
+  initializeProfileForContinuousMode();
 }
 
 /* Return buffer length that is required to store the current profile
  * filename with PID and hostname substitutions. */
 /* The length to hold uint64_t followed by 2 digit pool id including '_' */
 #define SIGLEN 24
 static int getCurFilenameLength() {
   int Len;

 static const char *getCurFilename(char *FilenameBuf, int ForceUseBuf) {
   int I, J, PidLength, HostNameLength, FilenamePatLength;
   const char *FilenamePat = lprofCurFilename.FilenamePat;
 
   if (!lprofCurFilename.FilenamePat || !lprofCurFilename.FilenamePat[0])
     return 0;
 
   if (!(lprofCurFilename.NumPids || lprofCurFilename.NumHosts ||
-        lprofCurFilename.MergePoolSize)) {
+        lprofCurFilename.MergePoolSize ||
+        __llvm_profile_is_continuous_mode_enabled())) {
     if (!ForceUseBuf)
       return lprofCurFilename.FilenamePat;
 
     FilenamePatLength = strlen(lprofCurFilename.FilenamePat);
     memcpy(FilenameBuf, lprofCurFilename.FilenamePat, FilenamePatLength);
     FilenameBuf[FilenamePatLength] = '\0';
     return FilenameBuf;
   }

 }
 
 /* This API is directly called by the user application code. It has the
  * highest precedence compared with LLVM_PROFILE_FILE environment variable
  * and command line option -fprofile-instr-generate=<profile_name>.
  */
 COMPILER_RT_VISIBILITY
 void __llvm_profile_set_filename(const char *FilenamePat) {
+  if (__llvm_profile_is_continuous_mode_enabled())
+    return;
   parseAndSetFilename(FilenamePat, PNS_runtime_api, 1);
 }
 
 /* The public API for writing profile data into the file with name
  * set by previous calls to __llvm_profile_set_filename or
  * __llvm_profile_override_default_filename or
  * __llvm_profile_initialize_file. */
 COMPILER_RT_VISIBILITY
 int __llvm_profile_write_file(void) {
   int rc, Length;
   const char *Filename;
   char *FilenameBuf;
   int PDeathSig = 0;
 
-  if (lprofProfileDumped()) {
+  if (lprofProfileDumped() || __llvm_profile_is_continuous_mode_enabled()) {
     PROF_NOTE("Profile data not written to file: %s.\n", "already written");
     return 0;
   }
 
   Length = getCurFilenameLength();
   FilenameBuf = (char *)COMPILER_RT_ALLOCA(Length + 1);
   Filename = getCurFilename(FilenameBuf, 0);
 

compiler-rt/lib/profile/InstrProfilingPort.h

 
 #ifndef DIR_SEPARATOR_2
 #define IS_DIR_SEPARATOR(ch) ((ch) == DIR_SEPARATOR)
 #else /* DIR_SEPARATOR_2 */
 #define IS_DIR_SEPARATOR(ch)                                                   \
   (((ch) == DIR_SEPARATOR) || ((ch) == DIR_SEPARATOR_2))
 #endif /* DIR_SEPARATOR_2 */
 
+#if defined(_WIN32)
+static inline size_t getpagesize() {
+  SYSTEM_INFO S;
+  GetNativeSystemInfo(&S);
+  return S.dwPageSize;
+}
+#else /* defined(_WIN32) */
+#include <unistd.h>
+#endif /* defined(_WIN32) */
+
 #define PROF_ERR(Format, ...)                                                  \
   fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
 
 #define PROF_WARN(Format, ...)                                                 \
   fprintf(stderr, "LLVM Profile Warning: " Format, __VA_ARGS__);
 
 #define PROF_NOTE(Format, ...)                                                 \
   fprintf(stderr, "LLVM Profile Note: " Format, __VA_ARGS__);

compiler-rt/lib/profile/InstrProfilingRuntime.cpp

 COMPILER_RT_VISIBILITY int INSTR_PROF_PROFILE_RUNTIME_VAR;
 }
 
 namespace {
 
 class RegisterRuntime {
 public:
   RegisterRuntime() {
-    __llvm_profile_register_write_file_atexit();
     __llvm_profile_initialize_file();
+    if (!__llvm_profile_is_continuous_mode_enabled())
+      __llvm_profile_register_write_file_atexit();
   }
 };
 
 RegisterRuntime Registration;
 
 }

compiler-rt/lib/profile/InstrProfilingWriter.c

 /*===- InstrProfilingWriter.c - Write instrumentation to a file or buffer -===*\
 |*
 |* Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 |* See https://llvm.org/LICENSE.txt for license information.
 |* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 |*
 \*===----------------------------------------------------------------------===*/
 
 #ifdef _MSC_VER
 /* For _alloca */
 #include <malloc.h>
 #endif
 #include <string.h>
 
 #include "InstrProfiling.h"
 #include "InstrProfilingInternal.h"
+#include "InstrProfilingPort.h"
 
 #define INSTR_PROF_VALUE_PROF_DATA
 #include "InstrProfData.inc"
 
 COMPILER_RT_VISIBILITY void (*FreeHook)(void *) = NULL;
 static ProfBufferIO TheBufferIO;
 #define VP_BUFFER_SIZE 8 * 1024
 static uint8_t BufferIOBuffer[VP_BUFFER_SIZE];
lprofWriteDataImpl(ProfDataWriter *Writer, const __llvm_profile_data *DataBegin,
                    const uint64_t *CountersBegin, const uint64_t *CountersEnd,
                    VPDataReaderType *VPDataReader, const char *NamesBegin,
                    const char *NamesEnd, int SkipNameDataWrite) {
 
   /* Calculate size of sections. */
   const uint64_t DataSize = __llvm_profile_get_data_size(DataBegin, DataEnd);
   const uint64_t CountersSize = CountersEnd - CountersBegin;
   const uint64_t NamesSize = NamesEnd - NamesBegin;
-  const uint64_t Padding = __llvm_profile_get_num_padding_bytes(NamesSize);
 
   /* Enough zeroes for padding. */
-  const char Zeroes[sizeof(uint64_t)] = {0};
+  unsigned PageSize = getpagesize();
+  char Zeroes[PageSize];
+  memset(Zeroes, 0, PageSize);
 
   /* Create the header. */
   __llvm_profile_header Header;
 
   if (!DataSize)
     return 0;
 
+  /* Determine how much padding is needed before/after the counters and after
+   * the names. */
+  uint64_t PaddingBytesBeforeCounters, PaddingBytesAfterCounters,
+      PaddingBytesAfterNames;
+  __llvm_profile_get_padding_sizes_for_counters(
+      DataSize, CountersSize, NamesSize, &PaddingBytesBeforeCounters,
+      &PaddingBytesAfterCounters, &PaddingBytesAfterNames);
+
 /* Initialize header structure.  */
 #define INSTR_PROF_RAW_HEADER(Type, Name, Init) Header.Name = Init;
 #include "InstrProfData.inc"
 
   /* Write the data. */
   ProfDataIOVec IOVec[] = {
       {&Header, sizeof(__llvm_profile_header), 1},
       {DataBegin, sizeof(__llvm_profile_data), DataSize},
+      {Zeroes, sizeof(uint8_t), PaddingBytesBeforeCounters},
       {CountersBegin, sizeof(uint64_t), CountersSize},
+      {Zeroes, sizeof(uint8_t), PaddingBytesAfterCounters},
       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
-      {Zeroes, sizeof(uint8_t), Padding}};
+      {Zeroes, sizeof(uint8_t), PaddingBytesAfterNames}};
   if (Writer->Write(Writer, IOVec, sizeof(IOVec) / sizeof(*IOVec)))
     return -1;
 
+  /* Value profiling is not yet supported in continuous mode. */
+  if (__llvm_profile_is_continuous_mode_enabled())
+    return 0;
+
   return writeValueProfData(Writer, VPDataReader, DataBegin, DataEnd);
 }

compiler-rt/test/profile/ContinuousSyncMode/basic.c

+// RUN: %clang -fprofile-instr-generate -fcoverage-mapping -o %t.exe %s
+// RUN: echo "garbage" > %t.profraw
+// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
+// RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s -check-prefix=CHECK-COUNTS
+// RUN: llvm-profdata merge -o %t.profdata %t.profraw
+// RUN: llvm-cov report %t.exe -instr-profile %t.profdata | FileCheck %s -check-prefix=CHECK-COVERAGE
+
+// CHECK-COUNTS: Counters:
+// CHECK-COUNTS-NEXT:   main:
+// CHECK-COUNTS-NEXT:     Hash: 0x{{.*}}
+// CHECK-COUNTS-NEXT:     Counters: 2
+// CHECK-COUNTS-NEXT:     Function count: 1
+// CHECK-COUNTS-NEXT:     Block counts: [1]
+// CHECK-COUNTS-NEXT: Instrumentation level: Front-end
+// CHECK-COUNTS-NEXT: Functions shown: 1
+// CHECK-COUNTS-NEXT: Total functions: 1
+// CHECK-COUNTS-NEXT: Maximum function count: 1
+// CHECK-COUNTS-NEXT: Maximum internal block count: 1
+
+// CHECK-COVERAGE: Filename    Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover
+// CHECK-COVERAGE-NEXT: ---
+// CHECK-COVERAGE-NEXT: basic.c      4                 1    75.00%           1                 0   100.00%           5                 2    60.00%
+// CHECK-COVERAGE-NEXT: ---
+// CHECK-COVERAGE-NEXT: TOTAL        4                 1    75.00%           1                 0   100.00%           5                 2    60.00%
+
+extern int __llvm_profile_is_continuous_mode_enabled(void);
+
+int main() {
+  if (__llvm_profile_is_continuous_mode_enabled())
+    return 0;
+  return 1;
+}

compiler-rt/test/profile/ContinuousSyncMode/darwin-proof-of-concept.c

+// Test whether mmap'ing profile counters onto an open file is feasible. As
+// this involves some platform-specific logic, this test is designed to be a
+// minimum viable proof-of-concept: it may be useful when porting the mmap()
+// mode to a new platform, but is not in and of itself a test of the profiling
+// runtime.
+
+// REQUIRES: darwin
+
+// Align counters and data to the maximum expected page size (16K).
+// RUN: %clang -g -o %t %s \
+// RUN:   -Wl,-sectalign,__DATA,__pcnts,0x4000 \
+// RUN:   -Wl,-sectalign,__DATA,__pdata,0x4000
+
+// Create a 'profile' using mmap() and validate it.
+// RUN: %run %t create %t.tmpfile
+// RUN: %run %t validate %t.tmpfile
+
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+
+__attribute__((section("__DATA,__pcnts"))) int counters[] = {0xbad};
+extern int cnts_start __asm("section$start$__DATA$__pcnts");
+const size_t cnts_len = 0x4000;
+
+__attribute__((section("__DATA,__pdata"))) int data[] = {1, 2, 3};
+extern int data_start __asm("section$start$__DATA$__pdata");
+const size_t data_len = sizeof(int) * 3;
+
+int create_tmpfile(char *path) {
+  // Create a temp file.
+  int fd = open(path, O_RDWR | O_TRUNC | O_CREAT, 0666);
+  if (fd == -1) {
+    perror("open");
+    return EXIT_FAILURE;
+  }
+
+  // Grow the file to hold data and counters.
+  if (0 != ftruncate(fd, cnts_len + data_len)) {
+    perror("ftruncate");
+    return EXIT_FAILURE;
+  }
+
+  // Write the data first (at offset 0x4000, after the counters).
+  if (data_len != pwrite(fd, &data, data_len, 0x4000)) {
+    perror("write");
+    return EXIT_FAILURE;
+  }
+
+  // Map the counters into the file, before the data.
+  //
+  // Requirements (on Darwin):
+  // - &cnts_start must be page-aligned.
+  // - The length and offset-into-fd must be page-aligned.
+  int *counter_map = (int *)mmap(&cnts_start, 0x4000, PROT_READ | PROT_WRITE,
+      MAP_FIXED | MAP_SHARED, fd, 0);
+  if (counter_map != &cnts_start) {
+    perror("mmap");
+    return EXIT_FAILURE;
+  }
+
+  // Update counters 1..9. These updates should be visible in the file.
+  // Expect counter 0 (0xbad), which is not updated, to be zero in the file.
+  for (int i = 1; i < 10; ++i)
+    counter_map[i] = i;
+
+  // Intentionally do not msync(), munmap(), or close().
+  return EXIT_SUCCESS;
+}
+
+int validate_tmpfile(char *path) {
+  int fd = open(path, O_RDONLY);
+  if (fd == -1) {
+    perror("open");
+    return EXIT_FAILURE;
+  }
+
+  // Verify that the file length is: sizeof(counters) + sizeof(data).
+  const size_t num_bytes = cnts_len + data_len;
+  int buf[num_bytes];
+  if (num_bytes != read(fd, &buf, num_bytes)) {
+    perror("read");
+    return EXIT_FAILURE;
+  }
+
+  // Verify the values of counters 1..9 (i.e. that the mmap() worked).
+  for (int i = 0; i < 10; ++i) {
+    if (buf[i] != i) {
+      fprintf(stderr,
+          "validate_tmpfile: Expected '%d' at pos=%d, but got '%d' instead.\n",
+          i, i, buf[i]);
+      return EXIT_FAILURE;
+    }
+  }
+
+  // Verify that the rest of the counters (after counter 9) are 0.
+  const int num_cnts = 0x4000 / sizeof(int);
+  for (int i = 10; i < num_cnts; ++i) {
+    if (buf[i] != 0) {
+      fprintf(stderr,
+          "validate_tmpfile: Expected '%d' at pos=%d, but got '%d' instead.\n",
+          0, i, buf[i]);
+      return EXIT_FAILURE;
+    }
+  }
+
+  // Verify that the data written after the counters is equal to the "data[]"
+  // array (i.e. {1, 2, 3}).
+  for (int i = num_cnts; i < num_cnts + 3; ++i) {
+    if (buf[i] != (i - num_cnts + 1)) {
+      fprintf(stderr,
+          "validate_tmpfile: Expected '%d' at pos=%d, but got '%d' instead.\n",
+          i - num_cnts + 1, i, buf[i]);
+      return EXIT_FAILURE;
+    }
+  }
+
+  // Intentionally do not close().
+  return EXIT_SUCCESS;
+}
+
+int main(int argc, char **argv) {
+  intptr_t cnts_start_int = (intptr_t)&cnts_start;
+  intptr_t data_start_int = (intptr_t)&data_start;
+  int pagesz = getpagesize();
+
+  if (cnts_start_int % pagesz != 0) {
+    fprintf(stderr, "__pcnts is not page-aligned: 0x%lx.\n", cnts_start_int);
+    return EXIT_FAILURE;
+  }
+  if (data_start_int % pagesz != 0) {
+    fprintf(stderr, "__pdata is not page-aligned: 0x%lx.\n", data_start_int);
+    return EXIT_FAILURE;
+  }
+  if (cnts_start_int + 0x4000 != data_start_int) {
+    fprintf(stderr, "__pdata not ordered after __pcnts.\n");
+    return EXIT_FAILURE;
+  }
+
+  char *action = argv[1];
+  char *path = argv[2];
+  if (0 == strcmp(action, "create"))
+    return create_tmpfile(path);
+  else if (0 == strcmp(action, "validate"))
+    return validate_tmpfile(path);
+  else
+    return EXIT_FAILURE;
+}

compiler-rt/test/profile/ContinuousSyncMode/lit.local.cfg.py

+import subprocess
+
+def getRoot(config):
+  if not config.parent:
+    return config
+  return getRoot(config.parent)
+
+root = getRoot(config)
+
+# As this has not been tested extensively on non-Darwin platforms,
+# only Darwin support is enabled for the moment. However, continuous mode
+# may "just work" without modification on Linux and other UNIX-likes (AIUI
+# the default value for the GNU linker's `--section-alignment` flag is
+# 0x1000, which is the size of a page on many systems).
+#
+# Please add supported configs to this list.
+if root.host_os not in ['Darwin']:
+  config.unsupported = True

compiler-rt/test/profile/ContinuousSyncMode/multiple-DSOs.c

+// RUN: echo "void dso1(void) {}" > %t.dso1.c
+// RUN: echo "void dso2(void) {}" > %t.dso2.c
+// RUN: %clang_pgogen -dynamiclib -o %t.dso1.dylib %t.dso1.c
+// RUN: %clang_pgogen -dynamiclib -o %t.dso2.dylib %t.dso2.c
+// RUN: %clang_pgogen -o %t.exe %s %t.dso1.dylib %t.dso2.dylib
+// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe
+// RUN: llvm-profdata show --counts --all-functions %t.profraw | FileCheck %s
+
+// CHECK-LABEL: Counters:
+// CHECK-NEXT:   dso1:
+// CHECK-NEXT:     Hash: 0x{{.*}}
+// CHECK-NEXT:     Counters: 1
+// CHECK-NEXT:     Block counts: [1]
+// CHECK-NEXT:   dso2:
+// CHECK-NEXT:     Hash: 0x{{.*}}
+// CHECK-NEXT:     Counters: 1
+// CHECK-NEXT:     Block counts: [1]
+// CHECK-NEXT:   main:
+// CHECK-NEXT:     Hash: 0x{{.*}}
+// CHECK-NEXT:     Counters: 1
+// CHECK-NEXT:     Block counts: [1]
+// CHECK-NEXT: Instrumentation level: IR
+// CHECK-NEXT: Functions shown: 3
+// CHECK-NEXT: Total functions: 3
+// CHECK-NEXT: Maximum function count: 1
+// CHECK-NEXT: Maximum internal block count: 0
+
+void dso1(void);
+void dso2(void);
+
+int main() {
+  dso1();
+  dso2();
+  return 0;
+}

compiler-rt/test/profile/ContinuousSyncMode/pid-substitution.c

+// RUN: rm -rf %t.dir && mkdir -p %t.dir
+// RUN: %clang_pgogen -o %t.exe %s
+//
+// Note: %%p is needed here, not %p, because of lit's path substitution.
+// RUN: env LLVM_PROFILE_FILE="%t.dir/%c-%%p" %run %t.exe
+
+#include <stdlib.h>
+#include <string.h>
+
+extern int __llvm_profile_is_continuous_mode_enabled(void);
+extern const char *__llvm_profile_get_filename(void);
+extern int getpid(void);
+
+int main() {
+  // Check that continuous mode is enabled.
+  if (!__llvm_profile_is_continuous_mode_enabled())
+    return 1;
+
+  // Check that the PID is actually in the filename.
+  const char *Filename = __llvm_profile_get_filename();
+
+  int Len = strlen(Filename);
+  --Len;
+  while (Filename[Len] != '-')
+    --Len;
+
+  const char *PidStr = Filename + Len + 1;
+  int Pid = atoi(PidStr);
+
+  if (Pid != getpid())
+    return 1;
+
+  return 0;
+}

compiler-rt/test/profile/ContinuousSyncMode/set-file-object.c

+// RUN: %clang_pgogen -o %t.exe %s
+// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.bad 2>&1 | FileCheck %s
+
+// CHECK: __llvm_profile_set_file_object(fd={{[0-9]+}}) not supported
+// CHECK: Profile data not written to file: already written.
+
+#include <stdio.h>
+
+extern int __llvm_profile_is_continuous_mode_enabled(void);
+extern void __llvm_profile_set_file_object(FILE *, int);
+extern int __llvm_profile_write_file(void);
+
+int main(int argc, char **argv) {
+  if (!__llvm_profile_is_continuous_mode_enabled())
+    return 1;
+
+  FILE *f = fopen(argv[1], "a+b");
+  if (!f)
+    return 1;
+
+  __llvm_profile_set_file_object(f, 0); // Try to set the file to "%t.bad".
+
+  if (__llvm_profile_write_file() != 0)
+    return 1;
+
+  f = fopen(argv[1], "r");
+  if (!f)
+    return 1;
+
+  fseek(f, 0, SEEK_END);
+  return ftell(f); // Check that the "%t.bad" is empty.
+}

compiler-rt/test/profile/ContinuousSyncMode/set-filename.c

+// RUN: %clang_pgogen -o %t.exe %s
+// RUN: env LLVM_PROFILE_FILE="%c%t.profraw" %run %t.exe %t.profraw %t.bad
+
+#include <string.h>
+
+extern int __llvm_profile_is_continuous_mode_enabled(void);
+extern void __llvm_profile_set_filename(const char *);
+extern const char *__llvm_profile_get_filename();
+
+int main(int argc, char **argv) {
+  if (!__llvm_profile_is_continuous_mode_enabled())
+    return 1;
+
+  __llvm_profile_set_filename(argv[2]); // Try to set the filename to "%t.bad".
+  const char *Filename = __llvm_profile_get_filename();
+  return strcmp(Filename, argv[1]); // Check that the filename is "%t.profraw".
+}

llvm/include/llvm/ProfileData/InstrProfData.inc

 #ifndef INSTR_PROF_RAW_HEADER
 #define INSTR_PROF_RAW_HEADER(Type, Name, Initializer)
 #else
 #define INSTR_PROF_DATA_DEFINED
 #endif
 INSTR_PROF_RAW_HEADER(uint64_t, Magic, __llvm_profile_get_magic())
 INSTR_PROF_RAW_HEADER(uint64_t, Version, __llvm_profile_get_version())
 INSTR_PROF_RAW_HEADER(uint64_t, DataSize, DataSize)
+INSTR_PROF_RAW_HEADER(uint64_t, PaddingBytesBeforeCounters, PaddingBytesBeforeCounters)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersSize, CountersSize)
+INSTR_PROF_RAW_HEADER(uint64_t, PaddingBytesAfterCounters, PaddingBytesAfterCounters)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesSize,  NamesSize)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersDelta, (uintptr_t)CountersBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesDelta, (uintptr_t)NamesBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, ValueKindLast, IPVK_Last)
 #undef INSTR_PROF_RAW_HEADER
 /* INSTR_PROF_RAW_HEADER  end */
 
 /* VALUE_PROF_FUNC_PARAM start */

 #define INSTR_PROF_RAW_MAGIC_64 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'r' << 8 | (uint64_t)129
 #define INSTR_PROF_RAW_MAGIC_32 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'R' << 8 | (uint64_t)129
 
 /* Raw profile format version (start from 1). */
-#define INSTR_PROF_RAW_VERSION 4
+#define INSTR_PROF_RAW_VERSION 5
 /* Indexed profile format version (start from 1). */
 #define INSTR_PROF_INDEX_VERSION 5
 /* Coverage mapping format vresion (start from 0). */
 #define INSTR_PROF_COVMAP_VERSION 2
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentaiton
typedef struct InstrProfValueData {
   uint64_t Value;
   /* Number of times the value appears in the training run. */
   uint64_t Count;
 } InstrProfValueData;
 
 #endif /* INSTR_PROF_DATA_INC */
 
 #ifndef INSTR_ORDER_FILE_INC
-// The maximal # of functions: 128*1024 (the buffer size will be 128*4 KB).
+/* The maximal # of functions: 128*1024 (the buffer size will be 128*4 KB). */
 #define INSTR_ORDER_FILE_BUFFER_SIZE 131072
 #define INSTR_ORDER_FILE_BUFFER_BITS 17
 #define INSTR_ORDER_FILE_BUFFER_MASK 0x1ffff
 #endif /* INSTR_ORDER_FILE_INC */
 #else
 #undef INSTR_PROF_DATA_DEFINED
 #endif

llvm/lib/ProfileData/InstrProfReader.cpp

Error RawInstrProfReader<IntPtrT>::readHeader(
     const RawInstrProf::Header &Header) {
   Version = swap(Header.Version);
   if (GET_VERSION(Version) != RawInstrProf::Version)
     return error(instrprof_error::unsupported_version);
 
   CountersDelta = swap(Header.CountersDelta);
   NamesDelta = swap(Header.NamesDelta);
   auto DataSize = swap(Header.DataSize);
+  auto PaddingBytesBeforeCounters = swap(Header.PaddingBytesBeforeCounters);
   auto CountersSize = swap(Header.CountersSize);
+  auto PaddingBytesAfterCounters = swap(Header.PaddingBytesAfterCounters);
   NamesSize = swap(Header.NamesSize);
   ValueKindLast = swap(Header.ValueKindLast);
 
   auto DataSizeInBytes = DataSize * sizeof(RawInstrProf::ProfileData<IntPtrT>);
   auto PaddingSize = getNumPaddingBytes(NamesSize);
 
   ptrdiff_t DataOffset = sizeof(RawInstrProf::Header);
-  ptrdiff_t CountersOffset = DataOffset + DataSizeInBytes;
-  ptrdiff_t NamesOffset = CountersOffset + sizeof(uint64_t) * CountersSize;
+  ptrdiff_t CountersOffset =
+      DataOffset + DataSizeInBytes + PaddingBytesBeforeCounters;
+  ptrdiff_t NamesOffset = CountersOffset + (sizeof(uint64_t) * CountersSize) +
+                          PaddingBytesAfterCounters;
   ptrdiff_t ValueDataOffset = NamesOffset + NamesSize + PaddingSize;
 
   auto *Start = reinterpret_cast<const char *>(&Header);
   if (Start + ValueDataOffset > DataBuffer->getBufferEnd())
     return error(instrprof_error::bad_header);
 
   Data = reinterpret_cast<const RawInstrProf::ProfileData<IntPtrT> *>(
       Start + DataOffset);

llvm/test/tools/llvm-profdata/c-general.test

 You need a checkout of clang with compiler-rt to generate the
 binary file here.  These shell commands can be used to regenerate
 it.
 $ SRC=path/to/llvm
 $ CFE=$SRC/tools/clang
 $ TESTDIR=$SRC/test/tools/llvm-profdata
 $ CFE_TESTDIR=$CFE/test/Profile
-$ clang -o a.out -fprofile-instr-generate $CFE_TESTDIR/c-general.c
+$ clang -o a.out -fprofile-instr-generate $CFE_TESTDIR/c-general.c -mllvm -enable-name-compression=false
 $ LLVM_PROFILE_FILE=$TESTDIR/Inputs/c-general.profraw ./a.out
 
 RUN: llvm-profdata show %p/Inputs/c-general.profraw -o - | FileCheck %s
 RUN: llvm-profdata show %p/Inputs/c-general.profraw --topn=3 -o - | FileCheck %s --check-prefix=TOPN
 RUN: llvm-profdata show %p/Inputs/c-general.profraw -o - --function=switches | FileCheck %s -check-prefix=SWITCHES -check-prefix=CHECK
 
 SWITCHES-LABEL: Counters:
 SWITCHES-NEXT:   switches:
-SWITCHES-NEXT:     Hash: 0x2618e4f23f2e8daa
+SWITCHES-NEXT:     Hash: 0xa50a07f391ae4be5
 SWITCHES-NEXT:     Counters: 19
 SWITCHES-NEXT:     Function count: 1
 SWITCHES-LABEL: Functions shown: 1
 
 CHECK-LABEL: Total functions: 12
 CHECK-NEXT: Maximum function count: 1
 CHECK-NEXT: Maximum internal block count: 100
 TOPN: boolean_operators, max count = 100
 TOPN-NEXT: simple_loops, max count = 100
 TOPN-NEXT:  conditionals, max count = 100

llvm/test/tools/llvm-profdata/raw-32-bits-be.test

 RUN: printf '\377lprofR\201' > %t
-RUN: printf '\0\0\0\0\0\0\0\4' >> %t
+RUN: printf '\0\0\0\0\0\0\0\5' >> %t
 RUN: printf '\0\0\0\0\0\0\0\2' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\3' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\20' >> %t
 RUN: printf '\0\0\0\0\1\0\0\0' >> %t
 RUN: printf '\0\0\0\0\2\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 
 RUN: printf '\134\370\302\114\333\030\275\254' >> %t
 RUN: printf '\0\0\0\0\0\0\0\1' >> %t
 RUN: printf '\1\0\0\0' >> %t

llvm/test/tools/llvm-profdata/raw-32-bits-le.test

 RUN: printf '\201Rforpl\377' > %t
-RUN: printf '\4\0\0\0\0\0\0\0' >> %t
+RUN: printf '\5\0\0\0\0\0\0\0' >> %t
 RUN: printf '\2\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\3\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\20\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\1\0\0\0\0' >> %t
 RUN: printf '\0\0\0\2\0\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 
 RUN: printf '\254\275\030\333\114\302\370\134' >> %t
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\1' >> %t

llvm/test/tools/llvm-profdata/raw-64-bits-be.test

 RUN: printf '\377lprofr\201' > %t
-RUN: printf '\0\0\0\0\0\0\0\4' >> %t
+RUN: printf '\0\0\0\0\0\0\0\5' >> %t
 RUN: printf '\0\0\0\0\0\0\0\2' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\3' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\20' >> %t
 RUN: printf '\0\0\0\1\0\4\0\0' >> %t
 RUN: printf '\0\0\0\2\0\4\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 
 RUN: printf '\134\370\302\114\333\030\275\254' >> %t
 RUN: printf '\0\0\0\0\0\0\0\1' >> %t
 RUN: printf '\0\0\0\1\0\4\0\0' >> %t

llvm/test/tools/llvm-profdata/raw-64-bits-le.test

 RUN: printf '\201rforpl\377' > %t
-RUN: printf '\4\0\0\0\0\0\0\0' >> %t
+RUN: printf '\5\0\0\0\0\0\0\0' >> %t
 RUN: printf '\2\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\3\0\0\0\0\0\0\0' >> %t
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 RUN: printf '\20\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\4\0\1\0\0\0' >> %t
 RUN: printf '\0\0\4\0\2\0\0\0' >> %t
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t
 
 RUN: printf '\254\275\030\333\114\302\370\134' >> %t
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t
 RUN: printf '\0\0\4\0\1\0\0\0' >> %t

llvm/test/tools/llvm-profdata/raw-two-profiles.test

 RUN: printf '\201rforpl\377' > %t-foo.profraw
-RUN: printf '\4\0\0\0\0\0\0\0' >> %t-foo.profraw
+RUN: printf '\5\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t-foo.profraw
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t-foo.profraw
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\10\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\4\0\1\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\4\0\2\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t-foo.profraw
 
 RUN: printf '\254\275\030\333\114\302\370\134' >> %t-foo.profraw
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\4\0\1\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t-foo.profraw
 
 RUN: printf '\023\0\0\0\0\0\0\0' >> %t-foo.profraw
 RUN: printf '\3\0foo\0\0\0' >> %t-foo.profraw
 
 RUN: printf '\201rforpl\377' > %t-bar.profraw
-RUN: printf '\4\0\0\0\0\0\0\0' >> %t-bar.profraw
+RUN: printf '\5\0\0\0\0\0\0\0' >> %t-bar.profraw
 RUN: printf '\1\0\0\0\0\0\0\0' >> %t-bar.profraw
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t-bar.profraw
 RUN: printf '\2\0\0\0\0\0\0\0' >> %t-bar.profraw
+RUN: printf '\0\0\0\0\0\0\0\0' >> %t-bar.profraw
 RUN: printf '\10\0\0\0\0\0\0\0' >> %t-bar.profraw
 RUN: printf '\0\0\6\0\1\0\0\0' >> %t-bar.profraw
 RUN: printf '\0\0\6\0\2\0\0\0' >> %t-bar.profraw
 RUN: printf '\0\0\0\0\0\0\0\0' >> %t-bar.profraw
 
 RUN: printf '\067\265\035\031\112\165\023\344' >> %t-bar.profraw
 RUN: printf '\02\0\0\0\0\0\0\0' >> %t-bar.profraw
 RUN: printf '\0\0\6\0\1\0\0\0' >> %t-bar.profraw