[profile] Support for memory-mapping counters to a raw profile

[vsk]

Summary

Summary

Using memory-mapped profile counters makes it possible to take snapshots of a running process's profiling information without changing the program. This is useful if the process exits abnormally, or if profiling data needs to be collected periodically.

Add the compiler-rt support required to create instrumented programs which memory-map their counters directly onto a raw profile.

More details

The memory-mapped counters feature works by writing out an empty raw profile at program start time, and then setting up an mmap. If this fails, we clean up and fall back to the normal atexit() profile writer.

One complication arises with profile truncation. The atexit() writer is able to truncate profiles every time __llvm_profile_initialize_file() is called. The mmap() writer can't do this because it needs to preserve the profiling data created by shared objects. The solution I picked is to use a lock file to synchronize truncation efforts.

Depends on: http://reviews.llvm.org/D19293

[vsk]

• Fix a memory bug I introduced in lprofWriteDataImpl. This will let me remove a hacky for-loop in D19293.
• Remove an extraneous pair of parentheses.

[vsk]

• Remove some extraneous changes (e.g, f(void) -> f()).

[davidxl]

lib/profile/InstrProfilingFile.c

431

This is another reason there is need to let driver teach the linker to align the combined counter variable section properly. Silently failing like this is not user friendly.

436

I think we should try to avoid bloating the writer interfaces (which makes it really hard to read)

1. the header offset can be obtained using 'stat' call before the write
2. There is no need to reuse the stream -- this can get rid of two other parameters.

3. can probably also use a static variable to set Write alignment -- so that pagesize parameter can be removed.

442

Can you explain what are these file lockings all about?

Is it to support multi-process concurrent update? or multi-threads with cocurrent dlopen/dlclose? Note that file synchronization is not supported with existing atexit file writer. For this reason, I think I suggest removing file locking code in this patch and we can discuss as a follow up. (Note the to be submitted online profile merging support also needs file locking, so there should be some sync up there).

lib/profile/InstrProfilingWriter.c

191

Since tail paddings are explicitly written out here, why is there a need to emit padding bytes in Instrumentation ?

193

This is to make sure the next header is properly aligned?

[vsk]

lib/profile/InstrProfilingFile.c

431

In D19295, I make some attempt to align the counters section to the target's page size. It's true that we silently fail here if the alignment is wrong, but falling back to the atexit() writer seems like a reasonable behavior.

436

Ok, I'll make these changes.

442

It's to support concurrent dlopen()'s. I'll remove the extra truncation lock related changes, and we can discuss it later if it's useful.

lib/profile/InstrProfilingWriter.c

191

If the size of the counters section is not a multiple of the target page size, the mmap() call will map in data from the section _following_ the counters section. On Darwin, this creates a problem because the mmap'd data becomes invalid after exit() is called, but before the atexit() handlers are run. This strange behavior results in data corruption and crashes in the removeTruncateLock atexit() handler.

Since I will get rid of the removeTruncateLock atexit() handler, the padding at the end of the counters section should not be necessary.

193

Yes, although with a few tweaks to RawInstrProfReader it might not be necessary. I will try this out and add a comment.

lib/profile/InstrProfData.inc

 /* Definition of member fields of the raw profile header data structure. */
 #ifndef INSTR_PROF_RAW_HEADER
 #define INSTR_PROF_RAW_HEADER(Type, Name, Initializer)
 #else
 #define INSTR_PROF_DATA_DEFINED
 #endif
 INSTR_PROF_RAW_HEADER(uint64_t, Magic, __llvm_profile_get_magic())
 INSTR_PROF_RAW_HEADER(uint64_t, Version, __llvm_profile_get_version())
+INSTR_PROF_RAW_HEADER(uint64_t, CounterSectionAlignment, CounterSectionAlignment)
 INSTR_PROF_RAW_HEADER(uint64_t, DataSize, DataSize)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersSize, CountersSize)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesSize,  NamesSize)
 INSTR_PROF_RAW_HEADER(uint64_t, CountersDelta, (uintptr_t)CountersBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, NamesDelta, (uintptr_t)NamesBegin)
 INSTR_PROF_RAW_HEADER(uint64_t, ValueKindLast, IPVK_Last)
 INSTR_PROF_RAW_HEADER(uint64_t, ValueDataSize, ValueDataSize)
 INSTR_PROF_RAW_HEADER(uint64_t, ValueDataDelta, (uintptr_t)ValueDataBegin)

 #define INSTR_PROF_RAW_MAGIC_64 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'r' << 8 | (uint64_t)129
 #define INSTR_PROF_RAW_MAGIC_32 (uint64_t)255 << 56 | (uint64_t)'l' << 48 | \
        (uint64_t)'p' << 40 | (uint64_t)'r' << 32 | (uint64_t)'o' << 24 |  \
         (uint64_t)'f' << 16 | (uint64_t)'R' << 8 | (uint64_t)129
 
 /* Raw profile format version (start from 1). */
-#define INSTR_PROF_RAW_VERSION 3
+#define INSTR_PROF_RAW_VERSION 4
 /* Indexed profile format version (start from 1). */
 #define INSTR_PROF_INDEX_VERSION 4
 /* Coverage mapping format vresion (start from 0). */
 #define INSTR_PROF_COVMAP_VERSION 1
 
 /* Profile version is always of type uint64_t. Reserve the upper 8 bits in the
  * version for other variants of profile. We set the lowest bit of the upper 8
  * bits (i.e. bit 56) to 1 to indicate if this is an IR-level instrumentaiton

lib/profile/InstrProfiling.h

 int __llvm_profile_write_file(void);
 
 /*!
  * \brief Set the filename for writing instrumentation data.
  *
  * Sets the filename to be used for subsequent calls to
  * \a __llvm_profile_write_file().
  *
+ * This is incompatible with LLVM_PROFILE_ENABLE_MMAP.
+ *
  * \c Name is not copied, so it must remain valid.  Passing NULL resets the
  * filename logic to the default behaviour.
  */
 void __llvm_profile_set_filename(const char *Name);
 
 /*!
  * \brief Set the filename for writing instrumentation data, unless the
  * \c LLVM_PROFILE_FILE environment variable was set.
  *
  * Unless overridden, sets the filename to be used for subsequent calls to
  * \a __llvm_profile_write_file().
  *
+ * This is incompatible with LLVM_PROFILE_ENABLE_MMAP.
+ *
  * \c Name is not copied, so it must remain valid.  Passing NULL resets the
  * filename logic to the default behaviour (unless the \c LLVM_PROFILE_FILE
  * was set in which case it has no effect).
  */
 void __llvm_profile_override_default_filename(const char *Name);
 
-/*! \brief Register to write instrumentation data to file at exit. */
+/*! \brief Register a memory map to emit counter data continuously.
+ *
+ * Returns a non-zero value to indicate failure.
+ */
+int __llvm_profile_register_mmap(void);
+
+/*! \brief Register to write instrumentation data to file at exit.
+ *
+ * Returns a non-zero value to indicate failure.
+ */
 int __llvm_profile_register_write_file_atexit(void);
 
 /*! \brief Initialize file handling. */
 void __llvm_profile_initialize_file(void);
 
 /*! \brief Get the magic token for the file format. */
 uint64_t __llvm_profile_get_magic(void);
 
 /*! \brief Get the version of the file format. */
 uint64_t __llvm_profile_get_version(void);
 
 /*! \brief Get the number of entries in the profile data section. */
 uint64_t __llvm_profile_get_data_size(const __llvm_profile_data *Begin,
                                       const __llvm_profile_data *End);
 
 #endif /* PROFILE_INSTRPROFILING_H_ */

lib/profile/InstrProfilingBuffer.c

uint64_t __llvm_profile_get_size_for_buffer_internal(
   const uint8_t Padding = __llvm_profile_get_num_padding_bytes(NamesSize);
   return sizeof(__llvm_profile_header) +
          (__llvm_profile_get_data_size(DataBegin, DataEnd) *
           sizeof(__llvm_profile_data)) +
          (CountersEnd - CountersBegin) * sizeof(uint64_t) + NamesSize + Padding;
 }
 
 COMPILER_RT_VISIBILITY int __llvm_profile_write_buffer(char *Buffer) {
-  return lprofWriteData(lprofBufferWriter, Buffer, 0, 0);
+  return lprofWriteData(lprofBufferWriter, Buffer, 1, 0, 0);
 }
 
 COMPILER_RT_VISIBILITY int __llvm_profile_write_buffer_internal(
     char *Buffer, const __llvm_profile_data *DataBegin,
     const __llvm_profile_data *DataEnd, const uint64_t *CountersBegin,
     const uint64_t *CountersEnd, const char *NamesBegin, const char *NamesEnd) {
   return lprofWriteDataImpl(lprofBufferWriter, Buffer, DataBegin, DataEnd,
-                            CountersBegin, CountersEnd, 0, 0, NamesBegin,
+                            CountersBegin, CountersEnd, 1, 0, 0, NamesBegin,
                             NamesEnd);
 }

lib/profile/InstrProfilingFile.c

 /*===- InstrProfilingFile.c - Write instrumentation to a file -------------===*\
 |*
 |*                     The LLVM Compiler Infrastructure
 |*
 |* This file is distributed under the University of Illinois Open Source
 |* License. See LICENSE.TXT for details.
 |*
 \*===----------------------------------------------------------------------===*/
 
 #include "InstrProfiling.h"
 #include "InstrProfilingInternal.h"
 #include "InstrProfilingUtil.h"
+
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <unistd.h>
+#include <stdbool.h>
+
+#if defined(_WIN32)
+#include "WindowsMMap.h"
+#else
+#include <sys/mman.h>
+#include <sys/file.h>
+#endif
+
+#ifdef COMPILER_RT_HAS_UNAME
+#include <sys/utsname.h>
+#endif
 
 #define UNCONST(ptr) ((void *)(uintptr_t)(ptr))
 
 /* Return 1 if there is an error, otherwise return  0.  */
 static uint32_t fileWriter(ProfDataIOVec *IOVecs, uint32_t NumIOVecs,
                            void **WriterCtx) {
   uint32_t I;
   FILE *File = (FILE *)*WriterCtx;
   for (I = 0; I < NumIOVecs; I++) {
     if (fwrite(IOVecs[I].Data, IOVecs[I].ElmSize, IOVecs[I].NumElm, File) !=
         IOVecs[I].NumElm)
       return 1;
   }
   return 0;
 }
 
 COMPILER_RT_VISIBILITY ProfBufferIO *
 lprofCreateBufferIOInternal(void *File, uint32_t BufferSz) {
   CallocHook = calloc;
   FreeHook = free;
   return lprofCreateBufferIO(fileWriter, File, BufferSz);
 }
 
-static int writeFile(FILE *File) {
+static int writeFile(FILE *File, uint64_t CounterSectionAlignment,
+                     bool EmitValueProf) {
   const char *BufferSzStr = 0;
   uint64_t ValueDataSize = 0;
-  struct ValueProfData **ValueDataArray =
-      __llvm_profile_gather_value_data(&ValueDataSize);
+  struct ValueProfData **ValueDataArray = NULL;
+
   FreeHook = &free;
   CallocHook = &calloc;
+
+  if (EmitValueProf) {
     BufferSzStr = getenv("LLVM_VP_BUFFER_SIZE");
     if (BufferSzStr && BufferSzStr[0])
       VPBufferSize = atoi(BufferSzStr);
-  return lprofWriteData(fileWriter, File, ValueDataArray, ValueDataSize);
+
+    ValueDataArray = __llvm_profile_gather_value_data(&ValueDataSize);
+  }
+
+  return lprofWriteData(fileWriter, File, CounterSectionAlignment,
+                        ValueDataArray, ValueDataSize);
 }
 
-static int writeFileWithName(const char *OutputName) {
+/*
+ * Write out a profile with the specificed counter section alignment.
+ *
+ * If \p EmitValueProf is true, emit any available value profile data.
+ *
+ * If \p CloseFile is true, the profile is closed after it's created.
+ *
+ * If \p CloseFile is false, the profile is kept locked and open. Its file
+ * stream is passed back to the caller via \p Stream.
+ *
+ * If \p HeaderOffset is non-null, it's set to the file offset to the header.
+ */
+static int writeFileWithName(const char *OutputName,
+                             uint64_t CounterSectionAlignment,
+                             bool EmitValueProf, bool CloseFile,
+                             FILE **Stream, off_t *HeaderOffset) {
   int RetVal;
   FILE *OutputFile;
+  int OFD;
   if (!OutputName || !OutputName[0])
     return -1;
 
   /* Append to the file to support profiling multiple shared objects. */
-  OutputFile = fopen(OutputName, "ab");
+  OutputFile = fopen(OutputName, "a+");
   if (!OutputFile)
     return -1;
 
-  RetVal = writeFile(OutputFile);
+  OFD = fileno(OutputFile);
+
+  if (!CloseFile) {
+    if (flock(OFD, LOCK_EX)) {
+      fclose(OutputFile);
+      return -1;
+    }
+  }
+
+  if (HeaderOffset) {
+    *HeaderOffset = ftello(OutputFile);
+    if (*HeaderOffset < 0 || *HeaderOffset % CounterSectionAlignment) {
+      flock(OFD, LOCK_UN);
+      fclose(OutputFile);
+      return -1;
+    }
+  }
+
+  RetVal = writeFile(OutputFile, CounterSectionAlignment, EmitValueProf);
+  if (RetVal) {
+    fclose(OutputFile);
+    return RetVal;
+  }
 
+  if (CloseFile)
     fclose(OutputFile);
+  else if (!CloseFile && Stream)
+    *Stream = OutputFile;
+
   return RetVal;
 }
 
 COMPILER_RT_WEAK int __llvm_profile_OwnsFilename = 0;
 COMPILER_RT_WEAK const char *__llvm_profile_CurrentFilename = NULL;
+COMPILER_RT_WEAK FILE *__llvm_profile_TruncateLockFile = NULL;
+COMPILER_RT_WEAK const char *__llvm_profile_TruncateLockFilename = NULL;
+
+static const char *getTruncateLockFilename(const char *Filename) {
+  unsigned TLFLen = strlen(Filename) + strlen(".lock") + 1;
+  char *TLF = malloc(TLFLen);
+  snprintf(TLF, TLFLen, "%s.lock", Filename);
+  return TLF;
+}
+
+static void removeTruncateLock(void) {
+  if (__llvm_profile_TruncateLockFile) {
+    flock(fileno(__llvm_profile_TruncateLockFile), LOCK_UN);
+    fclose(__llvm_profile_TruncateLockFile);
+    __llvm_profile_TruncateLockFile = NULL;
+  }
+
+  if (__llvm_profile_TruncateLockFilename) {
+    unlink(__llvm_profile_TruncateLockFilename);
+    free((void *)__llvm_profile_TruncateLockFilename);
+    __llvm_profile_TruncateLockFilename = NULL;
+  }
+}
 
 static void truncateCurrentFile(void) {
   const char *Filename;
   FILE *File;
 
   Filename = __llvm_profile_CurrentFilename;
   if (!Filename || !Filename[0])
     return;
 
   /* Create the directory holding the file, if needed. */
   if (strchr(Filename, '/') || strchr(Filename, '\\')) {
     char *Copy = malloc(strlen(Filename) + 1);
     strcpy(Copy, Filename);
     __llvm_profile_recursive_mkdir(Copy);
     free(Copy);
   }
 
+  /* Open the truncate lock file. */
+  const char *TLFilename = getTruncateLockFilename(Filename);
+  FILE *TLFile = fopen(TLFilename, "w+");
+  if (!TLFile) {
+    free((void *)TLFilename);
+    return;
+  }
+
+  /* If this is a new file, we may need to drop a stale truncate lock. */
+  if (__llvm_profile_TruncateLockFilename &&
+      strcmp(__llvm_profile_TruncateLockFilename, TLFilename))
+    removeTruncateLock();
+
+  /* Try to acquire the truncate lock. */
+  int TLFD = fileno(TLFile);
+  if (flock(TLFD, LOCK_EX | LOCK_NB)) {
+    /* A previous initialization already truncated the profile. */
+    free((void *)TLFilename);
+    fclose(TLFile);
+    return;
+  }
+
+  __llvm_profile_TruncateLockFile = TLFile;
+  __llvm_profile_TruncateLockFilename = TLFilename;
+  atexit(removeTruncateLock);
+
   /* Truncate the file.  Later we'll reopen and append. */
   File = fopen(Filename, "w");
   if (!File)
     return;
   fclose(File);
 }
 
 static void setFilename(const char *Filename, int OwnsFilename) {
   /* Check if this is a new filename and therefore needs truncation. */
   int NewFile = !__llvm_profile_CurrentFilename ||
                 (Filename && strcmp(Filename, __llvm_profile_CurrentFilename));
   if (__llvm_profile_OwnsFilename)
     free(UNCONST(__llvm_profile_CurrentFilename));
 
   __llvm_profile_CurrentFilename = Filename;
   __llvm_profile_OwnsFilename = OwnsFilename;
 
   /* If not a new file, append to support profiling multiple shared objects. */
   if (NewFile)

 }
 
 COMPILER_RT_VISIBILITY
 void __llvm_profile_initialize_file(void) {
   /* Check if the filename has been initialized. */
   if (__llvm_profile_CurrentFilename)
     return;
 
-  /* Detect the filename and truncate. */
+  /* Detect the filename and truncate the profile if we haven't already. */
   setFilenameAutomatically();
 }
 
 COMPILER_RT_VISIBILITY
 void __llvm_profile_set_filename(const char *Filename) {
   setFilenamePossiblyWithPid(Filename);
 }
 
 COMPILER_RT_VISIBILITY
 void __llvm_profile_override_default_filename(const char *Filename) {
   /* If the env var is set, skip setting filename from argument. */
   const char *Env_Filename = getenv("LLVM_PROFILE_FILE");
   if (Env_Filename && Env_Filename[0])
     return;
   setFilenamePossiblyWithPid(Filename);
 }
 
-COMPILER_RT_VISIBILITY
-int __llvm_profile_write_file(void) {
+static int writeFileImpl(uint64_t CounterSectionAlignment, bool EmitValueProf,
+                         bool CloseFile, FILE **Stream, off_t *HeaderOffset) {
   int rc;
 
   GetEnvHook = &getenv;
   /* Check the filename. */
   if (!__llvm_profile_CurrentFilename) {
     PROF_ERR("LLVM Profile: Failed to write file : %s\n", "Filename not set");
     return -1;
   }
 
   /* Check if there is llvm/runtime version mismatch.  */
   if (GET_VERSION(__llvm_profile_get_version()) != INSTR_PROF_RAW_VERSION) {
     PROF_ERR("LLVM Profile: runtime and instrumentation version mismatch : "
              "expected %d, but get %d\n",
              INSTR_PROF_RAW_VERSION,
              (int)GET_VERSION(__llvm_profile_get_version()));
     return -1;
   }
 
   /* Write the file. */
-  rc = writeFileWithName(__llvm_profile_CurrentFilename);
+  rc =
+      writeFileWithName(__llvm_profile_CurrentFilename, CounterSectionAlignment,
+                        EmitValueProf, CloseFile, Stream, HeaderOffset);
   if (rc)
     PROF_ERR("LLVM Profile: Failed to write file \"%s\": %s\n",
              __llvm_profile_CurrentFilename, strerror(errno));
   return rc;
 }
 
+COMPILER_RT_VISIBILITY
+int __llvm_profile_write_file(void) {
+  return writeFileImpl(1, true, true, NULL, NULL);
+}
+
 static void writeFileWithoutReturn(void) { __llvm_profile_write_file(); }
 
 COMPILER_RT_VISIBILITY
 int __llvm_profile_register_write_file_atexit(void) {
   static int HasBeenRegistered = 0;
 
   if (HasBeenRegistered)
     return 0;
 
   HasBeenRegistered = 1;
   return atexit(writeFileWithoutReturn);
 }
+
+static size_t getPageSize() {
+#if _WIN32
+  SYSTEM_INFO SI;
+  GetSystemInfo(&SI);
+  return (size_t)SI.dwPageSize;
+#else
+  return (size_t)sysconf(_SC_PAGESIZE);
+#endif
+}
+
+static bool hasValueProfData(const __llvm_profile_data *DataBegin,
+                             const __llvm_profile_data *DataEnd) {
+  const __llvm_profile_data *I;
+  unsigned J;
+  for (I = DataBegin; I < DataEnd; ++I)
+    for (J = 0; J <= IPVK_Last; ++J)
+      if (I->NumValueSites[J])
+        return true;
+  return false;
+}
+
+COMPILER_RT_VISIBILITY
+int __llvm_profile_register_mmap() {
+  int FD;
+  FILE *Stream;
+  size_t PageSize;
+
+  off_t HeaderOffset = 0;
+
+  const __llvm_profile_data *DataBegin = __llvm_profile_begin_data();
+  const __llvm_profile_data *DataEnd = __llvm_profile_end_data();
+  size_t DataSize = __llvm_profile_get_data_size(DataBegin, DataEnd);
+  size_t DataSizeInBytes = DataSize * sizeof(__llvm_profile_data);
+
+  const uint64_t *CountersBegin = __llvm_profile_begin_counters();
+  const uint64_t *CountersEnd = __llvm_profile_end_counters();
+  size_t CountersSize = CountersEnd - CountersBegin;
+  off_t CountersOffset;
+  size_t CountersSizeInBytes;
+  void *CountersMMap;
+
+  CountersSizeInBytes = CountersSize * sizeof(uint64_t);
+
+  /* No data, no profile. */
+  if (!DataSize)
+    return 0;
+
+  /* We don't support mmap'ing out value profile data. */
+  if (hasValueProfData(DataBegin, DataEnd))
+    return -1;
+
+  PageSize = getPageSize();
+
+  if (!lprofCheckIfAligned((size_t)CountersBegin, PageSize))
+    return -1;
+
+  /* Write out an empty profile with a page-aligned counters section.
+   * Expect a locked file stream in \p Stream. */
+  if (writeFileImpl(PageSize, false, false, &Stream, &HeaderOffset))
+    return -1;
+
+  CountersOffset = lprofAlignTo(
+      HeaderOffset + sizeof(__llvm_profile_header) + DataSizeInBytes, PageSize);
+
+  FD = fileno(Stream);
+
+  CountersMMap =
+      mmap((void *)CountersBegin, lprofAlignTo(CountersSizeInBytes, PageSize),
+           PROT_READ | PROT_WRITE, MAP_FILE | MAP_FIXED | MAP_SHARED, FD,
+           CountersOffset);
+
+  if (CountersMMap == MAP_FAILED) {
+    /* The mmap failed. Truncate the profile back to its original length. */
+    ftruncate(FD, HeaderOffset);
+  }
+
+  flock(FD, LOCK_UN);
+  fclose(Stream);
+
+  return (CountersMMap == MAP_FAILED) ? -1 : 0;
+}

lib/profile/InstrProfilingInternal.h

 
 /* The low level interface to write data into a buffer. It is used as the
  * callback by other high level writer methods such as buffered IO writer
  * and profile data writer.  */
 uint32_t lprofBufferWriter(ProfDataIOVec *IOVecs, uint32_t NumIOVecs,
                            void **WriterCtx);
 
 int lprofWriteData(WriterCallback Writer, void *WriterCtx,
+                   uint64_t CounterSectionAlignment,
                    struct ValueProfData **ValueDataArray,
                    const uint64_t ValueDataSize);
 int lprofWriteDataImpl(WriterCallback Writer, void *WriterCtx,
                        const __llvm_profile_data *DataBegin,
                        const __llvm_profile_data *DataEnd,
                        const uint64_t *CountersBegin,
                        const uint64_t *CountersEnd,
+                       uint64_t CounterSectionAlignment,
                        struct ValueProfData **ValueDataBeginArray,
                        const uint64_t ValueDataSize, const char *NamesBegin,
                        const char *NamesEnd);
 /* Merge value profile data pointed to by SrcValueProfData into
  * in-memory profile counters pointed by to DstData.  */
 void lprofMergeValueProfData(struct ValueProfData *SrcValueProfData,
                              __llvm_profile_data *DstData);
 
 extern char *(*GetEnvHook)(const char *);
 extern void (*FreeHook)(void *);
 extern void *(*CallocHook)(size_t, size_t);
 extern void (*VPMergeHook)(struct ValueProfData *, __llvm_profile_data *);
 extern uint32_t VPBufferSize;
 
 #endif

lib/profile/InstrProfilingRuntime.cc

 //===- InstrProfilingRuntime.cpp - PGO runtime initialization -------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 extern "C" {
 
 #include "InstrProfiling.h"
 
+#include <stdlib.h>
+#include <string.h>
+
 COMPILER_RT_VISIBILITY int __llvm_profile_runtime;
 }
 
 namespace {
 
+bool isMMapEnabled() {
+  const char *EnableMMap = getenv("LLVM_PROFILE_ENABLE_MMAP");
+  return EnableMMap && strlen(EnableMMap) && EnableMMap[0] != '0' &&
+         EnableMMap[0] != 'f';
+}
+
 class RegisterRuntime {
 public:
   RegisterRuntime() {
-    __llvm_profile_register_write_file_atexit();
     __llvm_profile_initialize_file();
+
+    if (isMMapEnabled())
+      if (!__llvm_profile_register_mmap())
+        return;
+
+    __llvm_profile_register_write_file_atexit();
   }
 };
 
 RegisterRuntime Registration;
 
 }

lib/profile/InstrProfilingUtil.h

 /*===- InstrProfilingUtil.h - Support library for PGO instrumentation -----===*\
 |*
 |*                     The LLVM Compiler Infrastructure
 |*
 |* This file is distributed under the University of Illinois Open Source
 |* License. See LICENSE.TXT for details.
 |*
 \*===----------------------------------------------------------------------===*/
 
 #ifndef PROFILE_INSTRPROFILINGUTIL_H
 #define PROFILE_INSTRPROFILINGUTIL_H
 
 #include <stddef.h>
+#include <stdbool.h>
 
 /*! \brief Create a directory tree. */
 void __llvm_profile_recursive_mkdir(char *Pathname);
 
 /* PS4 doesn't have getenv. Define a shim. */
 #if __PS4__
 static inline char *getenv(const char *name) { return NULL; }
 #endif /* #if __PS4__ */
 
 int lprofGetHostName(char *Name, int Len);
 
 unsigned lprofBoolCmpXchg(void **Ptr, void *OldV, void *NewV);
 
+/* Round \p Offset to the nearest multiple of \p Alignment. \p Alignment must
+ * be a power of 2. */
+size_t lprofAlignTo(size_t Offset, size_t Alignment);
+
+/* Check if \p Offset is a multiple of \p Alignment. \p Alignment must be a
+ * power of 2. */
+bool lprofCheckIfAligned(size_t Offset, size_t Alignment);
+
 #endif  /* PROFILE_INSTRPROFILINGUTIL_H */

lib/profile/InstrProfilingUtil.c

int lprofGetHostName(char *Name, int Len) {
   struct utsname N;
   int R;
   if (!(R = uname(&N)))
     strncpy(Name, N.nodename, Len);
   return R;
 }
 #endif
 
+bool lprofCheckIfAligned(size_t Offset, size_t Alignment) {
+  return !(Offset & (Alignment - 1));
+}
+
+size_t lprofAlignTo(size_t Offset, size_t Alignment) {
+  if (lprofCheckIfAligned(Offset, Alignment))
+    return Offset;
 
+  return (Offset & ~(Alignment - 1)) + Alignment;
+}

lib/profile/InstrProfilingWriter.c

if (BufferIO->FileWriter(IO, 1, &BufferIO->File))
       return -1;
     BufferIO->CurOffset = 0;
   }
   return 0;
 }
 
 COMPILER_RT_VISIBILITY int lprofWriteData(WriterCallback Writer,
                                           void *WriterCtx,
+                                          uint64_t CounterSectionAlignment,
                                           ValueProfData **ValueDataArray,
                                           const uint64_t ValueDataSize) {
   /* Match logic in __llvm_profile_write_buffer(). */
   const __llvm_profile_data *DataBegin = __llvm_profile_begin_data();
   const __llvm_profile_data *DataEnd = __llvm_profile_end_data();
   const uint64_t *CountersBegin = __llvm_profile_begin_counters();
   const uint64_t *CountersEnd = __llvm_profile_end_counters();
   const char *NamesBegin = __llvm_profile_begin_names();
   const char *NamesEnd = __llvm_profile_end_names();
   return lprofWriteDataImpl(Writer, WriterCtx, DataBegin, DataEnd,
-                            CountersBegin, CountersEnd, ValueDataArray,
-                            ValueDataSize, NamesBegin, NamesEnd);
+                            CountersBegin, CountersEnd, CounterSectionAlignment,
+                            ValueDataArray, ValueDataSize, NamesBegin,
+                            NamesEnd);
 }
 
 #define VP_BUFFER_SIZE 8 * 1024
 static int writeValueProfData(WriterCallback Writer, void *WriterCtx,
                               ValueProfData **ValueDataBegin,
                               uint64_t NumVData) {
   ProfBufferIO *BufferIO;
   uint32_t I = 0, BufferSz;
static int writeValueProfData(WriterCallback Writer, void *WriterCtx,
 
   if (lprofBufferIOFlush(BufferIO) != 0)
     return -1;
   lprofDeleteBufferIO(BufferIO);
 
   return 0;
 }
 
+static int writePadding(WriterCallback Writer, void *WriterCtx,
+                 uint64_t NumPaddingBytes) {
+  uint64_t BytesRemaining = NumPaddingBytes;
+
+  /* Enough zeroes for padding. */
+  const char Zeroes[256] = {0};
+
+  while (BytesRemaining > 0) {
+    uint64_t BytesToWrite =
+        (BytesRemaining < sizeof(Zeroes)) ? BytesRemaining : sizeof(Zeroes);
+    ProfDataIOVec PadIO[] = {{Zeroes, sizeof(uint8_t), BytesToWrite}};
+    if (Writer(PadIO, sizeof(PadIO) / sizeof(ProfDataIOVec), &WriterCtx))
+      return -1;
+    BytesRemaining -= BytesToWrite;
+  }
+
+  return 0;
+}
+
 COMPILER_RT_VISIBILITY int
 lprofWriteDataImpl(WriterCallback Writer, void *WriterCtx,
                    const __llvm_profile_data *DataBegin,
                    const __llvm_profile_data *DataEnd,
                    const uint64_t *CountersBegin, const uint64_t *CountersEnd,
+                   uint64_t CounterSectionAlignment,
                    ValueProfData **ValueDataBegin, const uint64_t ValueDataSize,
                    const char *NamesBegin, const char *NamesEnd) {
 
   /* Calculate size of sections. */
   const uint64_t DataSize = __llvm_profile_get_data_size(DataBegin, DataEnd);
   const uint64_t CountersSize = CountersEnd - CountersBegin;
   const uint64_t NamesSize = NamesEnd - NamesBegin;
   const uint64_t Padding = __llvm_profile_get_num_padding_bytes(NamesSize);
 
-  /* Enough zeroes for padding. */
-  const char Zeroes[sizeof(uint64_t)] = {0};
-
   /* Create the header. */
   __llvm_profile_header Header;
 
   if (!DataSize)
     return 0;
 
   /* Initialize header struture.  */
 #define INSTR_PROF_RAW_HEADER(Type, Name, Init) Header.Name = Init;
 #include "InstrProfData.inc"
 
+  uint64_t DataEndOffset =
+      sizeof(__llvm_profile_header) + (DataSize * sizeof(__llvm_profile_data));
+  uint64_t PaddingBeforeCounters =
+      lprofAlignTo(DataEndOffset, CounterSectionAlignment) - DataEndOffset;
+  uint64_t CountersSizeInBytes = CountersSize * sizeof(uint64_t);
+  uint64_t PaddingAfterCounters =
+      lprofAlignTo(CountersSizeInBytes, CounterSectionAlignment) -
+      CountersSizeInBytes;
+  uint64_t PaddingAfterNames =
+      lprofAlignTo(NamesSize + Padding, CounterSectionAlignment) -
+      (NamesSize + Padding);
+
   /* Write the data. */
-  ProfDataIOVec IOVec[] = {{&Header, sizeof(__llvm_profile_header), 1},
-                           {DataBegin, sizeof(__llvm_profile_data), DataSize},
-                           {CountersBegin, sizeof(uint64_t), CountersSize},
-                           {NamesBegin, sizeof(uint8_t), NamesSize},
-                           {Zeroes, sizeof(uint8_t), Padding}};
-  if (Writer(IOVec, sizeof(IOVec) / sizeof(*IOVec), &WriterCtx))
+  ProfDataIOVec IO1[] = {{&Header, sizeof(__llvm_profile_header), 1},
+                         {DataBegin, sizeof(__llvm_profile_data), DataSize}};
+  if (Writer(IO1, sizeof(IO1) / sizeof(ProfDataIOVec), &WriterCtx))
+    return -1;
+  if (writePadding(Writer, WriterCtx, PaddingBeforeCounters))
+    return -1;
+  ProfDataIOVec IO2[] = {{CountersBegin, sizeof(uint64_t), CountersSize}};
+  if (Writer(IO2, sizeof(IO2) / sizeof(ProfDataIOVec), &WriterCtx))
+    return -1;
+  if (writePadding(Writer, WriterCtx, PaddingAfterCounters))
+    return -1;
+  ProfDataIOVec IO3[] = {{NamesBegin, sizeof(uint8_t), NamesSize}};
+  if (Writer(IO3, sizeof(IO3) / sizeof(ProfDataIOVec), &WriterCtx))
+    return -1;
+  if (writePadding(Writer, WriterCtx, Padding + PaddingAfterNames))
     return -1;
-
   return writeValueProfData(Writer, WriterCtx, ValueDataBegin, DataSize);
 }

test/lit.common.cfg

 import lit.util
 
 # Setup test format. Use bash on Unix and the lit shell on Windows.
 execute_external = (not sys.platform in ['win32'])
 config.test_format = lit.formats.ShTest(execute_external)
 if execute_external:
   config.available_features.add('shell')
 
+config.available_features.add('OS=%s' % platform.system())
+
 # Setup clang binary.
 compiler_path = getattr(config, 'clang', None)
 if (not compiler_path) or (not os.path.exists(compiler_path)):
   lit_config.fatal("Can't find compiler on path %r" % compiler_path)
 
 compiler_id = getattr(config, 'compiler_id', None)
 if compiler_id == "Clang":
   if platform.system() != 'Windows':

test/profile/CMakeLists.txt

 set(PROFILE_LIT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 set(PROFILE_LIT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
 
 set(PROFILE_TESTSUITES)
 set(PROFILE_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS})
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND PROFILE_TEST_DEPS profile llvm-profdata llvm-cov)
 endif()
 
 set(PROFILE_TEST_ARCH ${PROFILE_SUPPORTED_ARCH})
 if(APPLE)
   darwin_filter_host_archs(PROFILE_SUPPORTED_ARCH PROFILE_TEST_ARCH)
 endif()
 
+set(PROFILE_SUPPORTED_MMAP_OPTIONS "0;1")
+
 foreach(arch ${PROFILE_TEST_ARCH})
+  foreach(do_mmap ${PROFILE_SUPPORTED_MMAP_OPTIONS})
     set(PROFILE_TEST_TARGET_ARCH ${arch})
+    set(PROFILE_DO_MMAP ${do_mmap})
     if(${arch} MATCHES "arm|aarch64")
       # This is only true if we're cross-compiling.
       set(PROFILE_TEST_TARGET_CFLAGS ${COMPILER_RT_TEST_COMPILER_CFLAGS})
     else()
       get_target_flags_for_arch(${arch} PROFILE_TEST_TARGET_CFLAGS)
       string(REPLACE ";" " " PROFILE_TEST_TARGET_CFLAGS "${PROFILE_TEST_TARGET_CFLAGS}")
     endif()
-  set(CONFIG_NAME Profile-${arch})
+    set(CONFIG_NAME Profile-${arch}-mmap-${do_mmap})
     configure_lit_site_cfg(
       ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
       ${CMAKE_CURRENT_BINARY_DIR}/${CONFIG_NAME}/lit.site.cfg
      )
     list(APPEND PROFILE_TESTSUITES ${CMAKE_CURRENT_BINARY_DIR}/${CONFIG_NAME})
   endforeach()
+endforeach()
 
 add_lit_testsuite(check-profile "Running the profile tests"
   ${PROFILE_TESTSUITES}
   DEPENDS ${PROFILE_TEST_DEPS})
 set_target_properties(check-profile PROPERTIES FOLDER "Profile tests")

test/profile/Linux/coverage_ctors.cpp

 // RUN: %clangxx_profgen -std=c++11 -fuse-ld=gold -fcoverage-mapping -o %t %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: llvm-cov show %t -instr-profile %t.profdata -filename-equivalence 2>&1 | FileCheck %s
 
 struct Base {
   int B;
   Base() : B(0) {}
   Base(const Base &b2) {
     B = b2.B + 5;

test/profile/Linux/coverage_dtor.cpp

 // RUN: %clang_profgen -x c++ -fno-exceptions  -std=c++11 -fuse-ld=gold -fcoverage-mapping -o %t %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: llvm-cov show %t -instr-profile %t.profdata -filename-equivalence 2>&1 | FileCheck %s
 
 int g = 100;
 struct Base {
   int B;
   Base(int B_) : B(B_) {}
   ~Base() { g -= B; }

test/profile/Linux/coverage_shared.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fcoverage-mapping -c -o %t.d/a.shared.o -fPIC  %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -fcoverage-mapping -o %t.d/a.shared -fPIC -shared %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t-shared -fPIC -rpath %t.d %t.d/a.shared %S/../Inputs/instrprof-dynamic-b.cpp %S/../Inputs/instrprof-dynamic-main.cpp
 
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t-static %t.d/a.shared.o %S/../Inputs/instrprof-dynamic-b.cpp %S/../Inputs/instrprof-dynamic-main.cpp
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-shared.profdata %t-shared.profraw
 
 RUN: llvm-cov show -instr-profile %t-shared.profdata %t.d/a.shared | FileCheck --check-prefix=COV %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: llvm-cov show -instr-profile %t-static.profdata %t-static | FileCheck --check-prefix=COV %S/../Inputs/instrprof-dynamic-a.cpp
 

test/profile/Linux/coverage_test.cpp

 // RUN: %clang_profgen -fuse-ld=gold -O2 -fdata-sections -ffunction-sections -fcoverage-mapping -Wl,--gc-sections -o %t %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: llvm-cov show %t -instr-profile %t.profdata -filename-equivalence 2>&1 | FileCheck %s
 // BFD linker older than 2.26 has a bug that per-func profile data will be wrongly garbage collected when GC is turned on. We only do end-to-end test here without GC:
 // RUN: %clang_profgen -O2  -fcoverage-mapping  -o %t.2 %s
-// RUN: env LLVM_PROFILE_FILE=%t.2.profraw %run %t.2
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.2.profraw %run %t.2
 // RUN: llvm-profdata merge -o %t.2.profdata %t.2.profraw
 // RUN: llvm-cov show %t.2 -instr-profile %t.2.profdata -filename-equivalence 2>&1 | FileCheck %s
 // Check covmap is not garbage collected when GC is turned on with BFD linker. Due to the bug mentioned above, we can only
 // do the check with objdump:
 // RUN: %clang_profgen -O2  -fcoverage-mapping -Wl,--gc-sections -o %t.3 %s
 // RUN: llvm-objdump -h %t.3 | FileCheck --check-prefix COVMAP %s
 // Check PIE option
 // RUN: %clang_profgen -fuse-ld=gold -O2 -fdata-sections -ffunction-sections -fPIE -pie -fcoverage-mapping -Wl,--gc-sections -o %t.pie %s
-// RUN: env LLVM_PROFILE_FILE=%t.pie.profraw %run %t.pie
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.pie.profraw %run %t.pie
 // RUN: llvm-profdata merge -o %t.pie.profdata %t.pie.profraw
 // RUN: llvm-cov show %t.pie -instr-profile %t.pie.profdata -filename-equivalence 2>&1 | FileCheck %s
 
 void foo(bool cond) { // CHECK:  1| [[@LINE]]|void foo(
   if (cond) {         // CHECK:  1| [[@LINE]]|  if (cond) {
   }                   // CHECK:  0| [[@LINE]]|  }
 }                     // CHECK:  1| [[@LINE]]|}
 void bar() {          // CHECK:  1| [[@LINE]]|void bar() {

test/profile/Linux/instrprof-basic.c

 // RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t -O3 %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: %clang_profuse=%t.profdata -o - -S -emit-llvm %s | FileCheck %s
 
 int begin(int i) {
   // CHECK: br i1 %{{.*}}, label %{{.*}}, label %{{.*}}, !prof ![[PD1:[0-9]+]]
   if (i)
     return 0;
   return 1;

test/profile/Linux/instrprof-dlopen.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -o %t.d/func.shared -fPIC -shared -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections  %S/../Inputs/instrprof-dlopen-func.c
 RUN: %clang_profgen -o %t.d/func2.shared -fPIC -shared -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections  %S/../Inputs/instrprof-dlopen-func2.c
 RUN: %clang -o %t-local -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_LOCAL" %S/../Inputs/instrprof-dlopen-main.c
 RUN: %clang -o %t-global -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_GLOBAL" %S/../Inputs/instrprof-dlopen-main.c
 
 RUN: %clang -c -o %t.d/main.o %S/../Inputs/instrprof-dlopen-main.c
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections  -o %t-static %S/../Inputs/instrprof-dlopen-func.c %S/../Inputs/instrprof-dlopen-func2.c %t.d/main.o
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-local.profraw %run %t-local
-RUN: env LLVM_PROFILE_FILE=%t-global.profraw %run %t-global
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-local.profraw %run %t-local
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-global.profraw %run %t-global
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-local.profdata %t-local.profraw
 RUN: llvm-profdata merge -o %t-global.profdata %t-global.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-func.static.ll -S -emit-llvm %S/../Inputs/instrprof-dlopen-func.c
 RUN: %clang_profuse=%t-local.profdata -o %t-func.local.ll -S -emit-llvm %S/../Inputs/instrprof-dlopen-func.c
 RUN: %clang_profuse=%t-global.profdata -o %t-func.global.ll -S -emit-llvm %S/../Inputs/instrprof-dlopen-func.c

test/profile/Linux/instrprof-dynamic-one-shared.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t.d/a.shared -fPIC -shared %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t-shared -fPIC -rpath %t.d %t.d/a.shared %S/../Inputs/instrprof-dynamic-b.cpp %S/../Inputs/instrprof-dynamic-main.cpp
 
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t-static %S/../Inputs/instrprof-dynamic-a.cpp %S/../Inputs/instrprof-dynamic-b.cpp %S/../Inputs/instrprof-dynamic-main.cpp
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-shared.profdata %t-shared.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-a.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-a.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: diff %t-a.static.ll %t-a.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-b.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-b.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-b.cpp
 RUN: diff %t-b.static.ll %t-b.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-main.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-main.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-main.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-main.cpp
 RUN: diff %t-main.static.ll %t-main.shared.ll

test/profile/Linux/instrprof-dynamic-two-shared.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t.d/a.shared -fPIC -shared %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t.d/b.shared -fPIC -shared %S/../Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profgen -fdata-sections -ffunction-sections -fuse-ld=gold -Wl,--gc-sections -o %t-shared -fPIC -rpath %t.d %t.d/a.shared %t.d/b.shared %S/../Inputs/instrprof-dynamic-main.cpp
 
 RUN: %clang_profgen -o %t-static %S/../Inputs/instrprof-dynamic-a.cpp %S/../Inputs/instrprof-dynamic-b.cpp %S/../Inputs/instrprof-dynamic-main.cpp
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-shared.profdata %t-shared.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-a.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-a.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-a.cpp
 RUN: diff %t-a.static.ll %t-a.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-b.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-b.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-b.cpp
 RUN: diff %t-b.static.ll %t-b.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-main.static.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-main.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-main.shared.ll -S -emit-llvm %S/../Inputs/instrprof-dynamic-main.cpp
 RUN: diff %t-main.static.ll %t-main.shared.ll

test/profile/gcc-flag-compatibility.test

 RUN: rm -rf %t.d
 RUN: mkdir -p %t.d
 RUN: %clang_profgen_gcc=%t.d/d1/d2 -o %t.d/code %S/Inputs/gcc-flag-compatibility.c
 
 # Test that the instrumented code writes to %t.d/d1/d2/default.profraw
-RUN: %run %t.d/code
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap %run %t.d/code
 RUN: llvm-profdata merge -o %t.profdata %t.d/d1/d2/default.profraw
 
 # Test that we can override the directory and file name with LLVM_PROFILE_FILE.
-RUN: env LLVM_PROFILE_FILE=%t.d/x1/prof.raw %run %t.d/code
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.d/x1/prof.raw %run %t.d/code
 RUN: llvm-profdata merge -o %t.profdata %t.d/x1/prof.raw
 
 # Test that we can specify a directory with -fprofile-use.
 RUN: llvm-profdata merge -o %t.d/default.profdata %t.d/x1/prof.raw
 RUN: %clang_profuse_gcc=%t.d -o %t.d/code %S/Inputs/gcc-flag-compatibility.c
 
 # Test that we can specify a file with -fprofile-use.
 RUN: %clang_profuse_gcc=%t.profdata -o %t.d/code %S/Inputs/gcc-flag-compatibility.c

test/profile/instrprof-basic.c

 // RUN: %clang_profgen -o %t -O3 %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: %clang_profuse=%t.profdata -o - -S -emit-llvm %s | FileCheck %s
 
 int begin(int i) {
   // CHECK: br i1 %{{.*}}, label %{{.*}}, label %{{.*}}, !prof ![[PD1:[0-9]+]]
   if (i)
     return 0;
   return 1;

test/profile/instrprof-bufferio.c

 // RUN: %clang_profgen -O3 -o %t %s
-// RUN: %run %t %t.out.1 %t.out.2 %t.out.3 %t.out.4
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap %run %t %t.out.1 %t.out.2 %t.out.3 %t.out.4
 // RUN: cat %t.out.1 | FileCheck %s
 // RUN: diff %t.out.1 %t.out.2
 // RUN: diff %t.out.2 %t.out.3
 // RUN: diff %t.out.3 %t.out.4
 
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>

test/profile/instrprof-dlopen.test

 RUN: mkdir -p %t.d
-RUN: %clang_profgen -o %t.d/func.shared -fPIC -shared %S/Inputs/instrprof-dlopen-func.c
-RUN: %clang_profgen -o %t.d/func2.shared -fPIC -shared %S/Inputs/instrprof-dlopen-func2.c
-RUN: %clang -o %t-local -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_LOCAL" %S/Inputs/instrprof-dlopen-main.c
-RUN: %clang -o %t-global -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_GLOBAL" %S/Inputs/instrprof-dlopen-main.c
-
-RUN: %clang -c -o %t.d/main.o %S/Inputs/instrprof-dlopen-main.c
-RUN: %clang_profgen -o %t-static %S/Inputs/instrprof-dlopen-func.c %S/Inputs/instrprof-dlopen-func2.c %t.d/main.o
-
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-local.profraw %run %t-local
-RUN: env LLVM_PROFILE_FILE=%t-global.profraw %run %t-global
+RUN: %clang_profgen -g -o %t.d/func.shared -fPIC -shared %S/Inputs/instrprof-dlopen-func.c
+RUN: %clang_profgen -g -o %t.d/func2.shared -fPIC -shared %S/Inputs/instrprof-dlopen-func2.c
+RUN: %clang -g -o %t-local -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_LOCAL" %S/Inputs/instrprof-dlopen-main.c
+RUN: %clang -g -o %t-global -fPIC -DDLOPEN_FUNC_DIR=\"%t.d\" -DDLOPEN_FLAGS="RTLD_LAZY | RTLD_GLOBAL" %S/Inputs/instrprof-dlopen-main.c
+
+RUN: %clang -g -c -o %t.d/main.o %S/Inputs/instrprof-dlopen-main.c
+RUN: %clang_profgen -g -o %t-static %S/Inputs/instrprof-dlopen-func.c %S/Inputs/instrprof-dlopen-func2.c %t.d/main.o
+
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-local.profraw %run %t-local
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-global.profraw %run %t-global
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-local.profdata %t-local.profraw
 RUN: llvm-profdata merge -o %t-global.profdata %t-global.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-func.static.ll -S -emit-llvm %S/Inputs/instrprof-dlopen-func.c
 RUN: %clang_profuse=%t-local.profdata -o %t-func.local.ll -S -emit-llvm %S/Inputs/instrprof-dlopen-func.c
 RUN: %clang_profuse=%t-global.profdata -o %t-func.global.ll -S -emit-llvm %S/Inputs/instrprof-dlopen-func.c

test/profile/instrprof-dynamic-one-shared.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -o %t.d/a.shared -fPIC -shared %S/Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -o %t-shared -fPIC -rpath %t.d %t.d/a.shared %S/Inputs/instrprof-dynamic-b.cpp %S/Inputs/instrprof-dynamic-main.cpp
 
 RUN: %clang_profgen -o %t-static %S/Inputs/instrprof-dynamic-a.cpp %S/Inputs/instrprof-dynamic-b.cpp %S/Inputs/instrprof-dynamic-main.cpp
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-shared.profdata %t-shared.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-a.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-a.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-a.cpp
 RUN: diff %t-a.static.ll %t-a.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-b.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-b.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-b.cpp
 RUN: diff %t-b.static.ll %t-b.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-main.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-main.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-main.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-main.cpp
 RUN: diff %t-main.static.ll %t-main.shared.ll

test/profile/instrprof-dynamic-two-shared.test

 RUN: mkdir -p %t.d
 RUN: %clang_profgen -o %t.d/a.shared -fPIC -shared %S/Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profgen -o %t.d/b.shared -fPIC -shared %S/Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profgen -o %t-shared -fPIC -rpath %t.d %t.d/a.shared %t.d/b.shared %S/Inputs/instrprof-dynamic-main.cpp
 
 RUN: %clang_profgen -o %t-static %S/Inputs/instrprof-dynamic-a.cpp %S/Inputs/instrprof-dynamic-b.cpp %S/Inputs/instrprof-dynamic-main.cpp
 
-RUN: env LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
-RUN: env LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-static.profraw %run %t-static
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-shared.profraw %run %t-shared
 
 RUN: llvm-profdata merge -o %t-static.profdata %t-static.profraw
 RUN: llvm-profdata merge -o %t-shared.profdata %t-shared.profraw
 
 RUN: %clang_profuse=%t-static.profdata -o %t-a.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-a.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-a.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-a.cpp
 RUN: diff %t-a.static.ll %t-a.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-b.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-b.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-b.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-b.cpp
 RUN: diff %t-b.static.ll %t-b.shared.ll
 
 RUN: %clang_profuse=%t-static.profdata -o %t-main.static.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-main.cpp
 RUN: %clang_profuse=%t-shared.profdata -o %t-main.shared.ll -S -emit-llvm %S/Inputs/instrprof-dynamic-main.cpp
 RUN: diff %t-main.static.ll %t-main.shared.ll

test/profile/instrprof-error.c

 // RUN: %clang_profgen -o %t -O3 %s
-// RUN: env LLVM_PROFILE_FILE=%t/ LLVM_PROFILE_VERBOSE_ERRORS=1 %run %t 1 2>&1 | FileCheck %s
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t/ LLVM_PROFILE_VERBOSE_ERRORS=1 %run %t 1 2>&1 | FileCheck %s
 
 int main(int argc, const char *argv[]) {
   if (argc < 2)
     return 1;
   return 0;
 }
 // CHECK: LLVM Profile: Failed to write file 

test/profile/instrprof-hostname.c

 // RUN: %clang_profgen -o %t -O3 %s
-// RUN: env LLVM_PROFILE_FILE=%h.%t-%h.profraw_%h %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%h.%t-%h.profraw_%h %run %t
 // RUN: llvm-profdata merge -o %t.profdata `uname -n`.%t-`uname -n`.profraw_`uname -n`
 // RUN: %clang_profuse=%t.profdata -o - -S -emit-llvm %s | FileCheck %s
 // REQUIRES: shell
 
 int main(int argc, const char *argv[]) {
   // CHECK: br i1 %{{.*}}, label %{{.*}}, label %{{.*}}, !prof ![[PD1:[0-9]+]]
   if (argc > 2)
     return 1;
   return 0;
 }
 // CHECK: ![[PD1]] = !{!"branch_weights", i32 1, i32 2}

test/profile/instrprof-merge-match.test

 // RUN: mkdir -p %t.d
 // RUN: %clang_profgen  -o %t.d/libt.so -fPIC -shared %S/Inputs/instrprof-merge-match-lib.c
 // RUN: %clang_profgen  -o %t -L %t.d -rpath %t.d  %S/Inputs/instrprof-merge-match.c -lt
-// RUN: %run %t
+// RUN: LLVM_PROFILE_ENABLE_MMAP=%do_mmap %run %t
 

test/profile/instrprof-merge.c

 // RUN: %clang_profgen -O2 -o %t %s
-// RUN: %run %t %t.profraw 1 1
+// RUN: LLVM_PROFILE_ENABLE_MMAP=%do_mmap %run %t %t.profraw 1 1
 // RUN: llvm-profdata show --all-functions --counts %t.profraw  | FileCheck %s
 
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
 
 int __llvm_profile_runtime = 0;
 uint64_t __llvm_profile_get_size_for_buffer(void);

test/profile/instrprof-mmap-darwin.c

+// Darwin-specific test for mmap'd counter sections.
+//
+// REQUIRES: OS=Darwin
+// RUN: %clang_profgen -g -o %t %s
+// RUN: %run %t %t.profraw.mmap %t.profraw.atexit
+// RUN: llvm-profdata show --function=foo %t.profraw.mmap | FileCheck %s
+// RUN: llvm-profdata show --function=foo %t.profraw.atexit | FileCheck %s
+
+extern void __llvm_profile_set_filename(const char *Name);
+extern int __llvm_profile_register_mmap(void);
+
+// CHECK: foo
+void foo() {}
+
+int main(int argc, char **argv) {
+  if (argc != 3)
+    return -1;
+
+  __llvm_profile_set_filename(argv[1]);
+
+  if (__llvm_profile_register_mmap())
+    return -1;
+
+  __llvm_profile_set_filename(argv[2]);
+
+  foo();
+
+  return 0;
+}

test/profile/instrprof-reset-counters.c

 // RUN: %clang_profgen -o %t -O3 %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: %clang_profuse=%t.profdata -o - -S -emit-llvm %s | FileCheck %s
 
 void __llvm_profile_reset_counters(void);
 void foo(int);
 int main(void) {
   foo(0);
   __llvm_profile_reset_counters();

test/profile/instrprof-shared.test

 RUN: %clang -o %t-no-instr1-instr -L%t.d -rpath %t.d -lt-instr  %S/Inputs/instrprof-shared-main.c
 RUN: %clang -o %t-no-instr1-no-instr1 -L%t.d -rpath %t.d -lt-no-instr1  %S/Inputs/instrprof-shared-main.c
 RUN: %clang -o %t-no-instr1-no-instr2 -L%t.d -rpath %t.d -lt-no-instr2  %S/Inputs/instrprof-shared-main.c
 RUN: %clang -c -o %t.d/instrprof-shared-main-no-instr2.o  %S/Inputs/instrprof-shared-main.c
 RUN: %clang -o %t-no-instr2-instr -L%t.d -rpath %t.d -lt-instr  %t.d/instrprof-shared-main-no-instr2.o
 RUN: %clang -o %t-no-instr2-no-instr1 -L%t.d -rpath %t.d -lt-no-instr1  %t.d/instrprof-shared-main-no-instr2.o
 RUN: %clang -o %t-no-instr2-no-instr2 -L%t.d -rpath %t.d -lt-no-instr2  %t.d/instrprof-shared-main-no-instr2.o
 
-RUN: env LLVM_PROFILE_FILE=%t-instr-instr.profraw %run %t-instr-instr
-RUN: env LLVM_PROFILE_FILE=%t-instr-no-instr1.profraw %run %t-instr-no-instr1
-RUN: env LLVM_PROFILE_FILE=%t-instr-no-instr2.profraw %run %t-instr-no-instr2
-RUN: env LLVM_PROFILE_FILE=%t-no-instr1-instr.profraw %run %t-no-instr1-instr
-RUN: env LLVM_PROFILE_FILE=%t-no-instr2-instr.profraw %run %t-no-instr2-instr
-RUN: env LLVM_PROFILE_FILE=%t-no-instr1-no-instr1.profraw %run %t-no-instr1-no-instr1
-RUN: env LLVM_PROFILE_FILE=%t-no-instr1-no-instr2.profraw %run %t-no-instr1-no-instr2
-RUN: env LLVM_PROFILE_FILE=%t-no-instr2-no-instr1.profraw %run %t-no-instr2-no-instr1
-RUN: env LLVM_PROFILE_FILE=%t-no-instr2-no-instr2.profraw %run %t-no-instr2-no-instr2
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-instr-instr.profraw %run %t-instr-instr
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-instr-no-instr1.profraw %run %t-instr-no-instr1
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-instr-no-instr2.profraw %run %t-instr-no-instr2
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr1-instr.profraw %run %t-no-instr1-instr
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr2-instr.profraw %run %t-no-instr2-instr
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr1-no-instr1.profraw %run %t-no-instr1-no-instr1
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr1-no-instr2.profraw %run %t-no-instr1-no-instr2
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr2-no-instr1.profraw %run %t-no-instr2-no-instr1
+RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t-no-instr2-no-instr2.profraw %run %t-no-instr2-no-instr2
 
 RUN: llvm-profdata merge -o %t-instr-instr.profdata %t-instr-instr.profraw
 RUN: llvm-profdata merge -o %t-instr-no-instr1.profdata %t-instr-no-instr1.profraw
 RUN: llvm-profdata merge -o %t-instr-no-instr2.profdata %t-instr-no-instr2.profraw
 RUN: llvm-profdata merge -o %t-no-instr1-instr.profdata %t-no-instr1-instr.profraw
 RUN: llvm-profdata merge -o %t-no-instr2-instr.profdata %t-no-instr2-instr.profraw
 
 RUN: not llvm-profdata merge -o %t-no-instr1-no-instr1.profdata %t-no-instr1-no-instr1.profraw 2>&1 | FileCheck %s --check-prefix=MISSING-FILE

test/profile/instrprof-visibility.cpp

 // RUN: %clangxx_profgen -fcoverage-mapping %S/Inputs/instrprof-visibility-helper.cpp -o %t %s
-// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t
+// RUN: env LLVM_PROFILE_ENABLE_MMAP=%do_mmap LLVM_PROFILE_FILE=%t.profraw %run %t
 // RUN: llvm-profdata merge %t.profraw -o %t.profdata
 // RUN: llvm-profdata show --all-functions %t.profraw | FileCheck %s --check-prefix=PROFILE
 // RUN: llvm-cov show %t -instr-profile=%t.profdata | FileCheck %s --check-prefix=COV
 
 namespace {
 #define NO_WEAK
 #define NO_EXTERN
 #include "instrprof-visibility-kinds.inc"

test/profile/instrprof-without-libc.c

 // RUN: %clang_profgen -DCHECK_SYMBOLS -O3 -o %t.symbols %s
 // RUN: llvm-nm %t.symbols | FileCheck %s --check-prefix=CHECK-SYMBOLS
 // RUN: %clang_profgen -O3 -o %t %s
-// RUN: %run %t %t.profraw
+// RUN: LLVM_PROFILE_ENABLE_MMAP=%do_mmap %run %t %t.profraw
 // RUN: llvm-profdata merge -o %t.profdata %t.profraw
 // RUN: %clang_profuse=%t.profdata -o - -S -emit-llvm %s | FileCheck %s
 
 #include <stdint.h>
 #include <stdlib.h>
 
 #ifndef CHECK_SYMBOLS
 #include <stdio.h>

test/profile/lit.cfg

 # -*- Python -*-
 
 import os
 
 def get_required_attr(config, attr_name):
   attr_value = getattr(config, attr_name, None)
   if attr_value == None:
     lit_config.fatal(
       "No attribute %r in test configuration! You may need to run "
       "tests from your build directory or add this attribute "
       "to lit.site.cfg " % attr_name)
   return attr_value
 
 # Setup config name.
-config.name = 'Profile-' + config.target_arch
+config.name = '-'.join(('Profile', config.target_arch, 'mmap', config.do_mmap))
 
 # Setup source root.
 config.test_source_root = os.path.dirname(__file__)
 
 # Setup executable root.
 if hasattr(config, 'profile_lit_binary_dir') and \
         config.profile_lit_binary_dir is not None:
     config.test_exec_root = os.path.join(config.profile_lit_binary_dir, config.name)

 config.substitutions.append( ("%clang ", build_invocation(clang_cflags)) )
 config.substitutions.append( ("%clangxx ", build_invocation(clang_cxxflags)) )
 config.substitutions.append( ("%clang_profgen ", build_invocation(clang_cflags) + " -fprofile-instr-generate ") )
 config.substitutions.append( ("%clang_profuse=", build_invocation(clang_cflags) + " -fprofile-instr-use=") )
 config.substitutions.append( ("%clangxx_profgen ", build_invocation(clang_cxxflags) + " -fprofile-instr-generate ") )
 config.substitutions.append( ("%clangxx_profuse=", build_invocation(clang_cxxflags) + " -fprofile-instr-use=") )
 config.substitutions.append( ("%clang_profgen_gcc=", build_invocation(clang_cflags) + " -fprofile-generate=") )
 config.substitutions.append( ("%clang_profuse_gcc=", build_invocation(clang_cflags) + " -fprofile-use=") )
+config.substitutions.append( ("%do_mmap", config.do_mmap) )
 
 if config.host_os not in ['Darwin', 'FreeBSD', 'Linux']:
   config.unsupported = True
 
 if config.target_arch in ['armv7l']:
   config.unsupported = True

test/profile/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 # Tool-specific config options.
 config.profile_lit_binary_dir = "@PROFILE_LIT_BINARY_DIR@"
 config.target_cflags = "@PROFILE_TEST_TARGET_CFLAGS@"
 config.target_arch = "@PROFILE_TEST_TARGET_ARCH@"
+config.do_mmap = "@PROFILE_DO_MMAP@"
 
 # Load common config for all compiler-rt lit tests.
 lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
 
 # Load tool-specific config that would do the real work.
 lit_config.load_config(config, "@PROFILE_LIT_SOURCE_DIR@/lit.cfg")