[PGO] Suspend SIGKILL for PR_SET_PDEATHSIG in profile-write

[xur]

Summary

We found a nondeterministic behavior when doing online profile merging
for multi-process applications. The application forks a sub-process and
sub-process sets to get SIGKILL when the parent process exits,

The first process gets the lock, and dumps the profile. The second one
will mmap the file, do the merge and write out the file. Note that before
the merged write, we truncate the profile.

Depending on the timing, the child process might be terminated
abnormally when the parent exits first. If this happens:
(1) before the truncation, we will get the profile for the main process
(2) after the truncation, and before write-out the profile, we will get
0 size profile.
(3) after the merged write, we get merged profile.

This patch temporarily suspend the SIGKILL for PR_SET_PDEATHSIG
before profile-write and restore it after the write.

This patch only applies to Linux system.

[davidxl]

lib/profile/InstrProfilingFile.c

560

Put the platform dependent utility into InstProfilingUtil.c file.

void lprofSuspendSigKill(int *pdeathsig) {
#ifdef __linux__
...
#else 
#endif
}

void lprofRestoreSigKill(int pdeathsig) {
..
}

[xur]

move platform depended code to InstrProfilingUnil.c per David's suggestion.

[davidxl]

Can you create a test case? It can be put under the Linux subdir.

[xur]

Add a testcase

sing a fake count section to increase the profile size.
Main process gets to profile run-time first and dump the profile.
It will then exit when child process writes the profile.
Without this patch, there is a high chance of getting 0 size profile.

[davidxl]

test/profile/Linux/prctl.c

17 ↗

(On Diff #88595)

format the test in llvm style.

23 ↗

(On Diff #88595)

50 us or microsec

33 ↗

(On Diff #88595)

10us

40 ↗

(On Diff #88595)

Just check size of profile data? It is not guaranteed that child process will have a chance to dump.

[xur]

Integrated David's review comments.

An addition change is the adding of 'sleep 5' b/w run and llvm-profdata
because if we run llvm-profdata immediately after. It's possible to get a
0-size files as it's being truncated by the child process (ie. the main
process exit while the child process is still accessing the file).

[davidxl]

lgtm

test/profile/Linux/prctl.c

3 ↗

(On Diff #88616)

sleep 5 seconds can be a problem -- it greatly increases the total runtime of check-profile. Please tune this down to 1 second (by reducing the fake data size).

lib/profile/InstrProfilingFile.c

 /* For _chsize_s */
 #include <io.h>
 #else
 #include <sys/file.h>
 #include <sys/mman.h>
 #include <unistd.h>
 #if defined(__linux__)
 #include <sys/types.h>
+#include <signal.h>
+#include <sys/prctl.h>
 #endif
 #endif
 
 /* From where is profile name specified.
  * The order the enumerators define their
  * precedence. Re-order them may lead to
  * runtime behavior change. */ 
 typedef enum ProfileNameSpecifier {

  * set by previous calls to __llvm_profile_set_filename or
  * __llvm_profile_override_default_filename or
  * __llvm_profile_initialize_file. */
 COMPILER_RT_VISIBILITY
 int __llvm_profile_write_file(void) {
   int rc, Length;
   const char *Filename;
   char *FilenameBuf;
+#if defined(__linux__)
+   int PDeachSig = 0;
+#endif
 
   if (lprofProfileDumped()) {
     PROF_NOTE("Profile data not written to file: %s.\n", 
               "already written");
     return 0;
   }
 
   Length = getCurFilenameLength();
#endif
   if (GET_VERSION(__llvm_profile_get_version()) != INSTR_PROF_RAW_VERSION) {
     PROF_ERR("Runtime and instrumentation version mismatch : "
              "expected %d, but get %d\n",
              INSTR_PROF_RAW_VERSION,
              (int)GET_VERSION(__llvm_profile_get_version()));
     return -1;
   }
 
+#if defined(__linux__)
+  // Temporarily suspend getting SIGKILL when the parent exits.
+  if (prctl(PR_GET_PDEATHSIG, &PDeachSig) == 0 && PDeachSig == SIGKILL)
+    prctl(PR_SET_PDEATHSIG, 0);
+#endif
+
   /* Write profile data to the file. */
   rc = writeFile(Filename);
   if (rc)
     PROF_ERR("Failed to write file \"%s\": %s\n", Filename, strerror(errno));
+
+#if defined(__linux__)
+  // Restore PR_SET_PDEATHSIG.
+  if (PDeachSig == SIGKILL)
+    prctl(PR_SET_PDEATHSIG, SIGKILL);
+#endif
   return rc;
 }
 
 COMPILER_RT_VISIBILITY
 int __llvm_profile_dump(void) {
   if (!doMerging())
     PROF_WARN("Later invocation of __llvm_profile_dump can lead to clobbering "
               " of previously dumped profile data : %s. Either use %%m "