[profile] Enable on Solaris

[ro]

Summary

The test/asan/TestCases/asan_and_llvm_coverage_test.cc failed to link on Solaris
since libclang_rt.profile was missing. Instead of trying to document that requirement
in the test, I just tried to enable the library on Solaris instead.

Most changes are probably straightforward, but two bear explanation:

• The getpid prototype in lib/profile/InstrProfilingFile.c conflicted with the system one (returning pid_t instead of instead), so I removed it.

• While parts of the profile library prefer fcntl(F_SETLCKW) over flock(LOCK_EX) if available, others unconditionally use flock. flock just doesn't exist on Solaris, so I moved the locking to common code.

[vsk]

lib/profile/GCDAProfiling.c

39

Why not migrate O_BINARY to the common header as well?

lib/profile/InstrProfilingUtil.c

116

Is flock available on Windows? If not, please handle that case by reporting a warning, and document the limitation near the header declarations of lprof(Un)lockFd.

134

Please clang-format your changes.

[ro]

lib/profile/GCDAProfiling.c

39

Looking closer, I see that both MAP_FILE and O_BINARY
are problematic: InstrProfilingPort.h and/or InstrProfiling.h
are included before system headers, but such fallback definitions
need to come after those had a chance to provide the real
definitions. To fix this, either the system header includes
need to be moved to the InstProfiling{Port,}.h files or those
need to be included after system headers. WDYT?

lib/profile/InstrProfilingUtil.c

116

I guess so: even before my patch, it was used unconditionally
in GCDAProfiling.c which is common code.

134

Will do: I missed this because formatting errors in may
sanitizer patches were pointed out during the build, so I assumed
this applied to all of compiler-rt (or even llvm).

[vsk]

lib/profile/GCDAProfiling.c

39

Moving the InstrProfiling*.h imports after the system header imports sgtm, but please add a short comment there explaining why we're relying on that sort of textual #include (it goes against typical llvm practice).

[ro]

I've moved the InstrProfiling*.h after the system headers now (without a comment
yet because it's unclear where best to put it), but there are reasons to also include them
early:

COMPILER_RT_HAS_UNAME was defined in InstrProfilingPort.h and used to control
whether or not to include <sys/utsname.h>. When InstrProfilingPort.h is included later,
this won't work, so I moved the check for <sys/utsname.h> to CMakeLists.txt.
Initially, this didn't work as expected: I tried check_include_file. Although checking for
a header could be a compile-only test, that function tries to link, which fails since
cmake/config-ix.cmake adds -nodefaultlibs to CMAKE_REQUIRED_FLAGS, so both
the startup files and libc are missing and the link fails.

Besides, I now print getpid() as long (which it can be on Solaris), avoiding a warning.

uname(2) is documented in POSIX.1-2008 to be non-negative on success (and
Solaris returns 1), while InstrProfilingUtil.c (lprofGetHostName) only expects the
0 return value found on Linux.

Last but not least, I found that I'd overlooked to enable the profile tests. With
a version of Solaris ld that supports the non-standard GNU ld extension of adding
start_SECNAME and stop_SECNAME labels to sections whose names are valid
as C identifiers and an LLVM companion patch to enable that (https://reviews.llvm.org/D41111),
all tests pass.

[vsk]

Thanks! LGTM with a nit -- please add the comment we discussed in InstrProfilingPort.h.

lib/profile/InstrProfilingPort.h

9

This is a good place to add a comment stating that this header must be textually #included before all others.

lib/profile/InstrProfilingUtil.c

91

Nice catch.

[ro]

lib/profile/InstrProfilingPort.h

9

Included after all the others, actually :-)

[ro]

Added comment.

Could someone please commit the patch for me? Thanks.

cmake/config-ix.cmake

 if (COMPILER_RT_HAS_SANITIZER_COMMON AND MSAN_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Linux|NetBSD")
   set(COMPILER_RT_HAS_MSAN TRUE)
 else()
   set(COMPILER_RT_HAS_MSAN FALSE)
 endif()
 
 if (PROFILE_SUPPORTED_ARCH AND NOT LLVM_USE_SANITIZER AND
-    OS_NAME MATCHES "Darwin|Linux|FreeBSD|Windows|Android")
+    OS_NAME MATCHES "Darwin|Linux|FreeBSD|Windows|Android|SunOS")
   set(COMPILER_RT_HAS_PROFILE TRUE)
 else()
   set(COMPILER_RT_HAS_PROFILE FALSE)
 endif()
 
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND TSAN_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Darwin|Linux|FreeBSD|Android|NetBSD")
   set(COMPILER_RT_HAS_TSAN TRUE)

lib/profile/GCDAProfiling.c

 #include <stdlib.h>
 #include <string.h>
 
 #if defined(_WIN32)
 #include "WindowsMMap.h"
 #else
 #include <sys/mman.h>
 #include <sys/file.h>
-#ifndef MAP_FILE
-#define MAP_FILE 0
-#endif
 #ifndef O_BINARY
 #define O_BINARY 0
 #endif
 #endif
 
 #if defined(__FreeBSD__) && defined(__i386__)
 #define I386_FREEBSD 1
 #else
 #define I386_FREEBSD 0
 #endif
 
if (fd == -1) {
       }
     }
   }
 
   /* Try to flock the file to serialize concurrent processes writing out to the
    * same GCDA. This can fail if the filesystem doesn't support it, but in that
    * case we'll just carry on with the old racy behaviour and hope for the best.
    */
-  flock(fd, LOCK_EX);
+  lprofLockFd(fd);
   output_file = fdopen(fd, mode);
 
   /* Initialize the write buffer. */
   write_buffer = NULL;
   cur_buffer_size = 0;
   cur_pos = 0;
 
   if (new_file) {
if (output_file) {
 
     if (new_file) {
       fwrite(write_buffer, cur_pos, 1, output_file);
       free(write_buffer);
     } else {
       unmap_file();
     }
 
-    flock(fd, LOCK_UN);
+    lprofUnlockFd(fd);
     fclose(output_file);
     output_file = NULL;
     write_buffer = NULL;
   }
   free(filename);
 
 #ifdef DEBUG_GCDAPROFILING
   fprintf(stderr, "llvmgcda: -----\n");

lib/profile/InstrProfilingFile.c

typedef struct lprofFilename {
    * can only appear once at the end of the name pattern. */
   unsigned MergePoolSize;
   ProfileNameSpecifier PNS;
 } lprofFilename;
 
 COMPILER_RT_WEAK lprofFilename lprofCurFilename = {0, 0, 0, {0}, {0},
                                                    0, 0, 0, PNS_unknown};
 
-int getpid(void);
 static int getCurFilenameLength();
 static const char *getCurFilename(char *FilenameBuf);
 static unsigned doMerging() { return lprofCurFilename.MergePoolSize; }
 
 /* Return 1 if there is an error, otherwise return  0.  */
 static uint32_t fileWriter(ProfDataWriter *This, ProfDataIOVec *IOVecs,
                            uint32_t NumIOVecs) {
   uint32_t I;

lib/profile/InstrProfilingPlatformLinux.c

 /*===- InstrProfilingPlatformLinux.c - Profile data Linux platform ------===*\
 |*
 |*                     The LLVM Compiler Infrastructure
 |*
 |* This file is distributed under the University of Illinois Open Source
 |* License. See LICENSE.TXT for details.
 |*
 \*===----------------------------------------------------------------------===*/
 
 #include "InstrProfiling.h"
 
-#if defined(__linux__) || defined(__FreeBSD__)
+#if defined(__linux__) || defined(__FreeBSD__) || (defined(__sun__) && defined(__svr4__))
 #include <stdlib.h>
 
 #define PROF_DATA_START INSTR_PROF_SECT_START(INSTR_PROF_DATA_SECT_NAME)
 #define PROF_DATA_STOP INSTR_PROF_SECT_STOP(INSTR_PROF_DATA_SECT_NAME)
 #define PROF_NAME_START INSTR_PROF_SECT_START(INSTR_PROF_NAME_SECT_NAME)
 #define PROF_NAME_STOP INSTR_PROF_SECT_STOP(INSTR_PROF_NAME_SECT_NAME)
 #define PROF_CNTS_START INSTR_PROF_SECT_START(INSTR_PROF_CNTS_SECT_NAME)
 #define PROF_CNTS_STOP INSTR_PROF_SECT_STOP(INSTR_PROF_CNTS_SECT_NAME)

lib/profile/InstrProfilingPlatformOther.c

 /*===- InstrProfilingPlatformOther.c - Profile data default platform ------===*\
 |*
 |*                     The LLVM Compiler Infrastructure
 |*
 |* This file is distributed under the University of Illinois Open Source
 |* License. See LICENSE.TXT for details.
 |*
 \*===----------------------------------------------------------------------===*/
 
 #include "InstrProfiling.h"
 
-#if !defined(__APPLE__) && !defined(__linux__) && !defined(__FreeBSD__)
+#if !defined(__APPLE__) && !defined(__linux__) && !defined(__FreeBSD__) && !(defined(__sun__) && defined(__svr4__))
 
 #include <stdlib.h>
 
 static const __llvm_profile_data *DataFirst = NULL;
 static const __llvm_profile_data *DataLast = NULL;
 static const char *NamesFirst = NULL;
 static const char *NamesLast = NULL;
 static uint64_t *CountersFirst = NULL;

lib/profile/InstrProfilingPort.h

 /*===- InstrProfilingPort.h- Support library for PGO instrumentation ------===*\
 |*
 |*                     The LLVM Compiler Infrastructure
 |*
 |* This file is distributed under the University of Illinois Open Source
 |* License. See LICENSE.TXT for details.
 |*
 \*===----------------------------------------------------------------------===*/
 
 #ifndef PROFILE_INSTRPROFILING_PORT_H_
 #define PROFILE_INSTRPROFILING_PORT_H_
 
 #ifdef _MSC_VER
 #define COMPILER_RT_ALIGNAS(x) __declspec(align(x))
 #define COMPILER_RT_VISIBILITY
 /* FIXME: selectany does not have the same semantics as weak. */
 #define COMPILER_RT_WEAK __declspec(selectany)
#define PROF_ERR(Format, ...)                                                  \
   fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
 
 #define PROF_WARN(Format, ...)                                                 \
   fprintf(stderr, "LLVM Profile Warning: " Format, __VA_ARGS__);
 
 #define PROF_NOTE(Format, ...)                                                 \
   fprintf(stderr, "LLVM Profile Note: " Format, __VA_ARGS__);
 
+#ifndef MAP_FILE
+#define MAP_FILE 0
+#endif
+
 #if defined(__FreeBSD__)
 
 #include <inttypes.h>
 #include <sys/types.h>
 
 #else /* defined(__FreeBSD__) */
 
 #include <inttypes.h>
 #include <stdint.h>
 
 #endif /* defined(__FreeBSD__) && defined(__i386__) */
 
 #endif /* PROFILE_INSTRPROFILING_PORT_H_ */

lib/profile/InstrProfilingUtil.h

 #define PROFILE_INSTRPROFILINGUTIL_H
 
 #include <stddef.h>
 #include <stdio.h>
 
 /*! \brief Create a directory tree. */
 void __llvm_profile_recursive_mkdir(char *Pathname);
 
+int lprofLockFd(int fd);
+int lprofUnlockFd(int fd);
+
 /*! Open file \c Filename for read+write with write
  * lock for exclusive access. The caller will block
  * if the lock is already held by another process. */
 FILE *lprofOpenFileEx(const char *Filename);
 /* PS4 doesn't have getenv. Define a shim. */
 #if __ORBIS__
 static inline char *getenv(const char *name) { return NULL; }
 #endif /* #if __ORBIS__ */

lib/profile/InstrProfilingUtil.c

if (WideCharToMultiByte(CP_UTF8, 0, Buffer, -1, Name, Len, NULL, NULL) == 0)
     return -1;
   return 0;
 }
 #elif defined(COMPILER_RT_HAS_UNAME)
 COMPILER_RT_VISIBILITY int lprofGetHostName(char *Name, int Len) {
   struct utsname N;
   int R;
   if (!(R = uname(&N)))
     strncpy(Name, N.nodename, Len);
   return R;
 }
 #endif
 
-COMPILER_RT_VISIBILITY FILE *lprofOpenFileEx(const char *ProfileName) {
-  FILE *f;
-  int fd;
+COMPILER_RT_VISIBILITY int lprofLockFd(int fd) {
 #ifdef COMPILER_RT_HAS_FCNTL_LCK
   struct flock s_flock;
 
   s_flock.l_whence = SEEK_SET;
   s_flock.l_start = 0;
   s_flock.l_len = 0; /* Until EOF.  */
   s_flock.l_pid = getpid();
-
   s_flock.l_type = F_WRLCK;
-  fd = open(ProfileName, O_RDWR | O_CREAT, 0666);
-  if (fd < 0)
-    return NULL;
 
   while (fcntl(fd, F_SETLKW, &s_flock) == -1) {
     if (errno != EINTR) {
       if (errno == ENOLCK) {
-        PROF_WARN("Data may be corrupted during profile merging : %s\n",
-                  "Fail to obtain file lock due to system limit.");
+  return -1;
+      }
+      break;
+    }
+  }
+  return 0;
+#else
+  flock(fd, LOCK_EX);
+  return 0;
+#endif
+}
+
+COMPILER_RT_VISIBILITY int lprofUnlockFd(int fd) {
+#ifdef COMPILER_RT_HAS_FCNTL_LCK
+  struct flock s_flock;
+
+  s_flock.l_whence = SEEK_SET;
+  s_flock.l_start = 0;
+  s_flock.l_len = 0; /* Until EOF.  */
+  s_flock.l_pid = getpid();
+  s_flock.l_type = F_UNLCK;
+
+  while (fcntl(fd, F_SETLKW, &s_flock) == -1) {
+    if (errno != EINTR) {
+      if (errno == ENOLCK) {
+  return -1;
       }
       break;
     }
   }
+  return 0;
+#else
+  flock(fd, LOCK_UN);
+  return 0;
+#endif
+}
+
+COMPILER_RT_VISIBILITY FILE *lprofOpenFileEx(const char *ProfileName) {
+  FILE *f;
+  int fd;
+#ifdef COMPILER_RT_HAS_FCNTL_LCK
+  fd = open(ProfileName, O_RDWR | O_CREAT, 0666);
+  if (fd < 0)
+    return NULL;
+
+  if (lprofLockFd(fd) != 0)
+    PROF_WARN("Data may be corrupted during profile merging : %s\n",
+              "Fail to obtain file lock due to system limit.");
 
   f = fdopen(fd, "r+b");
 #elif defined(_WIN32)
   // FIXME: Use the wide variants to handle Unicode filenames.
   HANDLE h = CreateFileA(ProfileName, GENERIC_READ | GENERIC_WRITE, 0, 0,
                          OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
   if (h == INVALID_HANDLE_VALUE)
     return NULL;