[XRay][compiler-rt] Remove reliance on C++ ABI features

[dberris]

Summary

This fixes http://llvm.org/PR32274.

This change adds a test to ensure that we're able to link XRay modes and
the runtime to binaries that don't need to depend on the C++ standard
library or a C++ ABI library. In particular, we ensure that this will work
with C programs compiled+linked with XRay.

To make the test pass, we need to change a few things in the XRay
runtime implementations to remove the reliance on C++ ABI features. In
particular, we change the thread-safe function-local-static
initialisation to use pthread_* instead of the C++ features that ensure
non-trivial thread-local/function-local-static initialisation.

Depends on D47696.

[dblaikie]

"In the future we can revisit this when we have a better idea as to why not depending on the C++ ABI functionality is a better solution." - this was discussed previously in the thread linked from the bug.

A big thing, so far as I understand it, is that Clang doesn't require some specific C++ library, so wouldn't this break for users who didn't have the specific C++ library installed that compiler-rt for XRay was built against?

[dberris]

In D46998#1103397, @dblaikie wrote:

"In the future we can revisit this when we have a better idea as to why not depending on the C++ ABI functionality is a better solution." - this was discussed previously in the thread linked from the bug.

A big thing, so far as I understand it, is that Clang doesn't require some specific C++ library, so wouldn't this break for users who didn't have the specific C++ library installed that compiler-rt for XRay was built against?

We don't use anything from the C++ library directly, we depend on some features provided by the implementation. In some cases, this isn't a library:

extern int g();

int f() {
  static int val = g();
  return val;
}

This will emit calls to guards for the thread-safe initialisation of val.

The alternatives here are that XRay just avoids these completely, and use pthreads instead to do a pthread_once_init(...).

The other alternative is to avoid pthreads and use the sanitizer_common mutexes too, to make it self-contained in compiler-rt.

The other alternative is to just link in the ABI library (libcxxabi or something else) instead of the standard library.

I can make one of those changes work, but that's also a lot of work, which I suppose is worth doing anyway (unless I'm missing something simpler).

Any preferences/stack-ranking of which solution might be more palatable?

[dberris]

Retitle, and update to limit only to compiler-rt changes.

This is still a work in progress, but working towards a cleaner solution.

[dberris]

• fixup: revert unnecessary change to clang

[dberris]

Revert unnecessary change in clang.

[dberris]

It turns out it's not that bad to actually just go and remove the dependency to C++ ABI specific requirements in the XRay implementation, so I'll continue down this path and remove as much of the interface as possible. In particular, the remaining issues are now in the FDR implementation, and we should be able to continue refactoring that to remove the C++ ABI dependencies in a similar fashion there.

[dberris]

• fixup: update fdr implementation to remove C++ ABI reliance

[jfb]

compiler-rt/lib/xray/xray_fdr_logging.cc

600 ↗

(On Diff #147530)

If you want to guard signal handler recursion you have to use an atomic exchange, not the current if (set) set = true; pattern you have.

[dberris]

• fixup: Use atomic_exchange for signal-safety

compiler-rt/lib/xray/xray_fdr_logging.cc

600 ↗

(On Diff #147530)

Good catch! Fixed.

[dberris]

• fixup: Remove RTTI from XRay builds
• fixup: Remove dependency on array-placement new in buffer_queue

[dberris]

This is now ready for another look @dblaikie and/or @echristo -- apologies for the huge change.

[echristo]

One general inline comment. Otherwise, yes, any decoupling you can do from the C++ standard library is goodness.

compiler-rt/lib/xray/xray_basic_logging.cc

165 ↗

(On Diff #147730)

Mild preference to avoid u64 and still use uint64_t here.

[dberris]

• Rebase
• fixup: use uint64_t instead of u64

[dberris]

• fixup: run binary linked with xray to check it actually works

[dberris]

Rebase and use PTHREAD_ONCE_INIT.

[dberris]

Ping @dblaikie and/or @echristo?

[echristo]

No strong opinions. I'm probably not the right person to review this :)

[dberris]

Thanks, @echristo -- adding in @kpw who may have a bit more context on the XRay side and @eizan who had been looking at doing this a while back.

[dberris]

• fixup: use static instead of inline

[dberris]

Ping @kpw and @eizan?

[kpw]

compiler-rt/lib/xray/xray_basic_logging.cc

428–431 ↗

(On Diff #148779)

The pthread once block that initialized UseRealTSC doesn't have to be this thread. Don't we need synchronization (memory_order_acquire) to make sure it's visible?

compiler-rt/lib/xray/xray_buffer_queue.cc

24 ↗

(On Diff #148779)

I've reviewed up to here. Will come back to this.

[eizan]

compiler-rt/lib/xray/xray_basic_logging.cc

85 ↗

(On Diff #148779)

Why do we need to use atomic_load with a custom memory order rather than just using the default provided by operator=?

412 ↗

(On Diff #148779)

Is this a bugfix?

[dberris]

• fixup: use memory_order_acquire instead of relaxed
• fixup: Move out recursion guard, use in FDR and Basic mode

[dberris]

compiler-rt/lib/xray/xray_basic_logging.cc

85 ↗

(On Diff #148779)

UseRealTSC isn't a std::atomic<...> which will have the operator= overload. We're using the types in sanitizer_common/sanitizer_atomic.h here, which require that loads and stores have an explicit memory order.

412 ↗

(On Diff #148779)

You could call it that, though we don't have failing tests that indicate we've hit this. :)

428–431 ↗

(On Diff #148779)

I think a stronger memory order here doesn't hurt -- but thinking about this a little, the global synchronisation on BasicInitialized just allows us to have one of these functions running at the same time. These loads will only happen in either:

• The first thread that gets to run this, in which case we get the once initialisation done before any of these loads.

• Another thread that will be able to run this function will always see the release stores commit.

Note that this assumes x86, which is a bad assumption to make. So you're right, I should be using stronger loads here.

[dberris]

Breaking this up into smaller changes now. Some of the smaller changes are NFC (namespace cleanup, using internal_* instead of std::*, DCHECK instead of assert, etc.) and are largely OK to commit in isolation. I'll update the patch once the NFC changes have landed.

[dberris]

Rebase, reword, and split up into smaller changes.

[dberris]

Rebase, on top of D47696.

[dberris]

• fixup: use atomic_exchange instead of pthread once for error latch; also clang-format

[kpw]

compiler-rt/lib/xray/xray_basic_logging.cc

500–501 ↗

(On Diff #150081)

I don't get why there is a mix of static FakeTLD and thread_local variables. Won't this only access and subsequently destroy the thread_locals for only a single thread?

compiler-rt/lib/xray/xray_fdr_logging.cc

660 ↗

(On Diff #150081)

I'm a bit surprised that using static variables with their initialized exactly once by the first caller semantics doesn't depend on the ABI. Are you sure these are OK?

714 ↗

(On Diff #150081)

What's the ABI reason to change this to aligned_storage instead of just using the type?

785–786 ↗

(On Diff #150081)

Are you adding the ability to cycle through states? I'd say that's a separate patch than the ABI stuff.

789–791 ↗

(On Diff #150081)

Removing this does not look related to ABI. This should be a separate patch if it's a behavior change. The ABI change would be to translate to destructor + internal free if I'm not mistaken.

compiler-rt/test/xray/TestCases/Posix/c-test.cc

3–4 ↗

(On Diff #150081)

Shouldn't a different prefix run FDR to check that that mode works as well?

[dberris]

• fixup: address comments by @kpw

compiler-rt/lib/xray/xray_basic_logging.cc

500–501 ↗

(On Diff #150081)

Note, that this happens at program exit -- we want to make sure that at program exit we're cleaning up the data for the final exiting thread. This code only runs when the mode installed at premain init. In that case we can't rely on the finalisation routine being invoked, so we force the cleanup and flush one way or another.

compiler-rt/lib/xray/xray_fdr_logging.cc

660 ↗

(On Diff #150081)

Yes, because this is initialised with a constant -- __xray::NanosecondsPerSecond is a constexpr.

714 ↗

(On Diff #150081)

Because XRayFileHeader is not trivially constructible, initialising it is guarded to be thread-safe -- that thread-safe init calls C++ ABI specific functions. Using aligned storage here ensures that the storage is static, but that initialisation happens in the pthread_once(...) region.

compiler-rt/test/xray/TestCases/Posix/c-test.cc

3–4 ↗

(On Diff #150081)

Technically unnecessary, since we link in the FDR mode runtime anyway -- the test is just ensuring that we can link and run a C program.

[kpw]

compiler-rt/lib/xray/xray_basic_logging.cc

500–501 ↗

(On Diff #150081)

I see. This is to salvage something to log when exiting abnormally. It's not obvious that TLDDestructor calls similar routines to finalize and flush instead of just freeing memory and stuff like that.

If you can come up with a way to make this more obvious, be my guest, otherwise the comment will have to suffice.

compiler-rt/lib/xray/xray_fdr_logging.cc

844 ↗

(On Diff #150245)

Doesn't the "\_\_xray" namespace have an effective "using namespace \_\_sanitizer" everywhere? <--- differential formatting turns these into underlines.

Are you just doing this to make it obvious this is not a stdlib atomic?

660 ↗

(On Diff #150081)

So we rely on clang turning these into pre-populated addresses (in bss?) at compilation? Is it the standard or the implementation that lets us rely on that?

714 ↗

(On Diff #150081)

It would be great if you could add a writeup of the common ABI pitfalls you discovered and the xray way of accomplishing similar idioms. I'm not sure where, but otherwise, these details are not going to be obvious to future maintainers/contributors etc.

[dberris]

• fixup: remove more unncessary namespace qualifications

compiler-rt/lib/xray/xray_fdr_logging.cc

844 ↗

(On Diff #150245)

Nope, bad rebase/merge. :)

660 ↗

(On Diff #150081)

It's the standard that says, that they must be be constant-initialised.

In C++11 this is [stmt.decl] p4 (6.7.4) which references [basic.start.init]p2 bullet 3.

714 ↗

(On Diff #150081)

That's a good idea. The test is supposed to guard against those, because it will turn into a link failure for missing ABI functions.

compiler-rt/trunk/lib/xray/xray_basic_logging.cc

 // This file is a part of XRay, a dynamic runtime instrumentation system.
 //
 // Implementation of a simple in-memory log of XRay events. This defines a
 // logging function that's compatible with the XRay handler interface, and
 // routines for exporting data to files.
 //
 //===----------------------------------------------------------------------===//
 
-#include <cassert>
-#include <cstring>
 #include <errno.h>
 #include <fcntl.h>
 #include <pthread.h>
 #include <sys/stat.h>
 #include <sys/syscall.h>
 #include <sys/types.h>
 #include <time.h>
 #include <unistd.h>

 static pthread_key_t PThreadKey;
 
 static atomic_uint8_t BasicInitialized{0};
 
 BasicLoggingOptions GlobalOptions;
 
 thread_local atomic_uint8_t Guard{0};
 
-static uint64_t thresholdTicks() XRAY_NEVER_INSTRUMENT {
-  static uint64_t TicksPerSec = probeRequiredCPUFeatures()
-                                    ? getTSCFrequency()
-                                    : NanosecondsPerSecond;
-  static const uint64_t ThresholdTicks =
-      TicksPerSec * GlobalOptions.DurationFilterMicros / 1000000;
-  return ThresholdTicks;
-}
+static atomic_uint8_t UseRealTSC{0};
+static atomic_uint64_t ThresholdTicks{0};
+static atomic_uint64_t TicksPerSec{0};
+static atomic_uint64_t CycleFrequency{NanosecondsPerSecond};
 
 static int openLogFile() XRAY_NEVER_INSTRUMENT {
   int F = getLogFD();
   if (F == -1)
     return -1;
 
-  // Test for required CPU features and cache the cycle frequency
-  static bool TSCSupported = probeRequiredCPUFeatures();
-  static uint64_t CycleFrequency =
-      TSCSupported ? getTSCFrequency() : NanosecondsPerSecond;
+  static pthread_once_t DetectOnce = PTHREAD_ONCE_INIT;
+  pthread_once(&DetectOnce, +[] {
+    if (atomic_load(&UseRealTSC, memory_order_acquire))
+      atomic_store(&CycleFrequency, getTSCFrequency(), memory_order_release);
+  });
 
   // Since we're here, we get to write the header. We set it up so that the
   // header will only be written once, at the start, and let the threads
   // logging do writes which just append.
   XRayFileHeader Header;
   Header.Version = 2; // Version 2 includes tail exit records.
   Header.Type = FileTypes::NAIVE_LOG;
-  Header.CycleFrequency = CycleFrequency;
+  Header.CycleFrequency = atomic_load(&CycleFrequency, memory_order_acquire);
 
   // FIXME: Actually check whether we have 'constant_tsc' and 'nonstop_tsc'
   // before setting the values in the header.
   Header.ConstantTSC = 1;
   Header.NonstopTSC = 1;
   retryingWriteAll(F, reinterpret_cast<char *>(&Header),
                    reinterpret_cast<char *>(&Header) + sizeof(Header));
   return F;
 }
 
-int getGlobalFd() XRAY_NEVER_INSTRUMENT {
-  static int Fd = openLogFile();
+static int getGlobalFd() XRAY_NEVER_INSTRUMENT {
+  static pthread_once_t OnceInit = PTHREAD_ONCE_INIT;
+  static int Fd = 0;
+  pthread_once(&OnceInit, +[] { Fd = openLogFile(); });
   return Fd;
 }
 
-ThreadLocalData &getThreadLocalData() XRAY_NEVER_INSTRUMENT {
+static ThreadLocalData &getThreadLocalData() XRAY_NEVER_INSTRUMENT {
   thread_local ThreadLocalData TLD;
   thread_local bool UNUSED TOnce = [] {
     if (GlobalOptions.ThreadBufferSize == 0) {
       if (Verbosity())
         Report("Not initializing TLD since ThreadBufferSize == 0.\n");
       return false;
     }
     TLD.TID = GetTid();
if (GlobalOptions.MaxStackDepth == 0) {
       TLD.ShadowStack = nullptr;
       return false;
     }
     TLD.ShadowStack = reinterpret_cast<StackEntry *>(
         InternalAlloc(sizeof(StackEntry) * GlobalOptions.MaxStackDepth, nullptr,
                       alignof(StackEntry)));
     TLD.StackSize = GlobalOptions.MaxStackDepth;
     TLD.StackEntries = 0;
-    if (Verbosity() >= 2) {
-      static auto UNUSED Once = [] {
-        auto ticks = thresholdTicks();
-        Report("Ticks threshold: %d\n", ticks);
-        return false;
-      }();
-    }
     return false;
   }();
   return TLD;
 }
 
 template <class RDTSC>
 void InMemoryRawLog(int32_t FuncId, XRayEntryType Type,
                     RDTSC ReadTSC) XRAY_NEVER_INSTRUMENT {
case XRayEntryType::TAIL: {
     // record and move the record offset back.
     StackEntry StackTop;
     auto StackEntryPtr = static_cast<char *>(TLD.ShadowStack) +
                          (sizeof(StackEntry) * TLD.StackEntries);
     internal_memcpy(&StackTop, StackEntryPtr, sizeof(StackEntry));
     if (StackTop.FuncId == FuncId && StackTop.CPU == CPU &&
         StackTop.TSC < TSC) {
       auto Delta = TSC - StackTop.TSC;
-      if (Delta < thresholdTicks()) {
-        assert(TLD.BufferOffset > 0);
+      if (Delta < atomic_load(&ThresholdTicks, memory_order_relaxed)) {
+        DCHECK(TLD.BufferOffset > 0);
         TLD.BufferOffset -= StackTop.Type == XRayEntryType::ENTRY ? 1 : 2;
         return;
       }
     }
     break;
   }
   default:
     // Should be unreachable.
-    assert(false && "Unsupported XRayEntryType encountered.");
+    DCHECK(false && "Unsupported XRayEntryType encountered.");
     break;
   }
 
   // First determine whether the delta between the function's enter record and
   // the exit record is higher than the threshold.
   XRayRecord R;
   R.RecordType = RecordTypes::NORMAL;
   R.CPU = CPU;
static void TLDDestructor(void *P) XRAY_NEVER_INSTRUMENT {
   // thread exits.
   fsync(TLD.Fd);
 }
 
 XRayLogInitStatus basicLoggingInit(size_t BufferSize, size_t BufferMax,
                                    void *Options,
                                    size_t OptionsSize) XRAY_NEVER_INSTRUMENT {
   uint8_t Expected = 0;
-  if (!atomic_compare_exchange_strong(
-          &BasicInitialized, &Expected, 1, memory_order_acq_rel)) {
+  if (!atomic_compare_exchange_strong(&BasicInitialized, &Expected, 1,
+                                      memory_order_acq_rel)) {
     if (Verbosity())
       Report("Basic logging already initialized.\n");
     return XRayLogInitStatus::XRAY_LOG_INITIALIZED;
   }
 
-  static bool UNUSED Once = [] {
+  static pthread_once_t OnceInit = PTHREAD_ONCE_INIT;
+  pthread_once(&OnceInit, +[] {
     pthread_key_create(&PThreadKey, TLDDestructor);
-    return false;
-  }();
+    atomic_store(&UseRealTSC, probeRequiredCPUFeatures(), memory_order_release);
+    // Initialize the global TicksPerSec value.
+    atomic_store(&TicksPerSec,
+                 probeRequiredCPUFeatures() ? getTSCFrequency()
+                                            : NanosecondsPerSecond,
+                 memory_order_release);
+    if (!atomic_load(&UseRealTSC, memory_order_relaxed) && Verbosity())
+      Report("WARNING: Required CPU features missing for XRay instrumentation, "
+             "using emulation instead.\n");
+  });
 
   if (BufferSize == 0 && BufferMax == 0 && Options != nullptr) {
     FlagParser P;
     BasicFlags F;
     F.setDefaults();
     registerXRayBasicFlags(&P, &F);
     P.ParseString(useCompilerDefinedBasicFlags());
     auto *EnvOpts = GetEnv("XRAY_BASIC_OPTIONS");
if (internal_strlen(EnvOpts) == 0) {
       F.max_stack_depth = flags()->xray_naive_log_max_stack_depth;
       F.thread_buffer_size = flags()->xray_naive_log_thread_buffer_size;
     }
 
     P.ParseString(static_cast<const char *>(Options));
     GlobalOptions.ThreadBufferSize = F.thread_buffer_size;
     GlobalOptions.DurationFilterMicros = F.func_duration_threshold_us;
     GlobalOptions.MaxStackDepth = F.max_stack_depth;
+    *basicFlags() = F;
   } else if (OptionsSize != sizeof(BasicLoggingOptions)) {
     Report("Invalid options size, potential ABI mismatch; expected %d got %d",
            sizeof(BasicLoggingOptions), OptionsSize);
     return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
   } else {
     if (Verbosity())
       Report("XRay Basic: struct-based init is deprecated, please use "
              "string-based configuration instead.\n");
     GlobalOptions = *reinterpret_cast<BasicLoggingOptions *>(Options);
   }
 
-  static auto UseRealTSC = probeRequiredCPUFeatures();
-  if (!UseRealTSC && Verbosity())
-    Report("WARNING: Required CPU features missing for XRay instrumentation, "
-           "using emulation instead.\n");
-
-  __xray_set_handler_arg1(UseRealTSC ? basicLoggingHandleArg1RealTSC
-                                     : basicLoggingHandleArg1EmulateTSC);
-  __xray_set_handler(UseRealTSC ? basicLoggingHandleArg0RealTSC
-                                : basicLoggingHandleArg0EmulateTSC);
+  atomic_store(&ThresholdTicks,
+               atomic_load(&TicksPerSec, memory_order_acquire) *
+                   GlobalOptions.DurationFilterMicros / 1000000,
+               memory_order_release);
+  __xray_set_handler_arg1(atomic_load(&UseRealTSC, memory_order_acquire)
+                              ? basicLoggingHandleArg1RealTSC
+  __xray_set_handler(atomic_load(&UseRealTSC, memory_order_acquire)
+                         ? basicLoggingHandleArg0RealTSC
+
+  // TODO: Implement custom event and typed event handling support in Basic
+  // Mode.
   __xray_remove_customevent_handler();
   __xray_remove_typedevent_handler();
 
   return XRayLogInitStatus::XRAY_LOG_INITIALIZED;
 }
 
 XRayLogInitStatus basicLoggingFinalize() XRAY_NEVER_INSTRUMENT {
   uint8_t Expected = 0;
-  if (!atomic_compare_exchange_strong(
-          &BasicInitialized, &Expected, 0, memory_order_acq_rel) &&
+  if (!atomic_compare_exchange_strong(&BasicInitialized, &Expected, 0,
+                                      memory_order_acq_rel) &&
       Verbosity())
     Report("Basic logging already finalized.\n");
 
   // Nothing really to do aside from marking state of the global to be
   // uninitialized.
 
   return XRayLogInitStatus::XRAY_LOG_FINALIZED;
 }
if (flags()->xray_naive_log ||
     auto *Env = GetEnv("XRAY_BASIC_OPTIONS");
     auto InitResult =
         __xray_log_init_mode("xray-basic", Env == nullptr ? "" : Env);
     if (InitResult != XRayLogInitStatus::XRAY_LOG_INITIALIZED) {
       if (Verbosity())
         Report("Failed initializing XRay Basic Mode; error = %d\n", InitResult);
       return false;
     }
-    static auto UNUSED Once = [] {
-      static auto UNUSED &TLD = getThreadLocalData();
-      Atexit(+[] { TLDDestructor(&TLD); });
-      return false;
-    }();
+
+    // At this point we know that we've successfully initialized Basic mode
+    // tracing, and the only chance we're going to get for the current thread to
+    // clean-up may be at thread/program exit. To ensure that we're going to get
+    // the cleanup even without calling the finalization routines, we're
+    // registering a program exit function that will do the cleanup.
+    static pthread_once_t DynamicOnce = PTHREAD_ONCE_INIT;
+    pthread_once(&DynamicOnce, +[] {
+      static void *FakeTLD = nullptr;
+      FakeTLD = &getThreadLocalData();
+      Atexit(+[] { TLDDestructor(FakeTLD); });
+    });
   }
   return true;
 }
 
 } // namespace __xray
 
 static auto UNUSED Unused = __xray::basicLogDynamicInitializer();

compiler-rt/trunk/lib/xray/xray_fdr_logging.cc

 // This file is a part of XRay, a dynamic runtime instrumentation system.
 //
 // Here we implement the Flight Data Recorder mode for XRay, where we use
 // compact structures to store records in memory as well as when writing out the
 // data to files.
 //
 //===----------------------------------------------------------------------===//
 #include "xray_fdr_logging.h"
+#include <cassert>
 #include <errno.h>
 #include <limits>
-#include <pthread.h>
+#include <memory>
 #include <sys/syscall.h>
 #include <sys/time.h>
 #include <time.h>
 #include <unistd.h>
 
 #include "sanitizer_common/sanitizer_allocator_internal.h"
 #include "sanitizer_common/sanitizer_atomic.h"
 #include "sanitizer_common/sanitizer_common.h"
 #include "xray/xray_interface.h"
 #include "xray/xray_records.h"
-#include "xray_recursion_guard.h"
 #include "xray_buffer_queue.h"
 #include "xray_defs.h"
 #include "xray_fdr_flags.h"
 #include "xray_flags.h"
+#include "xray_recursion_guard.h"
 #include "xray_tsc.h"
 #include "xray_utils.h"
 
 namespace __xray {
 
 atomic_sint32_t LoggingStatus = {XRayLogInitStatus::XRAY_LOG_UNINITIALIZED};
 
 // Group together thread-local-data in a struct, then hide it behind a function
if (TLD.NumConsecutiveFnEnters > 0) {
       TLD.NumTailCalls = 0;
       TLD.NumConsecutiveFnEnters = 0;
     }
     FuncRecord.RecordKind =
         uint8_t(FunctionRecord::RecordKinds::FunctionTailExit);
     break;
   case XRayEntryType::CUSTOM_EVENT: {
     // This is a bug in patching, so we'll report it once and move on.
-    static bool Once = [&] {
+    static atomic_uint8_t ErrorLatch{0};
+    if (!atomic_exchange(&ErrorLatch, 1, memory_order_acq_rel))
       Report("Internal error: patched an XRay custom event call as a function; "
              "func id = %d\n",
              FuncId);
-      return true;
-    }();
-    (void)Once;
     return;
   }
   case XRayEntryType::TYPED_EVENT: {
-    static bool Once = [&] {
+    static atomic_uint8_t ErrorLatch{0};
+    if (!atomic_exchange(&ErrorLatch, 1, memory_order_acq_rel))
       Report("Internal error: patched an XRay typed event call as a function; "
              "func id = %d\n",
              FuncId);
-      return true;
-    }();
-    (void)Once;
     return;
   }
   }
 
   internal_memcpy(TLD.RecordPtr, &FuncRecord, sizeof(FunctionRecord));
   TLD.RecordPtr += sizeof(FunctionRecord);
   incrementExtents(sizeof(FunctionRecord));
 }
 
-static uint64_t thresholdTicks() {
-  static uint64_t TicksPerSec = probeRequiredCPUFeatures()
-                                    ? getTSCFrequency()
-                                    : __xray::NanosecondsPerSecond;
-  static const uint64_t ThresholdTicks =
-      TicksPerSec * fdrFlags()->func_duration_threshold_us / 1000000;
-  return ThresholdTicks;
-}
+static atomic_uint64_t TicksPerSec{0};
+static atomic_uint64_t ThresholdTicks{0};
 
 // Re-point the thread local pointer into this thread's Buffer before the recent
 // "Function Entry" record and any "Tail Call Exit" records after that.
 static void rewindRecentCall(uint64_t TSC, uint64_t &LastTSC,
                              uint64_t &LastFunctionEntryTSC, int32_t FuncId) {
   auto &TLD = getThreadLocalData();
   TLD.RecordPtr -= FunctionRecSize;
   decrementExtents(FunctionRecSize);
internal_memcpy(&ExpectedFunctionEntry, RewindingRecordPtr,
                     FunctionRecSize);
     DCHECK(ExpectedFunctionEntry.RecordKind ==
                uint8_t(FunctionRecord::RecordKinds::FunctionEnter) &&
            "Expected to find function entry when rewinding tail call.");
     DCHECK(ExpectedFunctionEntry.FuncId == ExpectedTailExit.FuncId &&
            "Expected funcids to match when rewinding tail call.");
 
     // This tail call exceeded the threshold duration. It will not be erased.
-    if ((TSC - RewindingTSC) >= thresholdTicks()) {
+    if ((TSC - RewindingTSC) >= atomic_load_relaxed(&ThresholdTicks)) {
       TLD.NumTailCalls = 0;
       return;
     }
 
     // We can erase a tail exit pair that we're exiting through since
     // its duration is under threshold.
     --TLD.NumTailCalls;
     RewindingRecordPtr -= FunctionRecSize;
static void processFunctionHook(int32_t FuncId, XRayEntryType Entry,
   case XRayEntryType::LOG_ARGS_ENTRY:
     // Update the thread local state for the next invocation.
     TLD.LastFunctionEntryTSC = TSC;
     break;
   case XRayEntryType::TAIL:
   case XRayEntryType::EXIT:
     // Break out and write the exit record if we can't erase any functions.
     if (TLD.NumConsecutiveFnEnters == 0 ||
-        (TSC - TLD.LastFunctionEntryTSC) >= thresholdTicks())
+        (TSC - TLD.LastFunctionEntryTSC) >=
+            atomic_load_relaxed(&ThresholdTicks))
       break;
     rewindRecentCall(TSC, TLD.LastTSC, TLD.LastFunctionEntryTSC, FuncId);
     return; // without writing log.
   case XRayEntryType::CUSTOM_EVENT: {
     // This is a bug in patching, so we'll report it once and move on.
-    static bool Once = [&] {
+    static atomic_uint8_t ErrorLatch{0};
+    if (!atomic_exchange(&ErrorLatch, 1, memory_order_acq_rel))
       Report("Internal error: patched an XRay custom event call as a function; "
-             "func id = %d",
+             "func id = %d\n",
              FuncId);
-      return true;
-    }();
-    (void)Once;
     return;
   }
   case XRayEntryType::TYPED_EVENT: {
-    static bool Once = [&] {
+    static atomic_uint8_t ErrorLatch{0};
+    if (!atomic_exchange(&ErrorLatch, 1, memory_order_acq_rel))
       Report("Internal error: patched an XRay typed event call as a function; "
              "func id = %d\n",
              FuncId);
-      return true;
-    }();
-    (void)Once;
     return;
   }
   }
 
   writeFunctionRecord(FuncId, RecordTSCDelta, Entry);
   if (Entry == XRayEntryType::LOG_ARGS_ENTRY)
     writeCallArgumentMetadata(Arg1);
 
   // If we've exhausted the buffer by this time, we then release the buffer to
   // make sure that other threads may start using this buffer.
   endBufferIfFull();
   __asm volatile("# LLVM-MCA-END");
 }
 
 // Global BufferQueue.
 BufferQueue *BQ = nullptr;
 
 atomic_sint32_t LogFlushStatus = {XRayLogFlushStatus::XRAY_LOG_NOT_FLUSHING};
 
 FDRLoggingOptions FDROptions;
 
 SpinMutex FDROptionsMutex;
 
-namespace {
-XRayFileHeader &fdrCommonHeaderInfo() {
-  static XRayFileHeader Header = [] {
-    XRayFileHeader H;
+static XRayFileHeader &fdrCommonHeaderInfo() {
+  static std::aligned_storage<sizeof(XRayFileHeader)>::type HStorage;
+  static pthread_once_t OnceInit = PTHREAD_ONCE_INIT;
+  static bool TSCSupported = true;
+  static uint64_t CycleFrequency = NanosecondsPerSecond;
+  pthread_once(&OnceInit, +[] {
+    XRayFileHeader &H = reinterpret_cast<XRayFileHeader &>(HStorage);
     // Version 2 of the log writes the extents of the buffer, instead of
     // relying on an end-of-buffer record.
     H.Version = 2;
     H.Type = FileTypes::FDR_LOG;
 
     // Test for required CPU features and cache the cycle frequency
-    static bool TSCSupported = probeRequiredCPUFeatures();
-    static uint64_t CycleFrequency =
-        TSCSupported ? getTSCFrequency() : __xray::NanosecondsPerSecond;
+    TSCSupported = probeRequiredCPUFeatures();
+    if (TSCSupported)
+      CycleFrequency = getTSCFrequency();
     H.CycleFrequency = CycleFrequency;
 
     // FIXME: Actually check whether we have 'constant_tsc' and
     // 'nonstop_tsc' before setting the values in the header.
     H.ConstantTSC = 1;
     H.NonstopTSC = 1;
-    return H;
-  }();
-  return Header;
+  });
+  return reinterpret_cast<XRayFileHeader &>(HStorage);
 }
 
-} // namespace
-
 // This is the iterator implementation, which knows how to handle FDR-mode
 // specific buffers. This is used as an implementation of the iterator function
 // needed by __xray_set_buffer_iterator(...). It maintains a global state of the
 // buffer iteration for the currently installed FDR mode buffers. In particular:
 //
 //   - If the argument represents the initial state of XRayBuffer ({nullptr, 0})
 //     then the iterator returns the header information.
 //   - If the argument represents the header information ({address of header
if (Verbosity())
           "XRay FDR: Failed global buffer queue is null or not finalizing!\n");
     return {nullptr, 0};
   }
 
   // We use a global scratch-pad for the header information, which only gets
   // initialized the first time this function is called. We'll update one part
   // of this information with some relevant data (in particular the number of
   // buffers to expect).
-  static XRayFileHeader Header = fdrCommonHeaderInfo();
+  static std::aligned_storage<sizeof(XRayFileHeader)>::type HeaderStorage;
+  static pthread_once_t HeaderOnce = PTHREAD_ONCE_INIT;
+  pthread_once(&HeaderOnce, +[] {
+    reinterpret_cast<XRayFileHeader &>(HeaderStorage) = fdrCommonHeaderInfo();
+  });
+
+  // We use a convenience alias for code referring to Header from here on out.
+  auto &Header = reinterpret_cast<XRayFileHeader &>(HeaderStorage);
   if (B.Data == nullptr && B.Size == 0) {
     Header.FdrData = FdrAdditionalHeaderData{BQ->ConfiguredBufferSize()};
     return XRayBuffer{static_cast<void *>(&Header), sizeof(Header)};
   }
 
   static BufferQueue::const_iterator It{};
   static BufferQueue::const_iterator End{};
   if (B.Data == static_cast<void *>(&Header) && B.Size == sizeof(Header)) {
-
     // From this point on, we provide raw access to the raw buffer we're getting
     // from the BufferQueue. We're relying on the iterators from the current
     // Buffer queue.
     It = BQ->cbegin();
     End = BQ->cend();
   }
 
   if (It == End)
XRayLogFlushStatus fdrLoggingFlush() XRAY_NEVER_INSTRUMENT {
   // we decide to do anything further with the global buffer queue.
   __xray_log_remove_buffer_iterator();
 
   if (fdrFlags()->no_file_flush) {
     if (Verbosity())
       Report("XRay FDR: Not flushing to file, 'no_file_flush=true'.\n");
 
     // Clean up the buffer queue, and do not bother writing out the files!
-    delete BQ;
+    BQ->~BufferQueue();
+    InternalFree(BQ);
     BQ = nullptr;
+
     atomic_store(&LogFlushStatus, XRayLogFlushStatus::XRAY_LOG_FLUSHED,
                  memory_order_release);
     return XRayLogFlushStatus::XRAY_LOG_FLUSHED;
   }
 
   // We write out the file in the following format:
   //
   //   1) We write down the XRay file header with version 1, type FDR_LOG.

 
 static TSCAndCPU getTimestamp() XRAY_NEVER_INSTRUMENT {
   // We want to get the TSC as early as possible, so that we can check whether
   // we've seen this CPU before. We also do it before we load anything else,
   // to allow for forward progress with the scheduling.
   TSCAndCPU Result;
 
   // Test once for required CPU features
-  static bool TSCSupported = probeRequiredCPUFeatures();
+  static pthread_once_t OnceProbe = PTHREAD_ONCE_INIT;
+  static bool TSCSupported = true;
+  pthread_once(&OnceProbe, +[] { TSCSupported = probeRequiredCPUFeatures(); });
 
   if (TSCSupported) {
     Result.TSC = __xray::readTSC(Result.CPU);
   } else {
     // FIXME: This code needs refactoring as it appears in multiple locations
     timespec TS;
     int result = clock_gettime(CLOCK_REALTIME, &TS);
     if (result != 0) {
void fdrLoggingHandleCustomEvent(void *Event,
                                  std::size_t EventSize) XRAY_NEVER_INSTRUMENT {
   auto TC = getTimestamp();
   auto &TSC = TC.TSC;
   auto &CPU = TC.CPU;
   RecursionGuard Guard{Running};
   if (!Guard)
     return;
   if (EventSize > std::numeric_limits<int32_t>::max()) {
-    using Empty = struct {};
-    static Empty Once = [&] {
-      Report("Event size too large = %zu ; > max = %d\n", EventSize,
-             std::numeric_limits<int32_t>::max());
-      return Empty();
-    }();
-    (void)Once;
+    static pthread_once_t Once = PTHREAD_ONCE_INIT;
+    pthread_once(&Once, +[] { Report("Event size too large.\n"); });
   }
   int32_t ReducedEventSize = static_cast<int32_t>(EventSize);
   auto &TLD = getThreadLocalData();
   if (!isLogInitializedAndReady(TLD.BQ, TSC, CPU, clock_gettime))
     return;
 
   // Here we need to prepare the log to handle:
   //   - The metadata record we're going to write. (16 bytes)
void fdrLoggingHandleTypedEvent(
     std::size_t EventSize) noexcept XRAY_NEVER_INSTRUMENT {
   auto TC = getTimestamp();
   auto &TSC = TC.TSC;
   auto &CPU = TC.CPU;
   RecursionGuard Guard{Running};
   if (!Guard)
     return;
   if (EventSize > std::numeric_limits<int32_t>::max()) {
-    using Empty = struct {};
-    static Empty Once = [&] {
-      Report("Event size too large = %zu ; > max = %d\n", EventSize,
-             std::numeric_limits<int32_t>::max());
-      return Empty();
-    }();
-    (void)Once;
+    static pthread_once_t Once = PTHREAD_ONCE_INIT;
+    pthread_once(&Once, +[] { Report("Event size too large.\n"); });
   }
   int32_t ReducedEventSize = static_cast<int32_t>(EventSize);
   auto &TLD = getThreadLocalData();
   if (!isLogInitializedAndReady(TLD.BQ, TSC, CPU, clock_gettime))
     return;
 
   // Here we need to prepare the log to handle:
   //   - The metadata record we're going to write. (16 bytes)
if (Verbosity())
              "string-based configuration instead.\n");
     SpinMutexLock Guard(&FDROptionsMutex);
     internal_memcpy(&FDROptions, Options, OptionsSize);
   }
 
   bool Success = false;
 
   if (BQ != nullptr) {
-    delete BQ;
+    BQ->~BufferQueue();
+    InternalFree(BQ);
     BQ = nullptr;
   }
 
-  if (BQ == nullptr)
-    BQ = new BufferQueue(BufferSize, BufferMax, Success);
+  if (BQ == nullptr) {
+    BQ = reinterpret_cast<BufferQueue *>(
+        InternalAlloc(sizeof(BufferQueue), nullptr, 64));
+    new (BQ) BufferQueue(BufferSize, BufferMax, Success);
+  }
 
   if (!Success) {
     Report("BufferQueue init failed.\n");
     if (BQ != nullptr) {
-      delete BQ;
+      BQ->~BufferQueue();
+      InternalFree(BQ);
       BQ = nullptr;
     }
     return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
   }
 
-  static bool UNUSED Once = [] {
+  static pthread_once_t OnceInit = PTHREAD_ONCE_INIT;
+  pthread_once(&OnceInit, +[] {
+    atomic_store(&TicksPerSec,
+                 probeRequiredCPUFeatures() ? getTSCFrequency()
+                                            : __xray::NanosecondsPerSecond,
+                 memory_order_release);
     pthread_key_create(&Key, +[](void *) {
       auto &TLD = getThreadLocalData();
       if (TLD.BQ == nullptr)
         return;
       auto EC = TLD.BQ->releaseBuffer(TLD.Buffer);
       if (EC != BufferQueue::ErrorCode::Ok)
         Report("At thread exit, failed to release buffer at %p; error=%s\n",
                TLD.Buffer.Data, BufferQueue::getErrorString(EC));
     });
-    return false;
-  }();
+  });
 
+  atomic_store(&ThresholdTicks,
+               atomic_load_relaxed(&TicksPerSec) *
+                   fdrFlags()->func_duration_threshold_us / 1000000,
+               memory_order_release);
   // Arg1 handler should go in first to avoid concurrent code accidentally
   // falling back to arg0 when it should have ran arg1.
   __xray_set_handler_arg1(fdrLoggingHandleArg1);
   // Install the actual handleArg0 handler after initialising the buffers.
   __xray_set_handler(fdrLoggingHandleArg0);
   __xray_set_customevent_handler(fdrLoggingHandleCustomEvent);
   __xray_set_typedevent_handler(fdrLoggingHandleTypedEvent);
 

compiler-rt/trunk/test/xray/TestCases/Posix/c-test.cc

+// RUN: %clang_xray -g -o %t %s
+// RUN: rm xray-log.c-test.* || true
+// RUN: XRAY_OPTIONS=patch_premain=true:verbosity=1:xray_mode=xray-basic %t \
+// RUN:     2>&1 | FileCheck %s
+// RUN: rm xray-log.c-test.* || true
+//
+// REQUIRES: x86_64-target-arch
+// REQUIRES: built-in-llvm-tree
+__attribute__((xray_always_instrument)) void always() {}
+
+int main() {
+  always();
+}
+
+// CHECK: =={{[0-9].*}}==XRay: Log file in '{{.*}}'