[XRay][compiler-rt+docs] Introduce __xray_log_init_mode(...).

[dberris]

Summary

This addresses http://llvm.org/PR36790.

The change Deprecates a number of functions and types in
include/xray/xray_log_interface.h to recommend using string-based
configuration of XRay through the __xray_log_init_mode(...) function. In
particular, this deprecates the following:

• __xray_set_log_impl(...) -- users should instead use the

__xray_log_register_mode(...) and __xray_log_select_mode(...) APIs.

• __xray_log_init(...) -- users should instead use the

__xray_log_init_mode(...) function, which also requires using the
__xray_log_register_mode(...) and __xray_log_select_mode(...)
functionality.

• __xray::FDRLoggingOptions -- in following patches, we'll be

migrating the FDR logging implementations (and tests) to use the
string-based configuration. In later stages we'll remove the
__xray::FDRLoggingOptions type, and ask users to migrate to using the
string-based configuration mechanism instead.

• __xray::BasicLoggingOptions -- same as __xray::FDRLoggingOptions,

we'll be removing this type later and instead rely exclusively on the
string-based configuration API.

We also update the documentation to reflect the new advice and remove
some of the deprecated notes.

[dberris]

Rebase

[kpw]

Everything looks good except I'd like to see us flexible for mode configuration that may contain nulls.

compiler-rt/include/xray/xray_log_interface.h

35–36

Is each mode expected to document its own flag options? Is there a shared option set among all modes?

43

If you're calling the sleds instrumentation points above, be consistent.

instrumentation sleds -> instrumentation points.

283–285

I would prefer an alternative (or perhaps having both) that provides the ArgsSize as length of char buffer. It could be nice to be able to encode protos or binary data without worrying about null bytes, since interpreting the configuration payload is entirely up to the mode.

compiler-rt/test/xray/TestCases/Posix/logging-modes.cc

60

Why volatile? I only see one thread of execution. If sync is needed, why volatile instead of atomic.

llvm/docs/XRay.rst

200

Kind of regrettable camelCase on this function, but it is correct. :(

[dberris]

• fixup: address comments by kpw

[dberris]

compiler-rt/include/xray/xray_log_interface.h

35–36

Yes, each mode is expected to document their own flags.

There is a shared set of "xray-specific" global options. Following patches define these global settings and separate the mode-specific settings.

43

Also changed globally in the file.

283–285

Makes sense -- added __xray_log_init_mode_bin(...) to support the pointer+length API for character buffers.

compiler-rt/test/xray/TestCases/Posix/logging-modes.cc

60

Yeah, this was an artifact of some debugging/exploration. Unnecessary change.

llvm/docs/XRay.rst

200

Yeah... someday, we will have the time to clean this all up. :)

[kpw]

LGTM.

compiler-rt/include/xray/xray_log_interface.h

 /// XRay allows users to implement their own logging handlers and install them
 /// to replace the default runtime-controllable implementation that comes with
 /// compiler-rt/xray. The "flight data recorder" (FDR) mode implementation uses
 /// this API to install itself in an XRay-enabled binary. See
 /// compiler-rt/lib/xray_fdr_logging.{h,cc} for details of that implementation.
 ///
 /// The high-level usage pattern for these APIs look like the following:
 ///
-///   // Before we try initializing the log implementation, we must set it as
-///   // the log implementation. We provide the function pointers that define
-///   // the various initialization, finalization, and other pluggable hooks
-///   // that we need.
-///   __xray_set_log_impl({...});
-///
-///   // Once that's done, we can now initialize the implementation. Each
-///   // implementation has a chance to let users customize the implementation
-///   // with a struct that their implementation supports. Roughly this might
-///   // look like:
-///   MyImplementationOptions opts;
-///   opts.enable_feature = true;
-///   ...
-///   auto init_status = __xray_log_init(
-///       BufferSize, MaxBuffers, &opts, sizeof opts);
-///   if (init_status != XRayLogInitStatus::XRAY_LOG_INITIALIZED) {
+///   // We choose the mode which we'd like to install, and check whether this
+///   // has succeeded.
+///   auto select_status = __xray_log_select_mode("xray-fdr");
+///   if (select_status != XRayLogRegisterStatus::XRAY_REGISTRATION_OK) {
+///     // This failed, we should not proceed with attempting to initialise
+///     // the currently selected mode.
+///     return;
+///   }
+///
+///   // Once that's done, we can now attempt to configure the implementation.
+///   // To do this, we provide the string flags configuration for the mode.
+///   auto config_status = __xray_log_init_mode(
+///       "xray-fdr", "verbosity=1 some_flag=1 another_flag=2");
+///   if (config_status != XRayLogInitStatus::XRAY_LOG_INITIALIZED) {
 ///     // deal with the error here, if there is one.
 ///   }
 ///
 ///   // When the log implementation has had the chance to initialize, we can
-///   // now patch the sleds.
+///   // now patch the instrumentation points. Note that we could have patched
+///   // the instrumentation sleds first, but there's no strict ordering to
+///   // these operations.
 ///   auto patch_status = __xray_patch();
 ///   if (patch_status != XRayPatchingStatus::SUCCESS) {
 ///     // deal with the error here, if it is an error.
 ///   }
 ///
 ///   // If we want to stop the implementation, we can then finalize it (before
 ///   // optionally flushing the log).
 ///   auto fin_status = __xray_log_finalize();
 ///   if (fin_status != XRayLogInitStatus::XRAY_LOG_FINALIZED) {
 ///     // deal with the error here, if it is an error.
 ///   }
 ///
 ///   // We can optionally wait before flushing the log to give other threads a
 ///   // chance to see that the implementation is already finalized. Also, at
 ///   // this point we can optionally unpatch the sleds to reduce overheads at
 ///   // runtime.
 ///   auto unpatch_status = __xray_unpatch();
 ///   if (unpatch_status != XRayPatchingStatus::SUCCESS) {
-//      // deal with the error here, if it is an error.
-//    }
+///     // deal with the error here, if it is an error.
+///   }
 ///
 ///   // If there are logs or data to be flushed somewhere, we can do so only
 ///   // after we've finalized the log. Some implementations may not actually
 ///   // have anything to log (it might keep the data in memory, or periodically
 ///   // be logging the data anyway).
 ///   auto flush_status = __xray_log_flushLog();
 ///   if (flush_status != XRayLogFlushStatus::XRAY_LOG_FLUSHED) {
 ///     // deal with the error here, if it is an error.
struct XRayLogImpl {
   /// The log implementation provided routine for when __xray_log_flushLog() is
   /// called.
   ///
   /// See XRayLogFlushStatus for details on what the implementation MUST return
   /// when called.
   XRayLogFlushStatus (*flush_log)();
 };
 
+/// DEPRECATED: Use the mode registration workflow instead with
+/// __xray_log_register_mode(...) and __xray_log_select_mode(...). See the
+/// documentation for those function.
+///
 /// This function installs a new logging implementation that XRay will use. In
 /// case there are any nullptr members in Impl, XRay will *uninstall any
 /// existing implementations*. It does NOT patch the instrumentation sleds.
 ///
 /// NOTE: This function does NOT attempt to finalize the currently installed
 /// implementation. Use with caution.
 ///
 /// It is guaranteed safe to call this function in the following states:

 ///   - When the implementation is UNINITIALIZED.
 ///   - When the implementation is FINALIZED.
 ///   - When there is no current implementation installed.
 ///
 /// It is logging implementation defined what happens when this function is
 /// called while in any other states.
 void __xray_remove_log_impl();
 
+/// DEPRECATED: Use __xray_log_init_mode() instead, and provide all the options
+/// in string form.
 /// Invokes the installed implementation initialization routine. See
 /// XRayLogInitStatus for what the return values mean.
 XRayLogInitStatus __xray_log_init(size_t BufferSize, size_t MaxBuffers,
                                   void *Args, size_t ArgsSize);
 
+/// Invokes the installed initialization routine, which *must* support the
+/// string based form.
+///
+/// NOTE: When this API is used, we still invoke the installed initialization
+/// routine, but we will call it with the following convention to signal that we
+/// are using the string form:
+///
+/// - BufferSize = 0
+/// - MaxBuffers = std::numeric_limit<size_t>::max()
+/// - ArgsSize = 0
+/// - Args will be the pointer to the null-terminated character buffer
+/// representing the configuration.
+///
+/// FIXME: Updating the XRayLogImpl struct is an ABI breaking change. When we
+/// are ready to make a breaking change, we should clean this up appropriately.
+XRayLogInitStatus __xray_log_init_mode(const char* Mode, const char* Config);
+
 /// Invokes the installed implementation finalization routine. See
 /// XRayLogInitStatus for what the return values mean.
 XRayLogInitStatus __xray_log_finalize();
 
 /// Invokes the install implementation log flushing routine. See
 /// XRayLogFlushStatus for what the return values mean.
 XRayLogFlushStatus __xray_log_flushLog();
 

 ///
 XRayLogFlushStatus __xray_log_process_buffers(void (*Processor)(const char *,
                                                                 XRayBuffer));
 
 } // extern "C"
 
 namespace __xray {
 
+/// DEPRECATED: Use __xray_log_init_mode(...) instead, and provide flag
+/// configuration strings to set the options instead.
 /// Options used by the LLVM XRay FDR logging implementation.
 struct FDRLoggingOptions {
   bool ReportErrors = false;
   int Fd = -1;
 };
 
+/// DEPRECATED: Use __xray_log_init_mode(...) instead, and provide flag
+/// configuration strings to set the options instead.
 /// Options used by the LLVM XRay Basic (Naive) logging implementation.
 struct BasicLoggingOptions {
   int DurationFilterMicros = 0;
   size_t MaxStackDepth = 0;
   size_t ThreadBufferSize = 0;
 };
 
 } // namespace __xray
 
 #endif // XRAY_XRAY_LOG_INTERFACE_H

compiler-rt/lib/xray/xray_log_interface.cc

XRayLogInitStatus __xray_log_init(size_t BufferSize, size_t MaxBuffers,
                                   void *Args,
                                   size_t ArgsSize) XRAY_NEVER_INSTRUMENT {
   __sanitizer::SpinMutexLock Guard(&XRayImplMutex);
   if (!GlobalXRayImpl)
     return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
   return GlobalXRayImpl->log_init(BufferSize, MaxBuffers, Args, ArgsSize);
 }
 
+XRayLogInitStatus __xray_log_init_mode(const char *Mode, const char *Config)
+    XRAY_NEVER_INSTRUMENT {
+  __sanitizer::SpinMutexLock Guard(&XRayImplMutex);
+  if (!GlobalXRayImpl)
+    return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
+
+  // Check first whether the current mode is the same as what we expect.
+  if (CurrentMode == nullptr || internal_strcmp(CurrentMode->Mode, Mode) != 0)
+    return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
+
+  // Here we do some work to coerce the pointer we're provided, so that
+  // the implementations that still take void* pointers can handle the
+  // data provided in the Config argument.
+  return GlobalXRayImpl->log_init(
+      0, 0, const_cast<void *>(static_cast<const void *>(Config)), 0);
+}
+
 XRayLogInitStatus __xray_log_finalize() XRAY_NEVER_INSTRUMENT {
   __sanitizer::SpinMutexLock Guard(&XRayImplMutex);
   if (!GlobalXRayImpl)
     return XRayLogInitStatus::XRAY_LOG_UNINITIALIZED;
   return GlobalXRayImpl->log_finalize();
 }
 
 XRayLogFlushStatus __xray_log_flushLog() XRAY_NEVER_INSTRUMENT {

compiler-rt/test/xray/TestCases/Posix/logging-modes.cc

 // Check that we can install an implementation associated with a mode.
 //
-// RUN: %clangxx_xray -std=c++11 %s -o %t
+// RUN: rm xray-log.logging-modes* || true
+// RUN: %clangxx_xray -std=c++11 %s -o %t -fxray-modes=none
 // RUN: %run %t | FileCheck %s
 //
 // UNSUPPORTED: target-is-mips64,target-is-mips64el
 
 #include "xray/xray_interface.h"
 #include "xray/xray_log_interface.h"
 #include <cassert>
 #include <cstdio>
[[clang::xray_never_instrument]] XRayBuffer next_buffer(XRayBuffer buffer) {
   if (buffer.Data == nullptr)
     return first_and_last;
   if (buffer.Data == first_and_last.Data)
     return XRayBuffer{nullptr, 0};
   assert(false && "Invalid buffer provided.");
 }
 
 [[clang::xray_never_instrument]] XRayLogInitStatus
-printing_init(size_t, size_t, void *, size_t) {
+printing_init(size_t BufferSize, size_t MaxBuffers, void *Config,
+              size_t ArgsSize) {
+  // We require that the printing init is called through the
+  // __xray_log_init_mode(...) implementation, and that the promised contract is
+  // enforced.
+  assert(BufferSize == 0);
+  assert(MaxBuffers == 0);
+  assert(Config != nullptr);
+  assert(ArgsSize == 0);
   __xray_log_set_buffer_iterator(next_buffer);
   return XRayLogInitStatus::XRAY_LOG_INITIALIZED;
 }
 
 [[clang::xray_never_instrument]] XRayLogInitStatus printing_finalize() {
   return XRayLogInitStatus::XRAY_LOG_FINALIZED;
 }
 
 [[clang::xray_never_instrument]] XRayLogFlushStatus printing_flush_log() {
   __xray_log_remove_buffer_iterator();
   return XRayLogFlushStatus::XRAY_LOG_FLUSHED;
 }
 
 [[clang::xray_always_instrument]] void callme() { std::printf("called me!\n"); }
 
-static auto buffer_counter = 0;
+static auto volatile buffer_counter = 0;
 
 void process_buffer(const char *, XRayBuffer) { ++buffer_counter; }
 
-static bool unused = [] {
+int main(int argc, char **argv) {
   assert(__xray_log_register_mode("custom",
                                   {printing_init, printing_finalize,
                                    printing_handler, printing_flush_log}) ==
          XRayLogRegisterStatus::XRAY_REGISTRATION_OK);
-  return true;
-}();
-
-int main(int argc, char **argv) {
   assert(__xray_log_select_mode("custom") ==
          XRayLogRegisterStatus::XRAY_REGISTRATION_OK);
   assert(__xray_log_get_current_mode() != nullptr);
   std::string current_mode = __xray_log_get_current_mode();
   assert(current_mode == "custom");
   assert(__xray_patch() == XRayPatchingStatus::SUCCESS);
-  assert(__xray_log_init(0, 0, nullptr, 0) ==
+  assert(__xray_log_init_mode("custom", "flags_config_here=true") ==
          XRayLogInitStatus::XRAY_LOG_INITIALIZED);
   // CHECK: printing {{.*}}
   callme(); // CHECK: called me!
   // CHECK: printing {{.*}}
   assert(__xray_log_finalize() == XRayLogInitStatus::XRAY_LOG_FINALIZED);
   assert(__xray_log_process_buffers(process_buffer) ==
          XRayLogFlushStatus::XRAY_LOG_FLUSHED);
   assert(buffer_counter == 1);
   assert(__xray_log_flushLog() == XRayLogFlushStatus::XRAY_LOG_FLUSHED);
   assert(__xray_log_select_mode("not-found") ==
          XRayLogRegisterStatus::XRAY_MODE_NOT_FOUND);
 }

llvm/docs/XRay.rst

 +-------------------+-----------------+---------------+------------------------+
 | xray_mode         | ``const char*`` | ``""``        | Default mode to        |
 |                   |                 |               | install and initialize |
 |                   |                 |               | before ``main``.       |
 +-------------------+-----------------+---------------+------------------------+
 | xray_logfile_base | ``const char*`` | ``xray-log.`` | Filename base for the  |
 |                   |                 |               | XRay logfile.          |
 +-------------------+-----------------+---------------+------------------------+
-| xray_naive_log    | ``bool``        | ``false``     | **DEPRECATED:** Use    |
-|                   |                 |               | xray_mode=xray-basic   |
-|                   |                 |               | instead. Whether to    |
-|                   |                 |               | install the basic log  |
-|                   |                 |               | the naive log          |
-|                   |                 |               | implementation.        |
-+-------------------+-----------------+---------------+------------------------+
-| xray_fdr_log      | ``bool``        | ``false``     | **DEPRECATED:** Use    |
-|                   |                 |               | xray_mode=xray-fdr     |
-|                   |                 |               | instead. Whether to    |
-|                   |                 |               | install the Flight     |
-|                   |                 |               | Data Recorder          |
-|                   |                 |               | (FDR) mode.            |
-+-------------------+-----------------+---------------+------------------------+
 | verbosity         | ``int``         | ``0``         | Runtime verbosity      |
 |                   |                 |               | level.                 |
 +-------------------+-----------------+---------------+------------------------+
 
 
 If you choose to not use the default logging implementation that comes with the
 XRay runtime and/or control when/how the XRay instrumentation runs, you may use
 the XRay APIs directly for doing so. To do this, you'll need to include the
-``xray_interface.h`` from the compiler-rt ``xray`` directory. The important API
+``xray_log_interface.h`` from the compiler-rt ``xray`` directory. The important API
 functions we list below:
 
-- ``__xray_set_handler(void (*entry)(int32_t, XRayEntryType))``: Install your
-  own logging handler for when an event is encountered. See
-  ``xray/xray_interface.h`` for more details.
-- ``__xray_remove_handler()``: Removes whatever the installed handler is.
-- ``__xray_patch()``: Patch all the instrumentation points defined in the
-  binary.
-- ``__xray_unpatch()``: Unpatch the instrumentation points defined in the
-  binary.
-
-There are some requirements on the logging handler to be installed for the
-thread-safety of operations to be performed by the XRay runtime library:
-
-- The function should be thread-safe, as multiple threads may be invoking the
-  function at the same time. If the logging function needs to do
-  synchronisation, it must do so internally as XRay does not provide any
-  synchronisation guarantees outside from the atomicity of updates to the
-  pointer.
-- The pointer provided to ``__xray_set_handler(...)`` must be live even after
-  calls to ``__xray_remove_handler()`` and ``__xray_unpatch()`` have succeeded.
-  XRay cannot guarantee that all threads that have ever gotten a copy of the
-  pointer will not invoke the function.
+- ``__xray_log_register_mode(...)``: Register a logging implementation against
+  a string Mode identifier. The implementation is an instance of
+  ``XRayLogImpl`` defined in ``xray/xray_log_interface.h``.
+- ``__xray_log_select_mode(...)``: Select the mode to install, associated with
+  a string Mode identifier. Only implementations registered with
+  ``__xray_log_register_mode(...)`` can be chosen with this function.
+- ``__xray_log_init_mode(...)``: This function allows for initializing and
+  re-initializing an installed logging implementation. See
+  ``xray/xray_log_interface.h`` for details, part of the XRay compiler-rt
+  installation.
+
+Once a logging implementation has been initialized, it can be "stopped" by
+finalizing the implementation through the ``__xray_log_finalize()`` function.
+The finalization routine is the opposite of the initialization. When finalized,
+an implementation's data can be cleared out through the
+``__xray_log_flushLog()`` function. For implementations that support in-memory
+processing, these should register an iterator function to provide access to the
+data via the ``__xray_log_set_buffer_iterator(...)`` which allows code calling
+the ``__xray_log_process_buffers(...)`` function to deal with the data in
+memory.
+
+All of this is better explained in the ``xray/xray_log_interface.h`` header.
+
+Basic Mode
+----------
+
+XRay supports a basic logging mode which will trace the application's
+execution, and periodically append to a single log. This mode can be
+installed/enabled by setting ``xray_mode=xray-basic`` in the ``XRAY_OPTIONS``
+environment variable. Combined with ``patch_premain=true`` this can allow for
+tracing applications from start to end.
+
+Like all the other modes installed through ``__xray_log_select_mode(...)``, the
+implementation can be configured through the ``__xray_log_init_mode(...)``
+function, providing the mode string and the flag options. Basic-mode specific
+defaults can be provided in the ``XRAY_BASIC_OPTIONS`` environment variable.
 
 Flight Data Recorder Mode
 -------------------------
 
 XRay supports a logging mode which allows the application to only capture a
 fixed amount of memory's worth of events. Flight Data Recorder (FDR) mode works
 very much like a plane's "black box" which keeps recording data to memory in a
 fixed-size circular queue of buffers, and have the data available
 programmatically until the buffers are finalized and flushed. To use FDR mode
-on your application, you may set the ``xray_fdr_log`` option to ``true`` in the
-``XRAY_OPTIONS`` environment variable (while also optionally setting the
-``xray_naive_log`` to ``false``).
+on your application, you may set the ``xray_mode`` variable to ``xray-fdr`` in
+the ``XRAY_OPTIONS`` environment variable. Additional options to the FDR mode
+implementation can be provided in the ``XRAY_FDR_OPTIONS`` environment
+variable. Programmatic configuration can be done by calling
+``__xray_log_init_mode("xray-fdr", <configuration string>)`` once it has been
+selected/installed.
 
 When the buffers are flushed to disk, the result is a binary trace format
 described by `XRay FDR format <XRayFDRFormat.html>`_
 
 When FDR mode is on, it will keep writing and recycling memory buffers until
 the logging implementation is finalized -- at which point it can be flushed and
 re-initialised later. To do this programmatically, we follow the workflow
 provided below:
.. code-block:: c++
   // At this point, we are sure that the log is finalized, so we may try
   // flushing the log.
   auto flush_status = __xray_log_flushLog();
   if (flush_status != XRAY_LOG_FLUSHED) {
     // maybe retry, or bail out.
   }
 
 The default settings for the FDR mode implementation will create logs named
-similarly to the naive log implementation, but will have a different log
+similarly to the basic log implementation, but will have a different log
 format. All the trace analysis tools (and the trace reading library) will
 support all versions of the FDR mode format as we add more functionality and
 record types in the future.
 
-  **NOTE:** We do not however promise perpetual support for when we update the
-  log versions we support going forward. Deprecation of the formats will be
+  **NOTE:** We do not promise perpetual support for when we update the log
+  versions we support going forward. Deprecation of the formats will be
   announced and discussed on the developers mailing list.
 
-XRay allows for replacing the default FDR mode logging implementation using the
-following API:
-
-- ``__xray_set_log_impl(...)``: This function takes a struct of type
-  ``XRayLogImpl``, which is defined in ``xray/xray_log_interface.h``, part of
-  the XRay compiler-rt installation.
-- ``__xray_log_register_mode(...)``: Register a logging implementation against
-  a string Mode. The implementation is an instance of ``XRayLogImpl`` defined
-  in ``xray/xray_log_interface.h``.
-- ``__xray_log_select_mode(...)``: Select the mode to install, associated with
-  a string Mode. Only implementations registered with
-  ``__xray_log_register_mode(...)`` can be chosen with this function. When
-  successful, has the same effects as calling ``__xray_set_log_impl(...)`` with
-  the registered logging implementation.
-- ``__xray_log_init(...)``: This function allows for initializing and
-  re-initializing an installed logging implementation. See
-  ``xray/xray_log_interface.h`` for details, part of the XRay compiler-rt
-  installation.
-
 Trace Analysis Tools
 --------------------
 
 We currently have the beginnings of a trace analysis tool in LLVM, which can be
 found in the ``tools/llvm-xray`` directory. The ``llvm-xray`` tool currently
 supports the following subcommands:
 
 - ``extract``: Extract the instrumentation map from a binary, and return it as
   YAML.
 - ``account``: Performs basic function call accounting statistics with various
   options for sorting, and output formats (supports CSV, YAML, and
   console-friendly TEXT).
 - ``convert``: Converts an XRay log file from one format to another. We can
-  convert from binary XRay traces (both naive and FDR mode) to YAML,
+  convert from binary XRay traces (both basic and FDR mode) to YAML,
   `flame-graph <https://github.com/brendangregg/FlameGraph>`_ friendly text
   formats, as well as `Chrome Trace Viewer (catapult)
   <https://github.com/catapult-project/catapult>` formats.
 - ``graph``: Generates a DOT graph of the function call relationships between
   functions found in an XRay trace.
 - ``stack``: Reconstructs function call stacks from a timeline of function
   calls in an XRay trace.