[llvm-xsan] Introduce llvm-xsan

[delcypher]

Summary

llvm-xsan is a tool that contains a set of subtools for working with the
Sanitizers. The goal here is to provide useful tools for working with
the Sanitizers that can be shipped as part of an LLVM toolchain.

This patch includes the following subtools

• version - Reports the tool version.
• asan get-schema - A tool to report the schema that describes ASan reports.
• asan symbolicate - A tool to symbolicate ASan reports.
• asan parse - A tool to parse ASan reports.

Other tools (e.g. kasan <subtool>, tsan <subtool>, ubsan <subtool>`)
could be added in the future.

Only the version tool contains a full implementation. The other tools
just contain stubs that will be implemented in future patches.

llvm-xsan will not be installed by default but can be installed by
setting the new LLVM_INSTALL_LLVM_XSAN CMake option to ON.

This patch also includes changes to allow testing of llvm-xsan from any
of the compiler-rt test suites. A llvm-xsan-misc test suite has also
been added that tests features that are independent of any particular
sanitizer.

rdar://problem/67436450

[vitalybuka]

This comment was removed by vitalybuka.

[vitalybuka]

I can't see "rdar" content and motivations for these scripts, and I don't expect myself extend my workflow with another layer between me and sanitizers.
As a negative side, if it's committed it will be community responsibility to maintain.
But I have no strong opinion so I am going to resign.

[delcypher]

In D88938#2607598, @vitalybuka wrote:

I can't see "rdar" content and motivations for these scripts, and I don't expect myself extend my workflow with another layer between me and sanitizers.
As a negative side, if it's committed it will be community responsibility to maintain.
But I have no strong opinion so I am going to resign.

@vitalybuka Fair enough. Sorry I think you had a question here that's now deleted that I missed. I think it was a question about why this is in compiler-rt and not llvm. The reason for that is because many of the tests will require parsing the output of the Sanitizers which is defined in the runtime. The most convenient place to run binaries built with Sanitizers to parse their output is compiler-rt because we already do that extensively there to test the Sanitizers.

[vitalybuka]

In D88938#2608062, @delcypher wrote:

In D88938#2607598, @vitalybuka wrote:

I can't see "rdar" content and motivations for these scripts, and I don't expect myself extend my workflow with another layer between me and sanitizers.
As a negative side, if it's committed it will be community responsibility to maintain.
But I have no strong opinion so I am going to resign.

@vitalybuka Fair enough. Sorry I think you had a question here that's now deleted that I missed. I think it was a question about why this is in compiler-rt and not llvm. The reason for that is because many of the tests will require parsing the output of the Sanitizers which is defined in the runtime. The most convenient place to run binaries built with Sanitizers to parse their output is compiler-rt because we already do that extensively there to test the Sanitizers.

Actually question was "Why this is a part of llvm and not compiler-rt?". But it's already in compiler-rt, so I've deleted the question :)

compiler-rt/test/CMakeLists.txt

if(COMPILER_RT_BUILD_XRAY)
     compiler_rt_test_runtime(xray)
   endif()
   if(COMPILER_RT_BUILD_CRT AND COMPILER_RT_HAS_CRT)
     add_subdirectory(crt)
   endif()
   # ShadowCallStack does not yet provide a runtime with compiler-rt, the tests
   # include their own minimal runtime
   add_subdirectory(shadowcallstack)
+
+  add_subdirectory(llvm-xsan-misc)
 endif()
 
 if(COMPILER_RT_STANDALONE_BUILD)
   # Now that we've traversed all the directories and know all the lit testsuites,
   # introduce a rule to run to run all of them.
   get_property(LLVM_LIT_TESTSUITES GLOBAL PROPERTY LLVM_LIT_TESTSUITES)
   get_property(LLVM_LIT_DEPENDS GLOBAL PROPERTY LLVM_LIT_DEPENDS)
   add_lit_target(check-compiler-rt

compiler-rt/test/lit.common.cfg.py

 # -*- Python -*-
 
 # Configuration file for 'lit' test runner.
 # This file contains common rules for various compiler-rt testsuites.
 # It is mostly copied from lit.cfg.py used by Clang.
 import os
 import platform
 import re
 import subprocess
 import json
 
 import lit.formats
 import lit.util
 
+# Get shlex.quote if available (added in 3.3), and fall back to pipes.quote if
+# it's not available.
+try:
+  import shlex
+  sh_quote = shlex.quote
+except:
+  import pipes
+  sh_quote = pipes.quote
+
+def get_required_attr(config, attr_name):
+  attr_value = getattr(config, attr_name, None)
+  if attr_value == None:
+    lit_config.fatal(
+      "No attribute %r in test configuration! You may need to run "
+      "tests from your build directory or add this attribute "
+      "to lit.site.cfg.py " % attr_name)
+  return attr_value
+
 # Choose between lit's internal shell pipeline runner and a real shell.  If
 # LIT_USE_INTERNAL_SHELL is in the environment, we use that as an override.
 use_lit_shell = os.environ.get("LIT_USE_INTERNAL_SHELL")
 if use_lit_shell:
     # 0 is external, "" is default, and everything else is internal.
     execute_external = (use_lit_shell == "0")
 else:
     # Otherwise we default to internal on Windows and external elsewhere, as

 
 if config.use_lld and config.has_lld and not config.use_lto:
   extra_cflags += ["-fuse-ld=lld"]
 elif config.use_lld and (not config.has_lld):
   config.unsupported = True
 
 config.clang = " " + " ".join(run_wrapper + [config.compile_wrapper, config.clang]) + " "
 config.target_cflags = " " + " ".join(target_cflags + extra_cflags) + " "
+
+# llvm-xsan
+python_exec = sh_quote(get_required_attr(config, "python_executable"))
+llvm_xsan_path = os.path.join(get_required_attr(config, "llvm_tools_dir"), "llvm-xsan")
+if not os.path.exists(llvm_xsan_path):
+  lit_config.fatal('Cannot find llvm-xsan at path: "{}"'.format(llvm_xsan_path))
+llvm_xsan_subst = "{} {}".format(
+  python_exec,
+  sh_quote(llvm_xsan_path)
+)
+config.substitutions.append(('%llvm-xsan%', llvm_xsan_subst))

compiler-rt/test/llvm-xsan-misc/CMakeLists.txt

+set(LLVM_XSAN_MISC_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
+set(TEST_DIR "${CMAKE_CURRENT_BINARY_DIR}/TestCases")
+configure_lit_site_cfg(
+  ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
+  ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
+)
+
+add_lit_testsuite(check-llvm-xsan-misc "Running llvm-xsan misc tests"
+  ${TEST_DIR}
+  DEPENDS ${ASAN_TEST_DEPS})
+set_target_properties(check-llvm-xsan-misc PROPERTIES FOLDER "Compiler-RT Misc")
+
+# TODO(dliew): Add a top-level check-llvm-xsan target once we have multiple
+# targets spread across the different sanitizers

compiler-rt/test/llvm-xsan-misc/TestCases/help.py

+# RUN: %llvm-xsan% --help

compiler-rt/test/llvm-xsan-misc/TestCases/version.py

+# String output
+# RUN: %llvm-xsan% version --format string | FileCheck --check-prefix=CHECK-STR %s
+# CHECK-STR: {{[0-9]+\.[0-9]+\.[0.9]}}
+
+# JSON output:
+# FIXME(dliew): Parse the JSON and validate it properly.
+# RUN: %llvm-xsan% version --format json | FileCheck --check-prefix=CHECK-JSON %s
+# CHECK-JSON: "major":{{ *[0-9]+}}
+# CHECK-JSON: "minor":{{ *[0-9]+}}
+# CHECK-JSON: "patch":{{ *[0-9]+}}
+
+# Subtool's help output
+# RUN: %llvm-xsan% version --help | FileCheck --check-prefix=CHECK-HELP %s
+# CHECK-HELP: llvm-xsan version
+# CHECK-HELP: Prints the tool version

compiler-rt/test/llvm-xsan-misc/lit.cfg.py

+import os
+import sys
+
+config.name = 'llvm-xsan-misc'
+config.test_source_root = os.path.dirname(__file__)
+config.suffixes = ['.c', '.cpp', '.py']
+
+# FIXME(dliew): lit shouldn't need to be told to ignore this.
+config.excludes.add('lit.cfg.py')
+
+# llvm-xsan does not support python 2
+if sys.version_info < (3,0):
+  config.unsupported = True

compiler-rt/test/llvm-xsan-misc/lit.site.cfg.py.in

+@LIT_SITE_CFG_IN_HEADER@
+
+# Load common config for all compiler-rt lit tests.
+lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
+
+# Load tool-specific config that would do the real work.
+lit_config.load_config(config, "@LLVM_XSAN_MISC_SOURCE_DIR@/lit.cfg.py")

compiler-rt/tools/CMakeLists.txt

 add_subdirectory(gwp_asan)
+add_subdirectory(llvm-xsan)

compiler-rt/tools/llvm-xsan/.flake8

+[flake8]
+max-line-length=80

compiler-rt/tools/llvm-xsan/.gitignore

+llvm_xsan.egg-info
+build

compiler-rt/tools/llvm-xsan/.pylintrc

+[MESSAGES CONTROL]
+disable=missing-function-docstring,
+  missing-module-docstring,
+  no-self-use,
+  too-few-public-methods,
+  missing-class-docstring,
+  # misfires for setup.py
+  cyclic-import,
+  unnecessary-pass

compiler-rt/tools/llvm-xsan/CMakeLists.txt

+# Put binary in tools directory
+set(LLVM_XSAN_VERSION_MAJOR ${LLVM_VERSION_MAJOR})
+set(LLVM_XSAN_VERSION_MINOR ${LLVM_VERSION_MINOR})
+set(LLVM_XSAN_VERSION_PATCH ${LLVM_VERSION_MINOR})
+configure_file(llvm-xsan.in "${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/llvm-xsan" @ONLY)
+
+# TODO: Set the default to ON once the tool is of sufficient quality.
+option(LLVM_INSTALL_LLVM_XSAN OFF "Install the llvm-xsan tool")
+
+if (LLVM_INSTALL_LLVM_XSAN)
+  if (NOT EXISTS "${Python3_EXECUTABLE}")
+    message(FATAL_ERROR "Python3_EXECUTABLE must be set")
+  endif()
+  set(PYTHON_EXECUTABLE_FOR_SETUP "${Python3_EXECUTABLE}")
+
+  # Specify files to copy over. This is to avoid polluting the
+  # source tree.
+  set(PATHS_TO_COPY
+    "${CMAKE_CURRENT_SOURCE_DIR}/setup.py"
+    "${CMAKE_CURRENT_SOURCE_DIR}/llvm_xsan"
+  )
+
+  set(WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}")
+  configure_file(llvm_xsan_install.cmake.in llvm_xsan_install.cmake @ONLY)
+  install(
+    SCRIPT "${CMAKE_CURRENT_BINARY_DIR}/llvm_xsan_install.cmake"
+    COMPONENT llvm-xsan
+  )
+endif()

compiler-rt/tools/llvm-xsan/llvm-xsan.in

+#!@Python3_EXECUTABLE@
+import os
+import sys
+
+# Make sure we can find the llvm_xsan package
+sys.path.insert(0, os.path.join('@CMAKE_CURRENT_SOURCE_DIR@'))
+
+if __name__=='__main__':
+    from llvm_xsan import driver
+    version = (
+        @LLVM_XSAN_VERSION_MAJOR@,
+        @LLVM_XSAN_VERSION_MINOR@,
+        @LLVM_XSAN_VERSION_PATCH@
+    )
+    driver.llvm_bin_dir_entry_point(version)

compiler-rt/tools/llvm-xsan/llvm_xsan/asan/subtool_asan.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+from .. import driver
+
+
+class ASanParentSubTool(driver.ParentSubTool):
+    """ A collection of tools for AddressSanitizer (ASan) """
+    def __init__(self):
+        # pylint: disable=import-outside-toplevel
+        super().__init__('asan')
+
+        # Register core tools
+        from . import subtool_parse
+        self.add_child_tool(subtool_parse.ASanParseSubTool())
+        from . import subtool_symbolicate
+        self.add_child_tool(subtool_symbolicate.ASanSymbolicateSubTool())
+        from . import subtool_get_schema
+        self.add_child_tool(subtool_get_schema.ASanGetSchemaSubTool())

compiler-rt/tools/llvm-xsan/llvm_xsan/asan/subtool_get_schema.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+from .. import driver
+
+
+class ASanGetSchemaSubTool(driver.EntryPointSubTool):
+    """
+        Outputs the Schema defining structed ASan reports.
+        The schema definition is itself defined by jsonschema.
+    """
+    def __init__(self):
+        super().__init__('get-schema')
+
+    def register_args(self, _):
+        pass
+
+    def entry_point(self, pargs):
+        raise NotImplementedError()

compiler-rt/tools/llvm-xsan/llvm_xsan/asan/subtool_parse.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+from .. import driver
+
+
+class ASanParseSubTool(driver.EntryPointSubTool):
+    """ Parses ASan reports"""
+    def __init__(self):
+        super().__init__('parse')
+
+    def register_args(self, _):
+        pass
+
+    def entry_point(self, pargs):
+        raise NotImplementedError()

compiler-rt/tools/llvm-xsan/llvm_xsan/asan/subtool_symbolicate.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+from .. import driver
+
+
+class ASanSymbolicateSubTool(driver.EntryPointSubTool):
+    """ Symbolicates ASan reports """
+    def __init__(self):
+        super().__init__('symbolicate')
+
+    def register_args(self, _):
+        pass
+
+    def entry_point(self, pargs):
+        raise NotImplementedError()

compiler-rt/tools/llvm-xsan/llvm_xsan/driver.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+"""
+llvm-xsan is a collection of tools for working with the Sanitizers.
+"""
+from enum import Enum
+import argparse
+import logging
+import sys
+from . import version_info
+
+_logger = logging.getLogger(__name__)
+
+
+def setuptools_entry_point():
+    """
+        This is the entry point used by the script generated by
+        setuptools.
+    """
+    # Get version from special file created by CMake at install time.
+    from . import version_info_install  # noqa: E501 ; pylint: disable=import-outside-toplevel,no-name-in-module
+    version_info_install._register_version()  # pylint: disable=protected-access
+    sys.exit(entry_point(sys.argv))
+
+
+def llvm_bin_dir_entry_point(ver):
+    """
+        This is the entry point used by the script placed in the LLVM build
+        tree.
+
+        It is expected to be passed `ver` which is a tuple containing the tool
+        version.
+    """
+    version_info._register_version(  # pylint: disable=protected-access
+        major=ver[0],
+        minor=ver[1],
+        patch=ver[2]
+    )
+    sys.exit(entry_point(sys.argv))
+
+
+def entry_point(args):
+    driver = Driver(args)
+    return driver.entry_point()
+
+
+class ExitCodes(Enum):
+    """
+        The different tool exit codes.
+    """
+    SUCCESS = 0
+    CMDLINE_ERROR = 1
+
+
+class Driver:
+    """
+        The top level driver.
+    """
+    def __init__(self, args):
+        self._original_args = args
+        self.arg_parser = None
+
+    def entry_point(self):
+        unparsed_args = self._original_args[1:]
+        unparsed_args = self._setup_logging(unparsed_args)
+
+        self.arg_parser = argparse.ArgumentParser(
+            description=__doc__,
+            prog=self._original_args[0]
+        )
+        # Add these args so they show up in `--help` even though we don't parse
+        # them here.
+        self._add_logging_args(self.arg_parser)
+
+        self.arg_parser.set_defaults(func=None)
+        subparsers = self.arg_parser.add_subparsers()
+        self._load_subtools(subparsers)
+
+        # Parse the remaining args
+        pargs = self.arg_parser.parse_args(unparsed_args)
+
+        # Call the sub-tool entry point
+        if pargs.func is None:
+            _logger.error('A subtool must be selected')
+            self.arg_parser.print_usage()
+            return ExitCodes.CMDLINE_ERROR.value
+        exit_code = pargs.func(pargs)
+        return exit_code
+
+    def _add_logging_args(self, parser):
+        parser.add_argument(
+            '--log-dest',
+            default=None,
+            help='Destination path for script logging (default stderr).',
+        )
+        parser.add_argument(
+            '--log-level',
+            choices=['debug', 'info', 'warning', 'error', 'critical'],
+            default='info',
+            help='Log level for script (default: %(default)s).'
+        )
+
+    def _setup_logging(self, args):
+        # Set up a parser just for parsing the logging arguments.
+        # This is necessary because logging should be configured before we
+        # perform the main argument parsing.
+        parser = argparse.ArgumentParser(add_help=False)
+        self._add_logging_args(parser)
+        pargs, unparsed_args = parser.parse_known_args(args)
+
+        log_level = getattr(logging, pargs.log_level.upper())
+        if log_level == logging.DEBUG:
+            # pylint: disable=line-too-long
+            log_format = '%(levelname)s: [%(funcName)s() %(filename)s:%(lineno)d] %(message)s'  # noqa: E501
+        else:
+            log_format = '%(levelname)s: %(message)s'
+        basic_config = {
+            'level': log_level,
+            'format': log_format
+        }
+        log_dest = pargs.log_dest
+        if log_dest:
+            basic_config['filename'] = log_dest
+        logging.basicConfig(**basic_config)
+        _logger.debug(
+            'Logging level set to "%s" and directing output to "%s"',
+            pargs.log_level,
+            'stderr' if log_dest is None else log_dest
+        )
+        return unparsed_args
+
+    def _load_subtools(self, subparsers):
+        # pylint: disable=import-outside-toplevel
+        from .subtool_version import VersionTool
+        VersionTool().register_subparser(subparsers)
+
+        from .asan.subtool_asan import ASanParentSubTool
+        ASanParentSubTool().register_subparser(subparsers)
+
+
+class SubTool:
+    """
+        Common parent class for all subtools.
+    """
+    def __init__(self, name, **kwargs):
+        self._name = name
+        self._parser = None
+        self._parent = None
+        self._kwargs = kwargs
+
+    @property
+    def name(self):
+        """
+            Return the unqualified name of the subtool
+        """
+        return self._name
+
+    @property
+    def parent(self):
+        """
+            Return the parent `SubTool` object or `None`
+            if this is the root `SubTool`.
+        """
+        return self._parent
+
+    @parent.setter
+    def parent(self, new_parent):
+        assert self._parent is None
+        assert isinstance(new_parent, SubTool)
+        self._parent = new_parent
+
+    def compute_full_name(self):
+        """
+            Return the fully qualified name of the subtool.
+        """
+        ancestry = [self.name]
+        parent = self.parent
+        while parent is not None:
+            ancestry.append(parent.name)
+            parent = parent.parent
+        full_name = " ".join(reversed(ancestry))
+        return full_name
+
+    def register_subparser(self, subparsers):
+        """
+            Register this subtool as a subparser.
+            `subparsers` should be the object returned by calling
+            `parser.add_subparsers()` where `parser` is the parent
+            parser.
+        """
+        _logger.debug('Registering subtool "%s"', self.compute_full_name())
+        self._parser = subparsers.add_parser(
+            self._name,
+            description=self.__doc__,
+            **self._kwargs
+        )
+        self.register_args(self._parser)
+
+    def register_args(self, parser):
+        """
+            Implementations should override this to register
+            their needed command line arguments.
+        """
+        pass
+
+
+class ParentSubTool(SubTool):
+    """
+        Parent class for subtools that are a parent for other
+        subtools.
+    """
+    def __init__(self, name, **kwargs):
+        super().__init__(name, **kwargs)
+        self._child_tools = []
+        self._register_subparser_called = False
+
+    def add_child_tool(self, subtool):
+        """
+            Add a child tool `subtool`. All calls to this method
+            should be made before calling `register_subparser()`.
+        """
+        assert isinstance(subtool, (ParentSubTool, EntryPointSubTool))
+        assert not self._register_subparser_called
+        subtool.parent = self
+        self._child_tools.append(subtool)
+
+    def register_subparser(self, subparsers):
+        self._register_subparser_called = True
+        super().register_subparser(subparsers)
+        self._parser.set_defaults(func=self.entry_point)
+
+        # Add the child tools
+        assert len(self._child_tools) > 0
+        child_subparser = self._parser.add_subparsers()
+        for child_tool in self._child_tools:
+            child_tool.register_subparser(child_subparser)
+
+    def entry_point(self, _):
+        # Parent subtools just print their usage and exit
+        self._parser.print_usage()
+        return ExitCodes.SUCCESS.value
+
+
+class EntryPointSubTool(SubTool):
+    """
+        Parent class for subtools with an entry-point.  Implementations must
+        define an `entry_point` method and a docstring.
+
+        Implementations can optionally implement the `register_args` method
+        to add command line arguments to the subtool.
+    """
+    def __init__(self, name, **kwargs):
+        super().__init__(name, **kwargs)
+        assert hasattr(self, 'entry_point')
+
+    def register_subparser(self, subparsers):
+        # pylint: disable=no-member
+        super().register_subparser(subparsers)
+        self._parser.set_defaults(func=self.entry_point)

compiler-rt/tools/llvm-xsan/llvm_xsan/subtool_version.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+import json
+from . import driver
+from . import version_info
+
+
+class VersionTool(driver.EntryPointSubTool):
+    """Prints the tool version"""
+
+    def __init__(self):
+        super().__init__('version')
+
+    def register_args(self, parser):
+        parser.add_argument(
+            '--format',
+            choices=['string', 'json'],
+            default='json',
+            help='Output format for version'
+        )
+
+    def entry_point(self, pargs):
+        if pargs.format == 'json':
+            version = version_info.get_version()
+            json_output = {
+                'major': version[0],
+                'minor': version[1],
+                'patch': version[2]
+            }
+            print(json.dumps(json_output))
+        elif pargs.format == 'string':
+            print(version_info.get_version_str())
+        else:
+            raise ValueError('format "{}" is not handled'.format(pargs.format))
+        return driver.ExitCodes.SUCCESS.value

compiler-rt/tools/llvm-xsan/llvm_xsan/version_info.py

+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+"""
+    Provides an interface to access the package version.
+"""
+
+_VERSION = (0, 0, 0)
+
+
+def get_version():
+    return _VERSION
+
+
+def get_version_str():
+    ver = get_version()
+    return "{}.{}.{}".format(ver[0], ver[1], ver[2])
+
+
+def _register_version(major, minor, patch):
+    global _VERSION  # pylint: disable=global-statement
+    _VERSION = (major, minor, patch)
+    if not all([isinstance(field, int) for field in _VERSION]):
+        raise TypeError('Arguments must be integers')

compiler-rt/tools/llvm-xsan/llvm_xsan/version_info_install.py.in

+"""
+This file defines the tool version for installed versions
+of llvm-xsan.
+"""
+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+from . import version_info
+
+def _register_version():
+    version_info._register_version(
+        @LLVM_XSAN_VERSION_MAJOR@,
+        @LLVM_XSAN_VERSION_MINOR@,
+        @LLVM_XSAN_VERSION_PATCH@
+    )

compiler-rt/tools/llvm-xsan/llvm_xsan_install.cmake.in

+message(STATUS "Installing llvm-xsan")
+set(WORKING_DIRECTORY "@WORKING_DIRECTORY@")
+message(STATUS "WORKING_DIRECTORY: \"${WORKING_DIRECTORY}\"")
+
+if (NOT IS_DIRECTORY "${WORKING_DIRECTORY}")
+  message(FATAL_ERROR "WORKING_DIRECTORY (${WORKING_DIRECTORY}) is not a directory")
+endif()
+
+set(SETUP_ARGS "")
+
+set(LLVM_XSAN_VERSION_MAJOR @LLVM_XSAN_VERSION_MAJOR@)
+set(LLVM_XSAN_VERSION_MINOR @LLVM_XSAN_VERSION_MINOR@)
+set(LLVM_XSAN_VERSION_PATCH @LLVM_XSAN_VERSION_PATCH@)
+
+# Respect DESTDIR environment variable by changing installation root if
+# necessary.
+set(DESTDIR "$ENV{DESTDIR}")
+message(STATUS "DESTDIR = \"${DESTDIR}\"")
+if (DESTDIR)
+  list(APPEND SETUP_ARGS "--root" "${DESTDIR}")
+endif()
+
+set(PYTHON_EXECUTABLE "@PYTHON_EXECUTABLE_FOR_SETUP@")
+message(STATUS "PYTHON_EXECUTABLE: \"${PYTHON_EXECUTABLE}\"")
+if (NOT EXISTS "${PYTHON_EXECUTABLE}")
+  message(FATAL_ERROR "PYTHON_EXECUTABLE does not exist")
+endif()
+
+# Copy over paths so the source tree can remain clean
+foreach (PATH_TO_COPY @PATHS_TO_COPY@)
+  message(STATUS "Copying \"${PATH_TO_COPY}\"")
+  file(COPY "${PATH_TO_COPY}"
+    DESTINATION "${WORKING_DIRECTORY}/"
+  )
+endforeach()
+
+# Configure version file
+configure_file(
+  "${WORKING_DIRECTORY}/llvm_xsan/version_info_install.py.in"
+  "${WORKING_DIRECTORY}/llvm_xsan/version_info_install.py"
+  @ONLY
+)
+
+set(PATH_TO_SETUP_PY "${WORKING_DIRECTORY}/setup.py")
+message(STATUS "PATH_TO_SETUP_PY: \"${PATH_TO_SETUP_PY}\"")
+if (NOT EXISTS "${PATH_TO_SETUP_PY}")
+  message(FATAL_ERROR "PATH_TO_SETUP_PY executable does not exist")
+endif()
+
+execute_process(
+  COMMAND
+    "${PYTHON_EXECUTABLE}"
+    "${PATH_TO_SETUP_PY}"
+    install
+    ${SETUP_ARGS}
+  WORKING_DIRECTORY "${WORKING_DIRECTORY}"
+  RESULT_VARIABLE SETUP_RESULT
+  COMMAND_ECHO STDOUT
+)
+
+if (NOT ${SETUP_RESULT} EQUAL 0)
+  message(STATUS "Setup failed")
+else()
+  message(STATUS "Setup Succeeded")
+endif()

compiler-rt/tools/llvm-xsan/setup.py

+#!/usr/bin/env python
+# vim: set sw=4 ts=4 softtabstop=4 expandtab:
+import os
+from setuptools import setup, find_packages
+import llvm_xsan.version_info
+
+
+try:
+    # This module is generated by CMake during install
+    # pylint: disable=protected-access,no-member
+    import llvm_xsan.version_info_install
+    llvm_xsan.version_info_install._register_version()
+except ImportError as excep:
+    print("ERROR: Could not get install version.")
+    if os.getenv('LLVM_XSAN_SKIP_VERSION_CHECK') is None:
+        raise excep
+
+setup(
+    name='llvm_xsan',
+    version=llvm_xsan.version_info.get_version_str(),
+    description='Tools for the LLVM Sanitizers',
+    url='https://llvm.org',
+    author='Dan Liew',
+    author_email='[e-mail]',
+    license='Apache License v2.0 with LLVM Exceptions',
+    zip_safe=False,
+    include_package_data=True,
+    packages=find_packages(),
+    install_requires=[],
+    entry_points={
+        'console_scripts': [
+            'llvm-xsan=llvm_xsan.driver:setuptools_entry_point',
+        ]
+    },
+)

compiler-rt/tools/llvm-xsan/utils/lint.sh

+#!/bin/bash
+set -e
+set -o pipefail
+
+SCRIPT_DIR=$( cd ${BASH_SOURCE[0]%/*} ; pwd)
+ROOT_DIR="${SCRIPT_DIR}/../"
+
+################################################################################
+# Flake8
+################################################################################
+
+FLAKE8=${FLAKE8:=flake8}
+FLAKE8=$(which ${FLAKE8})
+
+# Check we're using python3
+echo "Running flake8..."
+FLAKE8_PYTHON_VERSION=$(${FLAKE8} --version | sed -E  '/^.*CPython.*/!d;s/.*(CPython [0-9]+).*/\1/;s/CPython ([0-9]+)/\1/')
+echo "flake8's python version: \"${FLAKE8_PYTHON_VERSION}\""
+if [ "${FLAKE8_PYTHON_VERSION}" -lt 3 ]; then
+  echo "Python 2 is not supported"
+  exit 1
+fi
+
+set -x
+cd "${ROOT_DIR}"
+${FLAKE8} $(find . -iname '*.py')
+set +x
+
+################################################################################
+# pylint
+################################################################################
+
+PYLINT=${PYLINT:=pylint}
+PYLINT=$(which "${PYLINT}")
+
+echo "Running pylint..."
+# Check we're using python3
+PYLINT_PYTHON_VERSION=$(${PYLINT} --version 2> /dev/null | sed -E '/^Python/!d;s/Python ([0-9]+)\..+/\1/')
+echo "pylint's python verison: \"${PYLINT_PYTHON_VERSION}\""
+if [ "${PYLINT_PYTHON_VERSION}" -lt 3 ]; then
+  echo "Python 2 is not supported"
+  exit 1
+fi
+
+set -x
+${PYLINT} $(find . -iname '*.py')
+set +x