Comprehensive Static Instrumentation (1/2): LLVM pass

[tdenniston]

Summary

The Comprehensive Static Instrumentation (CSI) framework provides static instrumentation that a compiler inserts into a program-under-test so that dynamic-analysis tools -- memory checkers, race detectors, cache simulators, performance profilers, code-coverage analyzers, etc. -- can observe and investigate runtime behavior. Heretofore, tools based on compiler instrumentation would each separately modify the compiler to insert their own instrumentation. In contrast, CSI inserts a standard collection of instrumentation hooks into the program-under-test. Each CSI-tool is implemented as a library that defines relevant hooks, and the remaining hooks are "nulled'" out and elided during link-time optimization (LTO), resulting in instrumented runtimes on par with custom instrumentation. CSI allows many compiler-based tools to be written as simple libraries without modifying the compiler, greatly lowering the bar for developing dynamic-analysis tools.

This diff encompasses a skeleton compiler pass for CSI. For an extended usage doc, please see the docs/CSI.rst document submitted with the clang diff. We will be submitting further diffs that incrementally add functionality to CSI.

[bruening]

lib/Transforms/Instrumentation/ComprehensiveStaticInstrumentation.cpp

36

I would suggest a FIXME comment to make it clear that this is not finished and is an incremental step.

[tdenniston]

lib/Transforms/Instrumentation/ComprehensiveStaticInstrumentation.cpp

36

Will do.

[mehdi_amini]

Each CSI-tool is implemented as a library that defines relevant hooks, and the remaining hooks are "nulled'" out and elided during link-time optimization (LTO)

The thread on llvm-dev explained why LTO should be orthogonal to the design.

include/llvm/InitializePasses.h

 void initializeDomViewerPass(PassRegistry&);
 void initializeDominanceFrontierWrapperPassPass(PassRegistry&);
 void initializeDominatorTreeWrapperPassPass(PassRegistry&);
 void initializeDwarfEHPreparePass(PassRegistry&);
 void initializeEarlyCSELegacyPassPass(PassRegistry &);
 void initializeEarlyIfConverterPass(PassRegistry&);
 void initializeEdgeBundlesPass(PassRegistry&);
 void initializeEfficiencySanitizerPass(PassRegistry&);
+void initializeComprehensiveStaticInstrumentationPass(PassRegistry&);
 void initializeEliminateAvailableExternallyLegacyPassPass(PassRegistry &);
 void initializeExpandISelPseudosPass(PassRegistry&);
 void initializeExpandPostRAPass(PassRegistry&);
 void initializeExternalAAWrapperPassPass(PassRegistry&);
 void initializeFinalizeMachineBundlesPass(PassRegistry&);
 void initializeFlattenCFGPassPass(PassRegistry&);
 void initializeFloat2IntLegacyPassPass(PassRegistry&);
 void initializeForceFunctionAttrsLegacyPassPass(PassRegistry&);

include/llvm/Transforms/Instrumentation.h

 #if defined(__GNUC__) && defined(__linux__) && !defined(ANDROID)
 inline ModulePass *createDataFlowSanitizerPassForJIT(
     const std::vector<std::string> &ABIListFiles = std::vector<std::string>()) {
   return createDataFlowSanitizerPass(ABIListFiles, getDFSanArgTLSPtrForJIT,
                                      getDFSanRetValTLSPtrForJIT);
 }
 #endif
 
+// Insert ComprehensiveStaticInstrumentation instrumentation
+ModulePass *createComprehensiveStaticInstrumentationPass();
+
 // BoundsChecking - This pass instruments the code to perform run-time bounds
 // checking on loads, stores, and other memory intrinsics.
 FunctionPass *createBoundsCheckingPass();
 
 /// \brief Calculate what to divide by to scale counts.
 ///
 /// Given the maximum count, calculate a divisor that will scale all the
 /// weights to strictly less than UINT32_MAX.

lib/Transforms/Instrumentation/CMakeLists.txt

 add_llvm_library(LLVMInstrumentation
   AddressSanitizer.cpp
   BoundsChecking.cpp
   DataFlowSanitizer.cpp
   GCOVProfiling.cpp
   MemorySanitizer.cpp
   IndirectCallPromotion.cpp
   Instrumentation.cpp
   InstrProfiling.cpp
   PGOInstrumentation.cpp
   SanitizerCoverage.cpp
   ThreadSanitizer.cpp
   EfficiencySanitizer.cpp
+  ComprehensiveStaticInstrumentation.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/Transforms
   )
 
 add_dependencies(LLVMInstrumentation intrinsics_gen)

lib/Transforms/Instrumentation/ComprehensiveStaticInstrumentation.cpp

+#include "llvm/IR/Module.h"
+#include "llvm/Transforms/Instrumentation.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "csi"
+
+namespace {
+/// The Comprehensive Static Instrumentation pass.
+/// Inserts calls to user-defined hooks at predefined points in the IR.
+struct ComprehensiveStaticInstrumentation : public ModulePass {
+  static char ID;
+
+  ComprehensiveStaticInstrumentation() : ModulePass(ID) {}
+  const char *getPassName() const override;
+  bool runOnModule(Module &M) override;
+};
+} // End anonymous namespace
+
+char ComprehensiveStaticInstrumentation::ID = 0;
+
+INITIALIZE_PASS(ComprehensiveStaticInstrumentation, "csi",
+                "ComprehensiveStaticInstrumentation: inserts calls to "
+                "user-defined hooks in the IR.",
+                false, false)
+
+const char *ComprehensiveStaticInstrumentation::getPassName() const {
+  return "ComprehensiveStaticInstrumentation";
+}
+
+ModulePass *llvm::createComprehensiveStaticInstrumentationPass() {
+  return new ComprehensiveStaticInstrumentation();
+}
+
+bool ComprehensiveStaticInstrumentation::runOnModule(Module &M) {
+  return false;
+}

lib/Transforms/Instrumentation/Instrumentation.cpp

void llvm::initializeInstrumentation(PassRegistry &Registry) {
   initializePGOInstrumentationUseLegacyPassPass(Registry);
   initializePGOIndirectCallPromotionLegacyPassPass(Registry);
   initializeInstrProfilingLegacyPassPass(Registry);
   initializeMemorySanitizerPass(Registry);
   initializeThreadSanitizerPass(Registry);
   initializeSanitizerCoverageModulePass(Registry);
   initializeDataFlowSanitizerPass(Registry);
   initializeEfficiencySanitizerPass(Registry);
+  initializeComprehensiveStaticInstrumentationPass(Registry);
 }
 
 /// LLVMInitializeInstrumentation - C binding for
 /// initializeInstrumentation.
 void LLVMInitializeInstrumentation(LLVMPassRegistryRef R) {
   initializeInstrumentation(*unwrap(R));
 }