[Clang] Introduce -fexperimental-sanitize-metadata=

[melver]

Summary

Introduces the frontend flag -fexperimental-sanitize-metadata=, which
enables SanitizerBinaryMetadata instrumentation.

The first intended user of the binary metadata emitted will be a variant
of GWP-TSan [1]. The plan is to open source a stable and production
quality version of GWP-TSan. The development of which, however, requires
upstream compiler support.

[1] https://llvm.org/devmtg/2020-09/slides/Morehouse-GWP-Tsan.pdf

Until the tool has been open sourced, we mark this kind of
instrumentation as "experimental", and reserve the option to change
binary format, remove features, and similar.

Depends on D130887

[MaskRay]

clang/include/clang/Driver/Options.td

5505

Let CC1 options use the same spelling as the driver options.

clang/lib/Driver/SanitizerArgs.cpp

848

Use Args.getLastArg(...)

clang/test/Driver/fsanitize-metadata.c

2

This RUN line is redundant. For other opt-in features, we don't check that the cc1 command line doesn't have an option.

5

-target is legacy. Use --target=. If a feature isn't Linux specific, use --target=x86_64.

[melver]

Address comments.

clang/lib/Driver/SanitizerArgs.cpp

848

This won't work if someone does:

-fsanitize-metadata=feature1 -fsanitize-metadata=feature2

(instead of '-fsanitize-metadata=feature1,feature2')

Added a test case.

clang/test/Driver/fsanitize-metadata.c

2

I've made the negative-presence test more useful by checking -fno- option works.

[melver]

Rebase.

[melver]

Rebase.

[vitalybuka]

Another test under llvm-project/clang/test/CodeGen/ is needed which tests IR generated from C/C++ contains expected info.
Random example clang/test/CodeGen/attr-noundef.cpp

[melver]

Add CodeGen test.

PTAL.

[vitalybuka]

clang/test/Driver/fsanitize-metadata.c

25

If possible, don't split long RUN: line
it's much easier to see a difference between different invocations

clang format is configured to ignore line length in tests nowadays anyway

[melver]

Don't split RUN lines.

[MaskRay]

clang/include/clang/Driver/Options.td

1687

You can use f_Group instead of f_clang_Group. There are so many clang-specific (not in gcc) options, so the group is not useful.

Remove NoXarchOption.

clang/test/Driver/fsanitize-metadata.c

2

-fno-experimental-sanitize-metadata= works on bitmasks. You can change this run line to remove one bit instead of all.

[melver]

• Options.td: s/f_clang_Group/f_Group/, remove NoXarchOption
• Driver test: also test -fexperimental-sanitize-metadata=all -fno-experimental-metadata=<some option>

clang/test/Driver/fsanitize-metadata.c

2

Added more tests to check removing one bit after =all, but left this in place for the extra coverage.

clang/include/clang/Basic/CodeGenOptions.h

#include "clang/Basic/CodeGenOptions.def"
   }
 
   // Check if any one of SanitizeCoverage* is enabled.
   bool hasSanitizeCoverage() const {
     return SanitizeCoverageType || SanitizeCoverageIndirectCalls ||
            SanitizeCoverageTraceCmp || SanitizeCoverageTraceLoads ||
            SanitizeCoverageTraceStores;
   }
+
+  // Check if any one of SanitizeBinaryMetadata* is enabled.
+  bool hasSanitizeBinaryMetadata() const {
+    return SanitizeBinaryMetadataCovered || SanitizeBinaryMetadataAtomics;
+  }
 };
 
 }  // end namespace clang
 
 #endif

clang/include/clang/Basic/CodeGenOptions.def

CODEGENOPT(SanitizeCoverageTracePCGuard, 1, 0) ///< Enable PC tracing with guard
                                                ///< in sanitizer coverage.
 CODEGENOPT(SanitizeCoverageInline8bitCounters, 1, 0) ///< Use inline 8bit counters.
 CODEGENOPT(SanitizeCoverageInlineBoolFlag, 1, 0) ///< Use inline bool flag.
 CODEGENOPT(SanitizeCoveragePCTable, 1, 0) ///< Create a PC Table.
 CODEGENOPT(SanitizeCoverageNoPrune, 1, 0) ///< Disable coverage pruning.
 CODEGENOPT(SanitizeCoverageStackDepth, 1, 0) ///< Enable max stack depth tracing
 CODEGENOPT(SanitizeCoverageTraceLoads, 1, 0) ///< Enable tracing of loads.
 CODEGENOPT(SanitizeCoverageTraceStores, 1, 0) ///< Enable tracing of stores.
+CODEGENOPT(SanitizeBinaryMetadataCovered, 1, 0) ///< Emit PCs for covered functions.
+CODEGENOPT(SanitizeBinaryMetadataAtomics, 1, 0) ///< Emit PCs for atomic operations.
 CODEGENOPT(SanitizeStats     , 1, 0) ///< Collect statistics for sanitizers.
 CODEGENOPT(SimplifyLibCalls  , 1, 1) ///< Set when -fbuiltin is enabled.
 CODEGENOPT(SoftFloat         , 1, 0) ///< -soft-float.
 CODEGENOPT(SpeculativeLoadHardening, 1, 0) ///< Enable speculative load hardening.
 CODEGENOPT(FineGrainedBitfieldAccesses, 1, 0) ///< Enable fine-grained bitfield accesses.
 CODEGENOPT(StrictEnums       , 1, 0) ///< Optimize based on strict enum definition.
 CODEGENOPT(StrictVTablePointers, 1, 0) ///< Optimize based on the strict vtable pointers
 CODEGENOPT(TimePasses        , 1, 0) ///< Set when -ftime-report or -ftime-report= is enabled.

clang/include/clang/Driver/Options.td

def fsanitize_coverage_allowlist : Joined<["-"], "fsanitize-coverage-allowlist=">,
     Group<f_clang_Group>, Flags<[CoreOption, NoXarchOption]>,
     HelpText<"Restrict sanitizer coverage instrumentation exclusively to modules and functions that match the provided special case list, except the blocked ones">,
     MarshallingInfoStringVector<CodeGenOpts<"SanitizeCoverageAllowlistFiles">>;
 def fsanitize_coverage_ignorelist : Joined<["-"], "fsanitize-coverage-ignorelist=">,
     Group<f_clang_Group>, Flags<[CoreOption, NoXarchOption]>,
     HelpText<"Disable sanitizer coverage instrumentation for modules and functions "
              "that match the provided special case list, even the allowed ones">,
     MarshallingInfoStringVector<CodeGenOpts<"SanitizeCoverageIgnorelistFiles">>;
+def fexperimental_sanitize_metadata_EQ : CommaJoined<["-"], "fexperimental-sanitize-metadata=">,
+  Group<f_clang_Group>,
+  HelpText<"Specify the type of metadata to emit for binary analysis sanitizers">;
+def fno_experimental_sanitize_metadata_EQ : CommaJoined<["-"], "fno-experimental-sanitize-metadata=">,
+  Group<f_clang_Group>, Flags<[CoreOption, NoXarchOption]>,
+  HelpText<"Disable emitting metadata for binary analysis sanitizers">;
 def fsanitize_memory_track_origins_EQ : Joined<["-"], "fsanitize-memory-track-origins=">,
                                         Group<f_clang_Group>,
                                         HelpText<"Enable origins tracking in MemorySanitizer">,
                                         MarshallingInfoInt<CodeGenOpts<"SanitizeMemoryTrackOrigins">>;
 def fsanitize_memory_track_origins : Flag<["-"], "fsanitize-memory-track-origins">,
                                      Group<f_clang_Group>,
                                      HelpText<"Enable origins tracking in MemorySanitizer">;
 def fno_sanitize_memory_track_origins : Flag<["-"], "fno-sanitize-memory-track-origins">,

 def fsanitize_coverage_trace_loads
     : Flag<["-"], "fsanitize-coverage-trace-loads">,
       HelpText<"Enable tracing of loads">,
       MarshallingInfoFlag<CodeGenOpts<"SanitizeCoverageTraceLoads">>;
 def fsanitize_coverage_trace_stores
     : Flag<["-"], "fsanitize-coverage-trace-stores">,
       HelpText<"Enable tracing of stores">,
       MarshallingInfoFlag<CodeGenOpts<"SanitizeCoverageTraceStores">>;
+def fexperimental_sanitize_metadata_EQ_covered
+    : Flag<["-"], "fexperimental-sanitize-metadata=covered">,
+      HelpText<"Emit PCs for code covered with binary analysis sanitizers">,
+      MarshallingInfoFlag<CodeGenOpts<"SanitizeBinaryMetadataCovered">>;
+def fexperimental_sanitize_metadata_EQ_atomics
+    : Flag<["-"], "fexperimental-sanitize-metadata=atomics">,
+      HelpText<"Emit PCs for atomic operations used by binary analysis sanitizers">,
+      MarshallingInfoFlag<CodeGenOpts<"SanitizeBinaryMetadataAtomics">>;
 def fpatchable_function_entry_offset_EQ
     : Joined<["-"], "fpatchable-function-entry-offset=">, MetaVarName<"<M>">,
       HelpText<"Generate M NOPs before function entry">,
       MarshallingInfoInt<CodeGenOpts<"PatchableFunctionEntryOffset">>;
 def fprofile_instrument_EQ : Joined<["-"], "fprofile-instrument=">,
     HelpText<"Enable PGO instrumentation">, Values<"none,clang,llvm,csllvm">,
     NormalizedValuesScope<"CodeGenOptions">,
     NormalizedValues<["ProfileNone", "ProfileClangInstr", "ProfileIRInstr", "ProfileCSIRInstr"]>,

clang/include/clang/Driver/SanitizerArgs.h

class SanitizerArgs {
   SanitizerSet RecoverableSanitizers;
   SanitizerSet TrapSanitizers;
 
   std::vector<std::string> UserIgnorelistFiles;
   std::vector<std::string> SystemIgnorelistFiles;
   std::vector<std::string> CoverageAllowlistFiles;
   std::vector<std::string> CoverageIgnorelistFiles;
   int CoverageFeatures = 0;
+  int BinaryMetadataFeatures = 0;
   int MsanTrackOrigins = 0;
   bool MsanUseAfterDtor = true;
   bool MsanParamRetval = false;
   bool CfiCrossDso = false;
   bool CfiICallGeneralizePointers = false;
   bool CfiCanonicalJumpTables = false;
   int AsanFieldPadding = 0;
   bool SharedRuntime = false;

clang/lib/CodeGen/BackendUtil.cpp

 #include "llvm/Transforms/Instrumentation/BoundsChecking.h"
 #include "llvm/Transforms/Instrumentation/DataFlowSanitizer.h"
 #include "llvm/Transforms/Instrumentation/GCOVProfiler.h"
 #include "llvm/Transforms/Instrumentation/HWAddressSanitizer.h"
 #include "llvm/Transforms/Instrumentation/InstrProfiling.h"
 #include "llvm/Transforms/Instrumentation/MemProfiler.h"
 #include "llvm/Transforms/Instrumentation/MemorySanitizer.h"
 #include "llvm/Transforms/Instrumentation/SanitizerCoverage.h"
+#include "llvm/Transforms/Instrumentation/SanitizerBinaryMetadata.h"
 #include "llvm/Transforms/Instrumentation/ThreadSanitizer.h"
 #include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Scalar/EarlyCSE.h"
 #include "llvm/Transforms/Scalar/GVN.h"
 #include "llvm/Transforms/Scalar/LowerMatrixIntrinsics.h"
 #include "llvm/Transforms/Utils.h"
 #include "llvm/Transforms/Utils/CanonicalizeAliases.h"
getSancovOptsFromCGOpts(const CodeGenOptions &CGOpts) {
   Opts.InlineBoolFlag = CGOpts.SanitizeCoverageInlineBoolFlag;
   Opts.PCTable = CGOpts.SanitizeCoveragePCTable;
   Opts.StackDepth = CGOpts.SanitizeCoverageStackDepth;
   Opts.TraceLoads = CGOpts.SanitizeCoverageTraceLoads;
   Opts.TraceStores = CGOpts.SanitizeCoverageTraceStores;
   return Opts;
 }
 
+static SanitizerBinaryMetadataOptions
+getSanitizerBinaryMetadataOptions(const CodeGenOptions &CGOpts) {
+  SanitizerBinaryMetadataOptions Opts;
+  Opts.Covered = CGOpts.SanitizeBinaryMetadataCovered;
+  Opts.Atomics = CGOpts.SanitizeBinaryMetadataAtomics;
+  return Opts;
+}
+
 // Check if ASan should use GC-friendly instrumentation for globals.
 // First of all, there is no point if -fdata-sections is off (expect for MachO,
 // where this is not a factor). Also, on ELF this feature requires an assembler
 // extension that only works with -integrated-as at the moment.
 static bool asanUseGlobalsGC(const Triple &T, const CodeGenOptions &CGOpts) {
   if (!CGOpts.SanitizeAddressGlobalsDeadStripping)
     return false;
   switch (T.getObjectFormat()) {
PB.registerOptimizerLastEPCallback([&](ModulePassManager &MPM,
                                          OptimizationLevel Level) {
     if (CodeGenOpts.hasSanitizeCoverage()) {
       auto SancovOpts = getSancovOptsFromCGOpts(CodeGenOpts);
       MPM.addPass(ModuleSanitizerCoveragePass(
           SancovOpts, CodeGenOpts.SanitizeCoverageAllowlistFiles,
           CodeGenOpts.SanitizeCoverageIgnorelistFiles));
     }
 
+    if (CodeGenOpts.hasSanitizeBinaryMetadata()) {
+      MPM.addPass(ModuleSanitizerBinaryMetadataPass(
+          getSanitizerBinaryMetadataOptions(CodeGenOpts)));
+    }
+
     auto MSanPass = [&](SanitizerMask Mask, bool CompileKernel) {
       if (LangOpts.Sanitize.has(Mask)) {
         int TrackOrigins = CodeGenOpts.SanitizeMemoryTrackOrigins;
         bool Recover = CodeGenOpts.SanitizeRecover.has(Mask);
 
         MemorySanitizerOptions options(TrackOrigins, Recover, CompileKernel,
                                        CodeGenOpts.SanitizeMemoryParamRetval);
         MPM.addPass(ModuleMemorySanitizerPass(options));

clang/lib/Driver/SanitizerArgs.cpp

enum CoverageFeature {
   CoverageInline8bitCounters = 1 << 12,
   CoveragePCTable = 1 << 13,
   CoverageStackDepth = 1 << 14,
   CoverageInlineBoolFlag = 1 << 15,
   CoverageTraceLoads = 1 << 16,
   CoverageTraceStores = 1 << 17,
 };
 
+enum BinaryMetadataFeature {
+  BinaryMetadataCovered = 1 << 0,
+  BinaryMetadataAtomics = 1 << 1,
+};
+
 /// Parse a -fsanitize= or -fno-sanitize= argument's values, diagnosing any
 /// invalid components. Returns a SanitizerMask.
 static SanitizerMask parseArgValues(const Driver &D, const llvm::opt::Arg *A,
                                     bool DiagnoseErrors);
 
 /// Parse -f(no-)?sanitize-coverage= flag values, diagnosing any invalid
 /// components. Returns OR of members of \c CoverageFeature enumeration.
 static int parseCoverageFeatures(const Driver &D, const llvm::opt::Arg *A,
                                  bool DiagnoseErrors);
 
+/// Parse -f(no-)?sanitize-metadata= flag values, diagnosing any invalid
+/// components. Returns OR of members of \c BinaryMetadataFeature enumeration.
+static int parseBinaryMetadataFeatures(const Driver &D, const llvm::opt::Arg *A,
+                                       bool DiagnoseErrors);
+
 /// Produce an argument string from ArgList \p Args, which shows how it
 /// provides some sanitizer kind from \p Mask. For example, the argument list
 /// "-fsanitize=thread,vptr -fsanitize=address" with mask \c NeedsUbsanRt
 /// would produce "-fsanitize=vptr".
 static std::string lastArgumentForMask(const Driver &D,
                                        const llvm::opt::ArgList &Args,
                                        SanitizerMask Mask);
 
parseSpecialCaseListArg(
         DiagnoseErrors);
     parseSpecialCaseListArg(
         D, Args, CoverageIgnorelistFiles,
         options::OPT_fsanitize_coverage_ignorelist, OptSpecifier(),
         clang::diag::err_drv_malformed_sanitizer_coverage_ignorelist,
         DiagnoseErrors);
   }
 
+  // Parse -f(no-)?sanitize-metadata.
+  for (const auto *Arg :
+       Args.filtered(options::OPT_fexperimental_sanitize_metadata_EQ,
+                     options::OPT_fno_experimental_sanitize_metadata_EQ)) {
+    if (Arg->getOption().matches(
+            options::OPT_fexperimental_sanitize_metadata_EQ)) {
+      Arg->claim();
+      BinaryMetadataFeatures |=
+          parseBinaryMetadataFeatures(D, Arg, DiagnoseErrors);
+    } else {
+      Arg->claim();
+      BinaryMetadataFeatures &=
+          ~parseBinaryMetadataFeatures(D, Arg, DiagnoseErrors);
+    }
+  }
+
   SharedRuntime =
       Args.hasFlag(options::OPT_shared_libsan, options::OPT_static_libsan,
                    TC.getTriple().isAndroid() || TC.getTriple().isOSFuchsia() ||
                        TC.getTriple().isOSDarwin());
 
   ImplicitCfiRuntime = TC.getTriple().isAndroid();
 
   if (AllAddedKinds & SanitizerKind::Address) {
for (auto F : CoverageFlags) {
     if (CoverageFeatures & F.first)
       CmdArgs.push_back(F.second);
   }
   addSpecialCaseListOpt(
       Args, CmdArgs, "-fsanitize-coverage-allowlist=", CoverageAllowlistFiles);
   addSpecialCaseListOpt(Args, CmdArgs, "-fsanitize-coverage-ignorelist=",
                         CoverageIgnorelistFiles);
 
+  // Translate available BinaryMetadataFeatures to corresponding clang-cc1
+  // flags. Does not depend on any other sanitizers.
+  const std::pair<int, std::string> BinaryMetadataFlags[] = {
+      std::make_pair(BinaryMetadataCovered, "covered"),
+      std::make_pair(BinaryMetadataAtomics, "atomics")};
+  for (const auto &F : BinaryMetadataFlags) {
+    if (BinaryMetadataFeatures & F.first)
+      CmdArgs.push_back(
+          Args.MakeArgString("-fexperimental-sanitize-metadata=" + F.second));
+  }
+
   if (TC.getTriple().isOSWindows() && needsUbsanRt()) {
     // Instruct the code generator to embed linker directives in the object file
     // that cause the required runtime libraries to be linked.
     CmdArgs.push_back(
         Args.MakeArgString("--dependent-lib=" +
                            TC.getCompilerRTBasename(Args, "ubsan_standalone")));
     if (types::isCXX(InputType))
       CmdArgs.push_back(Args.MakeArgString(
for (int i = 0, n = A->getNumValues(); i != n; ++i) {
     if (F == 0 && DiagnoseErrors)
       D.Diag(clang::diag::err_drv_unsupported_option_argument)
           << A->getOption().getName() << Value;
     Features |= F;
   }
   return Features;
 }
 
+int parseBinaryMetadataFeatures(const Driver &D, const llvm::opt::Arg *A,
+                                bool DiagnoseErrors) {
+  assert(
+      A->getOption().matches(options::OPT_fexperimental_sanitize_metadata_EQ) ||
+      A->getOption().matches(
+          options::OPT_fno_experimental_sanitize_metadata_EQ));
+  int Features = 0;
+  for (int i = 0, n = A->getNumValues(); i != n; ++i) {
+    const char *Value = A->getValue(i);
+    int F = llvm::StringSwitch<int>(Value)
+                .Case("covered", BinaryMetadataCovered)
+                .Case("atomics", BinaryMetadataAtomics)
+                .Case("all", ~0)
+                .Default(0);
+    if (F == 0 && DiagnoseErrors)
+      D.Diag(clang::diag::err_drv_unsupported_option_argument)
+          << A->getOption().getName() << Value;
+    Features |= F;
+  }
+  return Features;
+}
+
 std::string lastArgumentForMask(const Driver &D, const llvm::opt::ArgList &Args,
                                 SanitizerMask Mask) {
   for (llvm::opt::ArgList::const_reverse_iterator I = Args.rbegin(),
                                                   E = Args.rend();
        I != E; ++I) {
     const auto *Arg = *I;
     if (Arg->getOption().matches(options::OPT_fsanitize_EQ)) {
       SanitizerMask AddKinds =

clang/test/Driver/fsanitize-metadata.c

+// RUN: %clang --target=x86_64-linux-gnu \
+// RUN:   -fexperimental-sanitize-metadata=all \
+// RUN:   -fno-experimental-sanitize-metadata=all %s -### 2>&1 | FileCheck %s
+// CHECK-NOT: -fexperimental-sanitize-metadata
+
+// RUN: %clang --target=x86_64-linux-gnu -fexperimental-sanitize-metadata=bad_arg %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-INVALID %s
+// CHECK-INVALID: error: unsupported argument 'bad_arg' to option '-fexperimental-sanitize-metadata='
+
+// RUN: %clang --target=x86_64-linux-gnu -fexperimental-sanitize-metadata=covered %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-COVERED %s
+// RUN: %clang --target=x86_64-linux-gnu \
+// RUN:   -fexperimental-sanitize-metadata=atomics \
+// RUN:   -fno-experimental-sanitize-metadata=atomics \
+// RUN:   -fexperimental-sanitize-metadata=covered %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-COVERED %s
+// CHECK-COVERED: "-fexperimental-sanitize-metadata=covered"
+// CHECK-COVERED-NOT: "-fexperimental-sanitize-metadata=atomics"
+
+// RUN: %clang --target=x86_64-linux-gnu -fexperimental-sanitize-metadata=atomics %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-ATOMICS %s
+// CHECK-ATOMICS: "-fexperimental-sanitize-metadata=atomics"
+
+// RUN: %clang --target=x86_64-linux-gnu \
+// RUN:   -fexperimental-sanitize-metadata=covered,atomics %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-ALL %s
+// RUN: %clang --target=x86_64-linux-gnu \
+// RUN:   -fexperimental-sanitize-metadata=covered \
+// RUN:   -fexperimental-sanitize-metadata=atomics %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-ALL %s
+// RUN: %clang --target=x86_64-linux-gnu -fexperimental-sanitize-metadata=all %s -### 2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-ALL %s
+// CHECK-ALL: "-fexperimental-sanitize-metadata=covered"
+// CHECK-ALL: "-fexperimental-sanitize-metadata=atomics"