Add sanitizer blacklists to the rules generated with -M/-MM/-MD/-MMD.

[krasin]

Summary

Clang sanitizers, such as AddressSanitizer, ThreadSanitizer, MemorySanitizer,
Control Flow Integrity and others, use blacklists to specify which types / functions
should not be instrumented to avoid false positives or suppress known failures.

This change adds the blacklist filenames to the list of dependencies of the rules,
generated with -M/-MM/-MD/-MMD. This lets CMake/Ninja recognize that certain
C/C++/ObjC files need to be recompiled (if a blacklist is updated).

[pcc]

We should also make blacklists appear in the --show-includes output.

lib/Frontend/DependencyFile.cpp

416–420

If you move this code to the constructor you won't need to add an extra data member to this class.

test/Frontend/dependency-gen.c

26

Can we also test that the default blacklist appears in the -MD output if appropriate?

[krasin]

Add more test cases.

[krasin]

In D11968#222338, @pcc wrote:

We should also make blacklists appear in the --show-includes output.

Is it about Windows compatibility? Do you mean that the output of

bin/clang-cl /Zs /showIncludes ~/lala.cc -fsanitize=address

should include the default blacklist (and any explicit blacklists, if they were specified)?

lib/Frontend/DependencyFile.cpp

416–420

I considered that. In this case, extra deps will appear even before the source file itself.

Right now, for my local test code, it generates:

lala.o: /usr/local/google/home/krasin/lala.cc \
  /usr/local/google/home/krasin/foo.h \
  /usr/local/google/home/krasin/bar.h \
  /usr/local/google/home/krasin/blacklist.txt

If moved to the constructor, it will become:

lala.o: /usr/local/google/home/krasin/blacklist.txt \
  /usr/local/google/home/krasin/lala.cc \
  /usr/local/google/home/krasin/foo.h \
  /usr/local/google/home/krasin/bar.h

This decreases the readability of the generated rules, as the main source becomes harder to discover.

Please, let me know, if you think it's not a problem; I will move the code to the constructor and eliminate the extra member.

[pcc]

should include the default blacklist (and any explicit blacklists, if they were specified)?

Yes, exactly.

lib/Frontend/DependencyFile.cpp

416–420

These files are for machine consumption. If make and ninja accept this output, I don't think this should be a problem.

[krasin]

Windows compat

[krasin]

Please, take another look. The test for --show-includes is on the way.

[krasin]

More tests / fix tests.

[rsmith]

LGTM

[pcc]

LGTM

[krasin]

Thank you, Peter.

I will commit once I have restored my password (the email to Chris is sent)

include/clang/Frontend/DependencyOutputOptions.h

public:
   /// predefines buffer. If the output file is "-", output will be sent to
   /// stderr.
   std::string HeaderIncludeOutputFile;
 
   /// A list of names to use as the targets in the dependency file; this list
   /// must contain at least one entry.
   std::vector<std::string> Targets;
 
+  /// A list of filenames to be used as extra dependencies for every target.
+  std::vector<std::string> ExtraDeps;
+
   /// \brief The file to write GraphViz-formatted header dependencies to.
   std::string DOTOutputFile;
 
   /// \brief The directory to copy module dependencies to when collecting them.
   std::string ModuleDependencyOutputDir;
 
 public:
   DependencyOutputOptions() {

lib/Frontend/CompilerInvocation.cpp

static void ParseDependencyOutputArgs(DependencyOutputOptions &Opts,
   Opts.HeaderIncludeOutputFile = Args.getLastArgValue(OPT_header_include_file);
   Opts.AddMissingHeaderDeps = Args.hasArg(OPT_MG);
   Opts.PrintShowIncludes = Args.hasArg(OPT_show_includes);
   Opts.DOTOutputFile = Args.getLastArgValue(OPT_dependency_dot);
   Opts.ModuleDependencyOutputDir =
       Args.getLastArgValue(OPT_module_dependency_dir);
   if (Args.hasArg(OPT_MV))
     Opts.OutputFormat = DependencyOutputFormat::NMake;
+  // Add sanitizer blacklists as extra dependencies.
+  // They won't be discovered by the regular preprocessor, so
+  // we let make / ninja to know about this implicit dependency.
+  Opts.ExtraDeps = Args.getAllArgValues(OPT_fsanitize_blacklist);
 }
 
 bool clang::ParseDiagnosticArgs(DiagnosticOptions &Opts, ArgList &Args,
                                 DiagnosticsEngine *Diags) {
   using namespace options;
   bool Success = true;
 
   Opts.DiagnosticLogFile = Args.getLastArgValue(OPT_diagnostic_log_file);

lib/Frontend/DependencyFile.cpp

 namespace {
 /// Private implementation for DependencyFileGenerator
 class DFGImpl : public PPCallbacks {
   std::vector<std::string> Files;
   llvm::StringSet<> FilesSet;
   const Preprocessor *PP;
   std::string OutputFile;
   std::vector<std::string> Targets;
+  std::vector<std::string> ExtraDeps;
   bool IncludeSystemHeaders;
   bool PhonyTarget;
   bool AddMissingHeaderDeps;
   bool SeenMissingHeader;
   bool IncludeModuleFiles;
   DependencyOutputFormat OutputFormat;
 
 private:
   bool FileMatchesDepCriteria(const char *Filename,
                               SrcMgr::CharacteristicKind FileType);
   void OutputDependencyFile();
 
 public:
   DFGImpl(const Preprocessor *_PP, const DependencyOutputOptions &Opts)
     : PP(_PP), OutputFile(Opts.OutputFile), Targets(Opts.Targets),
+      ExtraDeps(Opts.ExtraDeps),
       IncludeSystemHeaders(Opts.IncludeSystemHeaders),
       PhonyTarget(Opts.UsePhonyTargets),
       AddMissingHeaderDeps(Opts.AddMissingHeaderDeps),
       SeenMissingHeader(false),
       IncludeModuleFiles(Opts.IncludeModuleFiles),
       OutputFormat(Opts.OutputFormat) {}
 
   void FileChanged(SourceLocation Loc, FileChangeReason Reason,
void DFGImpl::OutputDependencyFile() {
   std::error_code EC;
   llvm::raw_fd_ostream OS(OutputFile, EC, llvm::sys::fs::F_Text);
   if (EC) {
     PP->getDiagnostics().Report(diag::err_fe_error_opening) << OutputFile
                                                             << EC.message();
     return;
   }
 
+  // Add extra dependencies to the end of the list.
+  for (auto ExtraDep : ExtraDeps) {
+    AddFilename(ExtraDep);
+  }
+
   // Write out the dependency targets, trying to avoid overly long
   // lines when possible. We try our best to emit exactly the same
   // dependency file as GCC (4.2), assuming the included files are the
   // same.
   const unsigned MaxColumns = 75;
   unsigned Columns = 0;
 
   for (std::vector<std::string>::iterator

test/Frontend/dependency-gen.c

 // CHECK-THREE: {{ }}a/b{{[/\\]}}x.h
 // RUN: %clang -MD -MF - %s -fsyntax-only -I .//./a/b/ | FileCheck -check-prefix=CHECK-FOUR %s
 // CHECK-FOUR: {{ }}a/b{{[/\\]}}x.h
 // RUN: %clang -MD -MF - %s -fsyntax-only -I a/b/. | FileCheck -check-prefix=CHECK-FIVE %s
 // CHECK-FIVE: {{ }}a/b/.{{[/\\]}}x.h
 // RUN: cd a/b
 // RUN: %clang -MD -MF - %s -fsyntax-only -I ./ | FileCheck -check-prefix=CHECK-SIX %s
 // CHECK-SIX: {{ }}x.h
-
+// RUN: echo "fun:foo" > %t.blacklist
+// RUN: %clang -MD -MF - %s -fsyntax-only -fsanitize=cfi-vcall -flto -fsanitize-blacklist=%t.blacklist -I ./ | FileCheck -check-prefix=CHECK-SEVEN %s
+// CHECK-SEVEN: {{ }}x.h
+// CHECK-SEVEN: .blacklist
+// RUN: %clang -MD -MF - %s -fsyntax-only -fsanitize=address -flto -I . | FileCheck -check-prefix=CHECK-EIGHT %s
+// CHECK-EIGHT: {{ }}x.h
+// CHECK-EIGHT: asan_blacklist.txt
+// RUN: %clang -MD -MF - %s -fsyntax-only -fsanitize=address -flto -I . -fno-sanitize-blacklist | FileCheck -check-prefix=CHECK-NINE %s
+// CHECK-NINE: {{ }}x.h
+// CHECK-NINE-NOT: asan_blacklist.txt
 #ifndef INCLUDE_FLAG_TEST
 #include <x.h>
 #endif