[asan][clang] Add flag to outline instrumentation

[kstoimenov]

Summary

Summary This option can be used to reduce the size of the
binary. The trade-off in this case would be the run-time
performance.

[vitalybuka]

Why do we need this? I thought the plan to have instrumentation with better callbacks which will be small and close to efficiency to inlined instrumentation. In this case we will make it default and can avoid another frontend switch?
cc @kcc

[vitalybuka]

I took another looks and noticed:
also clang/docs/UsersManual.rst
and maybe clang/docs/AddressSanitizer.rst
also could you add some trivial test that the flag affects generated code, similar to clang/test/CodeGen/asan-use-after-return.cpp

[kstoimenov]

Modified UsersManual.rst and added a test. Couldn't find a relevant section in
AddressSanitizer.rst and adding a new one is outside of the scope of this change.

[vitalybuka]

In D105726#2875482, @kstoimenov wrote:

Modified UsersManual.rst and added a test. Couldn't find a relevant section in
AddressSanitizer.rst and adding a new one is outside of the scope of this change.

something like:
Limitations
....
AddressSanitizer increases binary size. Use -fsanitize-address-instrument... to reduce the overhead.

clang/docs/UsersManual.rst

1652 ↗

(On Diff #358422)

Here "Callback" here does not match common meaning of this term and it's just out internal detail.

-f[no-]sanitize-address-inline-instrumentation
or even
-f[no-]sanitize-address-inline
or
-f[no-]sanitize-address-outline-instrumentation

clang/include/clang/Driver/Options.td

1564

no-

clang/test/Driver/fsanitize.c

254

Maybe test pairs to make sure the last one is selected
"-fno-sanitize- -fsanitize-"
"-fsanitize- -no-fsanitize-"

[kstoimenov]

Addressed comments.

[vitalybuka]

Thanks, LGTM.
Do you have commit access?

clang/include/clang/Driver/Options.td

def fsanitize_memory_track_origins_EQ : Joined<["-"], "fsanitize-memory-track-origins=">,
                                         MarshallingInfoInt<CodeGenOpts<"SanitizeMemoryTrackOrigins">>;
 def fsanitize_memory_track_origins : Flag<["-"], "fsanitize-memory-track-origins">,
                                      Group<f_clang_Group>,
                                      HelpText<"Enable origins tracking in MemorySanitizer">;
 def fno_sanitize_memory_track_origins : Flag<["-"], "fno-sanitize-memory-track-origins">,
                                         Group<f_clang_Group>,
                                         Flags<[CoreOption, NoXarchOption]>,
                                         HelpText<"Disable origins tracking in MemorySanitizer">;
+def fsanitize_address_instrument_via_callback : Flag<["-"], "fsanitize-address-instrument-via-callback">,
+                                                Group<f_clang_Group>,
+                                                HelpText<"Always use callback for the address sanitizer">;
+def fnosanitize_address_instrument_via_callback : Flag<["-"], "fnosanitize-address-instrument-via-callback">,
+                                                  Group<f_clang_Group>,
+                                                  HelpText<"Use default logic for code inlining for the address sanitizer">;
 def fsanitize_hwaddress_experimental_aliasing
   : Flag<["-"], "fsanitize-hwaddress-experimental-aliasing">,
     Group<f_clang_Group>,
     HelpText<"Enable aliasing mode in HWAddressSanitizer">;
 def fno_sanitize_hwaddress_experimental_aliasing
   : Flag<["-"], "fno-sanitize-hwaddress-experimental-aliasing">,
     Group<f_clang_Group>, Flags<[CoreOption, NoXarchOption]>,
     HelpText<"Disable aliasing mode in HWAddressSanitizer">;

clang/include/clang/Driver/SanitizerArgs.h

class SanitizerArgs {
   int AsanFieldPadding = 0;
   bool SharedRuntime = false;
   bool AsanUseAfterScope = true;
   bool AsanPoisonCustomArrayCookie = false;
   bool AsanGlobalsDeadStripping = false;
   bool AsanUseOdrIndicator = false;
   bool AsanInvalidPointerCmp = false;
   bool AsanInvalidPointerSub = false;
+  bool AsanInstrumentViaCallback = false;
   llvm::AsanDtorKind AsanDtorKind = llvm::AsanDtorKind::Invalid;
   std::string HwasanAbi;
   bool LinkRuntimes = true;
   bool LinkCXXRuntimes = false;
   bool NeedPIE = false;
   bool SafeStackRuntime = false;
   bool Stats = false;
   bool TsanMemoryAccess = true;

clang/lib/Driver/SanitizerArgs.cpp

AsanUseAfterScope = Args.hasFlag(
         options::OPT_fsanitize_address_use_after_scope,
         options::OPT_fno_sanitize_address_use_after_scope, AsanUseAfterScope);
 
     AsanPoisonCustomArrayCookie = Args.hasFlag(
         options::OPT_fsanitize_address_poison_custom_array_cookie,
         options::OPT_fno_sanitize_address_poison_custom_array_cookie,
         AsanPoisonCustomArrayCookie);
 
+    AsanInstrumentViaCallback =
+        Args.hasFlag(options::OPT_fsanitize_address_instrument_via_callback,
+                     options::OPT_fnosanitize_address_instrument_via_callback,
+                     AsanInstrumentViaCallback);
+
     // As a workaround for a bug in gold 2.26 and earlier, dead stripping of
     // globals in ASan is disabled by default on ELF targets.
     // See https://sourceware.org/bugzilla/show_bug.cgi?id=19002
     AsanGlobalsDeadStripping =
         !TC.getTriple().isOSBinFormatELF() || TC.getTriple().isOSFuchsia() ||
         TC.getTriple().isPS4() ||
         Args.hasArg(options::OPT_fsanitize_address_globals_dead_stripping);
 
if (AsanInvalidPointerCmp) {
     CmdArgs.push_back("-asan-detect-invalid-pointer-cmp");
   }
 
   if (AsanInvalidPointerSub) {
     CmdArgs.push_back("-mllvm");
     CmdArgs.push_back("-asan-detect-invalid-pointer-sub");
   }
 
+  if (AsanInstrumentViaCallback) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
+  }
+
   // Only pass the option to the frontend if the user requested,
   // otherwise the frontend will just use the codegen default.
   if (AsanDtorKind != llvm::AsanDtorKind::Invalid) {
     CmdArgs.push_back(Args.MakeArgString("-fsanitize-address-destructor=" +
                                          AsanDtorKindToString(AsanDtorKind)));
   }
 
   if (AsanUseAfterReturn != llvm::AsanDetectStackUseAfterReturnMode::Invalid) {

clang/test/Driver/fsanitize.c

 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-globals-dead-stripping %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-ASAN-GLOBALS
 // RUN: %clang_cl --target=x86_64-windows-msvc -fsanitize=address -fsanitize-address-globals-dead-stripping -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang_cl --target=x86_64-windows-msvc -fsanitize=address -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang -target x86_64-scei-ps4  -fsanitize=address %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // CHECK-ASAN-GLOBALS: -cc1{{.*}}-fsanitize-address-globals-dead-stripping
 // CHECK-NO-ASAN-GLOBALS-NOT: -cc1{{.*}}-fsanitize-address-globals-dead-stripping
 
+// RUN: %clang -target x86_64-linux-gnu -fsanitize-address-instrument-via-callback %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-CALLBACK-WARN
+// CHECK-ASAN-CALLBACK-WARN: warning: argument unused during compilation: '-fsanitize-address-instrument-via-callback'
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-instrument-via-callback %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-CALLBACK-OK
+// CHECK-ASAN-CALLBACK-OK: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fnosanitize-address-instrument-via-callback %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-NOCALLBACK
+// CHECK-ASAN-NOCALLBACK-NOT: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-use-odr-indicator %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR
 // RUN: %clang_cl --target=x86_64-windows -fsanitize=address -fsanitize-address-use-odr-indicator -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR
 // CHECK-ASAN-ODR-INDICATOR: -cc1{{.*}}-fsanitize-address-use-odr-indicator
 
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fno-sanitize-address-use-odr-indicator %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR-OFF
 // RUN: %clang_cl --target=x86_64-windows -fsanitize=address -fno-sanitize-address-use-odr-indicator -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR-OFF
 // CHECK-ASAN-ODR-INDICATOR-OFF-NOT: -cc1{{.*}}address-generate-odr-globals