[asan][clang] Add flag to outline instrumentation

[kstoimenov]

Summary

Summary This option can be used to reduce the size of the
binary. The trade-off in this case would be the run-time
performance.

[vitalybuka]

Why do we need this? I thought the plan to have instrumentation with better callbacks which will be small and close to efficiency to inlined instrumentation. In this case we will make it default and can avoid another frontend switch?
cc @kcc

[vitalybuka]

I took another looks and noticed:
also clang/docs/UsersManual.rst
and maybe clang/docs/AddressSanitizer.rst
also could you add some trivial test that the flag affects generated code, similar to clang/test/CodeGen/asan-use-after-return.cpp

[kstoimenov]

Modified UsersManual.rst and added a test. Couldn't find a relevant section in
AddressSanitizer.rst and adding a new one is outside of the scope of this change.

[vitalybuka]

In D105726#2875482, @kstoimenov wrote:

Modified UsersManual.rst and added a test. Couldn't find a relevant section in
AddressSanitizer.rst and adding a new one is outside of the scope of this change.

something like:
Limitations
....
AddressSanitizer increases binary size. Use -fsanitize-address-instrument... to reduce the overhead.

clang/docs/UsersManual.rst

1652

Here "Callback" here does not match common meaning of this term and it's just out internal detail.

-f[no-]sanitize-address-inline-instrumentation
or even
-f[no-]sanitize-address-inline
or
-f[no-]sanitize-address-outline-instrumentation

clang/include/clang/Driver/Options.td

1564

no-

clang/test/Driver/fsanitize.c

254

Maybe test pairs to make sure the last one is selected
"-fno-sanitize- -fsanitize-"
"-fsanitize- -no-fsanitize-"

[kstoimenov]

Addressed comments.

[vitalybuka]

Thanks, LGTM.
Do you have commit access?

clang/docs/AddressSanitizer.rst

 
 which contains lines of the form `leak:<pattern>`. Memory leak will be
 suppressed if pattern matches any function name, source file name, or
 library name in the symbolized stack trace of the leak report. See
 `full documentation
 <https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions>`_
 for more details.
 
+Code generation control
+=======================
+
+Instrumentation code outlining
+------------------------------
+
+By default AddressSanitizer inlines the instumentation code to improve the
+run-time performance, which leads to increased binary size. Using the
+(clang flag ``-fsanitize-address-outline-instrumentation` default: ``false``)
+flag forces all code instumentation to be outlined, which reduces the size
+of the binary, but also reduces the run-time performace.
+
 Limitations
 ===========
 
 * AddressSanitizer uses more real memory than a native run. Exact overhead
   depends on the allocations sizes. The smaller the allocations you make the
   bigger the overhead is.
 * AddressSanitizer uses more stack memory. We have seen up to 3x increase.
 * On 64-bit platforms AddressSanitizer maps (but not reserves) 16+ Terabytes of

clang/docs/UsersManual.rst

.. option:: -fno-sanitize-ignorelist
 
    Don't use ignorelist file, if it was specified earlier in the command line.
 
 **-f[no-]sanitize-coverage=[type,features,...]**
 
    Enable simple code coverage in addition to certain sanitizers.
    See :doc:`SanitizerCoverage` for more details.
 
+**-f[no-]sanitize-address-outline-instrumentation**
+
+   Controls how address sanitizer code is generated. If enabled will always use
+   a function call instead of inlining the code. Turning this option on could
+   reduce the binary size, but might result in a worse run-time performance.
+
+   See :doc: `AddressSanitizer` for more details.
+
 **-f[no-]sanitize-stats**
 
    Enable simple statistics gathering for the enabled sanitizers.
    See :doc:`SanitizerStats` for more details.
 
 .. option:: -fsanitize-undefined-trap-on-error
 
    Deprecated alias for ``-fsanitize-trap=undefined``.

clang/include/clang/Driver/Options.td

def fsanitize_memory_track_origins_EQ : Joined<["-"], "fsanitize-memory-track-origins=">,
                                         MarshallingInfoInt<CodeGenOpts<"SanitizeMemoryTrackOrigins">>;
 def fsanitize_memory_track_origins : Flag<["-"], "fsanitize-memory-track-origins">,
                                      Group<f_clang_Group>,
                                      HelpText<"Enable origins tracking in MemorySanitizer">;
 def fno_sanitize_memory_track_origins : Flag<["-"], "fno-sanitize-memory-track-origins">,
                                         Group<f_clang_Group>,
                                         Flags<[CoreOption, NoXarchOption]>,
                                         HelpText<"Disable origins tracking in MemorySanitizer">;
+def fsanitize_address_outline_instrumentation : Flag<["-"], "fsanitize-address-outline-instrumentation">,
+                                                Group<f_clang_Group>,
+                                                HelpText<"Always generate function calls for address sanitizer instrumentation">;
+def fno_sanitize_address_outline_instrumentation : Flag<["-"], "fno-sanitize-address-outline-instrumentation">,
+                                                   Group<f_clang_Group>,
+                                                   HelpText<"Use default code inlining logic for the address sanitizer">;
 def fsanitize_hwaddress_experimental_aliasing
   : Flag<["-"], "fsanitize-hwaddress-experimental-aliasing">,
     Group<f_clang_Group>,
     HelpText<"Enable aliasing mode in HWAddressSanitizer">;
 def fno_sanitize_hwaddress_experimental_aliasing
   : Flag<["-"], "fno-sanitize-hwaddress-experimental-aliasing">,
     Group<f_clang_Group>, Flags<[CoreOption, NoXarchOption]>,
     HelpText<"Disable aliasing mode in HWAddressSanitizer">;

clang/include/clang/Driver/SanitizerArgs.h

class SanitizerArgs {
   int AsanFieldPadding = 0;
   bool SharedRuntime = false;
   bool AsanUseAfterScope = true;
   bool AsanPoisonCustomArrayCookie = false;
   bool AsanGlobalsDeadStripping = false;
   bool AsanUseOdrIndicator = false;
   bool AsanInvalidPointerCmp = false;
   bool AsanInvalidPointerSub = false;
+  bool AsanOutlineInstrumentation = false;
   llvm::AsanDtorKind AsanDtorKind = llvm::AsanDtorKind::Invalid;
   std::string HwasanAbi;
   bool LinkRuntimes = true;
   bool LinkCXXRuntimes = false;
   bool NeedPIE = false;
   bool SafeStackRuntime = false;
   bool Stats = false;
   bool TsanMemoryAccess = true;

clang/lib/Driver/SanitizerArgs.cpp

AsanUseAfterScope = Args.hasFlag(
         options::OPT_fsanitize_address_use_after_scope,
         options::OPT_fno_sanitize_address_use_after_scope, AsanUseAfterScope);
 
     AsanPoisonCustomArrayCookie = Args.hasFlag(
         options::OPT_fsanitize_address_poison_custom_array_cookie,
         options::OPT_fno_sanitize_address_poison_custom_array_cookie,
         AsanPoisonCustomArrayCookie);
 
+    AsanOutlineInstrumentation =
+        Args.hasFlag(options::OPT_fsanitize_address_outline_instrumentation,
+                     options::OPT_fno_sanitize_address_outline_instrumentation,
+                     AsanOutlineInstrumentation);
+
     // As a workaround for a bug in gold 2.26 and earlier, dead stripping of
     // globals in ASan is disabled by default on ELF targets.
     // See https://sourceware.org/bugzilla/show_bug.cgi?id=19002
     AsanGlobalsDeadStripping =
         !TC.getTriple().isOSBinFormatELF() || TC.getTriple().isOSFuchsia() ||
         TC.getTriple().isPS4() ||
         Args.hasArg(options::OPT_fsanitize_address_globals_dead_stripping);
 
if (AsanInvalidPointerCmp) {
     CmdArgs.push_back("-asan-detect-invalid-pointer-cmp");
   }
 
   if (AsanInvalidPointerSub) {
     CmdArgs.push_back("-mllvm");
     CmdArgs.push_back("-asan-detect-invalid-pointer-sub");
   }
 
+  if (AsanOutlineInstrumentation) {
+    CmdArgs.push_back("-mllvm");
+    CmdArgs.push_back("-asan-instrumentation-with-call-threshold=0");
+  }
+
   // Only pass the option to the frontend if the user requested,
   // otherwise the frontend will just use the codegen default.
   if (AsanDtorKind != llvm::AsanDtorKind::Invalid) {
     CmdArgs.push_back(Args.MakeArgString("-fsanitize-address-destructor=" +
                                          AsanDtorKindToString(AsanDtorKind)));
   }
 
   if (AsanUseAfterReturn != llvm::AsanDetectStackUseAfterReturnMode::Invalid) {

clang/test/CodeGen/asan-use-callbacks.cpp

+// RUN: %clang -S -fsanitize=address -emit-llvm -o - -fsanitize=address %s \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-NO-OUTLINE
+// RUN: %clang -S -fsanitize=address -emit-llvm -o - -fsanitize=address %s \
+// RUN:     -fsanitize-address-outline-instrumentation \
+// RUN:     | FileCheck %s --check-prefixes=CHECK-OUTLINE
+
+// CHECK-NO-OUTLINE-NOT: call{{.*}}@__asan_load4
+// CHECK-OUTLINE: call{{.*}}@__asan_load4
+
+int deref(int *p) {
+  return *p;
+}

clang/test/Driver/fsanitize.c

 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-globals-dead-stripping %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-NO-ASAN-GLOBALS
 // RUN: %clang_cl --target=x86_64-windows-msvc -fsanitize=address -fsanitize-address-globals-dead-stripping -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang_cl --target=x86_64-windows-msvc -fsanitize=address -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // RUN: %clang -target x86_64-scei-ps4  -fsanitize=address %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-GLOBALS
 // CHECK-ASAN-GLOBALS: -cc1{{.*}}-fsanitize-address-globals-dead-stripping
 // CHECK-NO-ASAN-GLOBALS-NOT: -cc1{{.*}}-fsanitize-address-globals-dead-stripping
 
+// RUN: %clang -target x86_64-linux-gnu -fsanitize-address-outline-instrumentation %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-OUTLINE-WARN
+// CHECK-ASAN-OUTLINE-WARN: warning: argument unused during compilation: '-fsanitize-address-outline-instrumentation'
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-outline-instrumentation %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-OUTLINE-OK
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fno-sanitize-address-outline-instrumentation -fsanitize-address-outline-instrumentation %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-ASAN-OUTLINE-OK
+// CHECK-ASAN-OUTLINE-OK: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fno-sanitize-address-outline-instrumentation %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-NO-CHECK-ASAN-CALLBACK
+// RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-outline-instrumentation -fno-sanitize-address-outline-instrumentation %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CHECK-NO-CHECK-ASAN-CALLBACK
+// CHECK-NO-CHECK-ASAN-CALLBACK-NOT: "-mllvm" "-asan-instrumentation-with-call-threshold=0"
+
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fsanitize-address-use-odr-indicator %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR
 // RUN: %clang_cl --target=x86_64-windows -fsanitize=address -fsanitize-address-use-odr-indicator -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR
 // CHECK-ASAN-ODR-INDICATOR: -cc1{{.*}}-fsanitize-address-use-odr-indicator
 
 // RUN: %clang -target x86_64-linux-gnu -fsanitize=address -fno-sanitize-address-use-odr-indicator %s -### 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR-OFF
 // RUN: %clang_cl --target=x86_64-windows -fsanitize=address -fno-sanitize-address-use-odr-indicator -### -- %s 2>&1 | FileCheck %s --check-prefix=CHECK-ASAN-ODR-INDICATOR-OFF
 // CHECK-ASAN-ODR-INDICATOR-OFF-NOT: -cc1{{.*}}address-generate-odr-globals