[ThinLTO, NewPM] Run OptimizerLastEPCallbacks from buildThinLTOPreLinkDefaultPipeline

[vitalybuka]

Summary

-O1 and above don't call real optimizer pipeline in ThinLTO PreLink.
Also clang can't add PostLink OptimizerLastEPCallbacks for in-process ThinLTO.
This results in the missing sanitizer instrumentation with ThinLTO.

This is the simplest working solution which just calls
OptimizerLastEPCallbacks at the end of buildThinLTOPreLinkDefaultPipeline.

[aeubanks]

This seems like the wrong fix to me.
ThinLTO only runs the simplification pipeline pre-link, then runs it again plus the optimization pipeline. The optimization pipeline is the only place that OptimizerLastEPCallback should run (aside from the -O0 pipeline). Basically, the OptimizerLastEPCallbacks should only ever run once throughout all of compilation for a given module.

So it seems like LTO should do the same and only run the simplification pipeline pre-link, rather than just forward to buildPerModuleDefaultPipeline(), which runs both.

[vitalybuka]

In D96320#2554493, @aeubanks wrote:

This seems like the wrong fix to me.
ThinLTO only runs the simplification pipeline pre-link, then runs it again plus the optimization pipeline. The optimization pipeline is the only place that OptimizerLastEPCallback should run (aside from the -O0 pipeline). Basically, the OptimizerLastEPCallbacks should only ever run once throughout all of compilation for a given module.

So it seems like LTO should do the same and only run the simplification pipeline pre-link, rather than just forward to buildPerModuleDefaultPipeline(), which runs both.

Alternative approach: register sanitizers on the backend only D96456

[vitalybuka]

restore

[vitalybuka]

I recovered this patch as PreLink alternative to D96456.

Short term either of them will fix internal google builds.
Long term D96320 is going to be easier to maintain, e.g. if other users, without distributed LTO, like android will need sanitizers.

[vitalybuka]

rebase

[tejohnson]

I tend to like this approach better for the reasons listed in D96456. @aeubanks any objections?

[aeubanks]

Sorry, just got back from vacation, I'll try to take a look tomorrow

[aeubanks]

I guess this is fine. I'm worried that various OptimizerLastEPCallbacks will only expect to be run once, and adding a special case of not adding those callbacks during thinlto post link will be confusing for other uses of OptimizerLastEPCallbacks, but maybe it won't be so bad

clang/lib/CodeGen/BackendUtil.cpp

1305

what about just IsThinLTOPostLink? The Skip implies that we should skip ThinLTO post link, not that we should skip the passes.

[vitalybuka]

rename variable

[tejohnson]

lgtm

clang/lib/CodeGen/BackendUtil.cpp

if (LangOpts.ObjCAutoRefCount) {
           [](FunctionPassManager &FPM, PassBuilder::OptimizationLevel Level) {
             if (Level != PassBuilder::OptimizationLevel::O0)
               FPM.addPass(ObjCARCOptPass());
           });
     }
 
     // If we reached here with a non-empty index file name, then the index
     // file was empty and we are not performing ThinLTO backend compilation
-    // (used in testing in a distributed build environment). Drop any the type
-    // test assume sequences inserted for whole program vtables so that
-    // codegen doesn't complain.
-    if (!CodeGenOpts.ThinLTOIndexFile.empty())
+    // (used in testing in a distributed build environment).
+    bool SkipThinLTOPostLink = !CodeGenOpts.ThinLTOIndexFile.empty();
+    // If so drop any the type test assume sequences inserted for whole program
+    // vtables so that codegen doesn't complain.
+    if (SkipThinLTOPostLink)
       PB.registerPipelineStartEPCallback(
           [](ModulePassManager &MPM, PassBuilder::OptimizationLevel Level) {
             MPM.addPass(LowerTypeTestsPass(/*ExportSummary=*/nullptr,
                                            /*ImportSummary=*/nullptr,
                                            /*DropTypeTests=*/true));
           });
 
     if (Level != PassBuilder::OptimizationLevel::O0) {
       PB.registerPipelineStartEPCallback(
           [](ModulePassManager &MPM, PassBuilder::OptimizationLevel Level) {
             MPM.addPass(createModuleToFunctionPassAdaptor(
                 EntryExitInstrumenterPass(/*PostInlining=*/false)));
           });
     }
 
     // Register callbacks to schedule sanitizer passes at the appropriate part
     // of the pipeline.
     if (LangOpts.Sanitize.has(SanitizerKind::LocalBounds))
       PB.registerScalarOptimizerLateEPCallback(
           [](FunctionPassManager &FPM, PassBuilder::OptimizationLevel Level) {
             FPM.addPass(BoundsCheckingPass());
           });
 
+    // Don't add sanitizers if we are here from ThinLTO PostLink. That already
+    // done on PreLink stage.
+    if (!SkipThinLTOPostLink)
       addSanitizers(TargetTriple, CodeGenOpts, LangOpts, PB);
 
     if (Optional<GCOVOptions> Options = getGCOVOptions(CodeGenOpts, LangOpts))
       PB.registerPipelineStartEPCallback(
           [Options](ModulePassManager &MPM,
                     PassBuilder::OptimizationLevel Level) {
             MPM.addPass(GCOVProfilerPass(*Options));
           });
     if (Optional<InstrProfOptions> Options =

clang/test/Driver/asan.c

 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-ASAN
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-ASAN
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-ASAN
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-ASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-ASAN
 
 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KASAN
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KASAN
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KASAN
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target i386-unknown-linux -fsanitize=kernel-address %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KASAN
 
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-HWASAN
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-HWASAN
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-HWASAN
 
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang     -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // RUN: %clang     -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
-// FIX: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target aarch64-unknown-linux -fsanitize=kernel-hwaddress %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefix=CHECK-KHWASAN
 // Verify that -fsanitize={address,hwaddres,kernel-address,kernel-hwaddress} invokes ASan, HWAsan, KASan or KHWASan instrumentation.
 
 int foo(int *a) { return *a; }
 // CHECK-ASAN: __asan_init
 // CHECK-KASAN: __asan_{{.*}}load4_noabort
 // CHECK-HWASAN: __hwasan_init
 // CHECK-KHWASAN: __hwasan_tls

clang/test/Driver/dfsan.c

 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto=thin -o - | FileCheck %s
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto -o - | FileCheck %s
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=dataflow %s -S -emit-llvm -flto -o - | FileCheck %s
 
 // Verify that -fsanitize=dataflow invokes DataFlowSanitizerPass instrumentation.
 
 int foo(int *a) { return *a; }
 // CHECK: __dfsan

clang/test/Driver/msan.c

 // REQUIRES: x86-registered-target
 
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O1
-
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O1
-
-// RUN: %clang -target mips64-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -target mips64el-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -target powerpc64-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -target powerpc64le-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s
+
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s
+
+// RUN: %clang -target mips64-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -target mips64el-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -target powerpc64-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
 
 // Verify that -fsanitize=memory and -fsanitize=kernel-memory invoke MSan/KMSAN instrumentation.
 
 // Also check that this works with the new pass manager with and without
 // optimization
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O0
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O1
-
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O0
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK,O1
-// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O0
-// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK,O1
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=memory %s -S -emit-llvm -flto -o - | FileCheck %s
+
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory  %s -S -emit-llvm -o - | FileCheck %s
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s --check-prefixes=CHECK0
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=kernel-memory %s -S -emit-llvm -flto -o - | FileCheck %s
 
 int foo(int *a) { return *a; }
-// O0: = alloca
-// O1-NOT: = alloca
+// CHECK0: = alloca
+// CHECK0: @__msan_
+
+// CHECK-NOT: = alloca
 // CHECK: @__msan_

clang/test/Driver/sancov.c

 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto=thin -o - | FileCheck %s
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto -o - | FileCheck %s
 
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O1 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O3 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang     -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto -o - | FileCheck %s
 // RUN: %clang -O2 -fno-experimental-new-pass-manager -target x86_64-unknown-linux -fsanitize-coverage=trace-pc-guard %s -S -emit-llvm -flto -o - | FileCheck %s
 
 // Verify that -fsanitize-coverage invokes ModuleSanitizerCoveragePass instrumentation.
 
 int foo(int *a) { return *a; }
-// CHECK: _sancov_
+// CHECK: _sancov_
 No newline at end of file

clang/test/Driver/tsan.c

 // Also check that this works with the new pass manager with and without
 // optimization
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O1 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang -O3 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -flto=thin -o - | FileCheck %s
-// FIX: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -flto=thin -o - | FileCheck %s
+// RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -flto=thin -o - | FileCheck %s
 // RUN: %clang     -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -flto -o - | FileCheck %s
 // RUN: %clang -O2 -fexperimental-new-pass-manager -target x86_64-unknown-linux -fsanitize=thread %s -S -emit-llvm -flto -o - | FileCheck %s
 
 int foo(int *a) { return *a; }
 // CHECK: __tsan_init

llvm/lib/Passes/PassBuilder.cpp

PassBuilder::buildThinLTOPreLinkDefaultPipeline(OptimizationLevel Level) {
   // coroutine intrinsics. To ensure ThinLTO optimization passes don't trip up
   // on these, we schedule the cleanup here.
   if (PTO.Coroutines)
     MPM.addPass(createModuleToFunctionPassAdaptor(CoroCleanupPass()));
 
   if (PGOOpt && PGOOpt->PseudoProbeForProfiling)
     MPM.addPass(PseudoProbeUpdatePass());
 
+  // Handle OptimizerLastEPCallbacks added by clang on PreLink. Actual
+  // optimization is going to be done in PostLink stage, but clang can't
+  // add callbacks there in case of in-process ThinLTO called by linker.
+  for (auto &C : OptimizerLastEPCallbacks)
+    C(MPM, Level);
+
   // Emit annotation remarks.
   addAnnotationRemarksPass(MPM);
 
   addRequiredLTOPreLinkPasses(MPM);
 
   return MPM;
 }