[CodeGen] Disable tail calls at -O0/-O1

[aeubanks]

Summary

-O1 is designed to impact debuggability as little as possible and tail
calls hurt debuggability, so turn off tail calls at -O0/-O1.

Add a new pass that adds the "disable-tail-calls"="true" function
attribute, which disables generating (non-musttail) tail calls in that
function. Add this pass to the -O0/-O1 codegen pipeline.

Motivation: D130374 inferred more tail calls even in -O1, causing
various internal symbolizers to regress.

[rnk]

The goal clearly makes sense to me, and I think this is a reasonable way to do it. Another option is to implement this in the frontend the way that we implement optnone at -O0.

+@probinson @dblaikie, this is kind of -Og / -O1 related if you have thoughts on how that should be handled.

[dblaikie]

In D132623#3747780, @rnk wrote:

The goal clearly makes sense to me, and I think this is a reasonable way to do it. Another option is to implement this in the frontend the way that we implement optnone at -O0.

+@probinson @dblaikie, this is kind of -Og / -O1 related if you have thoughts on how that should be handled.

I think it'd be good to have some more words about how exactly this improves debuggability - because at least at a cursory reading I was/am confused - DWARFv5 has debug info to more accurately model tail calls so back traces can be provided. Does that not work in some cases? If this is about debuggability in the absence of debug info - I'm not sure I'm in support of this direction. Tail calls lose stack trace info in a similar way to inlining, and we don't disable inlining at -O1 - so is there some reasoning that differentiates the impact of these cases & justifies treating one differently than the other?

[probinson]

causing various internal symbolizers to regress.

Regress in what way?

DWARFv5 has debug info to more accurately model tail calls so back traces can be provided. Does that not work in some cases?

If you're three levels down in a tail-call chain, I can imagine it would take a lot of effort to reconstruct the call sequence, even with the tail-call info.

[nikic]

Why does this need to use an attribute rather than querying optimization level in the backend?

[probinson]

In D132623#3756345, @nikic wrote:

Why does this need to use an attribute rather than querying optimization level in the backend?

The attribute already exists and is used in several places. This patch just adds the attribute to more functions.

[rnk]

In D132623#3756345, @nikic wrote:

Why does this need to use an attribute rather than querying optimization level in the backend?

It's also generally consistent with how LLVM has handled O0, Os, and Oz to date, as attributes: optnone, minsize, optsize. That was in turn driven by a desire to allow mixing O[2zs0] objects during classic (fat?) LTO, and have things work as they did in traditional compilation.

[efriedma]

In D132623#3756786, @probinson wrote:

In D132623#3756345, @nikic wrote:

Why does this need to use an attribute rather than querying optimization level in the backend?

The attribute already exists and is used in several places. This patch just adds the attribute to more functions.

Well, really, this is querying the optimizations level in the backend; that's what the code in TargetPassConfig::addISelPrepare does. The reason it feels weird to me (and probably to @nikic) is that it's querying it in a slightly weird place. If we're going to check it in the backend anyway, we might as well just check it in the same place we'd check for the attribute, instead of adding the attribute, then checking for it a couple passes later.

[rnk]

In D132623#3765673, @efriedma wrote:

Well, really, this is querying the optimizations level in the backend; that's what the code in TargetPassConfig::addISelPrepare does. The reason it feels weird to me (and probably to @nikic) is that it's querying it in a slightly weird place. If we're going to check it in the backend anyway, we might as well just check it in the same place we'd check for the attribute, instead of adding the attribute, then checking for it a couple passes later.

I see, sorry, I didn't read the patch closely. I think we should set this in the frontend, or check the optimization level in the codegenerator. Using an extra pass seems heavyweight.

[probinson]

In D132623#3772858, @rnk wrote:

In D132623#3765673, @efriedma wrote:

Well, really, this is querying the optimizations level in the backend; that's what the code in TargetPassConfig::addISelPrepare does. The reason it feels weird to me (and probably to @nikic) is that it's querying it in a slightly weird place. If we're going to check it in the backend anyway, we might as well just check it in the same place we'd check for the attribute, instead of adding the attribute, then checking for it a couple passes later.

I see, sorry, I didn't read the patch closely. I think we should set this in the frontend, or check the optimization level in the codegenerator. Using an extra pass seems heavyweight.

I think we are better off not having two ways to spell "don't tail-call here" (attribute & opt-level) that have to be checked every place we might tail-call. So, if doing it in the pass is weird, it should be set in the frontend.

[aeubanks]

The contract we have with the -O1 pipeline is to attempt to generate debuggable code as much as possible. For that reason I don't think the frontend should be responsible for adding the attribute.

As for a separate pass adding the attribute vs checking the opt level in the various isels, I like a separate pass better because it's more explicit and less magical, plus I'd have to go and touch all the various isels. Touching all the isels isn't a huge deal but it makes it easier to regress with future changes ("forgot to check opt level on top of disable-tail-calls"), and I'd have to make sure to not miss anything in the first place. Having only one way for a backend to check for disabling tail calls seems nicer.

[jrtc27]

In D132623#3775514, @aeubanks wrote:

The contract we have with the -O1 pipeline is to attempt to generate debuggable code as much as possible

No it's not. What we say is:

    :option:`-O0` Means "no optimization": this level compiles the fastest and
    generates the most debuggable code.

    :option:`-O1` Somewhere between :option:`-O0` and :option:`-O2`.

    :option:`-O2` Moderate level of optimization which enables most
    optimizations.

...


    :option:`-Og` Like :option:`-O1`. In future versions, this option might
    disable different optimizations in order to improve debuggability.

[aeubanks]

In D132623#3775541, @jrtc27 wrote:

In D132623#3775514, @aeubanks wrote:

The contract we have with the -O1 pipeline is to attempt to generate debuggable code as much as possible

No it's not. What we say is:

    :option:`-O0` Means "no optimization": this level compiles the fastest and
    generates the most debuggable code.

    :option:`-O1` Somewhere between :option:`-O0` and :option:`-O2`.

    :option:`-O2` Moderate level of optimization which enables most
    optimizations.

...


    :option:`-Og` Like :option:`-O1`. In future versions, this option might
    disable different optimizations in order to improve debuggability.

The -O1 pipeline has been tuned in the past to attempt to keep debuggability, and in OptimizationLevel.h we say

/// Optimize quickly without destroying debuggability.
///
/// This level is tuned to produce a result from the optimizer as quickly
/// as possible and to avoid destroying debuggability. This tends to result
/// in a very good development mode where the compiled code will be
/// immediately executed as part of testing. As a consequence, where
/// possible, we would like to produce efficient-to-execute code, but not
/// if it significantly slows down compilation or would prevent even basic
/// debugging of the resulting binary.
///
/// As an example, complex loop transformations such as versioning,
/// vectorization, or fusion don't make sense here due to the degree to
/// which the executed code differs from the source code, and the compile
/// time cost.
static const OptimizationLevel O1;

So I'd say that currently we do expect -O1 to generate debuggable code. If there's ever a push for a separate -Og pipeline that's fine, but we don't have that right now.

[dblaikie]

For that reason I don't think the frontend should be responsible for adding the attribute.

Not sure I follow - the "optnone at -O0" is implemented in the frontend, seems suitable to implement this one the same way, especially if we already have the attribute and infrastructure for it?

[aeubanks]

In D132623#3756786, @probinson wrote:

In D132623#3756345, @nikic wrote:

Why does this need to use an attribute rather than querying optimization level in the backend?

The attribute already exists and is used in several places. This patch just adds the attribute to more functions.

This seems less magical and more explicit. I also didn't want to make sure I had found every place that could possibly check the attribute and have potential future changes miss disabling tail calls for -O1.

And +1 to rnk's comment, although we only LTO at an IR level, not at an MIR/object file level.

Still trying to understand exactly what's going on with the symbolizer (which is just llvm-symbolizer)

In D132623#3779146, @dblaikie wrote:

For that reason I don't think the frontend should be responsible for adding the attribute.

Not sure I follow - the "optnone at -O0" is implemented in the frontend, seems suitable to implement this one the same way, especially if we already have the attribute and infrastructure for it?

This actually came up in a talk with some people recently.
It would be interesting to have some sort of monolithic pipeline where we embed -O0/1/2/3/s/z attributes per-function in the frontend and don't have per-optimization-level pipelines. Currently we have function attributes for -O0/s/z but not the others. -O1 is fairly distinct from -O2/3 in that we want to preserve debuggability. However, we don't have an -O1 function attribute yet, so I don't think what you're proposing makes sense with the current state of things where we express our intent via the optimization level parameter to the optimization pipeline.

[probinson]

It would be interesting to have some sort of monolithic pipeline where we embed -O0/1/2/3/s/z attributes per-function in the frontend and don't have per-optimization-level pipelines.

I would balk at this. A single generic pipeline at -O0 would be a compile-time performance hit, compared to the separate pipeline, and one of the explicit goals of -O0 is fast compilation. I'd expect a hit also at -O1 although probably less noticeable.
(You could experiment with this today, by compiling -emit-llvm -O0 to get IR with optnone everywhere, then compare compiling that IR at O0 vs O3.)

[dblaikie]

In D132623#3779146, @dblaikie wrote:

For that reason I don't think the frontend should be responsible for adding the attribute.

Not sure I follow - the "optnone at -O0" is implemented in the frontend, seems suitable to implement this one the same way, especially if we already have the attribute and infrastructure for it?

This actually came up in a talk with some people recently.
It would be interesting to have some sort of monolithic pipeline where we embed -O0/1/2/3/s/z attributes per-function in the frontend and don't have per-optimization-level pipelines. Currently we have function attributes for -O0/s/z but not the others. -O1 is fairly distinct from -O2/3 in that we want to preserve debuggability. However, we don't have an -O1 function attribute yet, so I don't think what you're proposing makes sense with the current state of things where we express our intent via the optimization level parameter to the optimization pipeline.

At least when I think optnone (or maybe optsize, I forget - I remember it was @chandlerc that was expressing these opinions most vocally) was proposed there was some distinction drawn between semantically meaningful differences (that "optimizing for size" and "not optimizing at all" were observable changes in behavior, in some sense - for debugging, for fitting in certain embedded devices, etc) and "varying shades of optimization" (essentially between -O2 and -O3) - and that the latter didn't seem suitable for per-function granularity expression (& so should be expressed only by the optimization pipeline, not embedded in the IR), but the former did.

As -O1 becomes/meanders near -Og, I guess we might end up with another of these "but this needs to be carried in the IR/is semantically meaningful to the end user" attributes ("optdebug"?) - but I don't know that we're there yet. I think if -O1/-Og's desire to be debuggable can be expressed by the frontend as much as possible with existing attributes in a similar way to -O0/optnone, that's a good thing - makes it more compatible with LTO, at least?

[nemanjai]

In D132623#3784292, @probinson wrote:

It would be interesting to have some sort of monolithic pipeline where we embed -O0/1/2/3/s/z attributes per-function in the frontend and don't have per-optimization-level pipelines.

I would balk at this. A single generic pipeline at -O0 would be a compile-time performance hit, compared to the separate pipeline, and one of the explicit goals of -O0 is fast compilation. I'd expect a hit also at -O1 although probably less noticeable.
(You could experiment with this today, by compiling -emit-llvm -O0 to get IR with optnone everywhere, then compare compiling that IR at O0 vs O3.)

+1
I would be very much against a single pipeline that would run a bunch of passes that check an attribute to decide whether they should do anything. Sounds like it would also prevent building dynamic pass pipelines which ORC JIT currently allows the user to do IIRC.

[aeubanks]

At least when I think optnone (or maybe optsize, I forget - I remember it was @chandlerc that was expressing these opinions most vocally) was proposed there was some distinction drawn between semantically meaningful differences (that "optimizing for size" and "not optimizing at all" were observable changes in behavior, in some sense - for debugging, for fitting in certain embedded devices, etc) and "varying shades of optimization" (essentially between -O2 and -O3) - and that the latter didn't seem suitable for per-function granularity expression (& so should be expressed only by the optimization pipeline, not embedded in the IR), but the former did.

The talk I had was actually with Chandler who liked the idea of a unified pipeline that looked at function attributes. (I seemed to remember that you had brought up Chandler being against this before and mentioned it but he had no recollection of being against something like this).

But anyway, we're getting off topic, I'm not actually proposing a unified optimization pipeline. I'll need to find time to understand the tail call vs inlining hurting stack traces issue.

[dblaikie]

In D132623#3832548, @aeubanks wrote:

At least when I think optnone (or maybe optsize, I forget - I remember it was @chandlerc that was expressing these opinions most vocally) was proposed there was some distinction drawn between semantically meaningful differences (that "optimizing for size" and "not optimizing at all" were observable changes in behavior, in some sense - for debugging, for fitting in certain embedded devices, etc) and "varying shades of optimization" (essentially between -O2 and -O3) - and that the latter didn't seem suitable for per-function granularity expression (& so should be expressed only by the optimization pipeline, not embedded in the IR), but the former did.

The talk I had was actually with Chandler who liked the idea of a unified pipeline that looked at function attributes. (I seemed to remember that you had brought up Chandler being against this before and mentioned it but he had no recollection of being against something like this).

Hmm - perhaps I'm misremembering and/or things just got jumbled up over the years and the people involved in different parts of the conversation, etc. :/

But anyway, we're getting off topic, I'm not actually proposing a unified optimization pipeline. I'll need to find time to understand the tail call vs inlining hurting stack traces issue.

*nod* Sounds good - basically, I think the issue is that DWARFv5-or-gcc-extension call site descriptions could solve some amount of tail call lossage in backtraces, but wouldn't guarantee all tail calls could get reconstructed - but it'd be nice to see how much we could get covered with that debug info. Maybe it'd be "enough"?

llvm/docs/ReleaseNotes.rst

 
 Changes to Sanitizers
 ---------------------
 
 
 Other Changes
 -------------
 
+* -O0 and -O1 no longer generate tail calls unless required.
+
 External Open Source Projects Using LLVM 15
 ===========================================
 
 * A project...
 
 Additional Information
 ======================
 

llvm/include/llvm/CodeGen/Passes.h

namespace llvm {
   /// If AbortOnFailedISel is true, abort compilation instead of resetting.
   MachineFunctionPass *createResetMachineFunctionPass(bool EmitFallbackDiag,
                                                       bool AbortOnFailedISel);
 
   /// createCodeGenPreparePass - Transform the code to expose more pattern
   /// matching during instruction selection.
   FunctionPass *createCodeGenPreparePass();
 
+  /// Pass to add the "disable-tail-calls" function attribute to all functions.
+  FunctionPass *createAddDisableTailCallsPass();
+
   /// AtomicExpandID -- Lowers atomic operations in terms of either cmpxchg
   /// load-linked/store-conditional loops.
   extern char &AtomicExpandID;
 
   /// MachineLoopInfo - This pass is a loop analysis pass.
   extern char &MachineLoopInfoID;
 
   /// MachineDominators - This pass is a machine dominators analysis pass.

llvm/include/llvm/InitializePasses.h

 void initializeGlobalISel(PassRegistry&);
 
 /// Initialize all passes linked into the CodeGen library.
 void initializeTarget(PassRegistry&);
 
 void initializeAAEvalLegacyPassPass(PassRegistry&);
 void initializeAAResultsWrapperPassPass(PassRegistry&);
 void initializeADCELegacyPassPass(PassRegistry&);
+void initializeAddDisableTailCallsPassPass(PassRegistry &);
 void initializeAddDiscriminatorsLegacyPassPass(PassRegistry&);
 void initializeAddFSDiscriminatorsPass(PassRegistry &);
 void initializeAggressiveInstCombinerLegacyPassPass(PassRegistry&);
 void initializeAliasSetPrinterPass(PassRegistry&);
 void initializeAlignmentFromAssumptionsPass(PassRegistry&);
 void initializeAlwaysInlinerLegacyPassPass(PassRegistry&);
 void initializeAssumeSimplifyPassLegacyPassPass(PassRegistry &);
 void initializeAssumeBuilderPassLegacyPassPass(PassRegistry &);

llvm/include/llvm/LinkAllPasses.h

ForcePassLinking() {
       (void)llvm::createScalarizeMaskedMemIntrinLegacyPass();
       (void) llvm::createWarnMissedTransformationsPass();
       (void) llvm::createHardwareLoopsPass();
       (void) llvm::createInjectTLIMappingsLegacyPass();
       (void) llvm::createUnifyLoopExitsPass();
       (void) llvm::createFixIrreduciblePass();
       (void)llvm::createFunctionSpecializationPass();
       (void)llvm::createSelectOptimizePass();
+      (void)llvm::createAddDisableTailCallsPass();
 
       (void)new llvm::IntervalPartition();
       (void)new llvm::ScalarEvolutionWrapperPass();
       llvm::Function::Create(nullptr, llvm::GlobalValue::ExternalLinkage)->viewCFGOnly();
       llvm::RGPassManager RGM;
       llvm::TargetLibraryInfoImpl TLII;
       llvm::TargetLibraryInfo TLI(TLII);
       llvm::AliasAnalysis AA(TLI);

llvm/lib/CodeGen/AddDisableTailCalls.cpp

+//===- AddDisableTailCalls.cpp --------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Pass to add the "disable-tail-calls" attribute to all functions to prevent
+// the backend from emitting tail calls unless required.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/Function.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+
+#define DEBUG_TYPE "add-disable-tail-calls"
+
+using namespace llvm;
+
+namespace {
+
+class AddDisableTailCallsPass : public FunctionPass {
+public:
+  static char ID; // Pass identification, replacement for typeid
+
+  AddDisableTailCallsPass() : FunctionPass(ID) {
+    initializeAddDisableTailCallsPassPass(*PassRegistry::getPassRegistry());
+  }
+
+  bool runOnFunction(Function &F) override {
+    if (!F.hasFnAttribute("disable-tail-calls")) {
+      F.addFnAttr("disable-tail-calls", "true");
+      return true;
+    }
+    return false;
+  }
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    // This pass doesn't affect any IR analyses.
+    AU.setPreservesAll();
+  }
+};
+} // namespace
+
+FunctionPass *llvm::createAddDisableTailCallsPass() {
+  return new AddDisableTailCallsPass();
+}
+
+char AddDisableTailCallsPass::ID = 0;
+INITIALIZE_PASS_BEGIN(AddDisableTailCallsPass, DEBUG_TYPE,
+                      "Add \"disable-tail-calls\" attribute to functions",
+                      false, false)
+INITIALIZE_PASS_END(AddDisableTailCallsPass, DEBUG_TYPE,
+                    "Add \"disable-tail-calls\" attribute to functions", false,
+                    false)

llvm/lib/CodeGen/CMakeLists.txt

if (DEFINED LLVM_HAVE_TF_AOT OR DEFINED LLVM_HAVE_TF_API)
   endif()
 
   if (DEFINED LLVM_HAVE_TF_API)
     list(APPEND MLLinkDeps ${tensorflow_c_api} ${tensorflow_fx})
   endif()
 endif()
 
 add_llvm_component_library(LLVMCodeGen
+  AddDisableTailCalls.cpp
   AggressiveAntiDepBreaker.cpp
   AllocationOrder.cpp
   Analysis.cpp
   AtomicExpandPass.cpp
   BasicTargetTransformInfo.cpp
   BranchFolding.cpp
   BranchRelaxation.cpp
   BreakFalseDeps.cpp

llvm/lib/CodeGen/CodeGen.cpp

 #include "llvm-c/Initialization.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/PassRegistry.h"
 
 using namespace llvm;
 
 /// initializeCodeGen - Initialize all passes linked into the CodeGen library.
 void llvm::initializeCodeGen(PassRegistry &Registry) {
+  initializeAddDisableTailCallsPassPass(Registry);
   initializeAtomicExpandPass(Registry);
   initializeBasicBlockSectionsPass(Registry);
   initializeBranchFolderPassPass(Registry);
   initializeBranchRelaxationPass(Registry);
   initializeCFGuardLongjmpPass(Registry);
   initializeCFIFixupPass(Registry);
   initializeCFIInstrInserterPass(Registry);
   initializeCheckDebugMachineModulePass(Registry);

llvm/lib/CodeGen/TargetPassConfig.cpp

static cl::opt<bool> DisablePostRAMachineSink("disable-postra-machine-sink",
     cl::Hidden,
     cl::desc("Disable PostRA Machine Sinking"));
 static cl::opt<bool> DisableLSR("disable-lsr", cl::Hidden,
     cl::desc("Disable Loop Strength Reduction Pass"));
 static cl::opt<bool> DisableConstantHoisting("disable-constant-hoisting",
     cl::Hidden, cl::desc("Disable ConstantHoisting"));
 static cl::opt<bool> DisableCGP("disable-cgp", cl::Hidden,
     cl::desc("Disable Codegen Prepare"));
+static cl::opt<bool>
+    DisableAddDisableTailCalls("disable-add-disable-tail-calls", cl::Hidden,
+                               cl::desc("Disable AddDisableTailCalls"));
 static cl::opt<bool> DisableCopyProp("disable-copyprop", cl::Hidden,
     cl::desc("Disable Copy Propagation pass"));
 static cl::opt<bool> DisablePartialLibcallInlining("disable-partial-libcall-inlining",
     cl::Hidden, cl::desc("Disable Partial Libcall Inlining"));
 static cl::opt<bool> EnableImplicitNullChecks(
     "enable-implicit-null-checks",
     cl::desc("Fold null checks into faulting memory operations"),
     cl::init(false), cl::Hidden);
void TargetPassConfig::addISelPrepare() {
   if (requiresCodeGenSCCOrder())
     addPass(new DummyCGSCCPass);
 
   // Add both the safe stack and the stack protection passes: each of them will
   // only protect functions that have corresponding attributes.
   addPass(createSafeStackPass());
   addPass(createStackProtectorPass());
 
+  if (getOptLevel() <= CodeGenOpt::Less && !DisableAddDisableTailCalls)
+    addPass(createAddDisableTailCallsPass());
+
   if (PrintISelInput)
     addPass(createPrintFunctionPass(
         dbgs(), "\n\n*** Final LLVM Code input to ISel ***\n"));
 
   // All passes which modify the LLVM IR are now complete; run the verifier
   // to ensure that the IR is valid.
   if (!DisableVerify)
     addPass(createVerifierPass());

llvm/test/CodeGen/AArch64/O0-pipeline.ll

 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       AArch64 Stack Tagging
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
+; CHECK-NEXT:       Add "disable-tail-calls" attribute to functions
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Analysis containing CSE Info
 ; CHECK-NEXT:       IRTranslator
 ; CHECK-NEXT:       Analysis for ComputingKnownBits
 ; CHECK-NEXT:       AArch64O0PreLegalizerCombiner
 ; CHECK-NEXT:       Analysis containing CSE Info
 ; CHECK-NEXT:       Legalizer
 ; CHECK-NEXT:       AArch64PostLegalizerLowering

llvm/test/CodeGen/AArch64/arm64-abi_align.ll

 ; RUN: llc -aarch64-load-store-renaming=true < %s -mtriple=arm64-apple-darwin -mcpu=cyclone -enable-misched=false -frame-pointer=all | FileCheck %s
-; RUN: llc -aarch64-load-store-renaming=true  < %s -mtriple=arm64-apple-darwin -O0 -frame-pointer=all -fast-isel | FileCheck -check-prefix=FAST %s
+; RUN: llc -aarch64-load-store-renaming=true  < %s -mtriple=arm64-apple-darwin -O0 -disable-add-disable-tail-calls -frame-pointer=all -fast-isel | FileCheck -check-prefix=FAST %s
 
 ; rdar://12648441
 ; Generated from arm64-arguments.c with -O2.
 ; Test passing structs with size < 8, < 16 and > 16
 ; with alignment of 16 and without
 
 ; Structs with size < 8
 %struct.s38 = type { i32, i16 }

llvm/test/CodeGen/AArch64/dllimport.ll

 ; RUN: llc -mtriple aarch64-unknown-windows-msvc -filetype asm -o - %s | FileCheck %s -check-prefixes=CHECK,DAG-ISEL
 ; RUN: llc -mtriple aarch64-unknown-windows-msvc -fast-isel -filetype asm -o - %s | FileCheck %s -check-prefixes=CHECK,FAST-ISEL
-; RUN: llc -mtriple aarch64-unknown-windows-msvc -verify-machineinstrs -O0 -filetype asm -o - %s | FileCheck %s -check-prefixes=CHECK,GLOBAL-ISEL,GLOBAL-ISEL-FALLBACK
+; RUN: llc -mtriple aarch64-unknown-windows-msvc -verify-machineinstrs -O0 -disable-add-disable-tail-calls -filetype asm -o - %s | FileCheck %s -check-prefixes=CHECK,GLOBAL-ISEL,GLOBAL-ISEL-FALLBACK
 
 @var = external dllimport global i32
 @ext = external global i32
 declare dllimport i32 @external()
 declare i32 @internal()
 
 define i32 @get_var() {
   %1 = load i32, i32* @var, align 4

llvm/test/CodeGen/AArch64/tailcall-fastisel.ll

-; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 -fast-isel | FileCheck %s
+; RUN: llc < %s -mtriple=arm64-apple-darwin -O0 -disable-add-disable-tail-calls -fast-isel | FileCheck %s
 
 ; CHECK: b _foo0
 
 define i32 @foo1() {
 entry:
   %call = tail call i32 @foo0()
   ret i32 %call
 }
 
 declare i32 @foo0()

llvm/test/CodeGen/AArch64/win64_vararg_float.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=aarch64-windows -verify-machineinstrs | FileCheck %s --check-prefixes=DAGISEL
-; RUN: llc < %s -mtriple=aarch64-windows -verify-machineinstrs -O0 -fast-isel | FileCheck %s --check-prefixes=O0,FASTISEL
-; RUN: llc < %s -mtriple=aarch64-windows -verify-machineinstrs -O0 -global-isel | FileCheck %s --check-prefixes=O0,GISEL
+; RUN: llc < %s -mtriple=aarch64-windows -verify-machineinstrs -O0 -disable-add-disable-tail-calls -fast-isel | FileCheck %s --check-prefixes=O0,FASTISEL
+; RUN: llc < %s -mtriple=aarch64-windows -verify-machineinstrs -O0 -disable-add-disable-tail-calls -global-isel | FileCheck %s --check-prefixes=O0,GISEL
 
 define void @float_va_fn(float %a, i32 %b, ...) nounwind {
 ; DAGISEL-LABEL: float_va_fn:
 ; DAGISEL:       // %bb.0: // %entry
 ; DAGISEL-NEXT:    str x30, [sp, #-64]! // 8-byte Folded Spill
 ; DAGISEL-NEXT:    add x8, sp, #16
 ; DAGISEL-NEXT:    fmov s0, w0
 ; DAGISEL-NEXT:    add x0, sp, #16

llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-assert-align.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-; RUN: llc -march=amdgcn -mcpu=fiji -O0 -stop-after=irtranslator -global-isel -verify-machineinstrs -o - %s | FileCheck %s
+; RUN: llc -march=amdgcn -mcpu=fiji -O0 -disable-add-disable-tail-calls -stop-after=irtranslator -global-isel -verify-machineinstrs -o - %s | FileCheck %s
 
 ; TODO: Could potentially insert it here
 define void @arg_align_8(i8 addrspace(1)* align 8 %arg0) {
   ; CHECK-LABEL: name: arg_align_8
   ; CHECK: bb.1 (%ir-block.0):
   ; CHECK-NEXT:   liveins: $vgpr0, $vgpr1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0

llvm/test/CodeGen/AMDGPU/llc-pipeline.ll

 ; GCN-O0-NEXT:        AMDGPU Annotate Uniform Values
 ; GCN-O0-NEXT:        SI annotate control flow
 ; GCN-O0-NEXT:        LCSSA Verifier
 ; GCN-O0-NEXT:        Loop-Closed SSA Form Pass
 ; GCN-O0-NEXT:      DummyCGSCCPass
 ; GCN-O0-NEXT:      FunctionPass Manager
 ; GCN-O0-NEXT:        Safe Stack instrumentation pass
 ; GCN-O0-NEXT:        Insert stack protectors
+; GCN-O0-NEXT:        Add "disable-tail-calls" attribute to functions
 ; GCN-O0-NEXT:        Dominator Tree Construction
 ; GCN-O0-NEXT:        Post-Dominator Tree Construction
 ; GCN-O0-NEXT:        Natural Loop Information
 ; GCN-O0-NEXT:        Legacy Divergence Analysis
 ; GCN-O0-NEXT:        AMDGPU DAG->DAG Pattern Instruction Selection
 ; GCN-O0-NEXT:        MachineDominator Tree Construction
 ; GCN-O0-NEXT:        SI Fix SGPR copies
 ; GCN-O0-NEXT:        MachinePostDominator Tree Construction

 ; GCN-O1-NEXT:        AMDGPU Annotate Uniform Values
 ; GCN-O1-NEXT:        SI annotate control flow
 ; GCN-O1-NEXT:        LCSSA Verifier
 ; GCN-O1-NEXT:        Loop-Closed SSA Form Pass
 ; GCN-O1-NEXT:      DummyCGSCCPass
 ; GCN-O1-NEXT:      FunctionPass Manager
 ; GCN-O1-NEXT:        Safe Stack instrumentation pass
 ; GCN-O1-NEXT:        Insert stack protectors
+; GCN-O1-NEXT:        Add "disable-tail-calls" attribute to functions
 ; GCN-O1-NEXT:        Dominator Tree Construction
 ; GCN-O1-NEXT:        Post-Dominator Tree Construction
 ; GCN-O1-NEXT:        Natural Loop Information
 ; GCN-O1-NEXT:        Legacy Divergence Analysis
 ; GCN-O1-NEXT:        Basic Alias Analysis (stateless AA impl)
 ; GCN-O1-NEXT:        Function Alias Analysis Results
 ; GCN-O1-NEXT:        Branch Probability Analysis
 ; GCN-O1-NEXT:        Lazy Branch Probability Analysis

 ; GCN-O1-OPTS-NEXT:        AMDGPU Annotate Uniform Values
 ; GCN-O1-OPTS-NEXT:        SI annotate control flow
 ; GCN-O1-OPTS-NEXT:        LCSSA Verifier
 ; GCN-O1-OPTS-NEXT:        Loop-Closed SSA Form Pass
 ; GCN-O1-OPTS-NEXT:      DummyCGSCCPass
 ; GCN-O1-OPTS-NEXT:      FunctionPass Manager
 ; GCN-O1-OPTS-NEXT:        Safe Stack instrumentation pass
 ; GCN-O1-OPTS-NEXT:        Insert stack protectors
+; GCN-O1-OPTS-NEXT:        Add "disable-tail-calls" attribute to functions
 ; GCN-O1-OPTS-NEXT:        Dominator Tree Construction
 ; GCN-O1-OPTS-NEXT:        Post-Dominator Tree Construction
 ; GCN-O1-OPTS-NEXT:        Natural Loop Information
 ; GCN-O1-OPTS-NEXT:        Legacy Divergence Analysis
 ; GCN-O1-OPTS-NEXT:        Basic Alias Analysis (stateless AA impl)
 ; GCN-O1-OPTS-NEXT:        Function Alias Analysis Results
 ; GCN-O1-OPTS-NEXT:        Branch Probability Analysis
 ; GCN-O1-OPTS-NEXT:        Lazy Branch Probability Analysis

llvm/test/CodeGen/AMDGPU/need-fp-from-vgpr-spills.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -O0 -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -O0 -disable-add-disable-tail-calls -verify-machineinstrs < %s | FileCheck %s
 
 ; FP is in CSR range, modified.
 define hidden fastcc void @callee_has_fp() #1 {
 ; CHECK-LABEL: callee_has_fp:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; CHECK-NEXT:    s_mov_b32 s4, s33
 ; CHECK-NEXT:    s_mov_b32 s33, s32

llvm/test/CodeGen/AMDGPU/sgpr-spills-split-regalloc.ll

-; RUN: llc -mtriple amdgcn-amd-amdhsa -mcpu=gfx803 -O0 -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
+; RUN: llc -mtriple amdgcn-amd-amdhsa -mcpu=gfx803 -O0 -disable-add-disable-tail-calls -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
 
 define void @child_function() #0 {
   call void asm sideeffect "", "~{vcc}" () #0
   ret void
 }
 
 ; GCN-LABEL: {{^}}spill_sgpr_with_no_lower_vgpr_available:
 ; GCN:  buffer_store_dword v255, off, s[0:3], s32

llvm/test/CodeGen/ARM/fast-tail-call.ll

-; RUN: llc -mtriple=thumbv7-linux-gnueabi -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=thumbv7-linux-gnueabi -disable-add-disable-tail-calls -O0 < %s | FileCheck %s
 ; RUN: llc -mtriple=thumbv8m.base-arm-none-eabi -filetype=obj < %s
 
 ; Primarily a non-crash test: Thumbv7 Linux does not have FastISel support,
 ; which led (via a convoluted route) to DAG nodes after a TC_RETURN that
 ; couldn't possibly work.
 
 declare i8* @g(i8*)
 
 define i8* @f(i8* %a) {
 entry:
   %0 = tail call i8* @g(i8* %a)
   ret i8* %0
 ; CHECK: b g
 ; CHECK-NOT: ldr
 ; CHECK-NOT: str
 }

llvm/test/CodeGen/ARM/none-macho.ll

 ; RUN: llc -mtriple=thumbv7m-none-macho %s -o - -relocation-model=pic -frame-pointer=all | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-NON-FAST
-; RUN: llc -mtriple=thumbv7m-none-macho -O0 %s -o - -relocation-model=pic -frame-pointer=all | FileCheck %s
+; RUN: llc -mtriple=thumbv7m-none-macho -O0 -disable-add-disable-tail-calls %s -o - -relocation-model=pic -frame-pointer=all | FileCheck %s
 ; RUN: llc -mtriple=thumbv7m-none-macho -filetype=obj %s -o /dev/null
 
 @var = external global i32
 
 define i32 @test_litpool() minsize {
 ; CHECK-LABEL: test_litpool:
   %val = load i32, i32* @var
   ret i32 %val

llvm/test/CodeGen/ARM/subtarget-no-movt.ll

 ; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - | \
 ; RUN:   FileCheck -check-prefixes=CHECK,NO-OPTION,NO-OPTION-COMMON %s
 ; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -mattr=-no-movt | \
 ; RUN:   FileCheck -check-prefixes=CHECK,USE-MOVT,USE-MOVT-COMMON %s
 ; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -mattr=+no-movt | \
 ; RUN:   FileCheck -check-prefixes=CHECK,NO-USE-MOVT,NO-USE-MOVT-COMMON %s
-; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 | \
+; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 -disable-add-disable-tail-calls | \
 ; RUN:   FileCheck -check-prefixes=CHECK,NO-OPTION-O0,NO-OPTION-COMMON %s
-; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 -mattr=-no-movt | \
+; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 -disable-add-disable-tail-calls -mattr=-no-movt | \
 ; RUN:  FileCheck -check-prefixes=CHECK,USE-MOVT-O0,USE-MOVT-COMMON %s
-; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 -mattr=+no-movt | \
+; RUN: llc -mcpu=cortex-a8 -relocation-model=static %s -o - -O0 -disable-add-disable-tail-calls -mattr=+no-movt | \
 ; RUN:   FileCheck -check-prefixes=CHECK,NO-USE-MOVT-O0,NO-USE-MOVT-COMMON %s
 
 target triple = "thumb-apple-darwin"
 
 ; NO-OPTION-COMMON-LABEL: {{_?}}foo0
 ; NO-OPTION-COMMON: ldr [[R0:r[0-9]+]], [[L0:.*]]
 ; NO-OPTION-COMMON: [[L0]]:
 ; NO-OPTION-COMMON: .long 2296237089

llvm/test/CodeGen/ARM/tail-call-float.ll

-; RUN: llc -mtriple armv7 -target-abi aapcs -float-abi soft -O0 -o - < %s \
+; RUN: llc -mtriple armv7 -target-abi aapcs -float-abi soft -O0 -disable-add-disable-tail-calls -o - < %s \
 ; RUN:   | FileCheck %s -check-prefix CHECK-SOFT -check-prefix CHECK
-; RUN: llc -mtriple armv7 -target-abi aapcs -float-abi hard -O0 -o - < %s \
+; RUN: llc -mtriple armv7 -target-abi aapcs -float-abi hard -O0 -disable-add-disable-tail-calls -o - < %s \
 ; RUN:   | FileCheck %s -check-prefix CHECK-HARD -check-prefix CHECK
 
 ; Tests for passing floating-point regs. Variadic functions will always use
 ; general-purpose registers. Standard functions will use the floating-point
 ; registers if there is hardware FP available.
 
 declare i1 @non_variadic(float, float, float, float)
 declare i1 @non_variadic_big(float, float, float, float, float, float)

llvm/test/CodeGen/ARM/tail-call.ll

-; RUN: llc -mtriple armv7 -target-abi apcs -O0 -o - < %s \
+; RUN: llc -mtriple armv7 -target-abi apcs -O0 -disable-add-disable-tail-calls -o - < %s \
 ; RUN:   | FileCheck %s -check-prefix CHECK-TAIL -check-prefix CHECK
-; RUN: llc -mtriple armv7 -target-abi apcs -O0 -disable-tail-calls -o - < %s \
+; RUN: llc -mtriple armv7 -target-abi apcs -O0 -disable-add-disable-tail-calls -disable-tail-calls -o - < %s \
 ; RUN:   | FileCheck %s -check-prefix CHECK-NO-TAIL -check-prefix CHECK
-; RUN: llc -mtriple armv7 -target-abi aapcs -O0 -o - < %s \
+; RUN: llc -mtriple armv7 -target-abi aapcs -O0 -disable-add-disable-tail-calls -o - < %s \
 ; RUN:   | FileCheck %s -check-prefix CHECK-TAIL-AAPCS -check-prefix CHECK
 
 declare i32 @callee(i32 %i)
 declare extern_weak fastcc void @callee_weak()
 
 define i32 @caller(i32 %i) {
 entry:
   %r = tail call i32 @callee(i32 %i)

llvm/test/CodeGen/Mips/tailcall/tail-call-arguments-clobber.ll

-; RUN: llc -march=mips -mcpu=mips32 -O0 -relocation-model=pic -mips-tail-calls=1 < %s | FileCheck \
+; RUN: llc -march=mips -mcpu=mips32 -O0 -disable-add-disable-tail-calls -relocation-model=pic -mips-tail-calls=1 < %s | FileCheck \
 ; RUN:     %s -check-prefix=MIPS32
-; RUN: llc -march=mips64 -mcpu=mips64 -O0 -relocation-model=pic -target-abi n64 \
+; RUN: llc -march=mips64 -mcpu=mips64 -O0 -disable-add-disable-tail-calls -relocation-model=pic -target-abi n64 \
 ; RUN:     -mips-tail-calls=1 < %s | FileCheck %s -check-prefix=MIPS64
-; RUN: llc -march=mips64 -mcpu=mips64 -O0 -relocation-model=pic -target-abi n32 \
+; RUN: llc -march=mips64 -mcpu=mips64 -O0 -disable-add-disable-tail-calls -relocation-model=pic -target-abi n32 \
 ; RUN:     -mips-tail-calls=1 < %s | FileCheck %s -check-prefix=MIPS64
 
 
 ; LLVM PR/30197
 ; Test that the scheduler does not order loads and stores of arguments that
 ; are passed on the stack such that the arguments of the caller are clobbered
 ; too early.
 

llvm/test/CodeGen/PowerPC/ppc64-sibcall.ll

-; RUN: llc < %s -relocation-model=static -O1 -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu | FileCheck %s -check-prefix=CHECK-SCO
-; RUN: llc < %s -relocation-model=static -O1 -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO
-; RUN: llc < %s -relocation-model=static -O1 -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO
-; RUN: llc < %s -relocation-model=static -O1 -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 -code-model=small | FileCheck %s -check-prefix=SCM
+; RUN: llc < %s -relocation-model=static -O1 -disable-add-disable-tail-calls -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu | FileCheck %s -check-prefix=CHECK-SCO
+; RUN: llc < %s -relocation-model=static -O1 -disable-add-disable-tail-calls -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO
+; RUN: llc < %s -relocation-model=static -O1 -disable-add-disable-tail-calls -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s -check-prefix=CHECK-SCO
+; RUN: llc < %s -relocation-model=static -O1 -disable-add-disable-tail-calls -disable-ppc-sco=false -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 -code-model=small | FileCheck %s -check-prefix=SCM
 
 ; No combination of "powerpc64le-unknown-linux-gnu" + "CHECK-SCO", because
 ; only Power8 (and later) fully support LE.
 
 %S_56 = type { [13 x i32], i32 }
 %S_64 = type { [15 x i32], i32 }
 %S_32 = type { [7 x i32], i32 }
 

llvm/test/CodeGen/RISCV/O0-pipeline.ll

 ; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
+; CHECK-NEXT:       Add "disable-tail-calls" attribute to functions
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       RISCV DAG->DAG Pattern Instruction Selection
 ; CHECK-NEXT:       Finalize ISel and expand pseudo-instructions
 ; CHECK-NEXT:       Local Stack Slot Allocation
 ; CHECK-NEXT:       RISCV Pre-RA pseudo instruction expansion pass
 ; CHECK-NEXT:       RISCV Insert VSETVLI pass
 ; CHECK-NEXT:       Eliminate PHI nodes for register allocation
 ; CHECK-NEXT:       Two-Address instruction pass

llvm/test/CodeGen/X86/O0-pipeline.ll

 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Expand vector predication intrinsics
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Expand indirectbr instructions
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
+; CHECK-NEXT:       Add "disable-tail-calls" attribute to functions
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       X86 DAG->DAG Instruction Selection
 ; CHECK-NEXT:       X86 PIC Global Base Reg Initialization
 ; CHECK-NEXT:       Finalize ISel and expand pseudo-instructions
 ; CHECK-NEXT:       Local Stack Slot Allocation
 ; CHECK-NEXT:       X86 speculative load hardening
 ; CHECK-NEXT:       MachineDominator Tree Construction
 ; CHECK-NEXT:       X86 EFLAGS copy lowering

llvm/test/CodeGen/X86/add-disable-tail-calls.ll

+; RUN: opt -add-disable-tail-calls < %s -mtriple=x86_64-- -S | FileCheck %s --check-prefix=DISABLE
+; RUN: llc -O0 -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=DISABLE
+; RUN: llc -O1 -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=DISABLE
+; RUN: llc -O0 -disable-add-disable-tail-calls -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=IGNORE
+; RUN: llc -O1 -disable-add-disable-tail-calls -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=IGNORE
+; RUN: llc -O2 -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=IGNORE
+; RUN: llc -O3 -stop-before=finalize-isel < %s -mtriple=x86_64--  | FileCheck %s --check-prefix=IGNORE
+
+define void @f() {
+  ret void
+}
+
+; IGNORE-NOT: "disable-tail-calls"
+; DISABLE: "disable-tail-calls"="true"

llvm/test/CodeGen/X86/atom-pad-short-functions.ll

-; RUN: llc < %s -O1 -mcpu=atom -mtriple=i686-linux  | FileCheck %s
+; RUN: llc < %s -O1 -disable-add-disable-tail-calls -mcpu=atom -mtriple=i686-linux  | FileCheck %s
 
 declare void @external_function(...)
 
 define i32 @test_return_val(i32 %a) nounwind {
 ; CHECK: test_return_val
 ; CHECK: movl
 ; CHECK: nop
 ; CHECK: nop

llvm/test/CodeGen/X86/fold-sext-trunc.ll

 ; RUN: llc < %s -mtriple=x86_64-- | FileCheck %s
-; RUN: llc < %s -O0 -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -stop-after livedebugvalues -o - | FileCheck %s -check-prefix=MIR
+; RUN: llc < %s -O0 -disable-add-disable-tail-calls -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -stop-after livedebugvalues -o - | FileCheck %s -check-prefix=MIR
 ; PR4050
 
 %0 = type { i64 }
 %struct.S1 = type { i16, i32 }
 
 @g_10 = external dso_local global %struct.S1
 
 declare void @func_28(i64, i64)

llvm/test/CodeGen/X86/fold-zext-trunc.ll

 ; RUN: llc < %s | FileCheck %s -check-prefix=ASM
-; RUN: llc < %s -O0 -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -stop-after livedebugvalues -o - | FileCheck %s -check-prefix=MIR
+; RUN: llc < %s -O0 -disable-add-disable-tail-calls -mtriple=x86_64-unknown-unknown -mcpu=x86-64 -stop-after livedebugvalues -o - | FileCheck %s -check-prefix=MIR
 ; PR9055
 target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:64:64-v128:128:128-a0:0:64-f80:32:32-n8:16:32"
 target triple = "i686-pc-linux-gnu"
 
 %struct.S0 = type { i32, [2 x i8], [2 x i8], [4 x i8] }
 
 @g_98 = common global %struct.S0 zeroinitializer, align 4
 

llvm/test/CodeGen/X86/ins_split_regalloc.ll

-; RUN: llc -O1 -regalloc=greedy -mtriple=x86_64-apple-macosx < %s -o - | FileCheck %s
+; RUN: llc -O1 -disable-add-disable-tail-calls -regalloc=greedy -mtriple=x86_64-apple-macosx < %s -o - | FileCheck %s
 ; Check that last chance split (RAGreedy::tryInstructonSplit) just split
 ; when this is beneficial, otherwise we end up with uncoalesced copies.
 ; <rdar://problem/15570057> 
 
 target datalayout = "e-i64:64-f80:128-s:64-n8:16:32:64-S128"
 
 @f = external constant ptr
 

llvm/test/CodeGen/X86/lvi-hardening-indirectbr.ll

 ; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -mattr=+lvi-cfi < %s | FileCheck %s --check-prefix=X64
-; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -mattr=+lvi-cfi -O0 < %s | FileCheck %s --check-prefix=X64FAST
+; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -mattr=+lvi-cfi -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --check-prefix=X64FAST
 ;
 ; Note that a lot of this code was lifted from retpoline.ll.
 
 declare dso_local void @bar(i32)
 
 ; Test a simple indirect call and tail call.
 define void @icall_reg(ptr %fp, i32 %x) {
 entry:

llvm/test/CodeGen/X86/mixed-ptr-sizes-i686.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s | FileCheck %s --check-prefixes=ALL,CHECK
-; RUN: llc -O0 < %s | FileCheck %s --check-prefixes=ALL,CHECK-O0
+; RUN: llc -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --check-prefixes=ALL,CHECK-O0
 
 ; Source to regenerate:
 ; struct Foo {
 ;   int * __ptr32 p32;
 ;   int * __ptr64 p64;
 ;   __attribute__((address_space(9))) int *p_other;
 ; };
 ; void use_foo(Foo *f);

llvm/test/CodeGen/X86/mixed-ptr-sizes.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s | FileCheck %s --check-prefixes=ALL,CHECK
-; RUN: llc -O0 < %s | FileCheck %s --check-prefixes=ALL,CHECK-O0
+; RUN: llc -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --check-prefixes=ALL,CHECK-O0
 
 ; Source to regenerate:
 ; struct Foo {
 ;   int * __ptr32 p32;
 ;   int * __ptr64 p64;
 ;   __attribute__((address_space(9))) int *p_other;
 ; };
 ; void use_foo(Foo *f);

llvm/test/CodeGen/X86/opt-pipeline.ll

 ; When EXPENSIVE_CHECKS are enabled, the machine verifier appears between each
 ; pass. Ignore it with 'grep -v'.
 ; RUN: llc -mtriple=x86_64-- -O1 -debug-pass=Structure < %s -o /dev/null 2>&1 \
-; RUN:   | grep -v 'Verify generated machine code' | FileCheck %s
+; RUN:   | grep -v 'Verify generated machine code' | FileCheck %s --check-prefixes=CHECK,O1
 ; RUN: llc -mtriple=x86_64-- -O2 -debug-pass=Structure < %s -o /dev/null 2>&1 \
 ; RUN:   | grep -v 'Verify generated machine code' | FileCheck %s
 ; RUN: llc -mtriple=x86_64-- -O3 -debug-pass=Structure < %s -o /dev/null 2>&1 \
 ; RUN:   | grep -v 'Verify generated machine code' | FileCheck %s
 ; RUN: llc -mtriple=x86_64-- -O3 -debug-pass=Structure < %s -o /dev/null 2>&1 \
 ; RUN:   | FileCheck %s --check-prefix=FPM
 
 ; REQUIRES: asserts

 ; CHECK-NEXT:       X86 Partial Reduction
 ; CHECK-NEXT:       Expand indirectbr instructions
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       CodeGen Prepare
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Exception handling preparation
 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
+; O1-NEXT:          Add "disable-tail-calls" attribute to functions
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Post-Dominator Tree Construction
 ; CHECK-NEXT:       Branch Probability Analysis
 ; CHECK-NEXT:       Lazy Branch Probability Analysis
 ; CHECK-NEXT:       Lazy Block Frequency Analysis

llvm/test/CodeGen/X86/pr1489.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -frame-pointer=all -O0 -mcpu=i486 | FileCheck %s
+; RUN: llc < %s -frame-pointer=all -O0 -disable-add-disable-tail-calls -mcpu=i486 | FileCheck %s
 ;; magic constants are 3.999f and half of 3.999
 ; ModuleID = '1489.c'
 target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:64:64-v128:128:128-a0:0:64"
 target triple = "i686-apple-darwin8"
 @.str = internal constant [13 x i8] c"%d %d %d %d\0A\00"    ; <ptr> [#uses=1]
 
 define i32 @quux() nounwind {
 ; CHECK-LABEL: quux:

llvm/test/CodeGen/X86/pr53243-tail-call-fastisel.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -O0 -fast-isel -mtriple=x86_64-- < %s | FileCheck %s
+; RUN: llc -O0 -disable-add-disable-tail-calls -fast-isel -mtriple=x86_64-- < %s | FileCheck %s
 
 define void @test() {
 ; CHECK-LABEL: test:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    jmp set_state@PLT # TAILCALL
   tail call void @set_state()
   call void @llvm.dbg.value(metadata i64 0, metadata !10, metadata !DIExpression()), !dbg !16
   ret void

llvm/test/CodeGen/X86/retpoline-external.ll

 ; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64
-; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -O0 < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64FAST
+; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64FAST
 
 ; RUN: llc -verify-machineinstrs -mtriple=i686-unknown < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86
-; RUN: llc -verify-machineinstrs -mtriple=i686-unknown -O0 < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86FAST
+; RUN: llc -verify-machineinstrs -mtriple=i686-unknown -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86FAST
 
 declare dso_local void @bar(i32)
 
 ; Test a simple indirect call and tail call.
 define void @icall_reg(ptr %fp, i32 %x) #0 {
 entry:
   tail call void @bar(i32 %x)
   tail call void %fp(i32 %x)

llvm/test/CodeGen/X86/retpoline.ll

 ; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64
-; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -O0 < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64FAST
+; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X64FAST
 
 ; RUN: llc -verify-machineinstrs -mtriple=i686-unknown < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86
-; RUN: llc -verify-machineinstrs -mtriple=i686-unknown -O0 < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86FAST
+; RUN: llc -verify-machineinstrs -mtriple=i686-unknown -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --implicit-check-not="jmp.*\*" --implicit-check-not="call.*\*" --check-prefix=X86FAST
 
 declare void @bar(i32)
 
 ; Test a simple indirect call and tail call.
 define void @icall_reg(ptr %fp, i32 %x) #0 {
 entry:
   tail call void @bar(i32 %x)
   tail call void %fp(i32 %x)

llvm/test/CodeGen/X86/swiftself-win64.ll

 ; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-windows-msvc -o - %s | FileCheck --check-prefix=CHECK --check-prefix=OPT %s
-; RUN: llc -O0 -verify-machineinstrs -mtriple=x86_64-unknown-windows-msvc -o - %s | FileCheck %s
+; RUN: llc -O0 -disable-add-disable-tail-calls -verify-machineinstrs -mtriple=x86_64-unknown-windows-msvc -o - %s | FileCheck %s
 
 ; Parameter with swiftself should be allocated to r13.
 ; CHECK-LABEL: swiftself_param:
 ; CHECK: movq %r13, %rax
 define ptr@swiftself_param(ptr swiftself %addr0) {
     ret ptr%addr0
 }
 

llvm/test/CodeGen/X86/swiftself.ll

 ; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -o - %s | FileCheck --check-prefix=CHECK --check-prefix=OPT %s
-; RUN: llc -O0 -verify-machineinstrs -mtriple=x86_64-unknown-unknown -o - %s | FileCheck %s
+; RUN: llc -O0 -disable-add-disable-tail-calls -verify-machineinstrs -mtriple=x86_64-unknown-unknown -o - %s | FileCheck %s
 
 ; Parameter with swiftself should be allocated to r13.
 ; CHECK-LABEL: swiftself_param:
 ; CHECK: movq %r13, %rax
 define ptr@swiftself_param(ptr swiftself %addr0) {
     ret ptr%addr0
 }
 

llvm/test/CodeGen/X86/tailcall-msvc-conventions.ll

-; RUN: llc -mtriple=i686-unknown-linux-gnu -O1 < %s | FileCheck %s
-; RUN: llc -mtriple=i686-unknown-linux-gnu -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-linux-gnu -O1 -disable-add-disable-tail-calls < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-linux-gnu -O0 -disable-add-disable-tail-calls < %s | FileCheck %s
 
 ; The MSVC family of x86 calling conventions makes tail calls really tricky.
 ; Tests of all the various combinations should live here.
 
 declare i32 @cdecl_i32()
 declare void @cdecl_void()
 
 ; Don't allow tail calling these cdecl functions, because we need to clear the

llvm/test/CodeGen/X86/win64_eh_leaf.ll

-; RUN: llc < %s -O1 -mtriple=x86_64-pc-win32 | FileCheck %s -check-prefix=ASM
-; RUN: llc < %s -O1 -mtriple=x86_64-pc-win32 -filetype=obj -o %t
+; RUN: llc < %s -O1 -disable-add-disable-tail-calls -mtriple=x86_64-pc-win32 | FileCheck %s -check-prefix=ASM
+; RUN: llc < %s -O1 -disable-add-disable-tail-calls -mtriple=x86_64-pc-win32 -filetype=obj -o %t
 ; RUN: llvm-readobj --unwind %t | FileCheck %s -check-prefix=READOBJ
 
 declare void @g(i32)
 
 define i32 @not_leaf(i32) uwtable {
 entry:
   call void @g(i32 42)
   ret i32 42

llvm/test/DebugInfo/COFF/tail-call-without-lexical-scopes.ll

-; RUN: llc -mcpu=core2 -mtriple=i686-pc-win32 -O0 < %s | FileCheck --check-prefix=X86 %s
+; RUN: llc -mcpu=core2 -mtriple=i686-pc-win32 -O0 -disable-add-disable-tail-calls < %s | FileCheck --check-prefix=X86 %s
 
 ; This LL file was generated by running clang on the following code:
 ; D:\test.cpp:
 ; 1 void foo();
 ; 2
 ; 3 static void bar(int arg, ...) {
 ; 4   foo();
 ; 5 }

llvm/test/DebugInfo/X86/dbg-declare-inalloca.ll

-; RUN: llc -O0 < %s | FileCheck %s --check-prefix=CHECK --check-prefix=DEBUG
+; RUN: llc -O0 -disable-add-disable-tail-calls < %s | FileCheck %s --check-prefix=CHECK --check-prefix=DEBUG
 ; RUN: llc < %s | FileCheck %s
-; RUN: llc -filetype=obj -O0 < %s | llvm-readobj --codeview - | FileCheck %s --check-prefix=OBJ
+; RUN: llc -filetype=obj -O0 -disable-add-disable-tail-calls < %s | llvm-readobj --codeview - | FileCheck %s --check-prefix=OBJ
 
 ; IR generated by the following source:
 ; struct NonTrivial {
 ;   NonTrivial();// : x(42) {}
 ;   ~NonTrivial();// {}
 ;   int x;
 ; };
 ; extern "C" void g(int);// {}

llvm/test/MC/ARM/arm-thumb-tail-call.ll

-; RUN: llc -O0 < %s -mtriple armv7-linux-gnueabi -o - \
+; RUN: llc -O0 -disable-add-disable-tail-calls < %s -mtriple armv7-linux-gnueabi -o - \
 ; RUN:   | llvm-mc -triple armv7-linux-gnueabi -filetype=obj -o - \
 ; RUN:    | llvm-readobj -r - | FileCheck %s
 
 target datalayout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
 target triple = "armv7--linux-gnueabihf"
 
 define internal i32 @arm_fn() #1 {
   %1 = tail call i32 @thumb_fn()

llvm/tools/opt/opt.cpp

std::vector<StringRef> PassNameExact = {
       "expand-reductions",    "indirectbr-expand",
       "generic-to-nvvm",      "expandmemcmp",
       "loop-reduce",          "lower-amx-type",
       "pre-amx-config",       "lower-amx-intrinsics",
       "polyhedral-info",      "print-polyhedral-info",
       "replace-with-veclib",  "jmc-instrument",
       "dot-regions",          "dot-regions-only",
       "view-regions",         "view-regions-only",
-      "select-optimize"};
+      "select-optimize",      "add-disable-tail-calls"};
   for (const auto &P : PassNamePrefix)
     if (Pass.startswith(P))
       return true;
   for (const auto &P : PassNameContain)
     if (Pass.contains(P))
       return true;
   return llvm::is_contained(PassNameExact, Pass);
 }
int main(int argc, char **argv) {
   initializeAnalysis(Registry);
   initializeTransformUtils(Registry);
   initializeInstCombine(Registry);
   initializeAggressiveInstCombine(Registry);
   initializeInstrumentation(Registry);
   initializeTarget(Registry);
   // For codegen passes, only passes that do IR to IR transformation are
   // supported.
+  initializeAddDisableTailCallsPassPass(Registry);
   initializeExpandMemCmpPassPass(Registry);
   initializeScalarizeMaskedMemIntrinLegacyPassPass(Registry);
   initializeSelectOptimizePass(Registry);
   initializeCodeGenPreparePass(Registry);
   initializeAtomicExpandPass(Registry);
   initializeRewriteSymbolsLegacyPassPass(Registry);
   initializeWinEHPreparePass(Registry);
   initializeDwarfEHPrepareLegacyPassPass(Registry);

llvm/utils/gn/secondary/llvm/lib/CodeGen/BUILD.gn

deps = [
     "//llvm/lib/MC",
     "//llvm/lib/ProfileData",
     "//llvm/lib/Support",
     "//llvm/lib/Target",
     "//llvm/lib/Transforms/Scalar",
     "//llvm/lib/Transforms/Utils",
   ]
   sources = [
+    "AddDisableTailCalls.cpp",
     "AggressiveAntiDepBreaker.cpp",
     "AllocationOrder.cpp",
     "Analysis.cpp",
     "AtomicExpandPass.cpp",
     "BasicBlockSections.cpp",
     "BasicBlockSectionsProfileReader.cpp",
     "BasicTargetTransformInfo.cpp",
     "BranchFolding.cpp",