Implemented [[clang::musttail]] attribute for guaranteed tail calls.

[haberman]

Summary

This is a Clang-only change and depends on the existing "musttail"
support already implemented in LLVM.

The [[clang::musttail]] attribute goes on a return statement, not
a function definition. There are several constraints that the user
must follow when using [[clang::musttail]], and these constraints
are verified by Sema.

Tail calls are supported on regular function calls, calls through a
function pointer, member function calls, and even pointer to member.

Future work would be to throw a warning if a users tries to pass
a pointer or reference to a local variable through a musttail call.

[rsmith]

clang/lib/CodeGen/CGCall.cpp

5315–5317

Yes, I think we should validate this by an assertion if we can. We can check this by walking the cleanup scope stack (walk from CurrentCleanupScopeDepth to EHScopeStack::stable_end()) and making sure that there is no "problematic" enclosing cleanup scope. Here, "problematic" would mean any scope other than an EHCleanupScope containing only CallLifetimeEnd cleanups.

Looking at the kinds of cleanups that we might encounter here, I think there may be a few more things that Sema needs to check in order to not get in the way of exception handling. In particular, I think we should reject if the callee is potentially-throwing and the musttail call is inside a try block or a function that's either noexcept or has a dynamic exception specification.

Oh, also, we should disallow musttail calls inside statement expressions, in order to defend against cleanups that exist transiently within an expression.

clang/lib/CodeGen/CGExpr.cpp

4829

The more I think about this, the more it makes me nervous: if any of the Emit*CallExpr functions below incidentally emit a call on the way to producing their results via the CGCall machinery, and do so without recursing through this function, that incidental call will be emitted as a tail call instead of the intended one. Specifically:

• I could imagine a block call involving multiple function calls, depending on the blocks ABI.
• I could imagine a member call performing a function call to convert from derived to virtual base in some ABIs.
• A CUDA kernel call in general involves calling a setup function before the actual function call happens (and it doesn't make sense for a CUDA kernel call to be a tail call anyway...)
• A call to a builtin can result in any number of function calls.
• If any expression in the function arguments emits a call without calling back into this function, we'll emit that call as a tail call instead of this one. Eg, [[clang::musttail]] return f(dynamic_cast<T*>(p)); might emit the call to __cxa_dynamic_cast as the tail call instead of emitting the call to f as the tail call, depending on whether the CGCall machinery is used when emitting the __cxa_dynamic_cast call.

Is it feasible to sink this check into the CodeGenFunction::EmitCall overload that takes a CallExpr, CodeGenFunction::EmitCXXMemberOrOperatorCall, and CodeGenFunction::EmitCXXMemberPointerCallExpr, after we've emitted the callee and call args? It looks like we might be able to check this immediately before calling the CGCall overload of EmitCall, so we could pass in the 'musttail' information as a flag or similar instead of using global state in the CodeGenFunction object; if so, it'd be much easier to be confident that we're applying the attribute to the right call.

clang/lib/Sema/SemaStmt.cpp

594

It's a bit awkward, but I think we should delay this check until after the others -- complaining about non-trivial destruction seems beside the point if the returned value isn't a function call.

Also, the diagnostic text for this error seems narrower than the cases it covers. For example:

void f(const char*);
void g(const char *s) {
  [[clang::musttail]] return f((s + "foo"s).c_str());
}

would be diagnosed as "attribute requires that the return type and all arguments are trivially destructible", and they are; the problem is that the return value creates a temporary object with non-trivial destruction.

clang/test/CodeGen/attr-musttail.cpp

72 ↗

(On Diff #334556)

For completeness, can we also get a CHECK-NEXT: ret void here too?

[haberman]

• Addressed more comments for musttail.
• Reject constructors and destructors from musttail.
• Fixed a few bugs and fixed the tests.
• Added Obj-C test.

[haberman]

I added tests for all the cases you mentioned. PTAL.

clang/lib/CodeGen/CGCall.cpp

5315–5317

I'm having trouble implementing the check because there doesn't appear to be any discriminator in EHScopeStack::Cleanup that will let you test if it is a CallLifetimeEnd. (The actual code just does virtual dispatch through EHScopeStack::Cleanup::Emit().

I temporarily implemented this by adding an extra virtual function to act as discriminator. The check fires if a VLA is in scope:

int Func14(int x) {
  int vla[x];
  [[clang::musttail]] return Bar(x);
}

Do we need to forbid VLAs or do I need to refine the check?

It appears that JumpDiagnostics.cpp is already diagnosing statement expressions and try. However I could not get testing to work. I tried adding a test with try but even with -fexceptions I am getting:

cannot use 'try' with exceptions disabled

clang/lib/CodeGen/CGExpr.cpp

4829

Done. It's feeling like IsMustTail, callOrInvoke, and Loc might want to get collapsed into an options struct, especially given the default parameters on the first two. Maybe could do as a follow up?

clang/test/CodeGen/attr-musttail.cpp

72 ↗

(On Diff #334556)

I turned on LLVM verification via opt so I think this should get verified by the IR verifier. Is that sufficient?

[haberman]

• Fixed unit test by running opt in a separate invocation.
• Formatting fixes.

[rsmith]

Thanks, I think this is looking really good.

@rjmccall, no explicit need to review; I just wanted to make sure you'd seen this and had a chance to express any concerns before we go ahead.

clang/include/clang/Basic/AttrDocs.td

452

One thing I'd add:

If the callee is a virtual function that is implemented by a thunk, there is no guarantee in general that the thunk tail-calls the implementation of the virtual function, so such a call in a recursive cycle can still result in unbounded stack growth.

clang/lib/CodeGen/CGCall.cpp

5315–5317

Do we need to forbid VLAs or do I need to refine the check?

Assuming that LLVM supports musttail calls from functions where a dynamic alloca is in scope, I think we should allow VLAs. The musttail documentation doesn't mention this, so I think its OK, and I can't think of a good reason why you wouldn't be able to musttail call due to a variably-sized frame.

Perhaps a good model would be to add a virtual function to permit asking a cleanup whether it's optional / skippable.

I could not get testing to work.

You need -fcxx-exceptions to use try. At the -cc1 level, we have essentially-orthogonal settings for "it's valid for exceptions to unwind through this code" (-fexceptions) and "C++ exception handling syntax is permitted" (-fcxx-exceptions), and you usually need to enable both for CodeGen tests involving exceptions.

5318–5319

Given the potential for mismatch between the JumpDiagnostics checks and this one, especially as new more exotic kinds of cleanup are added, I wonder if we should use an ErrorUnsupported here instead of an assert.

I strongly suspect we can still reach the problematic case here for a tail call in a statement expression. I don't think it's feasible to check for all the ways that an arbitrary expression context can have pending cleanups, which we'd need in order to produce precise Sema diagnostics for that, so either we handle that here or we blanket reject all musttail returns in statement expressions. I think either approach is probably acceptable.

clang/lib/Sema/SemaStmt.cpp

596–599

I would have thought this assert would fire for void f() { [[clang::musttail]] return; }. If so, we should reject this case with a diagnostic.

601

IgnoreUnlessSpelledInSource is a syntactic check that's only really intended for tooling use cases; I think we want something a bit more semantic here, so IgnoreImplicitAsWritten would be more appropriate.

I think it would be reasonable to also skip "parentheses" here (which we treat as also including things like C's _Generic). Would Ex->IgnoreImplicitAsWritten()->IgnoreParens() work?

If we're going to skip elidable copy construction of the result here (which I think we should), should we also reflect that in the AST? Perhaps we should strip the return value down to being just the call expression? I'm thinking in particular of things like building in C++14 or before with -fno-elide-constructors, where code generation for a by-value return of a class object will synthesize a local temporary to hold the result, with a final destination copy emitted after the call. (Testcase: struct A { A(const A&); }; A f(); A g() { [[clang::musttail]] return f(); } with -fno-elide-constructors.)

609

A call expression doesn't necessarily have a known callee declaration. I would expect this assert to fire on a case like:

void f() {
  void (*p)() = f;
  [[clang::musttail]] return p();
}

We should reject this with a diagnostic.

632–633

Please pass in a flag here so the diagnostic can %select and produce a more specific description of the problem.

644

There are a couple of other contexts that can include a return statement: the caller could also be an ObjCMethodDecl (an Objective-C method) or a CapturedDecl (the body of a #pragma omp parallel region). I'd probably use a specific diagnostic ("cannot be used from a block" / "cannot be used from an Objective-C function") for the block and ObjCMethod case, and a nonsepcific-but-correct "cannot be used from this context" for anything else.

clang/test/CodeGen/attr-musttail.cpp

72 ↗

(On Diff #334556)

We don't like Clang's tests depending on opt in general, but I think in this case it's an acceptable crutch until we fix Clang to run the verifier on its IR output again (as discussed offline, it looks like we lost that as part of the transition to the new pass manager). Please add a FIXME to remove the call to opt once that bug is fixed. Other than that, I'm fine with this approach.

[rsmith]

clang/lib/CodeGen/CGCall.cpp

5315–5317

Or maybe instead of "is optional / skippable", the right question is, "is this redundant if we're about to return?" That way we could potentially one day reuse the same mechanism to also skip emitting such cleanups when emitting a cleanup path into the return block.

[rjmccall]

CC'ing Varun Gandhi.

Is musttail actually supported generically on all LLVM backends, or does this need a target restriction?

You should structure this code so it's easy to add exceptions for certain calling conventions that can support tail calls with weaker restrictions (principally, callee-pop conventions). Mostly that probably means checking the calling convention first, or extracting the type restriction checks into a different function that you can skip. For example, I believe x86's fastcall convention can logically support any combination of prototypes as musttail as long as the return types are vaguely compatible.

clang/lib/CodeGen/CGCall.cpp

5318–5319

Yes, I think ErrorUnsupported is a much better idea.

clang/lib/Sema/SemaStmt.cpp

644

Blocks ought to be extremely straightforward to support. Just validate that the tail call is to a block pointer and then compare the underlying function types line up in the same way. You will need to be able to verify that there isn't a non-trivial conversion on the return types, even if the return type isn't known at this point in the function, but that's a problem in C++ as well due to lambdas and auto deduced return types.

Also, you can use isa<...> for checks like this instead of dyn_cast<...>.

[rsmith]

In D99517#2667025, @rjmccall wrote:

You should structure this code so it's easy to add exceptions for certain calling conventions that can support tail calls with weaker restrictions (principally, callee-pop conventions). Mostly that probably means checking the calling convention first, or extracting the type restriction checks into a different function that you can skip. For example, I believe x86's fastcall convention can logically support any combination of prototypes as musttail as long as the return types are vaguely compatible.

The LLVM musttail flag doesn't seem to allow for any target-specific loosening of the rules at the moment, so I don't think we can get any benefit from such restructuring right now; do you think it's OK to defer this restructuring and use the stricter rules across all targets for now?

I think there is also value in having a target-independent set of restrictions, even if we could actually guarantee tail calls in more circumstances on some (or maybe most!) targets, in order to allow people to make portable use of the attribute and as data towards something that we might be able to standardize. (For example, the people working on coroutines in C++ wanted something like this, but wanted feedback from implementers on what set of restrictions would be necessary in order to portably guarantee a tail call.) In order to strike a balance between portability and usefulness here, maybe we could plan to eventually accept any musttail call we know the target can support, but warn on musttail calls that don't satisfy the stricter rules and therefore may be non-portable?

[rjmccall]

In D99517#2667088, @rsmith wrote:

In D99517#2667025, @rjmccall wrote:

You should structure this code so it's easy to add exceptions for certain calling conventions that can support tail calls with weaker restrictions (principally, callee-pop conventions). Mostly that probably means checking the calling convention first, or extracting the type restriction checks into a different function that you can skip. For example, I believe x86's fastcall convention can logically support any combination of prototypes as musttail as long as the return types are vaguely compatible.

The LLVM musttail flag doesn't seem to allow for any target-specific loosening of the rules at the moment, so I don't think we can get any benefit from such restructuring right now; do you think it's OK to defer this restructuring and use the stricter rules across all targets for now?

Right, I wasn't suggesting that we needed to implement weaker rules right now, just that it'd be nice if the code didn't have to be totally restructured just to do it. Right now it's one big function that does all the checks.

I think there is also value in having a target-independent set of restrictions, even if we could actually guarantee tail calls in more circumstances on some (or maybe most!) targets, in order to allow people to make portable use of the attribute and as data towards something that we might be able to standardize. (For example, the people working on coroutines in C++ wanted something like this, but wanted feedback from implementers on what set of restrictions would be necessary in order to portably guarantee a tail call.) In order to strike a balance between portability and usefulness here, maybe we could plan to eventually accept any musttail call we know the target can support, but warn on musttail calls that don't satisfy the stricter rules and therefore may be non-portable?

I agree that we should not start loosening restrictions based on the vagaries of the platform CC, e.g. recognizing that a particular set of arguments happens to be passed solely in registers. I was thinking about callee-pop CCs, like fastcall and swiftasynccall, which are generally designed from the start to support almost unrestricted tail calls; e.g. the only restriction on tail calls between fastcall functions is that the return types are compatible. (IIRC — it's possible that highly-aligned arguments would change that.) Since tail calls are part of the designed feature set of these conventions, it seems appropriate to think about them when adding a tail-call feature.

Standard C conventions generally don't support unrestricted tail calls (because of variadics, unprototyped calls, easier assembly-writing, and history), so this would only apply as a target-specific extension when used in conjunction with a non-standard CC, which meshes well with your goals for standardization. I just want you to write the code so that maintainers can more easily skip some of the restrictions to cover a non-standard CC.

I'm not surprised that the C++ coroutine people want unrestricted tail calls; this is all pretty predictable, and it's essentially the point I made about generic coroutine lowering several years ago at LLVM dev. Really, they need to be asking for a standard calling convention that guarantees unrestricted tail calls. Of course, that would require the standard to admit the existence of calling conventions (other than language linkage :)).

[aaron.ballman]

clang/include/clang/Basic/AttrDocs.td

458

It'd be nice if we could nail down "similar" somewhat. I don't know if int and short are similar (due to promotions) or const int and int are similar, etc.

461–462

Not only is this not usable with K&R C declarations, but it's also not usable with ... variadic functions either, right?

clang/lib/Sema/SemaStmt.cpp

562–569

I disagree that ActOnAttributedStmt() is the correct place for this checking -- template checking should occur when the template is instantiated, same as happens for declaration attributes. I'd like to see this functionality moved to SemaStmtAttr.cpp. Keeping the attribute logic together and following the same patterns is what allows us to tablegenerate more of the attribute logic. Statement attributes are just starting to get more such automation.

[haberman]

• Addressed comments and tried moving check to SemaStmtAttr.cpp.

clang/lib/Sema/SemaStmt.cpp

562–569

I tried commenting out this code and adding the following code into handleMustTailAttr() in SemaStmtAttr.cpp:

if (!S.checkMustTailAttr(St, MTA))
  return nullptr;

This caused my test cases related to templates to fail. It also seemed to break test cases related to JumpDiagnostics. My interpretation of this is that handleMustTailAttr() is called during parsing only, and cannot catch errors at template instantiation time or that require a more complete AST.

What am I missing? Where in SemaStmtAttr.cpp are you suggesting that I put this check?

601

IgnoreImplicitAsWritten() doesn't skip ExprWithCleanups, and per your previous comment I was trying to find a CallExpr before doing the check prohibiting ExprWithCleanups with side effects.

I could write some custom ignore logic using clang::IgnoreExprNodes() directly.

If we're going to skip elidable copy construction of the result here (which I think we should)

To clarify, are you suggesting that we allow musttail through elidable copy constructors on the return value, even if -fno-elide-constructors is set? ie. we consider that musttail overrides the -fno-elide-constructors option on the command line?

[haberman]

• Added missing S.setFunctionHasMustTail().

[haberman]

clang/lib/Sema/SemaStmt.cpp

562–569

Scratch the part about JumpDiagnostics, that was me failing to call S.setFunctionHasMustTail(). I added that and now the JumpDiagnostics tests pass.

But the template test cases still fail, and I can't find any hook point in SemaStmtAttr.cpp that will let me evaluate these checks at template instantiation time.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

I think there's a bit of an architectural mixup, but I'm curious if @rsmith agrees before anyone starts doing work to make changes.

When transforming declarations, RebuildWhatever() calls the ActOnWhatever() function which calls ProcessDeclAttributeList() so that attributes are processed. RebuildAttributedStmt() similarly calls ActOnAttributedStmt(). However, ActOnAttributedStmt() doesn't call ProcessStmtAttributes() -- the logic is reversed so that ProcessStmtAttributes() is what calls ActOnAttributedStmt().

I think the correct answer is to switch the logic so that ActOnAttributedStmt() calls ProcessStmtAttributes(), then the template logic should automatically work.

[haberman]

clang/lib/Sema/SemaStmt.cpp

562–569

I think the correct answer is to switch the logic so that ActOnAttributedStmt() calls ProcessStmtAttributes()

I think this would require ProcessStmtAttributes() to be split into two separate functions. Currently that function is doing two separate things:

1. Translation of ParsedAttr into various subclasses of Attr.
2. Validation that the attribute is semantically valid.

The function signature for ActOnAttributedStmt() uses Attr (not ParsedAttr), so (1) must happen during the parse, before ActOnAttributedStmt() is called. But (2) must be deferred until template instantiation time for some cases, like musttail.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

I don't think the signature for ActOnAttributedStmt() is correct to use Attr instead of ParsedAttr. I think it should be StmtResult ActOnAttributedStmt(const ParsedAttributesViewWithRange &AttrList, Stmt *SubStmt); -- this likely requires a fair bit of surgery to make work though, which is why I'd like to hear from @rsmith if he agrees with the approach. In the meantime, I'll play around with this idea locally in more depth.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

I think my suggestion wasn't quite right, but close. I've got a patch in progress that changes this the way I was thinking it should be changed, but it won't call ActOnAttributedStmt() when doing template instantiation. Instead, it will continue to instantiate attributes explicitly by calling TransformAttr() and any additional instantiation time checks will require you to add a TreeTransfor::TransformWhateverAttr() to do the actual instantiation work (which is similar to how the declaration attributes work in Sema::InstantiateAttrs()).

I hope to put up a patch for review for these changes today or tomorrow. It'd be interesting to know whether they make your life easier or harder though, if you don't mind taking a look and seeing how well (or poorly) they integrate with your changes here.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

You can find that review at https://reviews.llvm.org/D99896.

[rsmith]

clang/lib/Sema/SemaStmt.cpp

562–569

I think the ideal model would be that we form a FooAttr from the user-supplied attribute description in an ActOn* function from the parser, and have a separate template instantiation mechanism to instantiate FooAttr objects, and those methods are unaware of the subject of the attribute. Then we have a separate mechanism to attach an attribute to its subjects that is used by both parsing and template instantiation. But I suspect there are reasons that doesn't work in practice -- where we need to know something about the subject in order to know how to form the FooAttr. That being the case, it probably makes most sense to model the formation and application of a FooAttr as a single process.

it won't call ActOnAttributedStmt() when doing template instantiation

Good -- not calling ActOn* during template instantiation is the right choice in general -- the ActOn* functions are only supposed to be called from parsing, with a Build* added if the parsing and template instantiation paths would share code (we sometimes shortcut that when the ActOn* and Build* would be identical, but I think that's turned out to be a mistake).

any additional instantiation time checks will require you to add a TreeTransform::TransformWhateverAttr() to do the actual instantiation work

That sounds appropriate to me in general. Are you expecting that this function would also be given the (transformed and perhaps original) subject of the attribute?

601

IgnoreImplicitAsWritten() doesn't skip ExprWithCleanups

That sounds like a bug. Are you sure? It looks like IgnoreImplicitAsWrittenSingleStep calls IgnoreImplicitSingleStep which calls IgnoreImplicitCastsSingleStep which skips FullExpr, and ExprWithCleanups is a kind of FullExpr.

To clarify, are you suggesting that we allow musttail through elidable copy constructors on the return value, even if -fno-elide-constructors is set? ie. we consider that musttail overrides the -fno-elide-constructors option on the command line?

Yes, I think the musttail attribute should override -fno-elide-constructors, because that's necessary in order to provide the tail call the user requested (and the local setting should override the global one). This is probably worth adding to the documentation.

(Also, -fno-elide-constructors is only supposed to affect code generation, not language semantics or program validity, so I think either we should always reject if a constructor call is required for the return value, regardless of whether it's elidable, or we should never reject in that case, and either way this determination should be made independent of the setting of -fno-elide-constructors. Given that choice, it seems more useful to bias towards the common case (-felide-constructors).)

[haberman]

clang/lib/Sema/SemaStmt.cpp

562–569

Would it be possible to defer that refactoring until after this change is in? There are a lot of other issues to resolve on this review as it is, and throwing a potential refactoring into the mix is making it a lot harder to get this into a state where it can be landed.

Once it's in I'm happy to collaborate on the other review.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

I'm fine with that -- my suggestion would be to ignore the template instantiation validation for the moment (add tests with FIXME comments where the behavior isn't what you want) and then when I get you the functionality you need to have more unified checking, you can refactor it at that time.

[haberman]

• Returned validation to ActOnAttributedStmt() so it works with templates.
• Merge branch 'main' into musttail
• Address more review comments.

[haberman]

clang/include/clang/Basic/AttrDocs.td

458

Done. I tried to summarize the C++ concept of "similar" types as defined in https://eel.is/c++draft/conv.qual#2 and implemented in https://clang.llvm.org/doxygen/classclang_1_1ASTContext.html#a1b1b3b7a67a30fd817ba85454780d8ad

clang/lib/Sema/SemaStmt.cpp

562–569

I would strongly prefer to submit correct code (that validates templates) and leave a FIXME to make it pretty, rather than submit pretty code and leave a FIXME to make it correct.

609

I think this case will work actually, the callee decl in this case is just the function pointer, which seems appropriate and type checks correctly.

I added a test for this.

644

Tail calls to a block are indeed straightforward and are handled below. This check is for tail calls from a block, which I tried to add support for but didn't have much luck (in particular, during parsing of a block I wasn't able to get good type information for the block).

I'd probably use a specific diagnostic ("cannot be used from a block" / "cannot be used from an Objective-C function") for the block and ObjCMethod case, and a nonsepcific-but-correct "cannot be used from this context" for anything else.

I implemented this as requested. I wasn't able to test OpenMP as you apparently can't return from an OpenMP block.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

I'm okay with that so long as the follow-up work actually happens (not to suggest that you plan to ignore the request!). "This is functional but not pretty" has a risk of becoming enshrined behavior as priorities shift, whereas "this is incomplete" generally does not.

Please add a FIXME comment here just to make sure it's clear we want the code to move in the future.

[haberman]

• Added FIXME for attribute refactoring.

[haberman]

• Factored duplicated code into a method on MustTailAttr.

[haberman]

clang/lib/Sema/SemaStmt.cpp

562–569

I added a FIXME. Just to set expectations, I'm happy to work with you on updating this code to fit your planned refactoring (either by offering comments/suggestions on a review by you or creating my own follow-up review per your suggestions). But I'll need a fair amount of input from you, since I don't fully grok what you find objectionable about the current code or what your desired end state is.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

562–569

Thanks for the FIXME. I'm totally happy to iterate with you on the refactoring. Mostly, it involves testing whether https://reviews.llvm.org/D99983 provides you with enough contextual information when performing template instantiation for you to be able to put the attribute checking logic into the right places.

The objectionable bit about the current approach is that ActOnAttributedStmt()/BuildAttributedStmt() are general functions for attributed statements that should not be doing per-attribute diagnostic work (this won't scale well as more statement attributes get added). My preferred approach based on what you have already is to call checkMustTailAttr() from handleMustTailAttr(), and call it from TreeTransform.h in a new TransformMustTailAttr() function when doing template instantiation (this part is what requires the other patch to land first).

[haberman]

• Moved calling convention check to happen as early as possible.

[haberman]

In D99517#2667025, @rjmccall wrote:

You should structure this code so it's easy to add exceptions for certain calling conventions that can support tail calls with weaker restrictions (principally, callee-pop conventions). Mostly that probably means checking the calling convention first, or extracting the type restriction checks into a different function that you can skip. For example, I believe x86's fastcall convention can logically support any combination of prototypes as musttail as long as the return types are vaguely compatible.

I moved the calling convention check to be as early as possible.

[haberman]

• Formatted files with clang-format.

[haberman]

clang/lib/Sema/SemaStmt.cpp

562–569

Sounds good. I will follow up with you on https://reviews.llvm.org/D99983.

[rsmith]

clang/include/clang/Basic/DiagnosticSemaKinds.td

2828–2829

Can we somehow avoid talking about ARC where it's not relevant? While it'd be nice to be more precise here, my main concern is that we shouldn't be mentioning ARC to people for whom it's not a meaningful term (eg, when not compiling Objective-C or Objective-C++). Perhaps the simplest approach would be to only mention ARC if getLangOpts().ObjCAutoRefCount is set?

clang/lib/AST/AttrImpl.cpp

221–226 ↗

(On Diff #336203)

IgnoreImplicitAsWritten should already skip over implicit elidable constructors, so I would imagine this is skipping over elidable explicit constructor calls (eg, [[musttail]] return T(make()); would perform a tail-call to make()). Is that what we want?

clang/lib/CodeGen/CGStmt.cpp

668

In the case where we're forcibly eliding a constructor, we'll need to emit a return statement that returns musttail call expression here rather than emitting the original substatement. Otherwise the tail call we emit will be initializing a local temporary rather than initializing our return slot. Eg, given:

struct A {
  A(const A&);
  ~A();
  char data[32];
};
A f();
A g() {
  [[clang::musttail]] return f();
}

under -fno-elide-constructors when targeting C++11, say, we'll normally lower that into something like:

void f(A *return_slot);
void g(A *return_slot) {
  A temporary; //uninitialized
  f(&temporary); // call f
  A::A(return_slot, temporary); // call copy constructor to copy into return slot
}

... and with the current patch, it looks like we'll add a 'ret void' after the call to f, leaving g's return slot uninitialized and passing an address into f that refers to a variable that will no longer exist once f is called. We need to instead lower to:

void f(A *return_slot);
void g(A *return_slot) {
  f(return_slot); // call f
}

Probably the easiest way to do this would be to change the return value on the ReturnStmt to be the tail-called CallExpr when attaching the attribute.

[haberman]

• Refined the implicit constructor skipping code.

clang/include/clang/Basic/DiagnosticSemaKinds.td

2828–2829

I implemented this but I couldn't figure out how to actually trigger the ARC case, so I just removed that part of the diagnostic text for now.

clang/lib/AST/AttrImpl.cpp

221–226 ↗

(On Diff #336203)

As discussed offline, it appears that IgnoreImplicitAsWritten() was not skipping the implicit constructor in this case. Per our discussion, I created a new version of IgnoreImplicitAsWritten() that does, with a FIXME to land it in Expr, and I made it skip implicit constructors only (and added tests for this case).

clang/lib/CodeGen/CGStmt.cpp

668

Done.

I had to change your test case to remove the destructor, otherwise it fails the trivial destruction check.

Take a look at the CodeGen tests and see if the output looks correct to you.

[haberman]

• Rename and refine IgnoreElidableImplicitConstructorSingleStep().

[aaron.ballman]

Mostly just nits from me, but the attribute portions look good to me.

clang/include/clang/AST/IgnoreExpr.h

127

clang/lib/Sema/SemaStmt.cpp

628

636–637

This worries me slightly -- not all CallExpr objects have a callee declaration (https://github.com/llvm/llvm-project/blob/main/clang/lib/AST/Expr.cpp#L1367). That said, I'm struggling to come up with an example that isn't covered so this may be fine.

641

655

659

682

700

It'd be better not to go through the cast machinery twice -- you cast to the MemberPointerType and then cast to the same thing again (but in a different way).

clang/lib/Sema/SemaStmtAttr.cpp

214

This can be removed entirely.

[haberman]

• Simplified some casts and type declarations.

clang/lib/Sema/SemaStmt.cpp

636–637

That was my experience too, I wasn't able to find a case that isn't covered. I tried to avoid adding any diagnostics that I didn't know how to trigger or test.

700

I changed to auto, but I can't tell if you have another suggestion here also. I can't see how any of these casts can be removed.

[aaron.ballman]

clang/lib/Sema/SemaStmt.cpp

697–699

I'm not certain if I should take a shower after writing that code or not, but it's one potential way not to perform the cast twice.

If that code is too odious for others, we should at least change the dyn_cast<> in the else if to be an isa<>.

[haberman]

• Switch to isa<> for type check.
• Merge branch 'main' into musttail

[haberman]

clang/lib/Sema/SemaStmt.cpp

697–699

I changed dyn_cast<> to isa<>. If @rsmith concurs about the dyn_cast_or_null<> variant I'll switch to that.

[rsmith]

clang/lib/Sema/SemaStmt.cpp

603–604

I think this would be clearer, assuming it's equivalent (and if it's not equivalent, I think it'd be useful to include a comment explaining why).

605–609

This loop is problematic: it's generally not safe to modify an expression that is used as a subexpression of another expression. (Modifying the ReturnStmt is, by contrast, much less problematic because the properties of a statement have less complex dependencies on the properties of its subexpressions.) In particular, if there were any implicit conversions here that changed the type or value category or similar, the enclosing parentheses would have the wrong type / value category / similar. Also there are possibilities here other than CallExpr and ParenExpr, such as anything else that we consider to be "parentheses" (such as a GenericSelectionExpr).

But I think this loop should never be necessary, because all implicit conversions should always be on the outside of the parentheses. Do you have a testcase that needs it?

618

... would be more in line with our normal idioms.

636–637

This assert is incorrect. It would fail for a case like:

using T = int();
T *f();
int g() { [[clang::musttail]] return f()(); }

... where there is no declaration associated with the function pointer returned by f().

I think instead of looking for a callee declaration, you should instead inspect the callee expression. You can distinguish between a member function call and a non-member call by looking at the type of the callee. Perhaps the simplest way would be to distinguish between three cases:

(1) There is a callee declaration, which is a member function: this is a direct call to a member function; you can use the type of the callee declaration for your check.
(2) The callee expression is (after skipping parens) a pointer-to-member access operator (BinaryOperator::isPtrMemOp); you can use the type of the RHS operand (which will be a pointer to member function) for your check.
(3) Anything else: this is a non-member-function call, and you can directly inspect the type of the callee without caring about the callee declaration. (You might still find the type is not a function type at this stage, which indicates this is some kind of special form. In particular, it could be a BuiltinType::BoundMember for a pseudo-destructor call. I'm not sure if there are currently any other special cases that make it this far; there might not be, because most such cases are dependent.)

687

Use getAs rather than dyn_cast to look through type sugar. For example, in

void (f)() { [[clang::musttail]] return f(); }

... the type of f is a ParenType, not a FunctionProtoType.

697

You need to use getAs<MemberPointerType> here not isa in order to look through type sugar (eg, typedefs).

However, as noted above, a call via a member pointer doesn't necessarily have a CalleeDecl, so you'll need to do this check by looking for a callee expression that's the right kind of BinaryOperator instead.

clang/test/CodeGen/attr-musttail.cpp

1 ↗

(On Diff #336894)

This is a C++ test so it should be in CodeGenCXX.

178–181 ↗

(On Diff #336894)

It turns out that we consider p to be the callee decl in this case, so we'll need a better example :)

194 ↗

(On Diff #336894)

This doesn't include enough of the output to be able to tell if we've generated correct code. Can you also include the define ... line, showing that %agg.result is the name of the first parameter?

clang/test/Sema/attr-musttail.cpp

1 ↗

(On Diff #336894)

This should be in SemaCXX.

66 ↗

(On Diff #336894)

Please add a FIXME to this; it seems like a bug that we can't tell the difference between needing to run a destructor for the return value and needing to run a destructor for some other temporary created in the return statement.

78 ↗

(On Diff #336894)

The "is a member of different class (expected void" seems surprising here. Can we customize the diagnostic to instead say that we can't musttail from a non-member to a member (and vice versa for the other case)?

167–171 ↗

(On Diff #336894)

Please also test the pseudo-destructor case:

void f() {
  int n;
  using T = int;
  [[clang::musttail]] return n.~T();
}

[haberman]

• Addressed more review comments.

clang/lib/Sema/SemaStmt.cpp

605–609

I removed it and my test cases still pass. I'm glad to know this isn't necessary: I was coding defensively because I didn't know that I could count on this invariant:

all implicit conversions should always be on the outside of the parentheses.

[rsmith]

Functionally this looks good to me. I've suggested some minor cleanups and I understand you're doing some wordsmithing on the diagnostics; I think once those are complete this will be ready to land. Thank you!

clang/lib/CodeGen/CGExpr.cpp

4829

I agree, that sounds like a nice cleanup. Delaying this to a future change makes sense to me.

clang/lib/Sema/SemaStmt.cpp

616

You shouldn't need the const in the argument to cast, and we generally omit it; cast copies the pointer/referenceness and qualifiers from its argument anyway, and the explicit const in the type of R seems sufficient for readers. (I'm not even sure if cast intends to permit explicit qualfiiers here.)

664

I think this isa<CapturedDecl> check is redundant, because a CapturedDecl is not a FunctionDecl, so CallerDecl will always be null when CurContext is a CapturedDecl.

711

Even in invalid code we should never see a CallExpr whose callee has a null type; if Sema can't form an Expr that meets the normal expression invariants during error recovery, it doesn't build one at all. I think you can remove this if.

771

Given that we don't care about differences in qualifiers, it might be clearer to not include them in the diagnostics.

clang/test/CodeGenCXX/attr-musttail.cpp

212

[xbolva00]

Nice new feature! Please also update Release Notes for clang.

[haberman]

clang/lib/Sema/SemaStmt.cpp

711

Without this if(), I crash on this test case. What do you think?

struct TestBadPMF {
  int (TestBadPMF::*pmf)();
  void BadPMF() {
    [[clang::musttail]] return ((*this)->*pmf)(); // expected-error {{left hand operand to ->* must be a pointer to class compatible with the right hand operand, but is 'TestBadPMF'}}
  }
};

Dump of CalleeExpr is:

RecoveryExpr 0x106671e8 '<dependent type>' contains-errors lvalue
|-ParenExpr 0x10667020 'struct TestBadPMF' lvalue
| `-UnaryOperator 0x10667008 'struct TestBadPMF' lvalue prefix '*' cannot overflow
|   `-CXXThisExpr 0x10666ff8 'struct TestBadPMF *' this
`-MemberExpr 0x10667050 'int (struct TestBadPMF::*)(void)' lvalue ->pmf 0x10666ed0
  `-CXXThisExpr 0x10667040 'struct TestBadPMF *' implicit this

[rsmith]

clang/lib/Sema/SemaStmt.cpp

711

Ah, right, while the callee will always have a non-null type, that type might not be a pointer type.

I think what we're missing here is a check for a dependent callee; checking for a dependent context isn't enough to check for error-dependent constructs. Probably the simplest thing would be to change the isDependentContext() checks to also check if the return expression isInstantiationDependent(). (That would only help with the error-dependent cases for now, but we'd also need that extra check in the future if anything like http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2277r0.html goes forward, allowing dependent constructs in non-dependent contexts, especially in combination with http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1306r1.pdf.)

[haberman]

• Word-smithed diagnostics and addressed other review comments.

[haberman]

• More diagnostic wordsmithing.

[haberman]

• Added release note for [[clang::musttail]].

[haberman]

• Fixed release note escaping.

[haberman]

• Fixed several cases in CodeGen test.

[xbolva00]

Thanks, cool :)

[haberman]

• Fixed typo in comment.

[haberman]

Ok I think this is ready to land.

There are a few FIXME comments, I will follow up with some small changes to address them.

[thakis]

Looks like this breaks tests on mac/arm: http://45.33.8.238/macm1/7552/step_7.txt

Please take a look and revert for now if it takes a while to fix.

[rsmith]

In D99517#2693418, @thakis wrote:

Looks like this breaks tests on mac/arm: http://45.33.8.238/macm1/7552/step_7.txt

Should be fixed by rGf7c9de0de5804498085af973dc6bfc934a18f000.

[JonChesterfield]

That is a great feature, thank you. Compiling state machines and scheme programs to C is now much prettier.

[theraven]

The error message here is very confusing:

/home/theraven/snmalloc2/src/mem/../ds/../aal/../ds/defines.h:122:27: error: cannot perform a tail call to function 'error' because its signature is incompatible with the calling function
      [[clang::musttail]] return snmalloc::error(str);
                          ^
/home/theraven/snmalloc2/src/mem/../ds/../aal/../ds/defines.h:63:16: note: target function has different number of parameters (expected 2 but has 1)
  [[noreturn]] SNMALLOC_COLD void error(const char* const str);
               ^
/home/theraven/snmalloc2/src/mem/../ds/../aal/../ds/defines.h:21:25: note: expanded from macro 'SNMALLOC_COLD'
#  define SNMALLOC_COLD __attribute__((cold))
                        ^
/home/theraven/snmalloc2/src/mem/../ds/../aal/../ds/defines.h:122:9: note: tail call required by 'musttail' attribute here
      [[clang::musttail]] return snmalloc::error(str);
        ^

The caller and callee both have one argument, the error is because the enclosing function has two parameters. The error appears wrong anyway for two reasons in this particular context:

• The callee is [[noreturn]], so the stack layout doesn't make any difference, anything can be tail called if it's no-return.
• The enclosing function is always_inline, so checking its argument-frame layout does not give useful information because it's the caller's argument-frame layout that matters.

[haberman]

@theraven: Can you post a minimal repro of your case? I don't follow your distinction between "caller" and "enclosing function."

Regarding noreturn and always_inline: maybe the rules for musttail could be relaxed in cases like the one you mention, but it would require changing the backend (LLVM). Here I changed the front-end only and used LLVM's existing musttail support, which meant accepting its existing limitations.

I would love to see an exception for always_inline: my use case would benefit greatly from this. In my own project I had to change a bunch of always_inline functions to macros to work around this rule. Unfortunately this is complicated by the fact that always_inline does not actually guarantee that inlining occurs.

[theraven]

Here's a minimal test:

void tail(int, float);

__attribute__((always_inline))
void caller(float x)
{
  [[clang::musttail]]
  return tail(42, x);
}

void outer(int x, float y)
{
        return caller(y);
}

This raises this error:

tail.cc:7:3: error: cannot perform a tail call to function 'tail' because its signature is incompatible with the calling function
  return tail(42, x);
  ^
tail.cc:1:1: note: target function has different number of parameters (expected 1 but has 2)
void tail(int, float);
^
tail.cc:6:5: note: tail call required by 'musttail' attribute here
  [[clang::musttail]]
    ^

There's also an interesting counterexample:

void tail(int, float);

__attribute__((always_inline))
void caller(int a, float x)
{
  [[clang::musttail]]
  return tail(a, x);
}

void outer(float y)
{
        return caller(42, y);
}

This *is* accepted by clang, but then generates this IR at -O0:

define dso_local void @_Z5outerf(float %0) #2 {
  %2 = alloca i32, align 4
  %3 = alloca float, align 4
  %4 = alloca float, align 4
  store float %0, float* %4, align 4
  %5 = load float, float* %4, align 4
  store i32 42, i32* %2, align 4
  store float %5, float* %3, align 4
  %6 = load i32, i32* %2, align 4
  %7 = load float, float* %3, align 4
  call void @_Z4tailif(i32 %6, float %7)
  ret void
}

And this IR at -O1:

; Function Attrs: uwtable mustprogress
define dso_local void @_Z5outerf(float %0) local_unnamed_addr #2 {
  call void @_Z4tailif(i32 42, float %0)
  ret void
}

Note that in both cases, the alway-inline attribute is respected (even at -O0, the always-inline inliner runs) but the musttail annotation is lost. The inlining has inserted the call into a function with a different set of parameters and so it cannot have a musttail IR annotation.

[manojgupta]

Please also see https://bugs.llvm.org/show_bug.cgi?id=51416

[rjmccall]

It's not generically true that "anything can be tail-called if it's noreturn". For one, noreturn doesn't imply that the function doesn't exit by e.g. throwing or calling longjmp. For another, the most important user expectation of tail calls is that a long series of tail calls will exhibit zero overall stack growth; in a caller-pop calling convention, calling a function with more parameters may require growing the argument area in a way that cannot be reversed, so e.g. a long sequence of tail calls alternating between 1-argument and 2-argument functions will eventually exhaust the stack, which violates that user expectation.

[chfast]

clang/lib/CodeGen/CGCall.cpp

5319

I reported a related issue. I wander if this is easy to fix. https://github.com/llvm/llvm-project/issues/53087.

clang/include/clang/AST/IgnoreExpr.h

if (auto *MTE = dyn_cast<MaterializeTemporaryExpr>(E))
     return MTE->getSubExpr();
 
   if (auto *BTE = dyn_cast<CXXBindTemporaryExpr>(E))
     return BTE->getSubExpr();
 
   return E;
 }
 
+inline Expr *IgnoreElidableImplicitConstructorSingleStep(Expr *E) {
+  auto *CCE = dyn_cast<CXXConstructExpr>(E);
+  if (CCE && CCE->isElidable() && !isa<CXXTemporaryObjectExpr>(CCE)) {
+    unsigned NumArgs = CCE->getNumArgs();
+    if ((NumArgs == 1 ||
+         (NumArgs > 1 && CCE->getArg(1)->isDefaultArgument())) &&
+        !CCE->getArg(0)->isDefaultArgument() && !CCE->isListInitialization())
+      return CCE->getArg(0);
+  }
+  return E;
+}
+
 inline Expr *IgnoreImplicitAsWrittenSingleStep(Expr *E) {
   if (auto *ICE = dyn_cast<ImplicitCastExpr>(E))
     return ICE->getSubExprAsWritten();
 
   return IgnoreImplicitSingleStep(E);
 }
 
 inline Expr *IgnoreParensOnlySingleStep(Expr *E) {

clang/include/clang/Basic/Attr.td

def NoMerge : DeclOrStmtAttr {
   let Spellings = [Clang<"nomerge">];
   let Documentation = [NoMergeDocs];
   let InheritEvenIfAlreadyPresent = 1;
   let Subjects = SubjectList<[Function, Stmt], ErrorDiag,
                              "functions and statements">;
   let SimpleHandler = 1;
 }
 
+def MustTail : StmtAttr {
+  let Spellings = [Clang<"musttail">];
+  let Documentation = [MustTailDocs];
+  let Subjects = SubjectList<[ReturnStmt], ErrorDiag, "return statements">;
+}
+
 def FastCall : DeclOrTypeAttr {
   let Spellings = [GCC<"fastcall">, Keyword<"__fastcall">,
                    Keyword<"_fastcall">];
 //  let Subjects = [Function, ObjCMethod];
   let Documentation = [FastCallDocs];
 }
 
 def RegCall : DeclOrTypeAttr {

clang/include/clang/Basic/AttrDocs.td

 over the tradeoff between code size and debug information precision. 
 
 ``nomerge`` attribute can also be used as function attribute to prevent all 
 calls to the specified function from merging. It has no effect on indirect 
 calls.
   }];
 }
 
+def MustTailDocs : Documentation {
+  let Category = DocCatStmt;
+  let Content = [{
+If a ``return`` statement is marked ``musttail``, this indicates that the
+compiler must generate a tail call for the program to be correct, even when
+optimizations are disabled. This guarantees that the call will not cause
+unbounded stack growth if it is part of a recursive cycle in the call graph.
+
+If the callee is a virtual function that is implemented by a thunk, there is
+no guarantee in general that the thunk tail-calls the implementation of the
+virtual function, so such a call in a recursive cycle can still result in
+unbounded stack growth.
+
+``clang::musttail`` can only be applied to a ``return`` statement whose value
+is the result of a function call (even functions returning void must use
+``return``, although no value is returned). The target function must have the
+same number of arguments as the caller. The types of the return value and all
+arguments must be similar according to C++ rules (differing only in cv
+qualifiers or array size), including the implicit "this" argument, if any.
+Any variables in scope, including all arguments to the function and the
+return value must be trivially destructible. The calling convention of the
+caller and callee must match, and they must not be variadic functions or have
+old style K&R C function declarations.
+  }];
+}
+
 def AssertCapabilityDocs : Documentation {
   let Category = DocCatFunction;
   let Heading = "assert_capability, assert_shared_capability";
   let Content = [{
 Marks a function that dynamically tests whether a capability is held, and halts
 the program if it is not held.
   }];
 }

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_auto_var_is_id : Warning<
   "'auto' deduced as 'id' in declaration of %0">,
   InGroup<DiagGroup<"auto-var-id">>;
 
 // Attributes
 def warn_nomerge_attribute_ignored_in_stmt: Warning<
   "%0 attribute is ignored because there exists no call expression inside the "
   "statement">,
   InGroup<IgnoredAttributes>;
+def err_musttail_needs_trivial_args : Error<
+  "%0 attribute requires that the return value, all parameters, and any "
+  "temporaries created by the expression are trivially destructible">;
+def err_musttail_needs_call : Error<
+  "%0 attribute requires that the return value is the result of a function call"
+  >;
+def err_musttail_needs_prototype : Error<
+  "%0 attribute requires that both caller and callee functions have a "
+  "prototype">;
+def note_musttail_fix_non_prototype : Note<
+  "add 'void' to the parameter list to turn a old-style K&R function "
+  "declaration into a prototype">;
+def err_musttail_forbidden_from_this_context : Error<
+  "%0 attribute cannot be used from "
+  "%select{a block|an Objective-C function|this context}1">;
+def err_musttail_structors_forbidden : Error<
+  "%0 attribute cannot be used when caller or callee is a constructor or "
+  "destructor, as these can have unusual calling conventions">;
+def note_musttail_structors_forbidden : Note<"%select{caller|callee}0 is a "
+  "%select{constructor|destructor}1">;
+def err_musttail_member_mismatch : Error<
+  "cannot %0 from a %select{member|non-member}1 to a "
+  "%select{non-member|member}1 function">;
+def note_musttail_member_mismatch : Note<
+  "target function is "
+  "%select{a member function of class %1|not a member function}0">;
+def err_musttail_mismatch : Error<
+  "%0 attribute requires that caller and callee have compatible function "
+  "signatures">;
+def note_musttail_mismatch : Note<
+    "target function "
+    "%select{is a member of different class%diff{ (expected $ but has $)|}1,2"
+    "|has different number of parameters (expected %1 but has %2)"
+    "|has type mismatch at %ordinal3 parameter"
+    "%diff{ (expected $ but has $)|}1,2"
+    "|has different return type%diff{ ($ expected but has $)|}1,2}0">;
+def err_musttail_callconv_mismatch : Error<
+  "%0 attribute requires that caller and callee use the same calling convention"
+  >;
+def note_musttail_callconv_mismatch : Note<
+  "target function has calling convention %1 (expected %0)">;
+def err_musttail_scope : Error<
+  "cannot perform a tail call from this return statement">;
+def err_musttail_no_variadic : Error<
+  "%0 attribute may not be used with variadic functions">;
 def err_nsobject_attribute : Error<
   "'NSObject' attribute is for pointer types only">;
 def err_attributes_are_not_compatible : Error<
   "%0 and %1 attributes are not compatible">;
 def err_attribute_invalid_argument : Error<
   "%select{a reference type|an array type|a non-vector or "
   "non-vectorizable scalar type}0 is an invalid argument to attribute %1">;
 def err_attribute_wrong_number_arguments : Error<

clang/include/clang/Sema/ScopeInfo.h

public:
   bool HasBranchProtectedScope : 1;
 
   /// Whether this function contains any switches or direct gotos.
   bool HasBranchIntoScope : 1;
 
   /// Whether this function contains any indirect gotos.
   bool HasIndirectGoto : 1;
 
+  /// Whether this function contains any statement marked with
+  /// \c [[clang::musttail]].
+  bool HasMustTail : 1;
+
   /// Whether a statement was dropped because it was invalid.
   bool HasDroppedStmt : 1;
 
   /// True if current scope is for OpenMP declare reduction combiner.
   bool HasOMPDeclareReductionCombiner : 1;
 
   /// Whether there is a fallthrough statement in this function.
   bool HasFallthroughStmt : 1;
private:
   WeakObjectUseMap WeakObjectUses;
 
 protected:
   FunctionScopeInfo(const FunctionScopeInfo&) = default;
 
 public:
   FunctionScopeInfo(DiagnosticsEngine &Diag)
       : Kind(SK_Function), HasBranchProtectedScope(false),
-        HasBranchIntoScope(false), HasIndirectGoto(false),
+        HasBranchIntoScope(false), HasIndirectGoto(false), HasMustTail(false),
         HasDroppedStmt(false), HasOMPDeclareReductionCombiner(false),
         HasFallthroughStmt(false), UsesFPIntrin(false),
-        HasPotentialAvailabilityViolations(false),
-        ObjCShouldCallSuper(false), ObjCIsDesignatedInit(false),
-        ObjCWarnForNoDesignatedInitChain(false), ObjCIsSecondaryInit(false),
-        ObjCWarnForNoInitDelegation(false), NeedsCoroutineSuspends(true),
-        ErrorTrap(Diag) {}
+        HasPotentialAvailabilityViolations(false), ObjCShouldCallSuper(false),
+        ObjCIsDesignatedInit(false), ObjCWarnForNoDesignatedInitChain(false),
+        ObjCIsSecondaryInit(false), ObjCWarnForNoInitDelegation(false),
+        NeedsCoroutineSuspends(true), ErrorTrap(Diag) {}
 
   virtual ~FunctionScopeInfo();
 
   /// Determine whether an unrecoverable error has occurred within this
   /// function. Note that this may return false even if the function body is
   /// invalid, because the errors may be suppressed if they're caused by prior
   /// invalid declarations.
   ///
public:
   void setHasBranchProtectedScope() {
     HasBranchProtectedScope = true;
   }
 
   void setHasIndirectGoto() {
     HasIndirectGoto = true;
   }
 
+  void setHasMustTail() { HasMustTail = true; }
+
   void setHasDroppedStmt() {
     HasDroppedStmt = true;
   }
 
   void setHasOMPDeclareReductionCombiner() {
     HasOMPDeclareReductionCombiner = true;
   }
 
public:
   }
 
   void setHasSEHTry(SourceLocation TryLoc) {
     setHasBranchProtectedScope();
     FirstSEHTryLoc = TryLoc;
   }
 
   bool NeedsScopeChecking() const {
-    return !HasDroppedStmt &&
-        (HasIndirectGoto ||
+    return !HasDroppedStmt && (HasIndirectGoto || HasMustTail ||
                                (HasBranchProtectedScope && HasBranchIntoScope));
   }
 
   // Add a block introduced in this function.
   void addBlock(const BlockDecl *BD) {
     Blocks.insert(BD);
   }
 
   // Add a __block variable introduced in this function.

clang/include/clang/Sema/Sema.h

sema::FunctionScopeInfo *getCurFunction() const {
     return FunctionScopes.empty() ? nullptr : FunctionScopes.back();
   }
 
   sema::FunctionScopeInfo *getEnclosingFunction() const;
 
   void setFunctionHasBranchIntoScope();
   void setFunctionHasBranchProtectedScope();
   void setFunctionHasIndirectGoto();
+  void setFunctionHasMustTail();
 
   void PushCompoundScope(bool IsStmtExpr);
   void PopCompoundScope();
 
   sema::CompoundScopeInfo &getCurCompoundScope() const;
 
   bool hasAnyUnrecoverableErrorsInThisFunction() const;
 

 
   // Determines which VarArgKind fits an expression.
   VarArgKind isValidVarArgType(const QualType &Ty);
 
   /// Check to see if the given expression is a valid argument to a variadic
   /// function, issuing a diagnostic if not.
   void checkVariadicArgument(const Expr *E, VariadicCallType CT);
 
+  /// Check whether the given expression can have musttail applied to it,
+  /// issuing a diagnostic and returning false if not. In the success case,
+  /// the statement is rewritten to remove implicit nodes from the return
+  /// value.
+  bool checkAndRewriteMustTailAttr(Stmt *St, const Attr &MTA);
+
+private:
+  /// Check whether the given expression can have musttail applied to it,
+  /// issuing a diagnostic and returning false if not.
+  bool checkMustTailAttr(const Stmt *St, const Attr &MTA);
+
+public:
   /// Check to see if a given expression could have '.c_str()' called on it.
   bool hasCStrMethod(const Expr *E);
 
   /// GatherArgumentsForCall - Collector argument expressions for various
   /// form of call prototypes.
   bool GatherArgumentsForCall(SourceLocation CallLoc, FunctionDecl *FDecl,
                               const FunctionProtoType *Proto,
                               unsigned FirstParam, ArrayRef<Expr *> Args,

clang/lib/CodeGen/CGCall.cpp

 };
 
 } // namespace
 
 RValue CodeGenFunction::EmitCall(const CGFunctionInfo &CallInfo,
                                  const CGCallee &Callee,
                                  ReturnValueSlot ReturnValue,
                                  const CallArgList &CallArgs,
-                                 llvm::CallBase **callOrInvoke,
+                                 llvm::CallBase **callOrInvoke, bool IsMustTail,
                                  SourceLocation Loc) {
   // FIXME: We no longer need the types from CallArgs; lift up and simplify.
 
   assert(Callee.isOrdinary() || Callee.isVirtual());
 
   // Handle struct-return functions by passing a pointer to the
   // location that we would like to return into.
   QualType RetTy = CallInfo.getReturnType();
if (!CI->getCalledFunction())
     PGO.valueProfile(Builder, llvm::IPVK_IndirectCallTarget,
                      CI, CalleePtr);
 
   // In ObjC ARC mode with no ObjC ARC exception safety, tell the ARC
   // optimizer it can aggressively ignore unwind edges.
   if (CGM.getLangOpts().ObjCAutoRefCount)
     AddObjCARCExceptionMetadata(CI);
 
-  // Suppress tail calls if requested.
+  // Set tail call kind if necessary.
   if (llvm::CallInst *Call = dyn_cast<llvm::CallInst>(CI)) {
     if (TargetDecl && TargetDecl->hasAttr<NotTailCalledAttr>())
       Call->setTailCallKind(llvm::CallInst::TCK_NoTail);
+    else if (IsMustTail)
+      Call->setTailCallKind(llvm::CallInst::TCK_MustTail);
   }
 
   // Add metadata for calls to MSAllocator functions
   if (getDebugInfo() && TargetDecl &&
       TargetDecl->hasAttr<MSAllocatorAttr>())
     getDebugInfo()->addHeapAllocSiteMetadata(CI, RetTy->getPointeeType(), Loc);
 
   // 4. Finish the call.
if (CI->doesNotReturn()) {
     // generally are not ready to handle emitting expressions at unreachable
     // points.
     EnsureInsertPoint();
 
     // Return a reasonable RValue.
     return GetUndefRValue(RetTy);
   }
 
+  // If this is a musttail call, return immediately. We do not branch to the
+  // epilogue in this case.
+  if (IsMustTail) {
+    for (auto it = EHStack.find(CurrentCleanupScopeDepth); it != EHStack.end();
+         ++it) {
+      EHCleanupScope *Cleanup = dyn_cast<EHCleanupScope>(&*it);
+      if (!(Cleanup && Cleanup->getCleanup()->isRedundantBeforeReturn()))
+        CGM.ErrorUnsupported(MustTailCall, "tail call skipping over cleanups");
+    }
+    if (CI->getType()->isVoidTy())
+      Builder.CreateRetVoid();
+    else
+      Builder.CreateRet(CI);
+    Builder.ClearInsertionPoint();
+    EnsureInsertPoint();
+    return GetUndefRValue(RetTy);
+  }
+
   // Perform the swifterror writeback.
   if (swiftErrorTemp.isValid()) {
     llvm::Value *errorResult = Builder.CreateLoad(swiftErrorTemp);
     Builder.CreateStore(errorResult, swiftErrorArg);
   }
 
   // Emit any call-associated writebacks immediately.  Arguably this
   // should happen after any return-value munging.

clang/lib/CodeGen/CGClass.cpp

CGCXXABI::AddedStructorArgCounts ExtraArgs =
       CGM.getCXXABI().addImplicitConstructorArgs(*this, D, Type, ForVirtualBase,
                                                  Delegating, Args);
 
   // Emit the call.
   llvm::Constant *CalleePtr = CGM.getAddrOfCXXStructor(GlobalDecl(D, Type));
   const CGFunctionInfo &Info = CGM.getTypes().arrangeCXXConstructorCall(
       Args, D, Type, ExtraArgs.Prefix, ExtraArgs.Suffix, PassPrototypeArgs);
   CGCallee Callee = CGCallee::forDirect(CalleePtr, GlobalDecl(D, Type));
-  EmitCall(Info, Callee, ReturnValueSlot(), Args, nullptr, Loc);
+  EmitCall(Info, Callee, ReturnValueSlot(), Args, nullptr, false, Loc);
 
   // Generate vtable assumptions if we're constructing a complete object
   // with a vtable.  We don't do this for base subobjects for two reasons:
   // first, it's incorrect for classes with virtual bases, and second, we're
   // about to overwrite the vptrs anyway.
   // We also have to make sure if we can refer to vtable:
   // - Otherwise we can refer to vtable if it's safe to speculatively emit.
   // FIXME: If vtable is used by ctor/dtor, or if vtable is external and we are

clang/lib/CodeGen/CGDecl.cpp

struct DestroyNRVOVariableC final
     void emitDestructorCall(CodeGenFunction &CGF) {
       CGF.destroyNonTrivialCStruct(CGF, Loc, Ty);
     }
   };
 
   struct CallStackRestore final : EHScopeStack::Cleanup {
     Address Stack;
     CallStackRestore(Address Stack) : Stack(Stack) {}
+    bool isRedundantBeforeReturn() override { return true; }
     void Emit(CodeGenFunction &CGF, Flags flags) override {
       llvm::Value *V = CGF.Builder.CreateLoad(Stack);
       llvm::Function *F = CGF.CGM.getIntrinsic(llvm::Intrinsic::stackrestore);
       CGF.Builder.CreateCall(F, V);
     }
   };
 
   struct ExtendGCLifetime final : EHScopeStack::Cleanup {

clang/lib/CodeGen/CGExpr.cpp

 #include "llvm/ADT/StringExtras.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/MDBuilder.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/Path.h"
+#include "llvm/Support/SaveAndRestore.h"
 #include "llvm/Transforms/Utils/SanitizerStats.h"
 
 #include <string>
 
 using namespace clang;
 using namespace CodeGen;
 
 //===--------------------------------------------------------------------===//

 }
 
 //===--------------------------------------------------------------------===//
 //                             Expression Emission
 //===--------------------------------------------------------------------===//
 
 RValue CodeGenFunction::EmitCallExpr(const CallExpr *E,
                                      ReturnValueSlot ReturnValue) {
   // Builtins never have block type.
   if (E->getCallee()->getType()->isBlockPointerType())
     return EmitBlockCallExpr(E, ReturnValue);
 
   if (const auto *CE = dyn_cast<CXXMemberCallExpr>(E))
     return EmitCXXMemberCallExpr(CE, ReturnValue);
 
   if (const auto *CE = dyn_cast<CUDAKernelCallExpr>(E))
     return EmitCUDAKernelCallExpr(CE, ReturnValue);
if (CGM.getLangOpts().HIP && !CGM.getLangOpts().CUDAIsDevice &&
     llvm::Value *Handle = Callee.getFunctionPointer();
     auto *Cast =
         Builder.CreateBitCast(Handle, Handle->getType()->getPointerTo());
     auto *Stub = Builder.CreateLoad(Address(Cast, CGM.getPointerAlign()));
     Callee.setFunctionPointer(Stub);
   }
   llvm::CallBase *CallOrInvoke = nullptr;
   RValue Call = EmitCall(FnInfo, Callee, ReturnValue, Args, &CallOrInvoke,
-                         E->getExprLoc());
+                         E == MustTailCall, E->getExprLoc());
 
   // Generate function declaration DISuprogram in order to be used
   // in debug info about call sites.
   if (CGDebugInfo *DI = getDebugInfo()) {
     if (auto *CalleeDecl = dyn_cast_or_null<FunctionDecl>(TargetDecl))
       DI->EmitFuncDeclForCallSite(CallOrInvoke, QualType(FnType, 0),
                                   CalleeDecl);
   }

clang/lib/CodeGen/CGExprCXX.cpp

RValue CodeGenFunction::EmitCXXMemberOrOperatorCall(
     const CallExpr *CE, CallArgList *RtlArgs) {
   const FunctionProtoType *FPT = MD->getType()->castAs<FunctionProtoType>();
   CallArgList Args;
   MemberCallInfo CallInfo = commonEmitCXXMemberOrOperatorCall(
       *this, MD, This, ImplicitParam, ImplicitParamTy, CE, Args, RtlArgs);
   auto &FnInfo = CGM.getTypes().arrangeCXXMethodCall(
       Args, FPT, CallInfo.ReqArgs, CallInfo.PrefixSize);
   return EmitCall(FnInfo, Callee, ReturnValue, Args, nullptr,
+                  CE && CE == MustTailCall,
                   CE ? CE->getExprLoc() : SourceLocation());
 }
 
 RValue CodeGenFunction::EmitCXXDestructorCall(
     GlobalDecl Dtor, const CGCallee &Callee, llvm::Value *This, QualType ThisTy,
     llvm::Value *ImplicitParam, QualType ImplicitParamTy, const CallExpr *CE) {
   const CXXMethodDecl *DtorDecl = cast<CXXMethodDecl>(Dtor.getDecl());
 
if (SrcAS != DstAS) {
     This = getTargetHooks().performAddrSpaceCast(*this, This, SrcAS, DstAS,
                                                  NewType);
   }
 
   CallArgList Args;
   commonEmitCXXMemberOrOperatorCall(*this, DtorDecl, This, ImplicitParam,
                                     ImplicitParamTy, CE, Args, nullptr);
   return EmitCall(CGM.getTypes().arrangeCXXStructorDeclaration(Dtor), Callee,
-                  ReturnValueSlot(), Args, nullptr,
+                  ReturnValueSlot(), Args, nullptr, CE && CE == MustTailCall,
                   CE ? CE->getExprLoc() : SourceLocation{});
 }
 
 RValue CodeGenFunction::EmitCXXPseudoDestructorExpr(
                                             const CXXPseudoDestructorExpr *E) {
   QualType DestroyedType = E->getDestroyedType();
   if (DestroyedType.hasStrongOrWeakObjCLifetime()) {
     // Automatic Reference Counting:
CodeGenFunction::EmitCXXMemberPointerCallExpr(const CXXMemberCallExpr *E,
   Args.add(RValue::get(ThisPtrForCall), ThisType);
 
   RequiredArgs required = RequiredArgs::forPrototypePlus(FPT, 1);
 
   // And the rest of the call args
   EmitCallArgs(Args, FPT, E->arguments());
   return EmitCall(CGM.getTypes().arrangeCXXMethodCall(Args, FPT, required,
                                                       /*PrefixSize=*/0),
-                  Callee, ReturnValue, Args, nullptr, E->getExprLoc());
+                  Callee, ReturnValue, Args, nullptr, E == MustTailCall,
+                  E->getExprLoc());
 }
 
 RValue
 CodeGenFunction::EmitCXXOperatorMemberCallExpr(const CXXOperatorCallExpr *E,
                                                const CXXMethodDecl *MD,
                                                ReturnValueSlot ReturnValue) {
   assert(MD->isInstance() &&
          "Trying to emit a member call expr on a static method!");

clang/lib/CodeGen/CGStmt.cpp

 //===----------------------------------------------------------------------===//
 
 #include "CGDebugInfo.h"
 #include "CGOpenMPRuntime.h"
 #include "CodeGenFunction.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
 #include "clang/AST/Attr.h"
+#include "clang/AST/Expr.h"
+#include "clang/AST/Stmt.h"
 #include "clang/AST/StmtVisitor.h"
 #include "clang/Basic/Builtins.h"
 #include "clang/Basic/DiagnosticSema.h"
 #include "clang/Basic/PrettyStackTrace.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/StringExtras.h"

 
 void CodeGenFunction::EmitLabelStmt(const LabelStmt &S) {
   EmitLabel(S.getDecl());
   EmitStmt(S.getSubStmt());
 }
 
 void CodeGenFunction::EmitAttributedStmt(const AttributedStmt &S) {
   bool nomerge = false;
-  for (const auto *A : S.getAttrs())
+  const CallExpr *musttail = nullptr;
+
+  for (const auto *A : S.getAttrs()) {
     if (A->getKind() == attr::NoMerge) {
       nomerge = true;
-      break;
-    }
+    }
+    if (A->getKind() == attr::MustTail) {
+      const Stmt *Sub = S.getSubStmt();
+      const ReturnStmt *R = cast<ReturnStmt>(Sub);
+      musttail = cast<CallExpr>(R->getRetValue()->IgnoreParens());
+    }
   SaveAndRestore<bool> save_nomerge(InNoMergeAttributedStmt, nomerge);
+  SaveAndRestore<const CallExpr *> save_musttail(MustTailCall, musttail);
   EmitStmt(S.getSubStmt(), S.getAttrs());
 }
 
 void CodeGenFunction::EmitGotoStmt(const GotoStmt &S) {
   // If this code is reachable then emit a stop point (if generating
   // debug info). We have to do this ourselves because we are on the
   // "simple" statement path.
   if (HaveInsertPoint())
     EmitStopPoint(&S);

clang/lib/CodeGen/CodeGenFunction.h

public:
 
   /// True if CodeGen currently emits code inside presereved access index
   /// region.
   bool IsInPreservedAIRegion = false;
 
   /// True if the current statement has nomerge attribute.
   bool InNoMergeAttributedStmt = false;
 
+  // The CallExpr within the current statement that the musttail attribute
+  // applies to.  nullptr if there is no 'musttail' on the current statement.
+  const CallExpr *MustTailCall = nullptr;
+
   /// True if the current function should be marked mustprogress.
   bool FnIsMustProgress = false;
 
   /// True if the C++ Standard Requires Progress.
   bool CPlusPlusWithProgress() {
     if (CGM.getCodeGenOpts().getFiniteLoops() ==
         CodeGenOptions::FiniteLoopsKind::Never)
       return false;
public:
 
   EHScopeStack EHStack;
   llvm::SmallVector<char, 256> LifetimeExtendedCleanupStack;
   llvm::SmallVector<const JumpDest *, 2> SEHTryEpilogueStack;
 
   llvm::Instruction *CurrentFuncletPad = nullptr;
 
   class CallLifetimeEnd final : public EHScopeStack::Cleanup {
+    bool isRedundantBeforeReturn() override { return true; }
+
     llvm::Value *Addr;
     llvm::Value *Size;
 
   public:
     CallLifetimeEnd(Address addr, llvm::Value *size)
         : Addr(addr.getPointer()), Size(size) {}
 
     void Emit(CodeGenFunction &CGF, Flags flags) override {
public:
   //                         Scalar Expression Emission
   //===--------------------------------------------------------------------===//
 
   /// EmitCall - Generate a call of the given function, expecting the given
   /// result type, and using the given argument list which specifies both the
   /// LLVM arguments and the types they were derived from.
   RValue EmitCall(const CGFunctionInfo &CallInfo, const CGCallee &Callee,
                   ReturnValueSlot ReturnValue, const CallArgList &Args,
-                  llvm::CallBase **callOrInvoke, SourceLocation Loc);
+                  llvm::CallBase **callOrInvoke, bool IsMustTail,
+                  SourceLocation Loc);
   RValue EmitCall(const CGFunctionInfo &CallInfo, const CGCallee &Callee,
                   ReturnValueSlot ReturnValue, const CallArgList &Args,
-                  llvm::CallBase **callOrInvoke = nullptr) {
+                  llvm::CallBase **callOrInvoke = nullptr,
+                  bool IsMustTail = false) {
     return EmitCall(CallInfo, Callee, ReturnValue, Args, callOrInvoke,
-                    SourceLocation());
+                    IsMustTail, SourceLocation());
   }
   RValue EmitCall(QualType FnType, const CGCallee &Callee, const CallExpr *E,
                   ReturnValueSlot ReturnValue, llvm::Value *Chain = nullptr);
   RValue EmitCallExpr(const CallExpr *E,
                       ReturnValueSlot ReturnValue = ReturnValueSlot());
   RValue EmitSimpleCallExpr(const CallExpr *E, ReturnValueSlot ReturnValue);
   CGCallee EmitCallee(const Expr *E);
 

clang/lib/CodeGen/EHScopeStack.h

public:
   protected:
     ~Cleanup() = default;
 
   public:
     Cleanup(const Cleanup &) = default;
     Cleanup(Cleanup &&) {}
     Cleanup() = default;
 
+    virtual bool isRedundantBeforeReturn() { return false; }
+
     /// Generation flags.
     class Flags {
       enum {
         F_IsForEH = 0x1,
         F_IsNormalCleanupKind = 0x2,
         F_IsEHCleanupKind = 0x4,
         F_HasExitSwitch = 0x8,
       };

clang/lib/Sema/JumpDiagnostics.cpp

 //===--- JumpDiagnostics.cpp - Protected scope jump analysis ------*- C++ -*-=//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the JumpScopeChecker class, which is used to diagnose
 // jumps that enter a protected scope in an invalid way.
 //
 //===----------------------------------------------------------------------===//
 
-#include "clang/Sema/SemaInternal.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Expr.h"
 #include "clang/AST/ExprCXX.h"
 #include "clang/AST/StmtCXX.h"
 #include "clang/AST/StmtObjC.h"
 #include "clang/AST/StmtOpenMP.h"
+#include "clang/Basic/SourceLocation.h"
+#include "clang/Sema/SemaInternal.h"
 #include "llvm/ADT/BitVector.h"
 using namespace clang;
 
 namespace {
 
 /// JumpScopeChecker - This object is used by Sema to diagnose invalid jumps
 /// into VLA and other protected scopes.  For example, this rejects:
 ///    goto L;
 ///    int a[n];
 ///  L:
 ///
+/// We also detect jumps out of protected scopes when it's not possible to do
+/// cleanups properly. Indirect jumps and ASM jumps can't do cleanups because
+/// the target is unknown. Return statements with \c [[clang::musttail]] cannot
+/// handle any cleanups due to the nature of a tail call.
 class JumpScopeChecker {
   Sema &S;
 
   /// Permissive - True when recovering from errors, in which case precautions
   /// are taken to handle incomplete scope information.
   const bool Permissive;
 
   /// GotoScope - This is a record that we use to keep track of all of the
class JumpScopeChecker {
   };
 
   SmallVector<GotoScope, 48> Scopes;
   llvm::DenseMap<Stmt*, unsigned> LabelAndGotoScopes;
   SmallVector<Stmt*, 16> Jumps;
 
   SmallVector<Stmt*, 4> IndirectJumps;
   SmallVector<Stmt*, 4> AsmJumps;
+  SmallVector<AttributedStmt *, 4> MustTailStmts;
   SmallVector<LabelDecl*, 4> IndirectJumpTargets;
   SmallVector<LabelDecl*, 4> AsmJumpTargets;
 public:
   JumpScopeChecker(Stmt *Body, Sema &S);
 private:
   void BuildScopeInformation(Decl *D, unsigned &ParentScope);
   void BuildScopeInformation(VarDecl *D, const BlockDecl *BDecl,
                              unsigned &ParentScope);
   void BuildScopeInformation(CompoundLiteralExpr *CLE, unsigned &ParentScope);
   void BuildScopeInformation(Stmt *S, unsigned &origParentScope);
 
   void VerifyJumps();
   void VerifyIndirectOrAsmJumps(bool IsAsmGoto);
+  void VerifyMustTailStmts();
   void NoteJumpIntoScopes(ArrayRef<unsigned> ToScopes);
   void DiagnoseIndirectOrAsmJump(Stmt *IG, unsigned IGScope, LabelDecl *Target,
                                  unsigned TargetScope);
   void CheckJump(Stmt *From, Stmt *To, SourceLocation DiagLoc,
                  unsigned JumpDiag, unsigned JumpDiagWarning,
                  unsigned JumpDiagCXX98Compat);
   void CheckGotoStmt(GotoStmt *GS);
+  const Attr *GetMustTailAttr(AttributedStmt *AS);
 
   unsigned GetDeepestCommonScope(unsigned A, unsigned B);
 };
 } // end anonymous namespace
 
 #define CHECK_PERMISSIVE(x) (assert(Permissive || !(x)), (Permissive && (x)))
 
 JumpScopeChecker::JumpScopeChecker(Stmt *Body, Sema &s)
     : S(s), Permissive(s.hasAnyUnrecoverableErrorsInThisFunction()) {
   // Add a scope entry for function scope.
   Scopes.push_back(GotoScope(~0U, ~0U, ~0U, SourceLocation()));
 
   // Build information for the top level compound statement, so that we have a
   // defined scope record for every "goto" and label.
   unsigned BodyParentScope = 0;
   BuildScopeInformation(Body, BodyParentScope);
 
   // Check that all jumps we saw are kosher.
   VerifyJumps();
   VerifyIndirectOrAsmJumps(false);
   VerifyIndirectOrAsmJumps(true);
+  VerifyMustTailStmts();
 }
 
 /// GetDeepestCommonScope - Finds the innermost scope enclosing the
 /// two scopes.
 unsigned JumpScopeChecker::GetDeepestCommonScope(unsigned A, unsigned B) {
   while (A != B) {
     // Inner scopes are created after outer scopes and therefore have
     // higher indices.
void JumpScopeChecker::BuildScopeInformation(Stmt *S,
   }
 
   case Stmt::CaseStmtClass:
   case Stmt::DefaultStmtClass:
   case Stmt::LabelStmtClass:
     LabelAndGotoScopes[S] = ParentScope;
     break;
 
+  case Stmt::AttributedStmtClass: {
+    AttributedStmt *AS = cast<AttributedStmt>(S);
+    if (GetMustTailAttr(AS)) {
+      LabelAndGotoScopes[AS] = ParentScope;
+      MustTailStmts.push_back(AS);
+    }
+    break;
+  }
+
   default:
     if (auto *ED = dyn_cast<OMPExecutableDirective>(S)) {
       if (!ED->isStandaloneDirective()) {
         unsigned NewParentScope = Scopes.size();
         Scopes.emplace_back(ParentScope,
                             diag::note_omp_protected_structured_block,
                             diag::note_omp_exits_structured_block,
                             ED->getStructuredBlock()->getBeginLoc());
void JumpScopeChecker::CheckGotoStmt(GotoStmt *GS) {
   if (GS->getLabel()->isMSAsmLabel()) {
     S.Diag(GS->getGotoLoc(), diag::err_goto_ms_asm_label)
         << GS->getLabel()->getIdentifier();
     S.Diag(GS->getLabel()->getLocation(), diag::note_goto_ms_asm_label)
         << GS->getLabel()->getIdentifier();
   }
 }
 
+void JumpScopeChecker::VerifyMustTailStmts() {
+  for (AttributedStmt *AS : MustTailStmts) {
+    for (unsigned I = LabelAndGotoScopes[AS]; I; I = Scopes[I].ParentScope) {
+      if (Scopes[I].OutDiag) {
+        S.Diag(AS->getBeginLoc(), diag::err_musttail_scope);
+        S.Diag(Scopes[I].Loc, Scopes[I].OutDiag);
+      }
+    }
+  }
+}
+
+const Attr *JumpScopeChecker::GetMustTailAttr(AttributedStmt *AS) {
+  ArrayRef<const Attr *> Attrs = AS->getAttrs();
+  const auto *Iter =
+      llvm::find_if(Attrs, [](const Attr *A) { return isa<MustTailAttr>(A); });
+  return Iter != Attrs.end() ? *Iter : nullptr;
+}
+
 void Sema::DiagnoseInvalidJumps(Stmt *Body) {
   (void)JumpScopeChecker(Body, *this);
 }

clang/lib/Sema/Sema.cpp

if (!FunctionScopes.empty())
     FunctionScopes.back()->setHasBranchProtectedScope();
 }
 
 void Sema::setFunctionHasIndirectGoto() {
   if (!FunctionScopes.empty())
     FunctionScopes.back()->setHasIndirectGoto();
 }
 
+void Sema::setFunctionHasMustTail() {
+  if (!FunctionScopes.empty())
+    FunctionScopes.back()->setHasMustTail();
+}
+
 BlockScopeInfo *Sema::getCurBlock() {
   if (FunctionScopes.empty())
     return nullptr;
 
   auto CurBSI = dyn_cast<BlockScopeInfo>(FunctionScopes.back());
   if (CurBSI && CurBSI->TheDecl &&
       !CurBSI->TheDecl->Encloses(CurContext)) {
     // We have switched contexts due to template instantiation.

clang/lib/Sema/SemaStmt.cpp

 //===--- SemaStmt.cpp - Semantic Analysis for Statements ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file implements semantic analysis for statements.
 //
 //===----------------------------------------------------------------------===//
 
-#include "clang/Sema/Ownership.h"
-#include "clang/Sema/SemaInternal.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/ASTDiagnostic.h"
 #include "clang/AST/ASTLambda.h"
-#include "clang/AST/CharUnits.h"
 #include "clang/AST/CXXInheritance.h"
+#include "clang/AST/CharUnits.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/AST/EvaluatedExprVisitor.h"
 #include "clang/AST/ExprCXX.h"
 #include "clang/AST/ExprObjC.h"
+#include "clang/AST/IgnoreExpr.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/StmtCXX.h"
 #include "clang/AST/StmtObjC.h"
 #include "clang/AST/TypeLoc.h"
 #include "clang/AST/TypeOrdering.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Sema/Initialization.h"
 #include "clang/Sema/Lookup.h"
+#include "clang/Sema/Ownership.h"
 #include "clang/Sema/Scope.h"
 #include "clang/Sema/ScopeInfo.h"
+#include "clang/Sema/SemaInternal.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 
 using namespace clang;
if (!TheDecl->isGnuLocal()) {
     }
   }
   return LS;
 }
 
 StmtResult Sema::BuildAttributedStmt(SourceLocation AttrsLoc,
                                      ArrayRef<const Attr *> Attrs,
                                      Stmt *SubStmt) {
+  // FIXME: move this code should move when a planned refactoring around
+  // statement attributes lands.
+  for (const auto *A : Attrs) {
+    if (A->getKind() == attr::MustTail) {
+      if (!checkAndRewriteMustTailAttr(SubStmt, *A)) {
+        return SubStmt;
+      }
+      setFunctionHasMustTail();
+    }
+  }
+
   return AttributedStmt::Create(Context, AttrsLoc, Attrs, SubStmt);
 }
 
 StmtResult Sema::ActOnAttributedStmt(const ParsedAttributesWithRange &Attrs,
                                      Stmt *SubStmt) {
   SmallVector<const Attr *, 1> SemanticAttrs;
   ProcessStmtAttributes(SubStmt, Attrs, SemanticAttrs);
   if (!SemanticAttrs.empty())
     return BuildAttributedStmt(Attrs.Range.getBegin(), SemanticAttrs, SubStmt);
   // If none of the attributes applied, that's fine, we can recover by
   // returning the substatement directly instead of making an AttributedStmt
   // with no attributes on it.
   return SubStmt;
 }
 
+bool Sema::checkAndRewriteMustTailAttr(Stmt *St, const Attr &MTA) {
+  if (CurContext->isDependentContext())
+    // We have to suspend our check until template instantiation time.
+    return true;
+
+  if (!checkMustTailAttr(St, MTA))
+    return false;
+
+  // FIXME: Replace Expr::IgnoreImplicitAsWritten() with this function.
+  // Currently it does not skip implicit constructors in an initialization
+  // context.
+  auto IgnoreImplicitAsWritten = [](Expr *Ex) -> Expr * {
+    return IgnoreExprNodes(Ex, IgnoreImplicitAsWrittenSingleStep,
+                           IgnoreElidableImplicitConstructorSingleStep);
+  };
+
+  // Now that we have verified that 'musttail' is valid here, rewrite the
+  // return value to remove all implicit nodes, but retain parentheses.
+  ReturnStmt *R = cast<ReturnStmt>(St);
+  Expr *Ex = IgnoreImplicitAsWritten(R->getRetValue());
+  R->setRetValue(Ex);
+  return true;
+}
+
+bool Sema::checkMustTailAttr(const Stmt *St, const Attr &MTA) {
+  assert(!CurContext->isDependentContext() &&
+         "musttail cannot be checked from a dependent context");
+
+  const auto *R = cast<const ReturnStmt>(St);
+  const Expr *Ex = R->getRetValue();
+
+  // FIXME: Add Expr::IgnoreParenImplicitAsWritten() with this definition.
+  auto IgnoreParenImplicitAsWritten = [](const Expr *Ex) -> const Expr * {
+    return IgnoreExprNodes(const_cast<Expr *>(Ex), IgnoreParensSingleStep,
+                           IgnoreImplicitAsWrittenSingleStep,
+                           IgnoreElidableImplicitConstructorSingleStep);
+  };
+
+  const auto *CE = dyn_cast_or_null<CallExpr>(IgnoreParenImplicitAsWritten(Ex));
+
+  if (!CE) {
+    Diag(St->getBeginLoc(), diag::err_musttail_needs_call) << &MTA;
+    return false;
+  }
+
+  if (const auto *EWC = dyn_cast<ExprWithCleanups>(Ex)) {
+    if (EWC->cleanupsHaveSideEffects()) {
+      Diag(St->getBeginLoc(), diag::err_musttail_needs_trivial_args) << &MTA;
+      return false;
+    }
+  }
+
+  // We need to determine the full function type (including "this" type, if any)
+  // for both caller and callee.
+  struct FuncType {
+    QualType This;
+    const FunctionProtoType *Func;
+  } CallerType, CalleeType;
+
+  auto GetMethodType = [this, St, MTA](const CXXMethodDecl *CMD, FuncType &Type,
+                                       bool IsCallee) -> bool {
+    if (isa<CXXConstructorDecl, CXXDestructorDecl>(CMD)) {
+      Diag(St->getBeginLoc(), diag::err_musttail_structors_forbidden) << &MTA;
+      Diag(CMD->getBeginLoc(), diag::note_musttail_structors_forbidden)
+          << IsCallee << isa<CXXDestructorDecl>(CMD);
+      return false;
+    }
+    if (!CMD->isStatic())
+      Type.This = CMD->getThisType()->getPointeeType();
+    Type.Func = CMD->getType()->castAs<FunctionProtoType>();
+    return true;
+  };
+
+  const auto *CallerDecl = dyn_cast<FunctionDecl>(CurContext);
+
+  // Find caller function signature.
+  if (!CallerDecl || isa<CapturedDecl>(CurContext)) {
+    int ContextType;
+    if (isa<BlockDecl>(CurContext))
+      ContextType = 0;
+    else if (isa<ObjCMethodDecl>(CurContext))
+      ContextType = 1;
+    else
+      ContextType = 2;
+    Diag(St->getBeginLoc(), diag::err_musttail_forbidden_from_this_context)
+        << &MTA << ContextType;
+    return false;
+  } else if (const auto *CMD = dyn_cast<CXXMethodDecl>(CurContext)) {
+    // Caller is a class/struct method.
+    if (!GetMethodType(CMD, CallerType, false))
+      return false;
+  } else {
+    // Caller is a non-method function.
+    CallerType.Func = CallerDecl->getType()->getAs<FunctionProtoType>();
+  }
+
+  const Expr *CalleeExpr = CE->getCallee()->IgnoreParens();
+  const auto *CalleeBinOp = dyn_cast<BinaryOperator>(CalleeExpr);
+  SourceLocation CalleeLoc = CE->getDirectCallee()
+                                  ? CE->getDirectCallee()->getBeginLoc()
+                                  : St->getBeginLoc();
+
+  // Find callee function signature.
+  if (const CXXMethodDecl *CMD =
+          dyn_cast_or_null<CXXMethodDecl>(CE->getCalleeDecl())) {
+    // Call is: obj.method(), obj->method(), functor(), etc.
+    if (!GetMethodType(CMD, CalleeType, true))
+      return false;
+  } else if (CalleeBinOp && CalleeBinOp->isPtrMemOp()) {
+    // Call is: obj->*method_ptr or obj.*method_ptr
+    const auto *MPT =
+        CalleeBinOp->getRHS()->getType()->castAs<MemberPointerType>();
+    CalleeType.This = QualType(MPT->getClass(), 0);
+    CalleeType.Func = MPT->getPointeeType()->castAs<FunctionProtoType>();
+  } else if (isa<CXXPseudoDestructorExpr>(CalleeExpr)) {
+    Diag(St->getBeginLoc(), diag::err_musttail_structors_forbidden) << &MTA;
+    Diag(St->getBeginLoc(), diag::note_musttail_structors_forbidden)
+        << /* IsCallee = */ 1 << /* IsDestructor = */ 1;
+    return false;
+  } else {
+    // Non-method function.
+    QualType CalleeQualType = CalleeExpr->getType()->getPointeeType();
+    if (CalleeQualType.isNull()) {
+      // The function callee is invalid and already triggered an error.
+      // Avoid compounding errors.
+      assert(hasUncompilableErrorOccurred() && "expected previous error");
+      return false;
+    }
+    CalleeType.Func = CalleeQualType->getAs<FunctionProtoType>();
+  }
+
+  // Both caller and callee must have a prototype (no K&R declarations).
+  if (!CalleeType.Func || !CallerType.Func) {
+    Diag(St->getBeginLoc(), diag::err_musttail_needs_prototype) << &MTA;
+    if (!CalleeType.Func && CE->getDirectCallee()) {
+      Diag(CE->getDirectCallee()->getBeginLoc(),
+           diag::note_musttail_fix_non_prototype);
+    }
+    if (!CallerType.Func)
+      Diag(CallerDecl->getBeginLoc(), diag::note_musttail_fix_non_prototype);
+    return false;
+  }
+
+  // Caller and callee must have matching calling conventions.
+  //
+  // Some calling conventions are physically capable of supporting tail calls
+  // even if the function types don't perfectly match. LLVM is currently too
+  // strict to allow this, but if LLVM added support for this in the future, we
+  // could exit early here and skip the remaining checks if the functions are
+  // using such a calling convention.
+  if (CallerType.Func->getCallConv() != CalleeType.Func->getCallConv()) {
+    Diag(St->getBeginLoc(), diag::err_musttail_callconv_mismatch) << &MTA;
+    Diag(CalleeLoc, diag::note_musttail_callconv_mismatch)
+        << FunctionType::getNameForCallConv(CallerType.Func->getCallConv())
+        << FunctionType::getNameForCallConv(CalleeType.Func->getCallConv());
+    return false;
+  }
+
+  if (CalleeType.Func->isVariadic() || CallerType.Func->isVariadic()) {
+    Diag(St->getBeginLoc(), diag::err_musttail_no_variadic) << &MTA;
+    return false;
+  }
+
+  if (CallerType.This.isNull() != CalleeType.This.isNull()) {
+    Diag(St->getBeginLoc(), diag::err_musttail_member_mismatch)
+        << &MTA << CallerType.This.isNull();
+    Diag(CalleeLoc, diag::note_musttail_member_mismatch)
+        << CalleeType.This.isNull() << CalleeType.This;
+    return false;
+  }
+
+  auto CheckTypesMatch = [this](FuncType CallerType, FuncType CalleeType,
+                                PartialDiagnostic &PD) -> bool {
+    enum {
+      ft_different_class,
+      ft_parameter_arity,
+      ft_parameter_mismatch,
+      ft_return_type,
+    };
+
+    auto DoTypesMatch = [this, &PD](QualType A, QualType B,
+                                    unsigned Select) -> bool {
+      if (!Context.hasSimilarType(A, B)) {
+        PD << Select << A << B;
+        return false;
+      }
+      return true;
+    };
+
+    if (!CallerType.This.isNull() &&
+        !DoTypesMatch(CallerType.This, CalleeType.This, ft_different_class))
+      return false;
+
+    if (!DoTypesMatch(CallerType.Func->getReturnType(),
+                      CalleeType.Func->getReturnType(), ft_return_type))
+      return false;
+
+    if (CallerType.Func->getNumParams() != CalleeType.Func->getNumParams()) {
+      PD << ft_parameter_arity << CallerType.Func->getNumParams()
+         << CalleeType.Func->getNumParams();
+      return false;
+    }
+
+    ArrayRef<QualType> CalleeParams = CalleeType.Func->getParamTypes();
+    ArrayRef<QualType> CallerParams = CallerType.Func->getParamTypes();
+    size_t N = CallerType.Func->getNumParams();
+    for (size_t I = 0; I < N; I++) {
+      if (!DoTypesMatch(CalleeParams[I], CallerParams[I],
+                        ft_parameter_mismatch)) {
+        PD << static_cast<int>(I) + 1;
+        return false;
+      }
+    }
+
+    return true;
+  };
+
+  PartialDiagnostic PD = PDiag(diag::note_musttail_mismatch);
+  if (!CheckTypesMatch(CallerType, CalleeType, PD)) {
+    Diag(St->getBeginLoc(), diag::err_musttail_mismatch) << &MTA;
+    Diag(CalleeLoc, PD);
+    return false;
+  }
+
+  return true;
+}
+
 namespace {
 class CommaVisitor : public EvaluatedExprVisitor<CommaVisitor> {
   typedef EvaluatedExprVisitor<CommaVisitor> Inherited;
   Sema &SemaRef;
 public:
   CommaVisitor(Sema &SemaRef) : Inherited(SemaRef.Context), SemaRef(SemaRef) {}
   void VisitBinaryOperator(BinaryOperator *E) {
     if (E->getOpcode() == BO_Comma)

clang/lib/Sema/SemaStmtAttr.cpp

 //===--- SemaStmtAttr.cpp - Statement Attribute Handling ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  This file implements stmt-related attribute processing.
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/EvaluatedExprVisitor.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Sema/DelayedDiagnostic.h"
 #include "clang/Sema/Lookup.h"
 #include "clang/Sema/ScopeInfo.h"
 #include "clang/Sema/SemaInternal.h"
 #include "llvm/ADT/StringExtras.h"
 
 using namespace clang;
if (!CEF.foundCallExpr()) {
     S.Diag(St->getBeginLoc(), diag::warn_nomerge_attribute_ignored_in_stmt)
         << NMA.getSpelling();
     return nullptr;
   }
 
   return ::new (S.Context) NoMergeAttr(S.Context, A);
 }
 
+static Attr *handleMustTailAttr(Sema &S, Stmt *St, const ParsedAttr &A,
+                                SourceRange Range) {
+  // Validation is in Sema::ActOnAttributedStmt().
+  return ::new (S.Context) MustTailAttr(S.Context, A);
+}
+
 static Attr *handleLikely(Sema &S, Stmt *St, const ParsedAttr &A,
                           SourceRange Range) {
 
   if (!S.getLangOpts().CPlusPlus20 && A.isCXX11Attribute() && !A.getScopeName())
     S.Diag(A.getLoc(), diag::ext_cxx20_attr) << A << Range;
 
   return ::new (S.Context) LikelyAttr(S.Context, A);
 }
static Attr *ProcessStmtAttribute(Sema &S, Stmt *St, const ParsedAttr &A,
   case ParsedAttr::AT_LoopHint:
     return handleLoopHintAttr(S, St, A, Range);
   case ParsedAttr::AT_OpenCLUnrollHint:
     return handleOpenCLUnrollHint(S, St, A, Range);
   case ParsedAttr::AT_Suppress:
     return handleSuppressAttr(S, St, A, Range);
   case ParsedAttr::AT_NoMerge:
     return handleNoMergeAttr(S, St, A, Range);
+  case ParsedAttr::AT_MustTail:
+    return handleMustTailAttr(S, St, A, Range);
   case ParsedAttr::AT_Likely:
     return handleLikely(S, St, A, Range);
   case ParsedAttr::AT_Unlikely:
     return handleUnlikely(S, St, A, Range);
   default:
     // N.B., ClangAttrEmitter.cpp emits a diagnostic helper that ensures a
     // declaration attribute is not written on a statement, but this code is
     // needed for attributes in Attr.td that do not list any subjects.

clang/test/CodeGenCXX/attr-musttail.cpp

+// RUN: %clang_cc1 -fno-elide-constructors -S -emit-llvm %s -triple x86_64-unknown-linux-gnu -o - | FileCheck %s
+// RUN: %clang_cc1 -fno-elide-constructors -S -emit-llvm %s -triple x86_64-unknown-linux-gnu -o - | opt -verify
+// FIXME: remove the call to "opt" once the tests are running the Clang verifier automatically again.
+
+int Bar(int);
+int Baz(int);
+
+int Func1(int x) {
+  if (x) {
+    // CHECK: %call = musttail call i32 @_Z3Bari(i32 %1)
+    // CHECK-NEXT: ret i32 %call
+    [[clang::musttail]] return Bar(x);
+  } else {
+    [[clang::musttail]] return Baz(x); // CHECK: %call1 = musttail call i32 @_Z3Bazi(i32 %3)
+  }
+}
+
+int Func2(int x) {
+  {
+    [[clang::musttail]] return Bar(Bar(x));
+  }
+}
+
+// CHECK: %call1 = musttail call i32 @_Z3Bari(i32 %call)
+
+class Foo {
+public:
+  static int StaticMethod(int x);
+  int MemberFunction(int x);
+  int TailFrom(int x);
+  int TailFrom2(int x);
+  int TailFrom3(int x);
+};
+
+int Foo::TailFrom(int x) {
+  [[clang::musttail]] return MemberFunction(x);
+}
+
+// CHECK: %call = musttail call i32 @_ZN3Foo14MemberFunctionEi(%class.Foo* nonnull dereferenceable(1) %this1, i32 %0)
+
+int Func3(int x) {
+  [[clang::musttail]] return Foo::StaticMethod(x);
+}
+
+// CHECK: %call = musttail call i32 @_ZN3Foo12StaticMethodEi(i32 %0)
+
+int Func4(int x) {
+  Foo foo; // Object with trivial destructor.
+  [[clang::musttail]] return foo.StaticMethod(x);
+}
+
+// CHECK: %call = musttail call i32 @_ZN3Foo12StaticMethodEi(i32 %0)
+
+int (Foo::*pmf)(int);
+
+int Foo::TailFrom2(int x) {
+  [[clang::musttail]] return ((*this).*pmf)(x);
+}
+
+// CHECK: %call = musttail call i32 %8(%class.Foo* nonnull dereferenceable(1) %this.adjusted, i32 %9)
+
+int Foo::TailFrom3(int x) {
+  [[clang::musttail]] return (this->*pmf)(x);
+}
+
+// CHECK: %call = musttail call i32 %8(%class.Foo* nonnull dereferenceable(1) %this.adjusted, i32 %9)
+
+void ReturnsVoid();
+
+void Func5() {
+  [[clang::musttail]] return ReturnsVoid();
+}
+
+// CHECK: musttail call void @_Z11ReturnsVoidv()
+
+class HasTrivialDestructor {};
+
+int ReturnsInt(int x);
+
+int Func6(int x) {
+  HasTrivialDestructor foo;
+  [[clang::musttail]] return ReturnsInt(x);
+}
+
+// CHECK: %call = musttail call i32 @_Z10ReturnsInti(i32 %0)
+
+struct Data {
+  int (*fptr)(Data *);
+};
+
+int Func7(Data *data) {
+  [[clang::musttail]] return data->fptr(data);
+}
+
+// CHECK: %call = musttail call i32 %1(%struct.Data* %2)
+
+template <class T>
+T TemplateFunc(T) {
+  return 5;
+}
+
+int Func9(int x) {
+  [[clang::musttail]] return TemplateFunc<int>(x);
+}
+
+// CHECK: %call = musttail call i32 @_Z12TemplateFuncIiET_S0_(i32 %0)
+
+template <class T>
+int Func10(int x) {
+  T t;
+  [[clang::musttail]] return Bar(x);
+}
+
+int Func11(int x) {
+  return Func10<int>(x);
+}
+
+// CHECK: %call = musttail call i32 @_Z3Bari(i32 %0)
+
+template <class T>
+T Func12(T x) {
+  [[clang::musttail]] return ::Bar(x);
+}
+
+int Func13(int x) {
+  return Func12<int>(x);
+}
+
+// CHECK: %call = musttail call i32 @_Z3Bari(i32 %0)
+
+int Func14(int x) {
+  int vla[x];
+  [[clang::musttail]] return Bar(x);
+}
+
+// CHECK: %call = musttail call i32 @_Z3Bari(i32 %3)
+
+void TrivialDestructorParam(HasTrivialDestructor obj);
+
+void Func14(HasTrivialDestructor obj) {
+  [[clang::musttail]] return TrivialDestructorParam(obj);
+}
+
+// CHECK: musttail call void @_Z22TrivialDestructorParam20HasTrivialDestructor()
+
+struct Struct3 {
+  void ConstMemberFunction(const int *) const;
+  void NonConstMemberFunction(int *i);
+};
+void Struct3::NonConstMemberFunction(int *i) {
+  // The parameters are not identical, but they are compatible.
+  [[clang::musttail]] return ConstMemberFunction(i);
+}
+
+// CHECK: musttail call void @_ZNK7Struct319ConstMemberFunctionEPKi(%struct.Struct3* nonnull dereferenceable(1) %this1, i32* %0)
+
+struct HasNonTrivialCopyConstructor {
+  HasNonTrivialCopyConstructor(const HasNonTrivialCopyConstructor &);
+};
+HasNonTrivialCopyConstructor ReturnsClassByValue();
+HasNonTrivialCopyConstructor TestNonElidableCopyConstructor() {
+  [[clang::musttail]] return (((ReturnsClassByValue())));
+}
+
+// CHECK: musttail call void @_Z19ReturnsClassByValuev(%struct.HasNonTrivialCopyConstructor* sret(%struct.HasNonTrivialCopyConstructor) align 1 %agg.result)
+
+struct HasNonTrivialCopyConstructor2 {
+  // Copy constructor works even if it has extra default params.
+  HasNonTrivialCopyConstructor2(const HasNonTrivialCopyConstructor &, int DefaultParam = 5);
+};
+HasNonTrivialCopyConstructor2 ReturnsClassByValue2();
+HasNonTrivialCopyConstructor2 TestNonElidableCopyConstructor2() {
+  [[clang::musttail]] return (((ReturnsClassByValue2())));
+}
+
+// CHECK: musttail call void @_Z20ReturnsClassByValue2v()
+
+void TestFunctionPointer(int x) {
+  void (*p)(int) = nullptr;
+  [[clang::musttail]] return p(x);
+}
+
+// CHECK: musttail call void %0(i32 %1)
+
+struct LargeWithCopyConstructor {
+  LargeWithCopyConstructor(const LargeWithCopyConstructor &);
+  char data[32];
+};
+LargeWithCopyConstructor ReturnsLarge();
+LargeWithCopyConstructor TestLargeWithCopyConstructor() {
+  [[clang::musttail]] return ReturnsLarge();
+}
+
+// CHECK: define dso_local void @_Z28TestLargeWithCopyConstructorv(%struct.LargeWithCopyConstructor* noalias sret(%struct.LargeWithCopyConstructor) align 1 %agg.result)
+// CHECK: musttail call void @_Z12ReturnsLargev(%struct.LargeWithCopyConstructor* sret(%struct.LargeWithCopyConstructor) align 1 %agg.result)
+
+using IntFunctionType = int();
+IntFunctionType *ReturnsIntFunction();
+int TestRValueFunctionPointer() {
+  [[clang::musttail]] return ReturnsIntFunction()();  // expected-error{{'musttail' attribute requires that caller and callee have compatible function signatures}}  // expected-note{{target function has different return type ('long' expected but has 'int')}}
+}
+
+// CHECK: musttail call i32 %call()
+
+void (FuncWithParens)() {
+  [[clang::musttail]] return FuncWithParens();
+}
+
+// CHECK: musttail call i32 %call()
+
+int TestNonCapturingLambda() {
+  auto lambda = []() { return 12; };   // expected-note {{target function is a member function of class 'const (lambda at /usr/local/google/home/haberman/code/llvm-project/clang/test/SemaCXX/attr-musttail.cpp:142:17)'}}
+  [[clang::musttail]] return (+lambda)();
+}

clang/test/Sema/attr-musttail.c

+// RUN: %clang_cc1 -verify -fsyntax-only %s
+
+int NotAProtoType(); // expected-note{{add 'void' to the parameter list to turn a old-style K&R function declaration into a prototype}}
+int TestCalleeNotProtoType(void) {
+  __attribute__((musttail)) return NotAProtoType(); // expected-error{{'musttail' attribute requires that both caller and callee functions have a prototype}}
+}
+
+int ProtoType(void);
+int TestCallerNotProtoType() {                  // expected-note{{add 'void' to the parameter list to turn a old-style K&R function declaration into a prototype}}
+  __attribute__((musttail)) return ProtoType(); // expected-error{{'musttail' attribute requires that both caller and callee functions have a prototype}}
+}
+
+int TestProtoType(void) {
+  return ProtoType();
+}

clang/test/Sema/attr-musttail.m

+// RUN: %clang_cc1 -fsyntax-only -fblocks -Wno-objc-root-class -verify %s
+
+void TestObjcBlock(void) {
+  void (^x)(void) = ^(void) {
+    __attribute__((musttail)) return TestObjcBlock(); // expected-error{{'musttail' attribute cannot be used from a block}}
+  };
+  __attribute__((musttail)) return x();
+}
+
+void ReturnsVoid(void);
+void TestObjcBlockVar(void) {
+  __block int i = 0;                              // expected-note{{jump exits scope of __block variable}}
+  __attribute__((musttail)) return ReturnsVoid(); // expected-error{{cannot perform a tail call from this return statement}}
+}
+
+__attribute__((objc_root_class))
+@interface TestObjcClass
+@end
+
+@implementation TestObjcClass
+
+- (void)testObjCMethod {
+  __attribute__((musttail)) return ReturnsVoid(); // expected-error{{'musttail' attribute cannot be used from an Objective-C function}}
+}
+
+@end

clang/test/SemaCXX/attr-musttail.cpp

+// RUN: %clang_cc1 -verify -fsyntax-only -fms-extensions -fcxx-exceptions -fopenmp %s
+
+int ReturnsInt1();
+int Func1() {
+  [[clang::musttail]] ReturnsInt1();              // expected-error {{'musttail' attribute only applies to return statements}}
+  [[clang::musttail(1, 2)]] return ReturnsInt1(); // expected-error {{'musttail' attribute takes no arguments}}
+  [[clang::musttail]] return 5;                   // expected-error {{'musttail' attribute requires that the return value is the result of a function call}}
+  [[clang::musttail]] return ReturnsInt1();
+}
+
+void NoFunctionCall() {
+  [[clang::musttail]] return; // expected-error {{'musttail' attribute requires that the return value is the result of a function call}}
+}
+
+[[clang::musttail]] static int int_val = ReturnsInt1(); // expected-error {{'musttail' attribute cannot be applied to a declaration}}
+
+void NoParams(); // expected-note {{target function has different number of parameters (expected 1 but has 0)}}
+void TestParamArityMismatch(int x) {
+  [[clang::musttail]] return NoParams(); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+
+void LongParam(long x); // expected-note {{target function has type mismatch at 1st parameter (expected 'long' but has 'int')}}
+void TestParamTypeMismatch(int x) {
+  [[clang::musttail]] return LongParam(x); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+
+long ReturnsLong(); // expected-note {{target function has different return type ('int' expected but has 'long')}}
+int TestReturnTypeMismatch() {
+  [[clang::musttail]] return ReturnsLong(); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+
+struct Struct1 {
+  void MemberFunction(); // expected-note {{target function is a member function of class 'Struct1'}}
+};
+void TestNonMemberToMember() {
+  Struct1 st;
+  [[clang::musttail]] return st.MemberFunction(); // expected-error {{cannot 'musttail' from a non-member to a member function}}
+}
+
+void ReturnsVoid(); // expected-note {{target function is not a member function}}
+struct Struct2 {
+  void TestMemberToNonMember() {
+    [[clang::musttail]] return ReturnsVoid(); // expected-error{{cannot 'musttail' from a member to a non-member function}}
+  }
+};
+
+class HasNonTrivialDestructor {
+public:
+  ~HasNonTrivialDestructor() {}
+  int ReturnsInt();
+};
+
+void ReturnsVoid2();
+void TestNonTrivialDestructorInScope() {
+  HasNonTrivialDestructor foo;              // expected-note {{jump exits scope of variable with non-trivial destructor}}
+  [[clang::musttail]] return ReturnsVoid(); // expected-error {{cannot perform a tail call from this return statement}}
+}
+
+int NonTrivialParam(HasNonTrivialDestructor x);
+int TestNonTrivialParam(HasNonTrivialDestructor x) {
+  [[clang::musttail]] return NonTrivialParam(x); // expected-error {{'musttail' attribute requires that the return value, all parameters, and any temporaries created by the expression are trivially destructible}}
+}
+
+HasNonTrivialDestructor ReturnsNonTrivialValue();
+HasNonTrivialDestructor TestReturnsNonTrivialValue() {
+  // FIXME: the diagnostic cannot currently distinguish between needing to run a
+  // destructor for the return value and needing to run a destructor for some
+  // other temporary created in the return statement.
+  [[clang::musttail]] return (ReturnsNonTrivialValue()); // expected-error {{'musttail' attribute requires that the return value, all parameters, and any temporaries created by the expression are trivially destructible}}
+}
+
+HasNonTrivialDestructor TestReturnsNonTrivialNonFunctionCall() {
+  [[clang::musttail]] return HasNonTrivialDestructor(); // expected-error {{'musttail' attribute requires that the return value is the result of a function call}}
+}
+
+struct UsesPointerToMember {
+  void (UsesPointerToMember::*p_mem)();
+};
+void TestUsesPointerToMember(UsesPointerToMember *foo) {
+  // "this" pointer cannot double as first parameter.
+  [[clang::musttail]] return (foo->*(foo->p_mem))(); // expected-error {{cannot 'musttail' from a non-member to a member function}} expected-note{{target function is a member function}}
+}
+
+void ReturnsVoid2();
+void TestNestedClass() {
+  HasNonTrivialDestructor foo;
+  class Nested {
+    __attribute__((noinline)) static void NestedMethod() {
+      // Outer non-trivial destructor does not affect nested class.
+      [[clang::musttail]] return ReturnsVoid2();
+    }
+  };
+}
+
+template <class T>
+T TemplateFunc(T x) { // expected-note{{target function has different return type ('long' expected but has 'int')}}
+  return x ? 5 : 10;
+}
+int OkTemplateFunc(int x) {
+  [[clang::musttail]] return TemplateFunc<int>(x);
+}
+template <class T>
+T BadTemplateFunc(T x) {
+  [[clang::musttail]] return TemplateFunc<int>(x); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+long TestBadTemplateFunc(long x) {
+  return BadTemplateFunc<long>(x); // expected-note {{in instantiation of}}
+}
+
+void IntParam(int x);
+void TestVLA(int x) {
+  HasNonTrivialDestructor vla[x];         // expected-note {{jump exits scope of variable with non-trivial destructor}}
+  [[clang::musttail]] return IntParam(x); // expected-error {{cannot perform a tail call from this return statement}}
+}
+
+void TestNonTrivialDestructorSubArg(int x) {
+  [[clang::musttail]] return IntParam(NonTrivialParam(HasNonTrivialDestructor())); // expected-error {{'musttail' attribute requires that the return value, all parameters, and any temporaries created by the expression are trivially destructible}}
+}
+
+void VariadicFunction(int x, ...);
+void TestVariadicFunction(int x, ...) {
+  [[clang::musttail]] return VariadicFunction(x); // expected-error {{'musttail' attribute may not be used with variadic functions}}
+}
+
+int TakesIntParam(int x);     // expected-note {{target function has type mismatch at 1st parameter (expected 'int' but has 'short')}}
+int TakesShortParam(short x); // expected-note {{target function has type mismatch at 1st parameter (expected 'short' but has 'int')}}
+int TestIntParamMismatch(int x) {
+  [[clang::musttail]] return TakesShortParam(x); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+int TestIntParamMismatch2(short x) {
+  [[clang::musttail]] return TakesIntParam(x); // expected-error {{'musttail' attribute requires that caller and callee have compatible function signatures}}
+}
+
+__regcall int RegCallReturnsInt(); // expected-note {{target function has calling convention regcall (expected cdecl)}}
+int TestMismatchCallingConvention() {
+  [[clang::musttail]] return RegCallReturnsInt(); // expected-error {{'musttail' attribute requires that caller and callee use the same calling convention}}
+}
+
+int TestNonCapturingLambda() {
+  auto lambda = []() { return 12; };   // expected-note {{target function is a member function of class 'const (lambda at}}
+  [[clang::musttail]] return lambda(); // expected-error {{cannot 'musttail' from a non-member to a member function}}
+
+  // This works.
+  auto lambda_fptr = static_cast<int (*)()>(lambda);
+  [[clang::musttail]] return lambda_fptr();
+  [[clang::musttail]] return (+lambda)();
+}
+
+int TestCapturingLambda() {
+  int x;
+  auto lambda = [x]() { return 12; };  // expected-note {{target function is a member function}}
+  [[clang::musttail]] return lambda(); // expected-error {{cannot 'musttail' from a non-member to a member function}}
+}
+
+// These tests are merely verifying that we don't crash with incomplete or
+// erroneous ASTs. These cases crashed the compiler in early iterations.
+
+struct TestBadPMF {
+  int (TestBadPMF::*pmf)();
+  void BadPMF() {
+    [[clang::musttail]] return ((*this)->*pmf)(); // expected-error {{left hand operand to ->* must be a pointer to class compatible with the right hand operand, but is 'TestBadPMF'}}
+  }
+};
+
+namespace ns {}
+void TestCallNonValue() {
+  [[clang::musttail]] return ns; // expected-error {{unexpected namespace name 'ns': expected expression}}
+}
+
+int TestNonTrivialTemporary(int) {
+  [[clang::musttail]] return TakesIntParam(HasNonTrivialDestructor().ReturnsInt()); // expected-error {{'musttail' attribute requires that the return value, all parameters, and any temporaries created by the expression are trivially destructible}}
+}
+
+void ReturnsVoid();
+struct TestDestructor {
+  ~TestDestructor() {                         // expected-note {{caller is a destructor}}
+    [[clang::musttail]] return ReturnsVoid(); // expected-error {{destructor '~TestDestructor' must not return void expression}}  // expected-error {{'musttail' attribute cannot be used when caller or callee is a constructor or destructor, as these can have unusual calling conventions}}
+  }
+};
+
+struct ClassWithDestructor { // expected-note {{callee is a destructor}}
+  void TestExplicitDestructorCall() {
+    [[clang::musttail]] return this->~ClassWithDestructor(); // expected-error {{'musttail' attribute cannot be used when caller or callee is a constructor or destructor, as these can have unusual calling conventions}}
+  }
+};
+
+struct HasNonTrivialCopyConstructor {
+  HasNonTrivialCopyConstructor(const HasNonTrivialCopyConstructor &);
+};
+HasNonTrivialCopyConstructor ReturnsClassByValue();
+HasNonTrivialCopyConstructor TestNonElidableCopyConstructor() {
+  // This is an elidable constructor, but when it is written explicitly
+  // we decline to elide it.
+  [[clang::musttail]] return HasNonTrivialCopyConstructor(ReturnsClassByValue()); // expected-error{{'musttail' attribute requires that the return value is the result of a function call}}
+}
+
+struct ClassWithConstructor {}; // expected-note {{callee is a constructor}}
+void TestExplicitConstructorCall(ClassWithConstructor a) {
+  [[clang::musttail]] return a.ClassWithConstructor::ClassWithConstructor(); // expected-error{{'musttail' attribute cannot be used when caller or callee is a constructor or destructor, as these can have unusual calling conventions}}  expected-warning{{explicit constructor calls are a Microsoft extension}}
+}
+
+void TestStatementExpression() {
+  ({
+    HasNonTrivialDestructor foo;               // expected-note {{jump exits scope of variable with non-trivial destructor}}
+    [[clang::musttail]] return ReturnsVoid2(); // expected-error {{cannot perform a tail call from this return statement}}
+  });
+}
+
+struct MyException {};
+void TestTryBlock() {
+  try {                                        // expected-note {{jump exits try block}}
+    [[clang::musttail]] return ReturnsVoid2(); // expected-error {{cannot perform a tail call from this return statement}}
+  } catch (MyException &e) {
+  }
+}
+
+using IntFunctionType = int();
+IntFunctionType *ReturnsIntFunction();
+long TestRValueFunctionPointer() {
+  [[clang::musttail]] return ReturnsIntFunction()();  // expected-error{{'musttail' attribute requires that caller and callee have compatible function signatures}}  // expected-note{{target function has different return type ('long' expected but has 'int')}}
+}
+
+void TestPseudoDestructor() {
+  int n;
+  using T = int;
+  [[clang::musttail]] return n.~T();  // expected-error{{'musttail' attribute cannot be used when caller or callee is a constructor or destructor, as these can have unusual calling conventions}}  // expected-note{{callee is a destructor}}
+}