Treat `std::move`, `forward`, and `move_if_noexcept` as builtins.

[rsmith]

Summary

We still require these functions to be declared before they can be used,
but don't instantiate their definitions unless their addresses are
taken. Instead, code generation, constant evaluation, and static
analysis are given direct knowledge of their effect.

This change aims to reduce various costs associated with these functions

• per-instantiation memory costs, compile time and memory costs due to

creating out-of-line copies and inlining them, code size at -O0, and so
on -- so that they are not substantially more expensive than a cast.
Most of these improvements are very small, but I measured a 3% decrease
in -O0 object file size for a simple C++ source file using the standard
library after this change.

We now automatically infer the const and nothrow attributes on these
now-builtin functions, in particular meaning that we get a warning for
an unused call to one of these functions.

In C++20 onwards, we disallow taking the addresses of these functions,
per the C++20 "addressable function" rule. In earlier language modes, a
compatibility warning is produced but the address can still be taken.

The same infrastructure is extended to the existing MSVC builtin
__GetExceptionInfo, which is now only recognized in namespace std
like it always should have been.

[aaron.ballman]

Thank you for this, it's a great improvement! I think it's generally correct, but I did have some minor questions.

Most of these improvements are very small, but I measured a 3% decrease in -O0 object file size for a simple C++ source file using the standard library after this change.

Any chance you'd be interested in submitting this patch to http://llvm-compile-time-tracker.com/ (instructions at http://llvm-compile-time-tracker.com/about.php) so we can have an idea of how compile times are impacted? (It's not critical, but having some more compile time performance measurements would be helpful to validate that this actually saves compile time as expected.

clang/include/clang/Basic/DiagnosticSemaKinds.td

6577–6578

Thank you for making this one on by default :-)

clang/lib/Sema/SemaExpr.cpp

20282

I wonder how often these get passed around as function objects... e.g.,

template <typename Fn>
void func(Fn &&Call);

int main() {
  func(std::move<int>);
}

(I don't see evidence that this is a common code pattern, but C++ surprises me often enough that I wonder if this will cause issues.)

clang/lib/Sema/SemaInit.cpp

8192–8195

Do we need to make this call every time we've called FixOverloadedFunctionReference()? I guess I'm not seeing the changes that necessitated this call (or the one below).

clang/test/SemaCXX/builtin-std-move.cpp

12

Formatting looks like it's gone wonky in this file.

[joerg]

As is, I think this conflicts with -ffreestanding assumptions or at the very least the spirit.

[aaron.ballman]

In D123345#3441262, @joerg wrote:

As is, I think this conflicts with -ffreestanding assumptions or at the very least the spirit.

Why? These functions are in <utility> which is not required in freestanding, but implementations are allowed to support more anyway (http://eel.is/c++draft/compliance#2). As the codegen doesn't emit a call to a library function but is purely using language facilities to reimplement the functionality, these don't seem to be in conflict with freestanding to me. If you could expound on what problems you see, that'd be helpful.

[joerg]

The patch contains at least one user visible change that would be quite surprising: it is no longer possible to intentionally set a break point on std::move.

Thinking more about it, what about a slightly different implementation strategy? Provide a compiler built-in __builtin_std_move. If it is present, libc++ can alias it into namespace std using regular C++. Much of the name-matching and argument checking in various places disappears. The check to skip template instantiation can be done the same way. Over-all, it should be much less intrusive with the same performance benefits for an built-in-aware STL. It completely side-steps the question of ignoring the actual implementation of std::move in the source, because there is none.

[rsmith]

• Add support for -fno-builtin, -ffreestanding, -fno-builtin-std-move.

[rsmith]

• Document -fno-builtin-std-foo.

[rsmith]

In D123345#3440935, @aaron.ballman wrote:

Any chance you'd be interested in submitting this patch to http://llvm-compile-time-tracker.com/ (instructions at http://llvm-compile-time-tracker.com/about.php) so we can have an idea of how compile times are impacted? (It's not critical, but having some more compile time performance measurements would be helpful to validate that this actually saves compile time as expected.

Nice idea, done:

• Instructions executed while compiling reduced by 0.3-0.6% at -O0 on C++ benchmarks kimwitu++ and tramp3d-v4. Changes in the noise on other benchmarks and with optimizations enabled.
• Code size reduced by 0.51% at -O0 on those same benchmarks. No change on any others.
• Effect on compiler memory usage appears to be below the noise floor.

In D123345#3442809, @joerg wrote:

The patch contains at least one user visible change that would be quite surprising: it is no longer possible to intentionally set a break point on std::move.

Yeah, I think it's reasonable to support -fno-builtin-std-move and the like -- done. The intended model here is to treat these std:: functions just like how we treat the functions in the global namespace for which we provide built-in semantics, so I think -fno-builtin and -ffreestanding and the like should all treat these functions the same way they treat malloc and strlen and such.

Thinking more about it, what about a slightly different implementation strategy? Provide a compiler built-in __builtin_std_move. If it is present, libc++ can alias it into namespace std using regular C++. Much of the name-matching and argument checking in various places disappears. The check to skip template instantiation can be done the same way. Over-all, it should be much less intrusive with the same performance benefits for an built-in-aware STL. It completely side-steps the question of ignoring the actual implementation of std::move in the source, because there is none.

This is an interesting idea, but I don't think it would work out well. As a superficial problem, C++ aliases (eg, using declarations) don't let you change the name of a function. More fundamentally, I think this would end up being a lot more complicated: we'd need to implement the rules for how to deduce parameter types and transform them into return types ourselves, we'd need to somehow support std::forward's explicit template argument, and in C++17 and before, it's valid to take the address of std::move and co, which would require us to invent a function definition for them in IR generation. This would also be a major deviation from how we treat all other standard library functions for which we have built-in knowledge.

clang/lib/Sema/SemaExpr.cpp

20282

I expect people will tell us if it this fires a lot... if it does, we can consider making the C++20 error a DefaultError ExtWarn.

clang/lib/Sema/SemaInit.cpp

8192–8195

We only need to do this in places that assume that the result doesn't have a placeholder type. Specifically, FixOverloadedFunctionReference can now take us from an expression whose type is the "overloaded function" builtin type to an expression whose type is the "builtin function" builtin type, after overload resolution picks a specific template specialization, which these places were not prepared for (they thought they'd always get a real function type).

In this case, we first build an initialization sequence (for example, for int &&(*p)(int&) = std::move;, we'd build this sequence: (1) resolve std::move to std::move<int>, then (2) decay std::move<int> to a function pointer), and then we apply it. If step (1) leaves us with a builtin function, we end up decaying *that* to a pointer, producing <builtin-function-type>*, which is an invalid type. The other case is similar: we first build an implicit conversion sequence then apply it and assume it will make sense. Other places that call FixOverloadedFunctionReference don't assume that they can blindly perform operations on its result, so they don't need this handling.

This never happened before because we never before had a builtin function that both requires overload resolution and uses a placeholder type as the type of a reference to the function. We do the latter here so that we can detect uses of the function that aren't calls, in order to trigger a real instantiation of the body in the rare cases where they're needed.

clang/test/SemaCXX/builtin-std-move.cpp

12

What are you referring to? The lines longer than 80 columns, or something else?

FWIW, long lines are common in our test/ files. Substantially more than half of them go over 80 columns:

$ rgrep '.\{81\}' clang/test  --files-with-match | wc -l
12481
$ rgrep '.\{81\}' clang/test  --files-without-match | wc -l
7194

[rsmith]

• Switch to a different example with underscores.

[xbolva00]

clang/docs/CommandGuide/clang.rst

264 ↗

(On Diff #422366)

-fno-builtin=xxx,yyy,zzz

wdyt?

[rsmith]

clang/docs/CommandGuide/clang.rst

264 ↗

(On Diff #422366)

Seems reasonable to me to treat -fno-builtin-<blah> and -fno-builtin=<blah> as equivalent, and I certainly prefer the latter form. I think that's somewhat orthogonal to this patch, though: I'm not introducing -fno-builtin-blah here, only documenting it. (It's a GCC flag originally.)

[aaron.ballman]

In D123345#3446495, @rsmith wrote:

In D123345#3440935, @aaron.ballman wrote:

Any chance you'd be interested in submitting this patch to http://llvm-compile-time-tracker.com/ (instructions at http://llvm-compile-time-tracker.com/about.php) so we can have an idea of how compile times are impacted? (It's not critical, but having some more compile time performance measurements would be helpful to validate that this actually saves compile time as expected.

Nice idea, done:

• Instructions executed while compiling reduced by 0.3-0.6% at -O0 on C++ benchmarks kimwitu++ and tramp3d-v4. Changes in the noise on other benchmarks and with optimizations enabled.
• Code size reduced by 0.51% at -O0 on those same benchmarks. No change on any others.
• Effect on compiler memory usage appears to be below the noise floor.

Awesome, thank you for that information! Not a massive win, but definitely seems like an improvement worth having.

In D123345#3442809, @joerg wrote:

The patch contains at least one user visible change that would be quite surprising: it is no longer possible to intentionally set a break point on std::move.

Yeah, I think it's reasonable to support -fno-builtin-std-move and the like -- done. The intended model here is to treat these std:: functions just like how we treat the functions in the global namespace for which we provide built-in semantics, so I think -fno-builtin and -ffreestanding and the like should all treat these functions the same way they treat malloc and strlen and such.

I can live with it. :-)

Thinking more about it, what about a slightly different implementation strategy? Provide a compiler built-in __builtin_std_move. If it is present, libc++ can alias it into namespace std using regular C++. Much of the name-matching and argument checking in various places disappears. The check to skip template instantiation can be done the same way. Over-all, it should be much less intrusive with the same performance benefits for an built-in-aware STL. It completely side-steps the question of ignoring the actual implementation of std::move in the source, because there is none.

This is an interesting idea, but I don't think it would work out well. As a superficial problem, C++ aliases (eg, using declarations) don't let you change the name of a function. More fundamentally, I think this would end up being a lot more complicated: we'd need to implement the rules for how to deduce parameter types and transform them into return types ourselves, we'd need to somehow support std::forward's explicit template argument, and in C++17 and before, it's valid to take the address of std::move and co, which would require us to invent a function definition for them in IR generation. This would also be a major deviation from how we treat all other standard library functions for which we have built-in knowledge.

Ah drat, great point about not being able to change the name of the function and the extra complexity.

Do you have ideas on how we can improve the debugging checkpoint behavior (if at all)?

clang/lib/Sema/SemaInit.cpp

8192–8195

Ah, thank you for that explanation, that helps a bunch!

clang/test/SemaCXX/builtin-std-move.cpp

12

Oh, the line length doesn't bother me (we smash all over that in tests, as you've seen). It was more the extra indentation within a namespace -- we don't usually indent like that and it caught me off guard. It's not a major deal though (the same indentation happens in clang/test/SemaCXX/builtin-std-move-nobuiltin.cpp).

[rsmith]

In D123345#3452496, @aaron.ballman wrote:

Do you have ideas on how we can improve the debugging checkpoint behavior (if at all)?

I think we just live with it, like we do for other builtin functions. (There might be things we can do by emitting inlining info into the debug info. If we do that, we should presumably do it for all builtin lib functions.)

clang/test/SemaCXX/builtin-std-move.cpp

12

Gotcha.

This is just what my editor / fingers happened to do, but now I'm consciously thinking about it, I see a bit of nuance here: I think it makes sense to not indent a namespace that spans a whole file, because the indent doesn't add anything for the reader and loses two columns of useful screen real estate. But when the namespace is covering a relatively small scope it seems useful to me to indent it to contrast the contents of the namespace against the surrounding code that's outside the namespace, in the same way it makes sense to indent a class definition to contrast the class scope against surrounding code.

It looks like indenting namespace bodies is pretty common in the clang test suite: of the 1917 files that contain a line matching /^namespace.*$/, 1171 of those files have a next line that is indented, which I think makes sense because namespaces in tests will tend to be short, and intended to semantically separate a portion of the test from the rest of the file. (By contrast, only ~10% of the clang sources and headers have indented namespaces, and from a quick sampling it looks like they basically follow the same pattern: indented namespaces are mostly used for short scopes and unindented ones are mostly used for longer / whole-file scopes.)

[rnk]

Generally speaking, this sounds like a good idea to me. One time in 2019 I used -ftime-trace+ClangBuildAnalyzer and it told me that std::unique_ptr was the most expensive template because it is instantiated so much. Those results don't even capture the -O0 object file size impact.

In D123345#3452933, @rsmith wrote:

In D123345#3452496, @aaron.ballman wrote:

Do you have ideas on how we can improve the debugging checkpoint behavior (if at all)?

I think we just live with it, like we do for other builtin functions. (There might be things we can do by emitting inlining info into the debug info. If we do that, we should presumably do it for all builtin lib functions.)

Honestly, I don't think it's worth the debug info bytes to describe these inlined call sites. Debug info isn't free.

[aaron.ballman]

In D123345#3452933, @rsmith wrote:

In D123345#3452496, @aaron.ballman wrote:

Do you have ideas on how we can improve the debugging checkpoint behavior (if at all)?

I think we just live with it, like we do for other builtin functions. (There might be things we can do by emitting inlining info into the debug info. If we do that, we should presumably do it for all builtin lib functions.)

Okie dokie, so be it.

The changes LGTM aside from a previous question about diagnostic wording (feel free to accept the suggestion or not as you see fit). Thanks for working on this!

clang/docs/ReleaseNotes.rst

121–124

Probably worth moving down to the C++ language changes section because it's C++ specific.

clang/include/clang/Basic/DiagnosticSemaKinds.td

6577–6578

Thoughts on this one?

[dblaikie]

In D123345#3453037, @rnk wrote:

Generally speaking, this sounds like a good idea to me. One time in 2019 I used -ftime-trace+ClangBuildAnalyzer and it told me that std::unique_ptr was the most expensive template because it is instantiated so much. Those results don't even capture the -O0 object file size impact.

In D123345#3452933, @rsmith wrote:

In D123345#3452496, @aaron.ballman wrote:

Do you have ideas on how we can improve the debugging checkpoint behavior (if at all)?

I think we just live with it, like we do for other builtin functions. (There might be things we can do by emitting inlining info into the debug info. If we do that, we should presumably do it for all builtin lib functions.)

Honestly, I don't think it's worth the debug info bytes to describe these inlined call sites. Debug info isn't free.

+1 there - and also these operations/intrinsics produce no instructions, so far as I understand/know - so for now, LLVM's got to way to represent that anyway (there's some talk in DWARF about how to have multiple "states" for a single instruction location (so, eg, you could step in/out of an inlined function even though you stay at exactly the same instruction))

[rsmith]

• Address review comments.

clang/include/clang/Basic/DiagnosticSemaKinds.td

6577–6578

Sorry, I was looking at your comments over email and didn't see the suggestion. Done.

[vitalybuka]

Reverted with e75d8b70370435b0ad10388afba0df45fcf9bfcc to recover bots

[vitalybuka]

It's broken again here https://lab.llvm.org/buildbot/#/builders/74/builds/10427

[paulkirth]

Hi, this is also breaking Fuchsia's clang CI builders (https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8816531831869640417/overview). If this will be hard to address, would you mind reverting until a patch is ready?

[rsmith]

In D123345#3456964, @paulkirth wrote:

Hi, this is also breaking Fuchsia's clang CI builders (https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8816531831869640417/overview). If this will be hard to address, would you mind reverting until a patch is ready?

I'm working on a fix. I'll revert if it takes me more than a few minutes. If you'd prefer to not wait, please feel free to revert it yourself.

[rsmith]

In D123345#3457109, @rsmith wrote:

I'm working on a fix. I'll revert if it takes me more than a few minutes. If you'd prefer to not wait, please feel free to revert it yourself.

Thanks for your patience, should be fixed in rGe43c93dd63cca295ef26ab69cd305816a71d45fd.

[Jake-Egan]

Hi, unfortunately there's a build failure on AIX: https://lab.llvm.org/buildbot/#/builders/214/builds/779/steps/9/logs/stdio. Could you take a look?

[daltenty]

In D123345#3460639, @Jake-Egan wrote:

Hi, unfortunately there's a build failure on AIX: https://lab.llvm.org/buildbot/#/builders/214/builds/779/steps/9/logs/stdio. Could you take a look?

Hi, any feedback on the AIX break? Seems like this might be non-trivial to address (were on an older libc++ level), so perhaps this needs a revert till can be worked out.

[rsmith]

In D123345#3463224, @daltenty wrote:

In D123345#3460639, @Jake-Egan wrote:

Hi, unfortunately there's a build failure on AIX: https://lab.llvm.org/buildbot/#/builders/214/builds/779/steps/9/logs/stdio. Could you take a look?

Hi, any feedback on the AIX break? Seems like this might be non-trivial to address (were on an older libc++ level), so perhaps this needs a revert till can be worked out.

Sorry, I somehow missed this. What you're seeing is a symptom of a libc++ issue (PR12704) that was fixed >10 years ago (rGbff1bfc6be0615ba3036a861fd27b75c96e3297c). I don't think we have explicit rules on how old a version of libc++ we aim to support, but trying to use version 3.1 or earlier with trunk Clang seems at least a little unreasonable. That said, it's probably not too hard to work around this, so I will :) but I would encourage whoever is responsible for libc++ in AIX to look into an upgrade!

[thakis]

I think your revert to fix the aix CI broke CI everywhere: http://45.33.8.238/linux/74239/step_7.txt

Please take a look and reland for now if it takes a while to fix.

[rsmith]

In D123345#3463540, @thakis wrote:

I think your revert to fix the aix CI broke CI everywhere: http://45.33.8.238/linux/74239/step_7.txt

Please take a look and reland for now if it takes a while to fix.

Relanded with a workaround for super-old libc++ in rG72315d02c432a0fe0acae9c96c69eac8d8e1a9f6.

[thakis]

Thanks!

[daltenty]

Thanks for getting this fixed this so quickly and sorry for the messy revert.

What you're seeing is a symptom of a libc++ issue (PR12704) that was fixed >10 years ago (rGbff1bfc6be0615ba3036a861fd27b75c96e3297c). I don't think we have explicit rules on how old a version of libc++ we aim to support, but trying to use version 3.1 or earlier with trunk Clang seems at least a little unreasonable. That said, it's probably not too hard to work around this, so I will :) but I would encourage whoever is responsible for libc++ in AIX to look into an upgrade!

Yeah, I'd tend to agree with you :) Thanks for accommodating us and identifying the libc++ issue as well, we'll definitely be looking into this.

clang/docs/ReleaseNotes.rst

 
 Non-comprehensive list of changes in this release
 -------------------------------------------------
 - Improve __builtin_dump_struct:
   - Support bitfields in struct and union.
   - Improve the dump format, dump both bitwidth(if its a bitfield) and field value.
   - Remove anonymous tag locations.
   - Beautify dump format, add indent for nested struct and struct members.
+- Improved ``-O0`` code generation for calls to ``std::move``, ``std::forward``,
+  and ``std::move_if_noexcept``. These are now treated as compiler builtins and
+  implemented directly, rather than instantiating the definition from the
+  standard library.
 
 New Compiler Flags
 ------------------
 
 Deprecated Compiler Flags
 -------------------------
 
 Modified Compiler Flags

clang/include/clang/Basic/Builtins.h

public:
 
   /// Determines whether this builtin is a predefined compiler-rt/libgcc
   /// function, such as "__clear_cache", where we know the signature a
   /// priori.
   bool isPredefinedRuntimeFunction(unsigned ID) const {
     return strchr(getRecord(ID).Attributes, 'i') != nullptr;
   }
 
+  /// Determines whether this builtin is a C++ standard library function
+  /// that lives in (possibly-versioned) namespace std, possibly a template
+  /// specialization, where the signature is determined by the standard library
+  /// declaration.
+  bool isInStdNamespace(unsigned ID) const {
+    return strchr(getRecord(ID).Attributes, 'z') != nullptr;
+  }
+
   /// Determines whether this builtin has custom typechecking.
   bool hasCustomTypechecking(unsigned ID) const {
     return strchr(getRecord(ID).Attributes, 't') != nullptr;
   }
 
   /// Determines whether a declaration of this builtin should be recognized
   /// even if the type doesn't match the specified signature.
   bool allowTypeMismatch(unsigned ID) const {

clang/include/clang/Basic/Builtins.def

 //          in that it accepts its arguments as a va_list rather than
 //          through an ellipsis
 //  e -> const, but only when -fno-math-errno
 //  j -> returns_twice (like setjmp)
 //  u -> arguments are not evaluated for their side-effects
 //  V:N: -> requires vectors of at least N bits to be legal
 //  C<N,M_0,...,M_k> -> callback behavior: argument N is called with argument
 //                      M_0, ..., M_k as payload
+//  z -> this is a C++ standard library function in (possibly-versioned)
+//       namespace std; implied by STDBUILTIN
 //  FIXME: gcc has nonnull
 
 #if defined(BUILTIN) && !defined(LIBBUILTIN)
 #  define LIBBUILTIN(ID, TYPE, ATTRS, HEADER, BUILTIN_LANG) BUILTIN(ID, TYPE, ATTRS)
 #endif
 
 #if defined(BUILTIN) && !defined(LANGBUILTIN)
 #  define LANGBUILTIN(ID, TYPE, ATTRS, BUILTIN_LANG) BUILTIN(ID, TYPE, ATTRS)
 #endif
 
+#if defined(BUILTIN) && !defined(STDBUILTIN)
+#  define STDBUILTIN(ID, TYPE, ATTRS, HEADER) LIBBUILTIN(ID, TYPE, "z" ATTRS, HEADER, CXX_LANG)
+#endif
+
 // Standard libc/libm functions:
 BUILTIN(__builtin_atan2 , "ddd"  , "Fne")
 BUILTIN(__builtin_atan2f, "fff"  , "Fne")
 BUILTIN(__builtin_atan2l, "LdLdLd", "Fne")
 BUILTIN(__builtin_atan2f128, "LLdLLdLLd", "Fne")
 BUILTIN(__builtin_abs  , "ii"  , "ncF")
 BUILTIN(__builtin_copysign, "ddd", "ncF")
 BUILTIN(__builtin_copysignf, "fff", "ncF")

 LIBBUILTIN(_byteswap_uint64, "ULLiULLi", "fnc", "stdlib.h", ALL_MS_LANGUAGES)
 LANGBUILTIN(__debugbreak,     "v",   "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(__exception_code, "UNi", "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_exception_code,  "UNi", "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(__exception_info, "v*",  "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_exception_info,  "v*",  "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(__abnormal_termination, "i", "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_abnormal_termination,  "i", "n", ALL_MS_LANGUAGES)
-LANGBUILTIN(__GetExceptionInfo, "v*.", "ntu", ALL_MS_LANGUAGES)
+LANGBUILTIN(__GetExceptionInfo, "v*.", "zntu", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedAnd8,   "ccD*c",        "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedAnd16,  "ssD*s",        "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedAnd,    "NiNiD*Ni",     "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedCompareExchange8,   "ccD*cc",         "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedCompareExchange16,  "ssD*ss",         "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedCompareExchange,    "NiNiD*NiNi",     "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedCompareExchange64,  "LLiLLiD*LLiLLi", "n", ALL_MS_LANGUAGES)
 LANGBUILTIN(_InterlockedCompareExchangePointer, "v*v*D*v*v*", "n", ALL_MS_LANGUAGES)

 LIBBUILTIN(__exp10, "dd", "fne", "math.h", ALL_LANGUAGES)
 LIBBUILTIN(__exp10f, "ff", "fne", "math.h", ALL_LANGUAGES)
 
 // Blocks runtime Builtin math library functions
 LIBBUILTIN(_Block_object_assign, "vv*vC*iC", "f", "Blocks.h", ALL_LANGUAGES)
 LIBBUILTIN(_Block_object_dispose, "vvC*iC", "f", "Blocks.h", ALL_LANGUAGES)
 // FIXME: Also declare NSConcreteGlobalBlock and NSConcreteStackBlock.
 
+// C++11
+STDBUILTIN(move, "v&v&", "ncTh", "utility")
+STDBUILTIN(move_if_noexcept, "v&v&", "ncTh", "utility")
+STDBUILTIN(forward, "v&v&", "ncTh", "utility")
+
 // Annotation function
 BUILTIN(__builtin_annotation, "v.", "tn")
 
 // Invariants
 BUILTIN(__builtin_assume, "vb", "n")
 
 // Multiprecision Arithmetic Builtins.
 BUILTIN(__builtin_addcb, "UcUcCUcCUcCUc*", "n")

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_self_assignment_builtin : Warning<
   InGroup<SelfAssignment>, DefaultIgnore;
 def warn_self_assignment_overloaded : Warning<
   "explicitly assigning value of variable of type %0 to itself">,
   InGroup<SelfAssignmentOverloaded>, DefaultIgnore;
 def warn_self_move : Warning<
   "explicitly moving variable of type %0 to itself">,
   InGroup<SelfMove>, DefaultIgnore;
 
+def err_builtin_move_forward_unsupported : Error<
+  "unsupported signature for '%select{std::move|std::forward}0'">;
+def err_use_of_unaddressable_function : Error<
+  "taking address of non-addressable standard library function">;
+// FIXME: This should also be in -Wc++23-compat once we have it.
+def warn_use_of_unaddressable_function : Warning<
+  "taking address of non-addressable standard library function">,
+  InGroup<CXX20Compat>;
+
 def warn_redundant_move_on_return : Warning<
   "redundant move in return statement">,
   InGroup<RedundantMove>, DefaultIgnore;
 def warn_pessimizing_move_on_return : Warning<
   "moving a local object in a return statement prevents copy elision">,
   InGroup<PessimizingMove>, DefaultIgnore;
 def warn_pessimizing_move_on_initialization : Warning<
   "moving a temporary object prevents copy elision">,

clang/lib/AST/ExprConstant.cpp

class LValueExprEvaluator
   : public LValueExprEvaluatorBase<LValueExprEvaluator> {
 public:
   LValueExprEvaluator(EvalInfo &Info, LValue &Result, bool InvalidBaseOK) :
     LValueExprEvaluatorBaseTy(Info, Result, InvalidBaseOK) {}
 
   bool VisitVarDecl(const Expr *E, const VarDecl *VD);
   bool VisitUnaryPreIncDec(const UnaryOperator *UO);
 
+  bool VisitCallExpr(const CallExpr *E);
   bool VisitDeclRefExpr(const DeclRefExpr *E);
   bool VisitPredefinedExpr(const PredefinedExpr *E) { return Success(E); }
   bool VisitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *E);
   bool VisitCompoundLiteralExpr(const CompoundLiteralExpr *E);
   bool VisitMemberExpr(const MemberExpr *E);
   bool VisitStringLiteral(const StringLiteral *E) { return Success(E); }
   bool VisitObjCEncodeExpr(const ObjCEncodeExpr *E) { return Success(E); }
   bool VisitCXXTypeidExpr(const CXXTypeidExpr *E);
if (!V->hasValue()) {
     // adjust the diagnostic to say that.
     if (!Info.checkingPotentialConstantExpression())
       Info.FFDiag(E, diag::note_constexpr_use_uninit_reference);
     return false;
   }
   return Success(*V, E);
 }
 
+bool LValueExprEvaluator::VisitCallExpr(const CallExpr *E) {
+  switch (unsigned BuiltinOp = E->getBuiltinCallee()) {
+  case Builtin::BImove:
+  case Builtin::BImove_if_noexcept:
+  case Builtin::BIforward:
+    if (cast<FunctionDecl>(E->getCalleeDecl())->isConstexpr())
+      return Visit(E->getArg(0));
+    break;
+  }
+
+  return ExprEvaluatorBaseTy::VisitCallExpr(E);
+}
+
 bool LValueExprEvaluator::VisitMaterializeTemporaryExpr(
     const MaterializeTemporaryExpr *E) {
   // Walk through the expression to find the materialized temporary itself.
   SmallVector<const Expr *, 2> CommaLHSs;
   SmallVector<SubobjectAdjustment, 2> Adjustments;
   const Expr *Inner =
       E->getSubExpr()->skipRValueSubobjectAdjustments(CommaLHSs, Adjustments);
 

clang/lib/Analysis/BodyFarm.cpp

 #include "clang/AST/ASTContext.h"
 #include "clang/AST/CXXInheritance.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/Expr.h"
 #include "clang/AST/ExprCXX.h"
 #include "clang/AST/ExprObjC.h"
 #include "clang/AST/NestedNameSpecifier.h"
 #include "clang/Analysis/CodeInjector.h"
+#include "clang/Basic/Builtins.h"
 #include "clang/Basic/OperatorKinds.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/Debug.h"
 
 #define DEBUG_TYPE "body-farm"
 
 using namespace clang;
 
public:
   ImplicitCastExpr *
   makeLvalueToRvalue(const VarDecl *Decl,
                      bool RefersToEnclosingVariableOrCapture = false);
 
   /// Create an implicit cast of the given type.
   ImplicitCastExpr *makeImplicitCast(const Expr *Arg, QualType Ty,
                                      CastKind CK = CK_LValueToRValue);
 
+  /// Create a cast to reference type.
+  CastExpr *makeReferenceCast(const Expr *Arg, QualType Ty);
+
   /// Create an Objective-C bool literal.
   ObjCBoolLiteralExpr *makeObjCBool(bool Val);
 
   /// Create an Objective-C ivar reference.
   ObjCIvarRefExpr *makeObjCIvarRef(const Expr *Base, const ObjCIvarDecl *IVar);
 
   /// Create a Return statement.
   ReturnStmt *makeReturn(const Expr *RetVal);
ImplicitCastExpr *ASTMaker::makeImplicitCast(const Expr *Arg, QualType Ty,
   return ImplicitCastExpr::Create(C, Ty,
                                   /* CastKind=*/CK,
                                   /* Expr=*/const_cast<Expr *>(Arg),
                                   /* CXXCastPath=*/nullptr,
                                   /* ExprValueKind=*/VK_PRValue,
                                   /* FPFeatures */ FPOptionsOverride());
 }
 
+CastExpr *ASTMaker::makeReferenceCast(const Expr *Arg, QualType Ty) {
+  assert(Ty->isReferenceType());
+  return CXXStaticCastExpr::Create(
+      C, Ty.getNonReferenceType(),
+      Ty->isLValueReferenceType() ? VK_LValue : VK_XValue, CK_NoOp,
+      const_cast<Expr *>(Arg), /*CXXCastPath=*/nullptr,
+      /*Written=*/C.getTrivialTypeSourceInfo(Ty), FPOptionsOverride(),
+      SourceLocation(), SourceLocation(), SourceRange());
+}
+
 Expr *ASTMaker::makeIntegralCast(const Expr *Arg, QualType Ty) {
   if (Arg->getType() == Ty)
     return const_cast<Expr*>(Arg);
   return makeImplicitCast(Arg, Ty, CK_IntegralCast);
 }
 
 ImplicitCastExpr *ASTMaker::makeIntegralCastToBoolean(const Expr *Arg) {
   return makeImplicitCast(Arg, C.BoolTy, CK_IntegralToBoolean);
return CXXOperatorCallExpr::Create(
       /*AstContext=*/C, OO_Call, callOperatorDeclRef,
       /*Args=*/CallArgs,
       /*QualType=*/C.VoidTy,
       /*ExprValueType=*/VK_PRValue,
       /*SourceLocation=*/SourceLocation(),
       /*FPFeatures=*/FPOptionsOverride());
 }
 
+/// Create a fake body for 'std::move' or 'std::forward'. This is just:
+///
+/// \code
+/// return static_cast<return_type>(param);
+/// \endcode
+static Stmt *create_std_move_forward(ASTContext &C, const FunctionDecl *D) {
+  LLVM_DEBUG(llvm::dbgs() << "Generating body for std::move / std::forward\n");
+
+  ASTMaker M(C);
+
+  QualType ReturnType = D->getType()->castAs<FunctionType>()->getReturnType();
+  Expr *Param = M.makeDeclRefExpr(D->getParamDecl(0));
+  Expr *Cast = M.makeReferenceCast(Param, ReturnType);
+  return M.makeReturn(Cast);
+}
+
 /// Create a fake body for std::call_once.
 /// Emulates the following function body:
 ///
 /// \code
 /// typedef struct once_flag_s {
 ///   unsigned long __state = 0;
 /// } once_flag;
 /// template<class Callable>
if (D->getIdentifier() == nullptr)
     return nullptr;
 
   StringRef Name = D->getName();
   if (Name.empty())
     return nullptr;
 
   FunctionFarmer FF;
 
-  if (Name.startswith("OSAtomicCompareAndSwap") ||
+  if (unsigned BuiltinID = D->getBuiltinID()) {
+    switch (BuiltinID) {
+    case Builtin::BImove:
+    case Builtin::BImove_if_noexcept:
+    case Builtin::BIforward:
+      FF = create_std_move_forward;
+      break;
+    default:
+      FF = nullptr;
+      break;
+    }
+  } else if (Name.startswith("OSAtomicCompareAndSwap") ||
              Name.startswith("objc_atomicCompareAndSwap")) {
     FF = create_OSAtomicCompareAndSwap;
   } else if (Name == "call_once" && D->getDeclContext()->isStdNamespace()) {
     FF = create_call_once;
   } else {
     FF = llvm::StringSwitch<FunctionFarmer>(Name)
           .Case("dispatch_sync", create_dispatch_sync)
           .Case("dispatch_once", create_dispatch_once)
           .Default(nullptr);

clang/lib/Basic/Builtins.cpp

while (*EndPos == ',') {
     Encoding.push_back(PayloadIdx);
   }
 
   assert(*EndPos == '>' && "Callback callee specifier must end with a '>'");
   return true;
 }
 
 bool Builtin::Context::canBeRedeclared(unsigned ID) const {
-  return ID == Builtin::NotBuiltin ||
-         ID == Builtin::BI__va_start ||
-         (!hasReferenceArgsOrResult(ID) &&
-          !hasCustomTypechecking(ID));
+  return ID == Builtin::NotBuiltin || ID == Builtin::BI__va_start ||
+         (!hasReferenceArgsOrResult(ID) && !hasCustomTypechecking(ID)) ||
+         isInStdNamespace(ID);
 }

clang/lib/CodeGen/CGBuiltin.cpp

if (getTarget().getTriple().isOSMSVCRT() && E->getNumArgs() == 1 &&
       if (getTarget().getTriple().getArch() == llvm::Triple::x86)
         return EmitMSVCRTSetJmp(*this, MSVCSetJmpKind::_setjmp3, E);
       else if (getTarget().getTriple().getArch() == llvm::Triple::aarch64)
         return EmitMSVCRTSetJmp(*this, MSVCSetJmpKind::_setjmpex, E);
       return EmitMSVCRTSetJmp(*this, MSVCSetJmpKind::_setjmp, E);
     }
     break;
 
+  // C++ std:: builtins.
+  case Builtin::BImove:
+  case Builtin::BImove_if_noexcept:
+  case Builtin::BIforward:
+    return RValue::get(EmitLValue(E->getArg(0)).getPointer(*this));
   case Builtin::BI__GetExceptionInfo: {
     if (llvm::GlobalVariable *GV =
             CGM.getCXXABI().getThrowInfo(FD->getParamDecl(0)->getType()))
       return RValue::get(llvm::ConstantExpr::getBitCast(GV, CGM.Int8PtrTy));
     break;
   }
 
   case Builtin::BI__fastfail:

clang/lib/Sema/SemaChecking.cpp

case Builtin::BI__GetExceptionInfo:
     if (CheckCXXThrowOperand(
             TheCall->getBeginLoc(),
             Context.getExceptionObjectType(FDecl->getParamDecl(0)->getType()),
             TheCall))
       return ExprError();
 
     TheCall->setType(Context.VoidPtrTy);
     break;
+  case Builtin::BImove:
+  case Builtin::BImove_if_noexcept:
+  case Builtin::BIforward:
+    if (checkArgCount(*this, TheCall, 1))
+      return ExprError();
+    if (!Context.hasSameUnqualifiedType(TheCall->getType(),
+                                        TheCall->getArg(0)->getType())) {
+      Diag(TheCall->getBeginLoc(), diag::err_builtin_move_forward_unsupported)
+          << (BuiltinID == Builtin::BIforward);
+      return ExprError();
+    }
+    break;
   // OpenCL v2.0, s6.13.16 - Pipe functions
   case Builtin::BIread_pipe:
   case Builtin::BIwrite_pipe:
     // Since those two functions are declared with var args, we need a semantic
     // check for the argument.
     if (SemaBuiltinRWPipe(*this, TheCall))
       return ExprError();
     break;

clang/lib/Sema/SemaDecl.cpp

while (S->isClassScope() ||
          (LangOpts.CPlusPlus &&
           S->isFunctionPrototypeScope()) ||
          ((S->getFlags() & Scope::DeclScope) == 0) ||
          (S->getEntity() && S->getEntity()->isTransparentContext()))
     S = S->getParent();
   return S;
 }
 
+/// Determine whether a declaration matches a known function in namespace std.
+static bool isStdBuiltin(ASTContext &Ctx, FunctionDecl *FD,
+                         unsigned BuiltinID) {
+  switch (BuiltinID) {
+  case Builtin::BI__GetExceptionInfo:
+    // No type checking whatsoever.
+    return Ctx.getTargetInfo().getCXXABI().isMicrosoft();
+
+  case Builtin::BImove:
+  case Builtin::BImove_if_noexcept:
+  case Builtin::BIforward: {
+    // Ensure that we don't treat the algorithm
+    //   OutputIt std::move(InputIt, InputIt, OutputIt)
+    // as the builtin std::move.
+    const auto *FPT = FD->getType()->castAs<FunctionProtoType>();
+    return FPT->getNumParams() == 1 && !FPT->isVariadic();
+  }
+
+  default:
+    return false;
+  }
+}
+
 NamedDecl*
 Sema::ActOnFunctionDeclarator(Scope *S, Declarator &D, DeclContext *DC,
                               TypeSourceInfo *TInfo, LookupResult &Previous,
                               MultiTemplateParamsArg TemplateParamListsRef,
                               bool &AddToScope) {
   QualType R = TInfo->getType();
 
   assert(R->isFunctionType());
if (NewFD->isInvalidDecl()) {
       // generate them.
       Diag(NewFD->getLocation(), diag::ext_out_of_line_declaration)
         << D.getCXXScopeSpec().getRange();
     }
   }
 
   // If this is the first declaration of a library builtin function, add
   // attributes as appropriate.
-  if (!D.isRedeclaration() &&
-      NewFD->getDeclContext()->getRedeclContext()->isFileContext()) {
+  if (!D.isRedeclaration()) {
     if (IdentifierInfo *II = Previous.getLookupName().getAsIdentifierInfo()) {
       if (unsigned BuiltinID = II->getBuiltinID()) {
+        bool InStdNamespace = Context.BuiltinInfo.isInStdNamespace(BuiltinID);
+        if (!InStdNamespace &&
+            NewFD->getDeclContext()->getRedeclContext()->isFileContext()) {
           if (NewFD->getLanguageLinkage() == CLanguageLinkage) {
             // Validate the type matches unless this builtin is specified as
             // matching regardless of its declared type.
             if (Context.BuiltinInfo.allowTypeMismatch(BuiltinID)) {
               NewFD->addAttr(BuiltinAttr::CreateImplicit(Context, BuiltinID));
             } else {
               ASTContext::GetBuiltinTypeError Error;
               LookupNecessaryTypesForBuiltin(S, BuiltinID);
               QualType BuiltinType = Context.GetBuiltinType(BuiltinID, Error);
 
               if (!Error && !BuiltinType.isNull() &&
                   Context.hasSameFunctionTypeIgnoringExceptionSpec(
                       NewFD->getType(), BuiltinType))
                 NewFD->addAttr(BuiltinAttr::CreateImplicit(Context, BuiltinID));
             }
-        } else if (BuiltinID == Builtin::BI__GetExceptionInfo &&
-                   Context.getTargetInfo().getCXXABI().isMicrosoft()) {
-          // FIXME: We should consider this a builtin only in the std namespace.
+          }
+        } else if (InStdNamespace && NewFD->isInStdNamespace() &&
+                   isStdBuiltin(Context, NewFD, BuiltinID)) {
           NewFD->addAttr(BuiltinAttr::CreateImplicit(Context, BuiltinID));
         }
       }
     }
   }
 
   ProcessPragmaWeak(S, NewFD);
   checkAttributesAfterMerging(*this, *NewFD);
Decl *Sema::ActOnStartOfFunctionDef(Scope *FnBodyScope, Decl *D,
   } else {
     // Enter a new function scope
     PushFunctionScope();
   }
 
   // Builtin functions cannot be defined.
   if (unsigned BuiltinID = FD->getBuiltinID()) {
     if (!Context.BuiltinInfo.isPredefinedLibFunction(BuiltinID) &&
-        !Context.BuiltinInfo.isPredefinedRuntimeFunction(BuiltinID)) {
+        !Context.BuiltinInfo.isPredefinedRuntimeFunction(BuiltinID) &&
+        !Context.BuiltinInfo.isInStdNamespace(BuiltinID)) {
       Diag(FD->getLocation(), diag::err_builtin_definition) << FD;
       FD->setInvalidDecl();
     }
   }
 
   // The return type of a function definition must be complete
   // (C99 6.9.1p3, C++ [dcl.fct]p6).
   QualType ResultType = FD->getReturnType();

clang/lib/Sema/SemaExpr.cpp

ExprResult Sema::CheckPlaceholderExpr(Expr *E) {
   case BuiltinType::PseudoObject:
     return checkPseudoObjectRValue(E);
 
   case BuiltinType::BuiltinFn: {
     // Accept __noop without parens by implicitly converting it to a call expr.
     auto *DRE = dyn_cast<DeclRefExpr>(E->IgnoreParenImpCasts());
     if (DRE) {
       auto *FD = cast<FunctionDecl>(DRE->getDecl());
-      if (FD->getBuiltinID() == Builtin::BI__noop) {
+      unsigned BuiltinID = FD->getBuiltinID();
+      if (BuiltinID == Builtin::BI__noop) {
         E = ImpCastExprToType(E, Context.getPointerType(FD->getType()),
                               CK_BuiltinFnToFnPtr)
                 .get();
         return CallExpr::Create(Context, E, /*Args=*/{}, Context.IntTy,
                                 VK_PRValue, SourceLocation(),
                                 FPOptionsOverride());
       }
+
+      if (Context.BuiltinInfo.isInStdNamespace(BuiltinID)) {
+        // Any use of these other than a direct call is ill-formed as of C++20,
+        // because they are not addressable functions. In earlier language
+        // modes, warn and force an instantiation of the real body.
+        Diag(E->getBeginLoc(), getLangOpts().CPlusPlus20
+                                   ? diag::err_use_of_unaddressable_function
+                                   : diag::warn_use_of_unaddressable_function);
+        if (FD->isImplicitlyInstantiable()) {
+          // Require a definition here because a normal attempt at
+          // instantiation for a builtin will be ignored, and we won't try
+          // again later. We assume that the definition of the template
+          // precedes this use.
+          InstantiateFunctionDefinition(E->getBeginLoc(), FD,
+                                        /*Recursive=*/false,
+                                        /*DefinitionRequired=*/true,
+                                        /*AtEndOfTU=*/false);
+        }
+        // Produce a properly-typed reference to the function.
+        CXXScopeSpec SS;
+        SS.Adopt(DRE->getQualifierLoc());
+        TemplateArgumentListInfo TemplateArgs;
+        DRE->copyTemplateArgumentsInto(TemplateArgs);
+        return BuildDeclRefExpr(
+            FD, FD->getType(), VK_LValue, DRE->getNameInfo(),
+            DRE->hasQualifier() ? &SS : nullptr, DRE->getFoundDecl(),
+            DRE->getTemplateKeywordLoc(),
+            DRE->hasExplicitTemplateArgs() ? &TemplateArgs : nullptr);
+      }
     }
 
     Diag(E->getBeginLoc(), diag::err_builtin_fn_use);
     return ExprError();
   }
 
   case BuiltinType::IncompleteMatrixIdx:
     Diag(cast<MatrixSubscriptExpr>(E->IgnoreParens())

clang/lib/Sema/SemaExprCXX.cpp

FunctionDecl *Fn = ResolveAddressOfOverloadedFunction(From, ToType,
                                                           true, Found);
     if (!Fn)
       return ExprError();
 
     if (DiagnoseUseOfDecl(Fn, From->getBeginLoc()))
       return ExprError();
 
     From = FixOverloadedFunctionReference(From, Found, Fn);
+
+    // We might get back another placeholder expression if we resolved to a
+    // builtin.
+    ExprResult Checked = CheckPlaceholderExpr(From);
+    if (Checked.isInvalid())
+      return ExprError();
+
+    From = Checked.get();
     FromType = From->getType();
   }
 
   // If we're converting to an atomic type, first convert to the corresponding
   // non-atomic type.
   QualType ToAtomicType;
   if (const AtomicType *ToAtomic = ToType->getAs<AtomicType>()) {
     ToAtomicType = ToType;

clang/lib/Sema/SemaInit.cpp

case SK_ResolveAddressOfOverloadedFunction:
       // Overload resolution determined which function invoke; update the
       // initializer to reflect that choice.
       S.CheckAddressOfMemberAccess(CurInit.get(), Step->Function.FoundDecl);
       if (S.DiagnoseUseOfDecl(Step->Function.FoundDecl, Kind.getLocation()))
         return ExprError();
       CurInit = S.FixOverloadedFunctionReference(CurInit,
                                                  Step->Function.FoundDecl,
                                                  Step->Function.Function);
+      // We might get back another placeholder expression if we resolved to a
+      // builtin.
+      if (!CurInit.isInvalid())
+        CurInit = S.CheckPlaceholderExpr(CurInit.get());
       break;
 
     case SK_CastDerivedToBasePRValue:
     case SK_CastDerivedToBaseXValue:
     case SK_CastDerivedToBaseLValue: {
       // We have a derived-to-base cast that produces either an rvalue or an
       // lvalue. Perform that cast.
 

clang/lib/Sema/SemaLookup.cpp

if (II) {
         }
       }
 
       // If this is a builtin on this (or all) targets, create the decl.
       if (unsigned BuiltinID = II->getBuiltinID()) {
         // In C++ and OpenCL (spec v1.2 s6.9.f), we don't have any predefined
         // library functions like 'malloc'. Instead, we'll just error.
         if ((getLangOpts().CPlusPlus || getLangOpts().OpenCL) &&
-            Context.BuiltinInfo.isPredefinedLibFunction(BuiltinID))
+            (Context.BuiltinInfo.isPredefinedLibFunction(BuiltinID) ||
+             Context.BuiltinInfo.isInStdNamespace(BuiltinID)))
           return false;
 
         if (NamedDecl *D =
                 LazilyCreateBuiltin(II, BuiltinID, TUScope,
                                     R.isForRedeclaration(), R.getNameLoc())) {
           R.addDecl(D);
           return true;
         }

clang/lib/Sema/SemaOverload.cpp

if (FunctionDecl *Fn
           = S.Context.getTypeDeclType(Method->getParent()).getTypePtr();
         FromType = S.Context.getMemberPointerType(FromType, ClassType);
       } else if (isa<UnaryOperator>(From->IgnoreParens())) {
         assert(cast<UnaryOperator>(From->IgnoreParens())->getOpcode() ==
                UO_AddrOf &&
                "Non-address-of operator for overloaded function expression");
         FromType = S.Context.getPointerType(FromType);
       }
-
-      // Check that we've computed the proper type after overload resolution.
-      // FIXME: FixOverloadedFunctionReference has side-effects; we shouldn't
-      // be calling it from within an NDEBUG block.
-      assert(S.Context.hasSameType(
-        FromType,
-        S.FixOverloadedFunctionReference(From, AccessPair, Fn)->getType()));
     } else {
       return false;
     }
   }
   // Lvalue-to-rvalue conversion (C++11 4.1):
   //   A glvalue (3.10) of a non-function, non-array type T can
   //   be converted to a prvalue.
   bool argIsLValue = From->isGLValue();
if (CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(Fn)) {
             UnOp->getOperatorLoc(), false, CurFPFeatureOverrides());
       }
     }
     Expr *SubExpr = FixOverloadedFunctionReference(UnOp->getSubExpr(),
                                                    Found, Fn);
     if (SubExpr == UnOp->getSubExpr())
       return UnOp;
 
-    return UnaryOperator::Create(
-        Context, SubExpr, UO_AddrOf, Context.getPointerType(SubExpr->getType()),
-        VK_PRValue, OK_Ordinary, UnOp->getOperatorLoc(), false,
-        CurFPFeatureOverrides());
+    // FIXME: This can't currently fail, but in principle it could.
+    return CreateBuiltinUnaryOp(UnOp->getOperatorLoc(), UO_AddrOf, SubExpr)
+        .get();
   }
 
   if (UnresolvedLookupExpr *ULE = dyn_cast<UnresolvedLookupExpr>(E)) {
     // FIXME: avoid copy.
     TemplateArgumentListInfo TemplateArgsBuffer, *TemplateArgs = nullptr;
     if (ULE->hasExplicitTemplateArgs()) {
       ULE->copyTemplateArgumentsInto(TemplateArgsBuffer);
       TemplateArgs = &TemplateArgsBuffer;
     }
 
-    DeclRefExpr *DRE =
-        BuildDeclRefExpr(Fn, Fn->getType(), VK_LValue, ULE->getNameInfo(),
-                         ULE->getQualifierLoc(), Found.getDecl(),
-                         ULE->getTemplateKeywordLoc(), TemplateArgs);
+    QualType Type = Fn->getType();
+    ExprValueKind ValueKind = getLangOpts().CPlusPlus ? VK_LValue : VK_PRValue;
+
+    // FIXME: Duplicated from BuildDeclarationNameExpr.
+    if (unsigned BID = Fn->getBuiltinID()) {
+      if (!Context.BuiltinInfo.isPredefinedLibFunction(BID)) {
+        Type = Context.BuiltinFnTy;
+        ValueKind = VK_PRValue;
+      }
+    }
+
+    DeclRefExpr *DRE = BuildDeclRefExpr(
+        Fn, Type, ValueKind, ULE->getNameInfo(), ULE->getQualifierLoc(),
+        Found.getDecl(), ULE->getTemplateKeywordLoc(), TemplateArgs);
     DRE->setHadMultipleCandidates(ULE->getNumDecls() > 1);
     return DRE;
   }
 
   if (UnresolvedMemberExpr *MemExpr = dyn_cast<UnresolvedMemberExpr>(E)) {
     // FIXME: avoid copy.
     TemplateArgumentListInfo TemplateArgsBuffer, *TemplateArgs = nullptr;
     if (MemExpr->hasExplicitTemplateArgs()) {

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

void Sema::InstantiateFunctionDefinition(SourceLocation PointOfInstantiation,
 
   // Never instantiate an explicit specialization except if it is a class scope
   // explicit specialization.
   TemplateSpecializationKind TSK =
       Function->getTemplateSpecializationKindForInstantiation();
   if (TSK == TSK_ExplicitSpecialization)
     return;
 
+  // Never implicitly instantiate a builtin; we don't actually need a function
+  // body.
+  if (Function->getBuiltinID() && TSK == TSK_ImplicitInstantiation &&
+      !DefinitionRequired)
+    return;
+
   // Don't instantiate a definition if we already have one.
   const FunctionDecl *ExistingDefn = nullptr;
   if (Function->isDefined(ExistingDefn,
                           /*CheckForPendingFriendDefinition=*/true)) {
     if (ExistingDefn->isThisDeclarationADefinition())
       return;
 
     // If we're asked to instantiate a function whose body comes from an

clang/test/Analysis/use-after-move.cpp

if (i == 2) { // peaceful-note 2 {{Assuming 'i' is not equal to 2}}
                   // peaceful-note@-1 2 {{Taking false branch}}
       a.foo();    // no-warning
     }
   }
   {
     A a;
     if (i == 1) { // peaceful-note 2 {{'i' is not equal to 1}}
                   // peaceful-note@-1 2 {{Taking false branch}}
-      std::move(a);
+      (void)std::move(a);
     }
     if (i == 2) { // peaceful-note 2 {{'i' is not equal to 2}}
                   // peaceful-note@-1 2 {{Taking false branch}}
       a = A();
       a.foo();
     }
   }
   // The built-in assignment operator should also be recognized as a
void templateArgIsNotUseTest() {
     A a;
     functionTemplate<sizeof(A(std::move(a)))>(std::move(a)); // no-warning
   }
 }
 
 // Moves of global variables are not reported.
 A global_a;
 void globalVariablesTest() {
-  std::move(global_a);
+  (void)std::move(global_a);
   global_a.foo(); // no-warning
 }
 
 // Moves of member variables.
 class memberVariablesTest {
   A a;
   static A static_a;
 

clang/test/CodeGenCXX/builtin-std-move.cpp

+// RUN: %clang_cc1 -triple=x86_64-linux-gnu -emit-llvm -o - -std=c++17 %s | FileCheck %s --implicit-check-not=@_ZSt4move
+
+namespace std {
+  template<typename T> constexpr T &&move(T &val) { return static_cast<T&&>(val); }
+  template<typename T> constexpr T &&move_if_noexcept(T &val);
+  template<typename T> constexpr T &&forward(T &val);
+
+  // Not the builtin.
+  template<typename T, typename U> T move(U source, U source_end, T dest);
+}
+
+class T {};
+extern "C" void take(T &&);
+
+T a;
+
+// Check emission of a constant-evaluated call.
+// CHECK-DAG: @move_a = constant %[[T:.*]]* @a
+T &&move_a = std::move(a);
+// CHECK-DAG: @move_if_noexcept_a = constant %[[T]]* @a
+T &&move_if_noexcept_a = std::move_if_noexcept(a);
+// CHECK-DAG: @forward_a = constant %[[T]]* @a
+T &forward_a = std::forward<T&>(a);
+
+// Check emission of a non-constant call.
+// CHECK-LABEL: define {{.*}} void @test
+extern "C" void test(T &t) {
+  // CHECK: store %[[T]]* %{{.*}}, %[[T]]** %[[T_REF:[^,]*]]
+  // CHECK: %0 = load %[[T]]*, %[[T]]** %[[T_REF]]
+  // CHECK: call void @take(%[[T]]* {{.*}} %0)
+  take(std::move(t));
+  // CHECK: %1 = load %[[T]]*, %[[T]]** %[[T_REF]]
+  // CHECK: call void @take(%[[T]]* {{.*}} %1)
+  take(std::move_if_noexcept(t));
+  // CHECK: %2 = load %[[T]]*, %[[T]]** %[[T_REF]]
+  // CHECK: call void @take(%[[T]]* {{.*}} %2)
+  take(std::forward<T&&>(t));
+
+  // CHECK: call {{.*}} @_ZSt4moveI1TS0_ET_T0_S2_S1_
+  std::move(t, t, t);
+}
+
+// CHECK: declare {{.*}} @_ZSt4moveI1TS0_ET_T0_S2_S1_
+
+// Check that we instantiate and emit if the address is taken.
+// CHECK-LABEL: define {{.*}} @use_address
+extern "C" void *use_address() {
+  // CHECK: ret {{.*}} @_ZSt4moveIiEOT_RS0_
+  return (void*)&std::move<int>;
+}
+
+// CHECK: define {{.*}} i32* @_ZSt4moveIiEOT_RS0_(i32*

clang/test/CodeGenCXX/microsoft-abi-throw.cpp

 // RUN: %clang_cc1 -emit-llvm -o - -triple=i386-pc-win32 -std=c++11 %s -fcxx-exceptions -fms-extensions | FileCheck %s
-// RUN: %clang_cc1 -emit-llvm -o - -triple=i386-pc-win32 -std=c++11 %s -fcxx-exceptions -fms-extensions -DSTD | FileCheck %s
 
 // CHECK-DAG: @"??_R0?AUY@@@8" = linkonce_odr global %rtti.TypeDescriptor7 { i8** @"??_7type_info@@6B@", i8* null, [8 x i8] c".?AUY@@\00" }, comdat
 // CHECK-DAG: @"_CT??_R0?AUY@@@8??0Y@@QAE@ABU0@@Z8" = linkonce_odr unnamed_addr constant %eh.CatchableType { i32 4, i8* bitcast (%rtti.TypeDescriptor7* @"??_R0?AUY@@@8" to i8*), i32 0, i32 -1, i32 0, i32 8, i8* bitcast (%struct.Y* (%struct.Y*, %struct.Y*, i32)* @"??0Y@@QAE@ABU0@@Z" to i8*) }, section ".xdata", comdat
 // CHECK-DAG: @"??_R0?AUZ@@@8" = linkonce_odr global %rtti.TypeDescriptor7 { i8** @"??_7type_info@@6B@", i8* null, [8 x i8] c".?AUZ@@\00" }, comdat
 // CHECK-DAG: @"_CT??_R0?AUZ@@@81" = linkonce_odr unnamed_addr constant %eh.CatchableType { i32 0, i8* bitcast (%rtti.TypeDescriptor7* @"??_R0?AUZ@@@8" to i8*), i32 0, i32 -1, i32 0, i32 1, i8* null }, section ".xdata", comdat
 // CHECK-DAG: @"??_R0?AUW@@@8" = linkonce_odr global %rtti.TypeDescriptor7 { i8** @"??_7type_info@@6B@", i8* null, [8 x i8] c".?AUW@@\00" }, comdat
 // CHECK-DAG: @"_CT??_R0?AUW@@@8??0W@@QAE@ABU0@@Z44" = linkonce_odr unnamed_addr constant %eh.CatchableType { i32 4, i8* bitcast (%rtti.TypeDescriptor7* @"??_R0?AUW@@@8" to i8*), i32 4, i32 -1, i32 0, i32 4, i8* bitcast (%struct.W* (%struct.W*, %struct.W*, i32)* @"??0W@@QAE@ABU0@@Z" to i8*) }, section ".xdata", comdat
 // CHECK-DAG: @"??_R0?AUM@@@8" = linkonce_odr global %rtti.TypeDescriptor7 { i8** @"??_7type_info@@6B@", i8* null, [8 x i8] c".?AUM@@\00" }, comdat
void j(TemplateWithDefault &twd) {
   throw twd;
 }
 
 
 void h() {
   throw nullptr;
 }
 
-#ifdef STD
 namespace std {
 template <typename T>
 void *__GetExceptionInfo(T);
 }
-#else
-template <typename T>
-void *__GetExceptionInfo(T);
-#endif
 using namespace std;
 
 void *GetExceptionInfo_test0() {
 // CHECK-LABEL: @"?GetExceptionInfo_test0@@YAPAXXZ"
 // CHECK:  ret i8* bitcast (%eh.ThrowInfo* @_TI1H to i8*)
   return __GetExceptionInfo(0);
 }
 

clang/test/SemaCXX/builtin-std-move.cpp

+// RUN: %clang_cc1 -std=c++17 -verify %s
+// RUN: %clang_cc1 -std=c++17 -verify %s -DNO_CONSTEXPR
+// RUN: %clang_cc1 -std=c++20 -verify %s
+
+namespace std {
+#ifndef NO_CONSTEXPR
+#define CONSTEXPR constexpr
+#else
+#define CONSTEXPR
+#endif
+
+  template<typename T> CONSTEXPR T &&move(T &x) {
+    static_assert(T::moveable, "instantiated move"); // expected-error {{no member named 'moveable' in 'B'}}
+                                                     // expected-error@-1 {{no member named 'moveable' in 'C'}}
+    return static_cast<T&&>(x);
+  }
+
+  // Unrelated move functions are not the builtin.
+  template<typename T> CONSTEXPR int move(T, T) { return 5; }
+
+  template<typename T, bool Rref> struct ref { using type = T&; };
+  template<typename T> struct ref<T, true> { using type = T&&; };
+
+  template<typename T> CONSTEXPR auto move_if_noexcept(T &x) -> typename ref<T, noexcept(T(static_cast<T&&>(x)))>::type {
+    static_assert(T::moveable, "instantiated move_if_noexcept"); // expected-error {{no member named 'moveable' in 'B'}}
+    return static_cast<typename ref<T, noexcept(T(static_cast<T&&>(x)))>::type>(x);
+  }
+
+  template<typename T> struct remove_reference { using type = T; };
+  template<typename T> struct remove_reference<T&> { using type = T; };
+  template<typename T> struct remove_reference<T&&> { using type = T; };
+
+  template<typename T> CONSTEXPR T &&forward(typename remove_reference<T>::type &x) {
+    static_assert(T::moveable, "instantiated forward"); // expected-error {{no member named 'moveable' in 'B'}}
+                                                        // expected-error@-1 {{no member named 'moveable' in 'C'}}
+    return static_cast<T&&>(x);
+  }
+}
+
+// Note: this doesn't have a 'moveable' member. Instantiation of the above
+// functions will fail if it's attempted.
+struct A {};
+constexpr bool f(A a) { // #f
+  A &&move = std::move(a); // #call
+  A &&move_if_noexcept = std::move_if_noexcept(a);
+  A &&forward1 = std::forward<A>(a);
+  A &forward2 = std::forward<A&>(a);
+  return &move == &a && &move_if_noexcept == &a &&
+         &forward1 == &a && &forward2 == &a &&
+         std::move(a, a) == 5;
+}
+
+#ifndef NO_CONSTEXPR
+static_assert(f({}), "should be constexpr");
+#else
+// expected-error@#f {{never produces a constant expression}}
+// expected-note@#call {{}}
+#endif
+
+struct B {};
+B &&(*pMove)(B&) = std::move; // #1 expected-note {{instantiation of}}
+B &&(*pMoveIfNoexcept)(B&) = &std::move_if_noexcept; // #2 expected-note {{instantiation of}}
+B &&(*pForward)(B&) = &std::forward<B>; // #3 expected-note {{instantiation of}}
+int (*pUnrelatedMove)(B, B) = std::move;
+
+struct C {};
+C &&(&rMove)(C&) = std::move; // #4 expected-note {{instantiation of}}
+C &&(&rForward)(C&) = std::forward<C>; // #5 expected-note {{instantiation of}}
+int (&rUnrelatedMove)(B, B) = std::move;
+
+#if __cplusplus <= 201703L
+// expected-warning@#1 {{non-addressable}}
+// expected-warning@#2 {{non-addressable}}
+// expected-warning@#3 {{non-addressable}}
+// expected-warning@#4 {{non-addressable}}
+// expected-warning@#5 {{non-addressable}}
+#else
+// expected-error@#1 {{non-addressable}}
+// expected-error@#2 {{non-addressable}}
+// expected-error@#3 {{non-addressable}}
+// expected-error@#4 {{non-addressable}}
+// expected-error@#5 {{non-addressable}}
+#endif
+
+void attribute_const() {
+  int n;
+  std::move(n); // expected-warning {{ignoring return value}}
+  std::move_if_noexcept(n); // expected-warning {{ignoring return value}}
+  std::forward<int>(n); // expected-warning {{ignoring return value}}
+}

clang/test/SemaCXX/unqualified-std-call-fixits.cpp

 // RUN: %clang_cc1 -verify -std=c++20 -Wall %s
 // RUN: cp %s %t
 // RUN: %clang_cc1 -x c++ -std=c++20 -fixit %t
 // RUN: %clang_cc1 -Wall -Werror -x c++ -std=c++20 %t
 // RUN: cat %t | FileCheck %s
 
 namespace std {
 
-void move(auto &&a) {}
+int &&move(auto &&a) { return a; }
 
-void forward(auto &a) {}
+int &&forward(auto &a) { return a; }
 
 } // namespace std
 
 using namespace std;
 
 void f() {
   int i = 0;
-  move(i); // expected-warning {{unqualified call to std::move}}
-  // CHECK: {{^}}  std::
-  forward(i); // expected-warning {{unqualified call to std::forward}}
-              // CHECK: {{^}}  std::
+  (void)move(i); // expected-warning {{unqualified call to std::move}}
+  // CHECK: {{^}}  (void)std::move
+  (void)forward(i); // expected-warning {{unqualified call to std::forward}}
+  // CHECK: {{^}}  (void)std::forward
 }

clang/test/SemaCXX/unqualified-std-call.cpp

-// RUN: %clang_cc1 -fsyntax-only -verify -Wall -std=c++11 %s
+// RUN: %clang_cc1 -fsyntax-only -verify -Wall -std=c++11 %s -Wno-unused-value
 
 namespace std {
 
 template <typename T>
 void dummy(T &&) {}
 template <typename T>
-void move(T &&) {}
+T &&move(T &&x) { return x; }
 template <typename T, typename U>
 void move(T &&, U &&) {}
 
 inline namespace __1 {
 template <typename T>
-void forward(T &) {}
+T &forward(T &x) { return x; }
 } // namespace __1
 
 struct foo {};
 
 } // namespace std
 
 namespace global {
 

clang/test/SemaCXX/warn-consumed-analysis.cpp

 } // end namespace TestParens
 
 } // end namespace InitializerAssertionFailTest
 
 
 namespace std {
   void move();
   template<class T>
-  void move(T&&);
+  T &&move(T&);
 
   namespace __1 {
     void move();
     template<class T>
-    void move(T&&);
+    T &&move(T&);
   }
 }
 
 namespace PR18260 {
   class X {
     public:
       void move();
   } x;
 
   void test() {
     x.move();
     std::move();
-    std::move(x);
+    std::move(x); // expected-warning {{ignoring return value}}
     std::__1::move();
     std::__1::move(x);
   }
 } // end namespace PR18260