[Clang] Add __builtin_launder

[EricWF]

Summary

This patch adds __builtin_launder, which is required to implement std::launder. Additionally GCC provides __builtin_launder, so thing brings Clang in-line with GCC.

I'm not exactly sure what magic __builtin_launder requires, but based on previous discussions this patch applies a @llvm.invariant.group.barrier. As noted in previous discussions, this may not be enough to correctly handle vtables.

[EricWF]

• Remove incorrect FIXME comment.

[majnemer]

A test with restrict and __restrict might be interesting.

[jroelofs]

include/clang/Basic/Builtins.def

486

GCC's is type-generic:

#include <type_traits>

bool is_type_generic() {
  int v;
  return std::is_same<decltype(__builtin_launder(&v)),
                      decltype(&v)>::value;
}

However this BUILTIN line says that this one is not (yeah, I see it's being fixed up in Sema later in this patch). I think you need to use custom type checking here via t.

[EricWF]

include/clang/Basic/Builtins.def

486

The t is specified. It's in the third argument.

[jroelofs]

include/clang/Basic/Builtins.def

486

ohhh. duh. sorry.

[EricWF]

• Fix argument initialization.
• Make constexpr.

[EricWF]

• Improve quality of tests.
• Format code.

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1674

It would be nice to avoid this for types that contain no const subobjects / reference subobjects / vptrs. I think we can also omit this entirely if -fstrict-vtable-ptrs is disabled, since in that case we don't generate any invariant.group metadata.

I'd be OK with the former being left to a future change, but the latter should be part of this change so we don't generate unnecessarily-inefficient code in the default mode for uses of std::launder.

lib/Sema/SemaChecking.cpp

860–864

Please also check that the pointee type is an object type -- per [ptr.launder]p3, "the program is ill-formed if T is a function type or cv void", and we don't want our builtin to need to deal with such cases.

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1674

I'll put it in this change set.

[EricWF]

• Address inline comments about missing diagnostics for void pointers and function pointers.
• Address inline comments about only enabling when -fstrict-vtable-pointers is specified, and only on types with vtables.

[EricWF]

• Improve diagnostic handling.

[EricWF]

Ping.

[EricWF]

• Rebase off master.

[EricWF]

Ping.

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1947–1948

I think you also need to catch class types that contain dynamic classes as subobjects.

lib/Sema/SemaChecking.cpp

860

Might be a bit clearer to use llvm::Optional<unsigned> as the type of this.

test/CodeGenCXX/builtin-launder.cpp

94–97

I would note that this means adding optimizations for those cases later is an LTO ABI break. That's probably OK, but just something we're going to need to remember.

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1947–1948

Only by-value subobjects, or also reference subobjects?

[EricWF]

test/CodeGenCXX/builtin-launder.cpp

94–97

I added your note almost verbatim to the test case.

[EricWF]

• Launder types with subobjects of dynamic class type.
• Improve diagnostic selection using llvm::Optional<unsigned>.
• Add comment about LTO ABI concerns to test file.
• Merge with master.

[EricWF]

Ping.

[rsmith]

LGTM with a fix (and test) for pointer-to-array-of-dynamic-class-type handling.

lib/CodeGen/CGBuiltin.cpp

936–938

I think you also need to recurse through array types (use Ty->getBaseElementTypeUnsafe()->getAsCXXRecordDecl() or similar).

[rsmith]

Also we probably want to hold off on landing this until PR37458 is fixed, otherwise std::launder will start miscompiling code.

[EricWF]

• Handle array types as requested.

• Attempt to handle incomplete class types. They are always considered in need of laundering. Including class templates who's instantiation hasn't been forced yet. @rsmith is this the correct thing to do?

[erichkeane]

Bump! I don't see anything here that seems questionable (and this has extensive testing), but I presume @rsmith should be the final one to approve.

[Romain-Geissler-1A]

Hi,

Is there any news on this code review ? Is it ready to land ?

Cheers,
Romain

[EricWF]

Merge with upstream.

[EricWF]

In D40218#1298943, @Romain-Geissler-1A wrote:

Is there any news on this code review ? Is it ready to land ?

I think so, but there were some changes regarding incomplete types since Richard last looked at it.
I wanted him to sign off on the new behavior.

lib/CodeGen/CGBuiltin.cpp

1037

@rsmith I believe this changed since you approved the patch.

We talked about this offline, but could you give it a thumbs up?

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1425–1426

I think it's a bug that launder doesn't require T to be a complete type. Can you file an LWG issue?

We should also decide whether we want to proactively fix this issue (require the type to be complete from the Sema checking of the builtin and assert that it's defined here) or not.

1437–1438

Delete this commented-out code.

1451

Would SmallPtrSet be a better choice here?

lib/Sema/SemaChecking.cpp

853

Did you mean to add this blank line?

858–860

Instead of performing some of the conversions here and some of them as part of initialization, I think it'd be more obvious to compute the builtin's parameter type here (which is the type of the argument if it's not of array [or function] type, and the decayed type of the argument otherwise), and do the decay and lvalue-to-rvalue conversion as part of the parameter initialization below.

The current code arrangement (and especially this comment) leaves a reader thinking "but you *need* an lvalue-to-rvalue conversion if the argument is an lvalue".

test/CodeGen/builtins.c

404–409

This test is not robust against minor IR differences such as variable or basic block names changing (some of these change in a release build), and is testing things that are not related to this builtin (eg, that we produce an alloca for a function parameter and its relative order to an alloca for a local variable).

I would remove everything here other than the load and the store, and add an explicit check that we don't generate a launder call:

// CHECK: [[TMP:%.*]] = load i32*,
// CHECK-NOT: @llvm.launder
// CHECK: store i32* [[TMP]],

test/CodeGenCXX/builtin-launder.cpp

17

This is likewise likely to fail with a release build of clang.

test/SemaCXX/builtins.cpp

120

&i doesn't what?

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1425–1426

Apparently I misread the specification. The blanket wording in [res.on.functions] already prohibits this.

Though it's worth noting that GCC doesn't enforce this requirement (nor the one about function pointers).

1451

I have no idea. I'm ignorant to the internals of both containers. Since you asked the question I'll assume it is and make the change.
If you would like, I can do further investigation upon request.

lib/Sema/SemaChecking.cpp

853

Oh Richard, you're too kind. Of course I didn't mean to do that.

858–860

Ack.

I think I've implemented what you requested. Could you please verify?

test/CodeGen/builtins.c

404–409

If you're referring to the behavior of discarding value names, Clang CC1 only does that if -discard-value-names is explicitly passed, regardless of how the compiler is built, and so the names should be stable. (The Clang driver still conditionally passes -discard-value-names in release builds).

Are there other IR changes you have in mind?

None the less, I'll simply this test as requested.

test/CodeGenCXX/builtin-launder.cpp

17

I just verified that they pass in both release and debug builds. I previously changed Clang and LLVM to make the label names consistent between the two configurations for the purposes of testing.

I'll still attempt to clean unnecessary checks up.

test/SemaCXX/builtins.cpp

120

That's a great question. Not sure what I was up to when I wrote that. I tried a couple of things I though I might have been thinking, but they all passed.

Removing the comment.

[EricWF]

Address review comments.

@rsmith I think this is good to go. Just need your input on one change.

[EricWF]

Merging with upstream. Preparing to commit.

include/clang/Basic/Builtins.def

 BUILTIN(__builtin_setjmp, "iv**", "j")
 BUILTIN(__builtin_longjmp, "vv**i", "r")
 BUILTIN(__builtin_unwind_init, "v", "")
 BUILTIN(__builtin_eh_return_data_regno, "iIi", "nc")
 BUILTIN(__builtin_snprintf, "ic*zcC*.", "nFp:2:")
 BUILTIN(__builtin_vsprintf, "ic*cC*a", "nFP:1:")
 BUILTIN(__builtin_vsnprintf, "ic*zcC*a", "nFP:2:")
 BUILTIN(__builtin_thread_pointer, "v*", "nc")
+BUILTIN(__builtin_launder, "v*v*", "nt")
 
 // GCC exception builtins
 BUILTIN(__builtin_eh_return, "vzv*", "r") // FIXME: Takes intptr_t, not size_t!
 BUILTIN(__builtin_frob_return_addr, "v*v*", "n")
 BUILTIN(__builtin_dwarf_cfa, "v*", "n")
 BUILTIN(__builtin_init_dwarf_reg_size_table, "vv*", "n")
 BUILTIN(__builtin_dwarf_sp_column, "Ui", "n")
 BUILTIN(__builtin_extend_pointer, "ULLiv*", "n") // _Unwind_Word == uint64_t

include/clang/Basic/DiagnosticSemaKinds.td

   "__final is a GNU extension, consider using C++11 final">,
   InGroup<GccCompat>;
 
 def warn_shadow_field :
   Warning<"non-static data member '%0' of '%1' shadows member inherited from type '%2'">,
   InGroup<ShadowField>, DefaultIgnore;
 def note_shadow_field : Note<"declared here">;
 
+def err_builtin_launder_non_pointer_arg :
+  Error<"non-pointer argument to '__builtin_launder'">;
+def err_builtin_launder_function_pointer_arg :
+  Error<"argument to '__builtin_launder' cannot be a function pointer">;
+def err_builtin_launder_void_pointer_arg :
+  Error<"argument to '__builtin_launder' cannot be a void pointer">;
+
 def err_target_required_in_redecl : Error<
   "function declaration is missing 'target' attribute in a multiversioned "
   "function">;
 def note_multiversioning_caused_here : Note<
   "function multiversioning caused by this declaration">;
 def err_multiversion_after_used : Error<
   "function declaration cannot become a multiversioned function after first "
   "usage">;

lib/AST/ExprConstant.cpp

                      diag::note_constexpr_baa_value_insufficient_alignment))
         << (int)OffsetResult.Offset.getQuantity()
         << (unsigned)Align.getQuantity();
       return false;
     }
 
     return true;
   }
-
+  case Builtin::BI__builtin_launder:
+    return evaluatePointer(E->getArg(0), Result);
   case Builtin::BIstrchr:
   case Builtin::BIwcschr:
   case Builtin::BImemchr:
   case Builtin::BIwmemchr:
     if (Info.getLangOpts().CPlusPlus11)
       Info.CCEDiag(E, diag::note_constexpr_invalid_function)
         << /*isConstexpr*/0 << /*isConstructor*/0
         << (std::string("'") + Info.Ctx.BuiltinInfo.getName(BuiltinOp) + "'");

lib/CodeGen/CGBuiltin.cpp

   CGF.Builder.CreateStore(CGF.EmitToMemory(Result, ResultQTy), ResultPtr,
                           isVolatile);
   return RValue::get(Overflow);
 }
 
 RValue CodeGenFunction::EmitBuiltinExpr(const FunctionDecl *FD,
                                         unsigned BuiltinID, const CallExpr *E,
                                         ReturnValueSlot ReturnValue) {
   // See if we can constant fold this builtin.  If so, don't emit it at all.
   Expr::EvalResult Result;
   if (E->EvaluateAsRValue(Result, CGM.getContext()) &&
       !Result.hasSideEffects()) {
     if (Result.Val.isInt())
       return RValue::get(llvm::ConstantInt::get(getLLVMContext(),
                                                 Result.Val.getInt()));
     if (Result.Val.isFloat())
       return RValue::get(llvm::ConstantFP::get(getLLVMContext(),
                                                Result.Val.getFloat()));
   }

     case Builtin::BIfmin:
     case Builtin::BIfminf:
     case Builtin::BIfminl:
     case Builtin::BI__builtin_fmin:
     case Builtin::BI__builtin_fminf:
     case Builtin::BI__builtin_fminl:
       return RValue::get(emitBinaryBuiltin(*this, E, Intrinsic::minnum));
 
     // fmod() is a special-case. It maps to the frem instruction rather than an
     // LLVM intrinsic.
     case Builtin::BIfmod:
     case Builtin::BIfmodf:
     case Builtin::BIfmodl:
     case Builtin::BI__builtin_fmod:
     case Builtin::BI__builtin_fmodf:
     case Builtin::BI__builtin_fmodl: {
       Value *Arg1 = EmitScalarExpr(E->getArg(0));

     // FIXME: Technically these constants should of type 'int', yes?
     RW = (E->getNumArgs() > 1) ? EmitScalarExpr(E->getArg(1)) :
       llvm::ConstantInt::get(Int32Ty, 0);
     Locality = (E->getNumArgs() > 2) ? EmitScalarExpr(E->getArg(2)) :
       llvm::ConstantInt::get(Int32Ty, 3);
     Value *Data = llvm::ConstantInt::get(Int32Ty, 1);
     Value *F = CGM.getIntrinsic(Intrinsic::prefetch);
     return RValue::get(Builder.CreateCall(F, {Address, RW, Locality, Data}));
   }
   case Builtin::BI__builtin_readcyclecounter: {
     Value *F = CGM.getIntrinsic(Intrinsic::readcyclecounter);
     return RValue::get(Builder.CreateCall(F));
   }
   case Builtin::BI__builtin___clear_cache: {
     Value *Begin = EmitScalarExpr(E->getArg(0));
     Value *End = EmitScalarExpr(E->getArg(1));
     Value *F = CGM.getIntrinsic(Intrinsic::clear_cache);
     return RValue::get(Builder.CreateCall(F, {Begin, End}));
   }
   case Builtin::BI__builtin_trap:
     return RValue::get(EmitTrapCall(Intrinsic::trap));
   case Builtin::BI__debugbreak:
     return RValue::get(EmitTrapCall(Intrinsic::debugtrap));
   case Builtin::BI__builtin_unreachable: {
     EmitUnreachable(E->getExprLoc());
 
     // We do need to preserve an insertion point.
     EmitBlock(createBasicBlock("unreachable.cont"));
 
     return RValue::get(nullptr);
   }
 
   case Builtin::BI__builtin_powi:
   case Builtin::BI__builtin_powif:
   case Builtin::BI__builtin_powil: {
     Value *Base = EmitScalarExpr(E->getArg(0));
     Value *Exponent = EmitScalarExpr(E->getArg(1));
     llvm::Type *ArgType = Base->getType();
     Value *F = CGM.getIntrinsic(Intrinsic::powi, ArgType);
     return RValue::get(Builder.CreateCall(F, {Base, Exponent}));
   }
 
   case Builtin::BI__builtin_isgreater:

 
   case Builtin::BI__builtin___memcpy_chk: {
     // fold __builtin_memcpy_chk(x, y, cst1, cst2) to memcpy iff cst1<=cst2.
     llvm::APSInt Size, DstSize;
     if (!E->getArg(2)->EvaluateAsInt(Size, CGM.getContext()) ||
         !E->getArg(3)->EvaluateAsInt(DstSize, CGM.getContext()))
       break;
     if (Size.ugt(DstSize))
       break;
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
     Builder.CreateMemCpy(Dest, Src, SizeVal, false);
     return RValue::get(Dest.getPointer());
   }
 
   case Builtin::BI__builtin_objc_memmove_collectable: {

     // longjmp doesn't return; mark this as unreachable.
     Builder.CreateUnreachable();
 
     // We do need to preserve an insertion point.
     EmitBlock(createBasicBlock("longjmp.cont"));
 
     return RValue::get(nullptr);
   }
+  case Builtin::BI__builtin_launder: {
+    const Expr *Arg = E->getArg(0);
+    QualType ArgTy = Arg->getType()->getPointeeType();
+    Value *Ptr = EmitScalarExpr(Arg);
+    const auto *Record = ArgTy->getAsCXXRecordDecl();
+    if (CGM.getCodeGenOpts().StrictVTablePointers && Record &&
+        Record->isDynamicClass())
+      Ptr = Builder.CreateInvariantGroupBarrier(Ptr);
+
+    return RValue::get(Ptr);
+  }
   case Builtin::BI__sync_fetch_and_add:
   case Builtin::BI__sync_fetch_and_sub:
   case Builtin::BI__sync_fetch_and_or:
   case Builtin::BI__sync_fetch_and_and:
   case Builtin::BI__sync_fetch_and_xor:
   case Builtin::BI__sync_fetch_and_nand:
   case Builtin::BI__sync_add_and_fetch:
   case Builtin::BI__sync_sub_and_fetch:

   case Builtin::BI__sync_fetch_and_xor_2:
   case Builtin::BI__sync_fetch_and_xor_4:
   case Builtin::BI__sync_fetch_and_xor_8:
   case Builtin::BI__sync_fetch_and_xor_16:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Xor, E);
   case Builtin::BI__sync_fetch_and_nand_1:
   case Builtin::BI__sync_fetch_and_nand_2:
   case Builtin::BI__sync_fetch_and_nand_4:
   case Builtin::BI__sync_fetch_and_nand_8:
   case Builtin::BI__sync_fetch_and_nand_16:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Nand, E);
 
   // Clang extensions: not overloaded yet.
   case Builtin::BI__sync_fetch_and_min:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Min, E);
   case Builtin::BI__sync_fetch_and_max:
     return EmitBinaryAtomic(*this, llvm::AtomicRMWInst::Max, E);
   case Builtin::BI__sync_fetch_and_umin:

lib/Sema/SemaChecking.cpp

default:
     llvm_unreachable("Invalid builtin function");
   }
   Call->setType(S.Context.getPointerType(S.Context.getQualifiedType(
       RT.getUnqualifiedType(), Qual)));
 
   return false;
 }
 
+static ExprResult SemaBuiltinLaunder(Sema &S, CallExpr *TheCall) {
+  if (checkArgCount(S, TheCall, 1))
+    return ExprError();
+
+  QualType ArgTy = TheCall->getArg(0)->getType();
+  TheCall->setType(ArgTy);
+
+  auto DiagArg = [&](unsigned DiagID) {
+    S.Diag(TheCall->getLocStart(), DiagID)
+      << TheCall->getSourceRange();
+    return ExprError();
+  };
+
+  if (!ArgTy->isPointerType())
+    return DiagArg(diag::err_builtin_launder_non_pointer_arg);
+  else if (ArgTy->isFunctionPointerType())
+    return DiagArg(diag::err_builtin_launder_function_pointer_arg);
+  else if (ArgTy->isVoidPointerType())
+    return DiagArg(diag::err_builtin_launder_void_pointer_arg);
+
+  assert(ArgTy->getPointeeType()->isObjectType() && "Unhandled non-object pointer case");
+
+  InitializedEntity Entity =
+      InitializedEntity::InitializeParameter(S.Context, ArgTy, false);
+  ExprResult Arg = TheCall->getArg(0);
+  Arg = S.PerformCopyInitialization(Entity, SourceLocation(), Arg);
+  if (Arg.isInvalid())
+    return ExprError();
+  TheCall->setArg(0, Arg.get());
+
+  return TheCall;
+}
+
 ExprResult
 Sema::CheckBuiltinFunctionCall(FunctionDecl *FDecl, unsigned BuiltinID,
                                CallExpr *TheCall) {
   ExprResult TheCallResult(TheCall);
 
   // Find out if any arguments are required to be integer constant expressions.
   unsigned ICEArguments = 0;
   ASTContext::GetBuiltinTypeError Error;
Sema::CheckBuiltinFunctionCall(FunctionDecl *FDecl, unsigned BuiltinID,
   case Builtin::BI__builtin_classify_type:
     if (checkArgCount(*this, TheCall, 1)) return true;
     TheCall->setType(Context.IntTy);
     break;
   case Builtin::BI__builtin_constant_p:
     if (checkArgCount(*this, TheCall, 1)) return true;
     TheCall->setType(Context.IntTy);
     break;
+  case Builtin::BI__builtin_launder:
+    return SemaBuiltinLaunder(*this, TheCall);
   case Builtin::BI__sync_fetch_and_add:
   case Builtin::BI__sync_fetch_and_add_1:
   case Builtin::BI__sync_fetch_and_add_2:
   case Builtin::BI__sync_fetch_and_add_4:
   case Builtin::BI__sync_fetch_and_add_8:
   case Builtin::BI__sync_fetch_and_add_16:
   case Builtin::BI__sync_fetch_and_sub:
   case Builtin::BI__sync_fetch_and_sub_1:

test/CodeGen/builtins.c

#define V(n,args) p(#n #args, (__builtin_##n args, 0))
   P(bitreverse64, (N));
 
   // FIXME
   // V(clear_cache, (&N, &N+1));
   V(trap, ());
   R(extract_return_addr, (&N));
   P(signbit, (1.0));
 
+  R(launder, (&N));
+
   return 0;
 }
 
 
 
 void foo() {
  __builtin_strcat(0, 0);
 }

 #endif
 
 // CHECK-LABEL: define i64 @test_builtin_readcyclecounter
 long long test_builtin_readcyclecounter() {
   // CHECK: call i64 @llvm.readcyclecounter()
   return __builtin_readcyclecounter();
 }
 
+/// __builtin_launder should be a NOP in C since there are no vtables.
+// CHECK-LABEL: define void @test_builtin_launder
+void test_builtin_launder(int *p) {
+  // CHECK: entry
+  // CHECK-NEXT: %p.addr = alloca i32*
+  // CHECK-NEXT: %d = alloca i32*
+  // CHECK-NEXT: store i32* %p, i32** %p.addr, align 8
+  // CHECK-NEXT: [[TMP:%.*]] = load i32*, i32** %p.addr
+  // CHECK-NEXT: store i32* [[TMP]], i32** %d
+  int *d = __builtin_launder(p);
+}
+
 // Behavior of __builtin_os_log differs between platforms, so only test on X86
 #ifdef __x86_64__
 
 // CHECK-LABEL: define void @test_builtin_os_log
 // CHECK: (i8* %[[BUF:.*]], i32 %[[I:.*]], i8* %[[DATA:.*]])
 void test_builtin_os_log(void *buf, int i, const char *data) {
   volatile int len;
   // CHECK: %[[BUF_ADDR:.*]] = alloca i8*, align 8

test/CodeGenCXX/builtin-launder.cpp

+// RUN: %clang_cc1 -triple=x86_64-linux-gnu -emit-llvm -fstrict-vtable-pointers -o - %s \
+// RUN: | FileCheck --check-prefixes=CHECK,CHECK-STRICT %s
+// RUN: %clang_cc1 -triple=x86_64-linux-gnu -emit-llvm -o - %s \
+// RUN: | FileCheck --check-prefixes=CHECK,CHECK-NONSTRICT %s
+
+//===----------------------------------------------------------------------===//
+//                            Positive Cases
+//===----------------------------------------------------------------------===//
+
+struct TestVirtualFn {
+  virtual void foo() {}
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_virtual_fn
+extern "C" void test_builtin_launder_virtual_fn(TestVirtualFn *p) {
+  // CHECK: entry
+  // CHECK-NEXT: %p.addr = alloca [[TYPE:%.*]], align 8
+  // CHECK-NEXT: %d = alloca [[TYPE]]
+  // CHECK-NEXT: store [[TYPE]] %p, [[TYPE]]* %p.addr
+  // CHECK-NEXT: [[TMP0:%.*]] = load [[TYPE]], [[TYPE]]* %p.addr
+
+  // CHECK-NONSTRICT-NEXT: store [[TYPE]] [[TMP0]], [[TYPE]]* %d
+
+  // CHECK-STRICT-NEXT: [[TMP1:%.*]] = bitcast [[TYPE]] [[TMP0]] to i8*
+  // CHECK-STRICT-NEXT: [[TMP2:%.*]] = call i8* @llvm.invariant.group.barrier.p0i8(i8* [[TMP1]])
+  // CHECK-STRICT-NEXT: [[TMP3:%.*]] = bitcast i8* [[TMP2]] to [[TYPE]]
+  // CHECK-STRICT-NEXT: store [[TYPE]] [[TMP3]], [[TYPE]]* %d
+
+  // CHECK-NEXT: ret void
+  TestVirtualFn *d = __builtin_launder(p);
+}
+
+struct TestPolyBase : TestVirtualFn {
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_poly_base
+extern "C" void test_builtin_launder_poly_base(TestPolyBase *p) {
+  // CHECK-STRICT-NOT: ret void
+  // CHECK-STRICT: @llvm.invariant.group.barrier
+
+  // CHECK-NONSTRICT-NOT: @llvm.invariant.group.barrier
+
+  // CHECK: ret void
+  TestPolyBase *d = __builtin_launder(p);
+}
+
+struct TestBase {};
+struct TestVirtualBase : virtual TestBase {};
+
+// CHECK-LABEL: define void @test_builtin_launder_virtual_base
+extern "C" void test_builtin_launder_virtual_base(TestVirtualBase *p) {
+  // CHECK-STRICT-NOT: ret void
+  // CHECK-STRICT: @llvm.invariant.group.barrier
+
+  // CHECK-NONSTRICT-NOT: @llvm.invariant.group.barrier
+
+  // CHECK: ret void
+  TestVirtualBase *d = __builtin_launder(p);
+}
+
+//===----------------------------------------------------------------------===//
+//                            Negative Cases
+//===----------------------------------------------------------------------===//
+
+// CHECK-LABEL: define void @test_builtin_launder_ommitted_one
+extern "C" void test_builtin_launder_ommitted_one(int *p) {
+  // CHECK: entry
+  // CHECK-NEXT: %p.addr = alloca i32*
+  // CHECK-NEXT: %d = alloca i32*
+  // CHECK-NEXT: store i32* %p, i32** %p.addr, align 8
+  // CHECK-NEXT: [[TMP:%.*]] = load i32*, i32** %p.addr
+  // CHECK-NEXT: store i32* [[TMP]], i32** %d
+  // CHECK-NEXT: ret void
+  int *d = __builtin_launder(p);
+}
+
+struct TestNoInvariant {
+  int x;
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_ommitted_two
+extern "C" void test_builtin_launder_ommitted_two(TestNoInvariant *p) {
+  // CHECK: entry
+  // CHECK-NEXT: %p.addr = alloca [[TYPE:%.*]], align 8
+  // CHECK-NEXT: %d = alloca [[TYPE]]
+  // CHECK-NEXT: store [[TYPE]] %p, [[TYPE]]* %p.addr
+  // CHECK-NEXT: [[TMP:%.*]] = load [[TYPE]], [[TYPE]]* %p.addr
+  // CHECK-NEXT: store [[TYPE]] [[TMP]], [[TYPE]]* %d
+  // CHECK-NEXT: ret void
+  TestNoInvariant *d = __builtin_launder(p);
+}
+
+/// The test cases in this namespace technically need to be laundered according
+/// to the language in the standard (ie they have const or reference subobjects)
+/// but LLVM doesn't currently optimize on these cases -- so Clang emits
+/// __builtin_launder as a nop.
+namespace pessimizing_cases {
+
+struct TestConstMember {
+  const int x;
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_const_member
+extern "C" void test_builtin_launder_const_member(TestConstMember *p) {
+  // CHECK: entry
+  // CHECK-NOT: @llvm.invariant.group.barrier
+  // CHECK: ret void
+  TestConstMember *d = __builtin_launder(p);
+}
+
+struct TestConstSubobject {
+  TestConstMember x;
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_const_subobject
+extern "C" void test_builtin_launder_const_subobject(TestConstSubobject *p) {
+  // CHECK: entry
+  // CHECK-NOT: @llvm.invariant.group.barrier
+  // CHECK: ret void
+  TestConstSubobject *d = __builtin_launder(p);
+}
+
+struct TestConstObject {
+  const struct TestConstMember x;
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_const_object
+extern "C" void test_builtin_launder_const_object(TestConstObject *p) {
+  // CHECK: entry
+  // CHECK-NOT: @llvm.invariant.group.barrier
+  // CHECK: ret void
+  TestConstObject *d = __builtin_launder(p);
+}
+
+struct TestReferenceMember {
+  int &x;
+};
+
+// CHECK-LABEL: define void @test_builtin_launder_reference_member
+extern "C" void test_builtin_launder_reference_member(TestReferenceMember *p) {
+  // CHECK: entry
+  // CHECK-NOT: @llvm.invariant.group.barrier
+  // CHECK: ret void
+  TestReferenceMember *d = __builtin_launder(p);
+}
+
+} // namespace pessimizing_cases

test/Preprocessor/feature_tests.c

 // RUN: %clang_cc1 %s -triple=i686-apple-darwin9 -verify -DVERIFY
 // RUN: %clang_cc1 %s -E -triple=i686-apple-darwin9
 #ifndef __has_feature
 #error Should have __has_feature
 #endif
 
 
 #if __has_feature(something_we_dont_have)
 #error Bad
 #endif
 
 #if  !__has_builtin(__builtin_huge_val) || \
      !__has_builtin(__builtin_shufflevector) || \
      !__has_builtin(__builtin_convertvector) || \
      !__has_builtin(__builtin_trap) || \
      !__has_builtin(__c11_atomic_init) || \
+     !__has_builtin(__builtin_launder) || \
      !__has_feature(attribute_analyzer_noreturn) || \
      !__has_feature(attribute_overloadable)
 #error Clang should have these
 #endif
 
 #if __has_builtin(__builtin_insanity)
 #error Clang should not have this
 #endif

test/Sema/builtins.c

char * Test20(char *p, const char *in, unsigned n)
     __builtin___memcpy_chk (&buf[5], "abcde", 5, __builtin_object_size (&buf[5], 0));
 
     __builtin___memcpy_chk (&buf[5], "abcde", n, __builtin_object_size (&buf[5], 0));
 
     __builtin___memcpy_chk (&buf[6], "abcde", 5, __builtin_object_size (&buf[6], 0)); // expected-warning {{'__builtin___memcpy_chk' will always overflow destination buffer}}
 
     return buf;
 }
+
+typedef void (fn_t)(int);
+
+void test_builtin_launder(char *p, void *vp, const void *cvp,
+                          const volatile int *ip, float *restrict fp,
+                          fn_t *fn) {
+  __builtin_launder(); // expected-error {{too few arguments to function call, expected 1, have 0}}
+  __builtin_launder(p, p); // expected-error {{too many arguments to function call, expected 1, have 2}}
+  int x;
+  __builtin_launder(x); // expected-error {{non-pointer argument to '__builtin_launder'}}
+  char *d = __builtin_launder(p);
+  __builtin_launder(vp); // expected-error {{argument to '__builtin_launder' cannot be a void pointer}}
+  __builtin_launder(cvp); // expected-error {{argument to '__builtin_launder' cannot be a void pointer}}
+  const volatile int *id = __builtin_launder(ip);
+  int *id2 = __builtin_launder(ip); // expected-warning {{discards qualifiers}}
+  float *fd = __builtin_launder(fp);
+  __builtin_launder(fn); // expected-error {{argument to '__builtin_launder' cannot be a function pointer}}
+}

test/SemaCXX/builtins.cpp

 
 struct FILE;
 extern "C" int vfprintf(FILE *__restrict, const char *__restrict,
                         __builtin_va_list va);
 
 void synchronize_args() {
   __sync_synchronize(0); // expected-error {{too many arguments}}
 }
+
+namespace test_launder {
+
+struct Dummy {};
+
+using FnType = int(char);
+using MemFnType = int (Dummy::*)(char);
+using ConstMemFnType = int (Dummy::*)() const;
+
+void foo() {}
+
+void test_builtin_launder_diags(void *vp, const void *cvp, FnType *fnp,
+                                MemFnType mfp, ConstMemFnType cmfp) {
+  __builtin_launder(vp);   // expected-error {{argument to '__builtin_launder' cannot be a void pointer}}
+  __builtin_launder(cvp);  // expected-error {{argument to '__builtin_launder' cannot be a void pointer}}
+  __builtin_launder(fnp);  // expected-error {{argument to '__builtin_launder' cannot be a function pointer}}
+  __builtin_launder(mfp);  // expected-error {{non-pointer argument to '__builtin_launder'}}
+  __builtin_launder(cmfp); // expected-error {{non-pointer argument to '__builtin_launder'}}
+  (void)__builtin_launder(&fnp);
+  __builtin_launder(42);      // expected-error {{non-pointer argument to '__builtin_launder'}}
+  __builtin_launder(nullptr); // expected-error {{non-pointer argument to '__builtin_launder'}}
+  __builtin_launder(foo) // expected-error {{non-pointer argument to '__builtin_launder'}}
+}
+
+void test_builtin_launder(char *p, const volatile int *ip, const float *&fp,
+                          double *__restrict dp) {
+  int x;
+  __builtin_launder(x); // expected-error {{non-pointer argument to '__builtin_launder'}}
+#define TEST_TYPE(Ptr, Type) \
+  static_assert(__is_same(decltype(__builtin_launder(Ptr)), Type), "expected same type")
+  TEST_TYPE(p, char*);
+  TEST_TYPE(ip, const volatile int*);
+  TEST_TYPE(fp, const float*);
+  TEST_TYPE(dp, double *__restrict);
+#undef TEST_TYPE
+  char *d = __builtin_launder(p);
+  const volatile int *id = __builtin_launder(ip);
+  int *id2 = __builtin_launder(ip); // expected-error {{cannot initialize a variable of type 'int *' with an rvalue of type 'const volatile int *'}}
+  const float* fd = __builtin_launder(fp);
+}
+
+template <class Tp>
+constexpr Tp *test_constexpr_launder(Tp *tp) {
+  return __builtin_launder(tp);
+}
+constexpr int const_int = 42;
+constexpr int const_int2 = 101;
+constexpr const int *const_ptr = test_constexpr_launder(&const_int);
+static_assert(&const_int == const_ptr, "");
+static_assert(const_ptr != test_constexpr_launder(&const_int2), "");
+
+void test_non_constexpr() {
+  constexpr int i = 42;                            // expected-note {{declared here}}
+  constexpr const int *ip = __builtin_launder(&i); // expected-error {{constexpr variable 'ip' must be initialized by a constant expression}}
+  // expected-note@-1 {{pointer to 'i' is not a constant expression}}
+}
+
+constexpr bool test_in_constexpr(const int &i) {
+  return (__builtin_launder(&i) == &i);
+}
+static_assert(test_in_constexpr(const_int), "");
+void f() {
+  constexpr int i = 42;
+  // FIXME: Should this work? Since `&i` doesn't.
+  static_assert(test_in_constexpr(i), "");
+}
+
+} // end namespace test_launder