[Clang] Add __builtin_launder

[EricWF]

Summary

This patch adds __builtin_launder, which is required to implement std::launder. Additionally GCC provides __builtin_launder, so thing brings Clang in-line with GCC.

I'm not exactly sure what magic __builtin_launder requires, but based on previous discussions this patch applies a @llvm.invariant.group.barrier. As noted in previous discussions, this may not be enough to correctly handle vtables.

[EricWF]

• Remove incorrect FIXME comment.

[majnemer]

A test with restrict and __restrict might be interesting.

[jroelofs]

include/clang/Basic/Builtins.def

480

GCC's is type-generic:

#include <type_traits>

bool is_type_generic() {
  int v;
  return std::is_same<decltype(__builtin_launder(&v)),
                      decltype(&v)>::value;
}

However this BUILTIN line says that this one is not (yeah, I see it's being fixed up in Sema later in this patch). I think you need to use custom type checking here via t.

[EricWF]

include/clang/Basic/Builtins.def

480

The t is specified. It's in the third argument.

[jroelofs]

include/clang/Basic/Builtins.def

480

ohhh. duh. sorry.

[EricWF]

• Fix argument initialization.
• Make constexpr.

[EricWF]

• Improve quality of tests.
• Format code.

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1674

It would be nice to avoid this for types that contain no const subobjects / reference subobjects / vptrs. I think we can also omit this entirely if -fstrict-vtable-ptrs is disabled, since in that case we don't generate any invariant.group metadata.

I'd be OK with the former being left to a future change, but the latter should be part of this change so we don't generate unnecessarily-inefficient code in the default mode for uses of std::launder.

lib/Sema/SemaChecking.cpp

860–864

Please also check that the pointee type is an object type -- per [ptr.launder]p3, "the program is ill-formed if T is a function type or cv void", and we don't want our builtin to need to deal with such cases.

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1674

I'll put it in this change set.

[EricWF]

• Address inline comments about missing diagnostics for void pointers and function pointers.
• Address inline comments about only enabling when -fstrict-vtable-pointers is specified, and only on types with vtables.

[EricWF]

• Improve diagnostic handling.

[EricWF]

Ping.

[EricWF]

• Rebase off master.

[EricWF]

Ping.

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1947–1948

I think you also need to catch class types that contain dynamic classes as subobjects.

lib/Sema/SemaChecking.cpp

860

Might be a bit clearer to use llvm::Optional<unsigned> as the type of this.

test/CodeGenCXX/builtin-launder.cpp

93–96 ↗

(On Diff #134986)

I would note that this means adding optimizations for those cases later is an LTO ABI break. That's probably OK, but just something we're going to need to remember.

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1947–1948

Only by-value subobjects, or also reference subobjects?

[EricWF]

test/CodeGenCXX/builtin-launder.cpp

93–96 ↗

(On Diff #134986)

I added your note almost verbatim to the test case.

[EricWF]

• Launder types with subobjects of dynamic class type.
• Improve diagnostic selection using llvm::Optional<unsigned>.
• Add comment about LTO ABI concerns to test file.
• Merge with master.

[EricWF]

Ping.

[rsmith]

LGTM with a fix (and test) for pointer-to-array-of-dynamic-class-type handling.

lib/CodeGen/CGBuiltin.cpp

936–938

I think you also need to recurse through array types (use Ty->getBaseElementTypeUnsafe()->getAsCXXRecordDecl() or similar).

[rsmith]

Also we probably want to hold off on landing this until PR37458 is fixed, otherwise std::launder will start miscompiling code.

[EricWF]

• Handle array types as requested.

• Attempt to handle incomplete class types. They are always considered in need of laundering. Including class templates who's instantiation hasn't been forced yet. @rsmith is this the correct thing to do?

[erichkeane]

Bump! I don't see anything here that seems questionable (and this has extensive testing), but I presume @rsmith should be the final one to approve.

[Romain-Geissler-1A]

Hi,

Is there any news on this code review ? Is it ready to land ?

Cheers,
Romain

[EricWF]

Merge with upstream.

[EricWF]

In D40218#1298943, @Romain-Geissler-1A wrote:

Is there any news on this code review ? Is it ready to land ?

I think so, but there were some changes regarding incomplete types since Richard last looked at it.
I wanted him to sign off on the new behavior.

lib/CodeGen/CGBuiltin.cpp

1037

@rsmith I believe this changed since you approved the patch.

We talked about this offline, but could you give it a thumbs up?

[rsmith]

lib/CodeGen/CGBuiltin.cpp

1425–1426

I think it's a bug that launder doesn't require T to be a complete type. Can you file an LWG issue?

We should also decide whether we want to proactively fix this issue (require the type to be complete from the Sema checking of the builtin and assert that it's defined here) or not.

1437–1438

Delete this commented-out code.

1451

Would SmallPtrSet be a better choice here?

lib/Sema/SemaChecking.cpp

853

Did you mean to add this blank line?

858–860

Instead of performing some of the conversions here and some of them as part of initialization, I think it'd be more obvious to compute the builtin's parameter type here (which is the type of the argument if it's not of array [or function] type, and the decayed type of the argument otherwise), and do the decay and lvalue-to-rvalue conversion as part of the parameter initialization below.

The current code arrangement (and especially this comment) leaves a reader thinking "but you *need* an lvalue-to-rvalue conversion if the argument is an lvalue".

test/CodeGen/builtins.c

404–409

This test is not robust against minor IR differences such as variable or basic block names changing (some of these change in a release build), and is testing things that are not related to this builtin (eg, that we produce an alloca for a function parameter and its relative order to an alloca for a local variable).

I would remove everything here other than the load and the store, and add an explicit check that we don't generate a launder call:

// CHECK: [[TMP:%.*]] = load i32*,
// CHECK-NOT: @llvm.launder
// CHECK: store i32* [[TMP]],

test/CodeGenCXX/builtin-launder.cpp

16 ↗

(On Diff #174109)

This is likewise likely to fail with a release build of clang.

test/SemaCXX/builtins.cpp

120

&i doesn't what?

[EricWF]

lib/CodeGen/CGBuiltin.cpp

1425–1426

Apparently I misread the specification. The blanket wording in [res.on.functions] already prohibits this.

Though it's worth noting that GCC doesn't enforce this requirement (nor the one about function pointers).

1451

I have no idea. I'm ignorant to the internals of both containers. Since you asked the question I'll assume it is and make the change.
If you would like, I can do further investigation upon request.

lib/Sema/SemaChecking.cpp

853

Oh Richard, you're too kind. Of course I didn't mean to do that.

858–860

Ack.

I think I've implemented what you requested. Could you please verify?

test/CodeGen/builtins.c

404–409

If you're referring to the behavior of discarding value names, Clang CC1 only does that if -discard-value-names is explicitly passed, regardless of how the compiler is built, and so the names should be stable. (The Clang driver still conditionally passes -discard-value-names in release builds).

Are there other IR changes you have in mind?

None the less, I'll simply this test as requested.

test/CodeGenCXX/builtin-launder.cpp

16 ↗

(On Diff #174109)

I just verified that they pass in both release and debug builds. I previously changed Clang and LLVM to make the label names consistent between the two configurations for the purposes of testing.

I'll still attempt to clean unnecessary checks up.

test/SemaCXX/builtins.cpp

120

That's a great question. Not sure what I was up to when I wrote that. I tried a couple of things I though I might have been thinking, but they all passed.

Removing the comment.

[EricWF]

Address review comments.

@rsmith I think this is good to go. Just need your input on one change.

[EricWF]

Merging with upstream. Preparing to commit.

include/clang/Basic/Builtins.def

 BUILTIN(__builtin_setjmp, "iv**", "j")
 BUILTIN(__builtin_longjmp, "vv**i", "r")
 BUILTIN(__builtin_unwind_init, "v", "")
 BUILTIN(__builtin_eh_return_data_regno, "iIi", "nc")
 BUILTIN(__builtin_snprintf, "ic*zcC*.", "nFp:2:")
 BUILTIN(__builtin_vsprintf, "ic*cC*a", "nFP:1:")
 BUILTIN(__builtin_vsnprintf, "ic*zcC*a", "nFP:2:")
 BUILTIN(__builtin_thread_pointer, "v*", "nc")
+BUILTIN(__builtin_launder, "v*v*", "nt")
 
 // GCC exception builtins
 BUILTIN(__builtin_eh_return, "vzv*", "r") // FIXME: Takes intptr_t, not size_t!
 BUILTIN(__builtin_frob_return_addr, "v*v*", "n")
 BUILTIN(__builtin_dwarf_cfa, "v*", "n")
 BUILTIN(__builtin_init_dwarf_reg_size_table, "vv*", "n")
 BUILTIN(__builtin_dwarf_sp_column, "Ui", "n")
 BUILTIN(__builtin_extend_pointer, "ULLiv*", "n") // _Unwind_Word == uint64_t

include/clang/Basic/DiagnosticSemaKinds.td

   "__final is a GNU extension, consider using C++11 final">,
   InGroup<GccCompat>;
 
 def warn_shadow_field :
   Warning<"non-static data member '%0' of '%1' shadows member inherited from type '%2'">,
   InGroup<ShadowField>, DefaultIgnore;
 def note_shadow_field : Note<"declared here">;
 
+def err_builtin_launder_non_pointer_arg :
+  Error<"non-pointer argument to '__builtin_launder'">;
+
 } // end of sema component.

lib/CodeGen/CGBuiltin.cpp

     return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::trunc));
   }
   case Builtin::BI__builtin_rint:
   case Builtin::BI__builtin_rintf:
   case Builtin::BI__builtin_rintl: {
     return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::rint));
   }
   case Builtin::BI__builtin_nearbyint:
   case Builtin::BI__builtin_nearbyintf:
   case Builtin::BI__builtin_nearbyintl: {
     return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::nearbyint));
   }
   case Builtin::BI__builtin_round:
   case Builtin::BI__builtin_roundf:
   case Builtin::BI__builtin_roundl: {
     return RValue::get(emitUnaryBuiltin(*this, E, Intrinsic::round));
   }
   case Builtin::BI__builtin_fmin:
   case Builtin::BI__builtin_fminf:

     llvm::Type *ArgType = ArgValue->getType();
     Value *F = CGM.getIntrinsic(Intrinsic::cttz, ArgType);
 
     llvm::Type *ResultType = ConvertType(E->getType());
     Value *Tmp =
         Builder.CreateAdd(Builder.CreateCall(F, {ArgValue, Builder.getTrue()}),
                           llvm::ConstantInt::get(ArgType, 1));
     Value *Zero = llvm::Constant::getNullValue(ArgType);
     Value *IsZero = Builder.CreateICmpEQ(ArgValue, Zero, "iszero");
     Value *Result = Builder.CreateSelect(IsZero, Zero, Tmp, "ffs");
     if (Result->getType() != ResultType)
       Result = Builder.CreateIntCast(Result, ResultType, /*isSigned*/true,
                                      "cast");
     return RValue::get(Result);
   }
   case Builtin::BI__builtin_parity:
   case Builtin::BI__builtin_parityl:

     Value *Size = EmitScalarExpr(E->getArg(0));
     Value *AlignmentInBitsValue = EmitScalarExpr(E->getArg(1));
     auto *AlignmentInBitsCI = cast<ConstantInt>(AlignmentInBitsValue);
     unsigned AlignmentInBits = AlignmentInBitsCI->getZExtValue();
     unsigned AlignmentInBytes =
         CGM.getContext().toCharUnitsFromBits(AlignmentInBits).getQuantity();
     AllocaInst *AI = Builder.CreateAlloca(Builder.getInt8Ty(), Size);
     AI->setAlignment(AlignmentInBytes);
     return RValue::get(AI);
   }
 
   case Builtin::BIbzero:
   case Builtin::BI__builtin_bzero: {
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Value *SizeVal = EmitScalarExpr(E->getArg(1));
     EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
     Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
     return RValue::get(nullptr);
   }
   case Builtin::BImemcpy:
   case Builtin::BI__builtin_memcpy: {
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
     EmitNonNullArgCheck(RValue::get(Src.getPointer()), E->getArg(1)->getType(),
                         E->getArg(1)->getExprLoc(), FD, 1);
     Builder.CreateMemCpy(Dest, Src, SizeVal, false);
     return RValue::get(Dest.getPointer());
   }
 
   case Builtin::BI__builtin_char_memchr:
     BuiltinID = Builtin::BI__builtin_memchr;
     break;
 
   case Builtin::BI__builtin___memcpy_chk: {
     // fold __builtin_memcpy_chk(x, y, cst1, cst2) to memcpy iff cst1<=cst2.
     llvm::APSInt Size, DstSize;
     if (!E->getArg(2)->EvaluateAsInt(Size, CGM.getContext()) ||
         !E->getArg(3)->EvaluateAsInt(DstSize, CGM.getContext()))
       break;

     // longjmp doesn't return; mark this as unreachable.
     Builder.CreateUnreachable();
 
     // We do need to preserve an insertion point.
     EmitBlock(createBasicBlock("longjmp.cont"));
 
     return RValue::get(nullptr);
   }
+  case Builtin::BI__builtin_launder: {
+    Value *Ptr = EmitScalarExpr(E->getArg(0));
+    Ptr = Builder.CreateInvariantGroupBarrier(Ptr);
+    return RValue::get(Ptr);
+  }
   case Builtin::BI__sync_fetch_and_add:
   case Builtin::BI__sync_fetch_and_sub:
   case Builtin::BI__sync_fetch_and_or:
   case Builtin::BI__sync_fetch_and_and:
   case Builtin::BI__sync_fetch_and_xor:
   case Builtin::BI__sync_fetch_and_nand:
   case Builtin::BI__sync_add_and_fetch:
   case Builtin::BI__sync_sub_and_fetch:

       RMW->setVolatile(Volatile);
       Result->addIncoming(RMW, BBs[i]);
       Builder.CreateBr(ContBB);
     }
 
     SI->addCase(Builder.getInt32(0), BBs[0]);
     SI->addCase(Builder.getInt32(1), BBs[1]);
     SI->addCase(Builder.getInt32(2), BBs[1]);
     SI->addCase(Builder.getInt32(3), BBs[2]);
     SI->addCase(Builder.getInt32(4), BBs[3]);
     SI->addCase(Builder.getInt32(5), BBs[4]);
 
     Builder.SetInsertPoint(ContBB);
     return RValue::get(Builder.CreateIsNotNull(Result, "tobool"));
   }
 
   case Builtin::BI__atomic_clear: {
     QualType PtrTy = E->getArg(0)->IgnoreImpCasts()->getType();

lib/Sema/SemaChecking.cpp

default:
     llvm_unreachable("Invalid builtin function");
   }
   Call->setType(S.Context.getPointerType(S.Context.getQualifiedType(
       RT.getUnqualifiedType(), Qual)));
 
   return false;
 }
 
+static bool SemaBuiltinLaunder(Sema& S, CallExpr *TheCall) {
+  if (checkArgCount(S, TheCall, 1)) return true;
+  Expr *Arg = TheCall->getArg(0);
+  QualType ArgT = Arg->getType();
+  if (!ArgT->isPointerType()) {
+    S.Diag(TheCall->getLocStart(), diag::err_builtin_launder_non_pointer_arg)
+        << TheCall->getSourceRange();
+    return true;
+  }
+  TheCall->setType(Arg->getType());
+  return false; // OK
+}
+
 ExprResult
 Sema::CheckBuiltinFunctionCall(FunctionDecl *FDecl, unsigned BuiltinID,
                                CallExpr *TheCall) {
   ExprResult TheCallResult(TheCall);
 
   // Find out if any arguments are required to be integer constant expressions.
   unsigned ICEArguments = 0;
   ASTContext::GetBuiltinTypeError Error;
Sema::CheckBuiltinFunctionCall(FunctionDecl *FDecl, unsigned BuiltinID,
   case Builtin::BI__builtin_classify_type:
     if (checkArgCount(*this, TheCall, 1)) return true;
     TheCall->setType(Context.IntTy);
     break;
   case Builtin::BI__builtin_constant_p:
     if (checkArgCount(*this, TheCall, 1)) return true;
     TheCall->setType(Context.IntTy);
     break;
+  case Builtin::BI__builtin_launder:
+    if (SemaBuiltinLaunder(*this, TheCall))
+      return ExprError();
+    break;
   case Builtin::BI__sync_fetch_and_add:
   case Builtin::BI__sync_fetch_and_add_1:
   case Builtin::BI__sync_fetch_and_add_2:
   case Builtin::BI__sync_fetch_and_add_4:
   case Builtin::BI__sync_fetch_and_add_8:
   case Builtin::BI__sync_fetch_and_add_16:
   case Builtin::BI__sync_fetch_and_sub:
   case Builtin::BI__sync_fetch_and_sub_1:

test/CodeGen/builtins.c

#define V(n,args) p(#n #args, (__builtin_##n args, 0))
   P(bitreverse64, (N));
 
   // FIXME
   // V(clear_cache, (&N, &N+1));
   V(trap, ());
   R(extract_return_addr, (&N));
   P(signbit, (1.0));
 
+  R(launder, (&N));
+
   return 0;
 }
 
 
 
 void foo() {
  __builtin_strcat(0, 0);
 }

 #endif
 
 // CHECK-LABEL: define i64 @test_builtin_readcyclecounter
 long long test_builtin_readcyclecounter() {
   // CHECK: call i64 @llvm.readcyclecounter()
   return __builtin_readcyclecounter();
 }
 
+// CHECK-LABEL: define void @test_builtin_launder
+void test_builtin_launder(int *p) {
+  // CHECK: entry
+  // CHECK-NEXT: %p.addr = alloca i32*
+  // CHECK-NEXT: %d = alloca i32*
+  // CHECK-NEXT: store i32* %p, i32** %p.addr, align 8
+  // CHECK-NEXT: [[TMP:%.*]] = load i32*, i32** %p.addr
+  // CHECK-NEXT: [[TMP1:%.*]] = bitcast i32* [[TMP]] to i8*
+  // CHECK-NEXT: [[TMP2:%.*]] = call i8* @llvm.invariant.group.barrier.p0i8(i8* [[TMP1]])
+  // CHECK-NEXT: [[TMP3:%.*]] = bitcast i8* [[TMP2]] to i32*
+  // CHECK-NEXT: store i32* [[TMP3]], i32** %d
+  int *d = __builtin_launder(p);
+}
+
 // Behavior of __builtin_os_log differs between platforms, so only test on X86
 #ifdef __x86_64__
 
 // CHECK-LABEL: define void @test_builtin_os_log
 // CHECK: (i8* %[[BUF:.*]], i32 %[[I:.*]], i8* %[[DATA:.*]])
 void test_builtin_os_log(void *buf, int i, const char *data) {
   volatile int len;
   // CHECK: %[[BUF_ADDR:.*]] = alloca i8*, align 8

test/Preprocessor/feature_tests.c

 // RUN: %clang_cc1 %s -triple=i686-apple-darwin9 -verify -DVERIFY
 // RUN: %clang_cc1 %s -E -triple=i686-apple-darwin9
 #ifndef __has_feature
 #error Should have __has_feature
 #endif
 
 
 #if __has_feature(something_we_dont_have)
 #error Bad
 #endif
 
 #if  !__has_builtin(__builtin_huge_val) || \
      !__has_builtin(__builtin_shufflevector) || \
      !__has_builtin(__builtin_convertvector) || \
      !__has_builtin(__builtin_trap) || \
      !__has_builtin(__c11_atomic_init) || \
+     !__has_builtin(__builtin_launder) || \
      !__has_feature(attribute_analyzer_noreturn) || \
      !__has_feature(attribute_overloadable)
 #error Clang should have these
 #endif
 
 #if __has_builtin(__builtin_insanity)
 #error Clang should not have this
 #endif

test/Sema/builtins.c

char * Test20(char *p, const char *in, unsigned n)
     __builtin___memcpy_chk (&buf[5], "abcde", 5, __builtin_object_size (&buf[5], 0));
 
     __builtin___memcpy_chk (&buf[5], "abcde", n, __builtin_object_size (&buf[5], 0));
 
     __builtin___memcpy_chk (&buf[6], "abcde", 5, __builtin_object_size (&buf[6], 0)); // expected-warning {{'__builtin___memcpy_chk' will always overflow destination buffer}}
 
     return buf;
 }
+
+void test_builtin_launder(char *p, void *vp, const volatile int *ip) {
+  __builtin_launder(); // expected-error {{too few arguments to function call, expected 1, have 0}}
+  __builtin_launder(p, p); // expected-error {{too many arguments to function call, expected 1, have 2}}
+  int x;
+  __builtin_launder(x); // expected-error {{non-pointer argument to '__builtin_launder'}}
+  char *d = __builtin_launder(p);
+  void *vd = __builtin_launder(vp);
+  const volatile int *id = __builtin_launder(ip);
+  int *id2 = __builtin_launder(ip); // expected-warning {{discards qualifiers}}
+}

test/SemaCXX/builtins.cpp

 
 struct FILE;
 extern "C" int vfprintf(FILE *__restrict, const char *__restrict,
                         __builtin_va_list va);
 
 void synchronize_args() {
   __sync_synchronize(0); // expected-error {{too many arguments}}
 }
+
+#define TEST_TYPE(Ptr, Type) \
+static_assert(__is_same(decltype(__builtin_launder(Ptr)), Type), "expected same type")
+
+void test_builtin_launder(char *p, void *vp, const volatile int *ip, const float*& fp) {
+  int x;
+  __builtin_launder(x); // expected-error {{non-pointer argument to '__builtin_launder'}}
+  TEST_TYPE(p, char*);
+  TEST_TYPE(vp, void*);
+  TEST_TYPE(ip, const volatile int*);
+  TEST_TYPE(fp, const float*);
+  char *d = __builtin_launder(p);
+  void *vd = __builtin_launder(vp);
+  const volatile int *id = __builtin_launder(ip);
+  int *id2 = __builtin_launder(ip); // expected-error {{cannot initialize a variable of type 'int *' with an rvalue of type 'const volatile int *'}}
+  const float* fd = __builtin_launder(fp);
+}
+
+#undef TEST_TYPE