[clang] Add support for __builtin_memcpy_inline

[gchatelet]

Summary

This is a follow up on D61634 and the last step to implement http://lists.llvm.org/pipermail/llvm-dev/2019-April/131973.html

[arichardson]

clang/docs/LanguageExtensions.rst

2259

Typo: except

clang/test/Sema/builtins-memcpy-inline.c

24

Size can only be a constant right? Should there be a test for the error reported in that case?

[gchatelet]

• Address comments

clang/docs/LanguageExtensions.rst

2259

Thx :)

clang/test/Sema/builtins-memcpy-inline.c

24

Thx it caught a bug indeed :)

[gchatelet]

rebasing

[gchatelet]

@efriedma would you mind having a look?
It should be my last patch on that matter :)

[efriedma]

clang/lib/Sema/SemaChecking.cpp

1653

SemaBuiltinConstantArg. Or actually, you should probably use "I" in Builtins.def.

[gchatelet]

• Address comments

[gchatelet]

Thx!

[efriedma]

clang/docs/LanguageExtensions.rst

2252

This is in the wrong section of the documentation. We could constant-evaluate __builtin_memcpy_inline, I guess, but that isn't the primary purpose, and your patch doesn't implement constant evaluation anyway.

Might make sense to add a new section, if no existing section makes sense.

[efriedma]

clang/lib/Sema/SemaChecking.cpp

1655

You can EvaluateKnownConstInt here, instead of isIntegerConstantExpr.

[gchatelet]

• Address comments

[gchatelet]

clang/docs/LanguageExtensions.rst

2252

I added a new memory builtins section. Let me know what you think.

[gchatelet]

Anything else @efriedma ?

[efriedma]

Sorry about the delayed response, I was traveling.

clang/docs/LanguageExtensions.rst

2229

Not sure putting "memcpy" in this list makes sense. We did add support for constant-evaluating memcpy, but it was separately from the others, so the description of the feature detection is wrong.

2252

Makes sense.

[gchatelet]

• Remove unrelated memcpy in documentation

clang/docs/LanguageExtensions.rst

2229

Actually this is unrelated to this patch. The documentation is missing memcpy as a string builtin. It should go in a separate patch though.

[efriedma]

LGTM

[gchatelet]

Thx a lot !

[melver]

clang/test/Sema/builtins-memcpy-inline.c

8

It appears that the expected-warning check here is guarded by the #if as well. Moving it after the #endif results in a failing test.

I noticed this as I was trying to use has_feature(builtin_memcpy_inline), but it somehow does not work, even though the compiler clearly supports __builtin_memcpy_inline.

Any idea what's wrong with the __has_feature test?

Thanks!

[efriedma]

clang/test/Sema/builtins-memcpy-inline.c

8

Should be __has_builtin, I think? __has_feature only applies to features defined in clang/include/clang/Basic/Features.def.

[gchatelet]

clang/test/Sema/builtins-memcpy-inline.c

8

Thx for noticing. I'll send a patch to update the test.

[gchatelet]

clang/test/Sema/builtins-memcpy-inline.c

8

Submitted as rG12c8e3632edda3bc640867ba326b52b5b2220a57

clang/docs/LanguageExtensions.rst

 more information on the semantics.
 
 String builtins
 ---------------
 
 Clang provides constant expression evaluation support for builtins forms of
 the following functions from the C standard library ``<string.h>`` header:
 
+* ``memcpy``
 * ``memchr``
 * ``memcmp``
 * ``strchr``
 * ``strcmp``
 * ``strlen``
 * ``strncmp``
 * ``wcschr``
 * ``wcscmp``
 * ``wcslen``
 * ``wcsncmp``
 * ``wmemchr``
 * ``wmemcmp``
 
 In each case, the builtin form has the name of the C library function prefixed
 by ``__builtin_``. Example:
 
 .. code-block:: c
 
   void *p = __builtin_memchr("foobar", 'b', 5);
 
-In addition to the above, one further builtin is provided:
+In addition to the above, the following builtins are provided:
+
+ * ``__builtin_memcpy_inline``
+
+.. code-block:: c
+
+  void __builtin_memcpy_inline(void *dst, const void *src, size_t size);
+
+``__builtin_memcpy_inline(dst, src, size)`` is identical to
+``__builtin_memcpy(dst, src, size)`` except that the generated code is
+guaranteed not to call any external functions. See [LLVM IR ‘llvm.memcpy.inline’
+Intrinsic](https://llvm.org/docs/LangRef.html#llvm-memcpy-inline-intrinsic) for
+more information.
+
+
+ * ``__builtin_char_memchr``
 
 .. code-block:: c
 
   char *__builtin_char_memchr(const char *haystack, int needle, size_t size);
 
 ``__builtin_char_memchr(a, b, c)`` is identical to
 ``(char*)__builtin_memchr(a, b, c)`` except that its use is permitted within
 constant expressions in C++11 onwards (where a cast from ``void*`` to ``char*``

clang/include/clang/Basic/Builtins.def

 BUILTIN(__builtin_assume_aligned, "v*vC*z.", "nc")
 BUILTIN(__builtin_bcmp, "ivC*vC*z", "Fn")
 BUILTIN(__builtin_bcopy, "vv*v*z", "n")
 BUILTIN(__builtin_bzero, "vv*z", "nF")
 BUILTIN(__builtin_fprintf, "iP*cC*.", "Fp:1:")
 BUILTIN(__builtin_memchr, "v*vC*iz", "nF")
 BUILTIN(__builtin_memcmp, "ivC*vC*z", "nF")
 BUILTIN(__builtin_memcpy, "v*v*vC*z", "nF")
+BUILTIN(__builtin_memcpy_inline, "vv*vC*z", "nt")
 BUILTIN(__builtin_memmove, "v*v*vC*z", "nF")
 BUILTIN(__builtin_mempcpy, "v*v*vC*z", "nF")
 BUILTIN(__builtin_memset, "v*v*iz", "nF")
 BUILTIN(__builtin_printf, "icC*.", "Fp:0:")
 BUILTIN(__builtin_stpcpy, "c*c*cC*", "nF")
 BUILTIN(__builtin_stpncpy, "c*c*cC*z", "nF")
 BUILTIN(__builtin_strcasecmp, "icC*cC*", "nF")
 BUILTIN(__builtin_strcat, "c*c*cC*", "nF")

clang/lib/CodeGen/CGBuilder.h

public:
   }
   llvm::CallInst *CreateMemCpy(Address Dest, Address Src, uint64_t Size,
                                bool IsVolatile = false) {
     return CreateMemCpy(Dest.getPointer(), Dest.getAlignment().getAsAlign(),
                         Src.getPointer(), Src.getAlignment().getAsAlign(), Size,
                         IsVolatile);
   }
 
+  using CGBuilderBaseTy::CreateMemCpyInline;
+  llvm::CallInst *CreateMemCpyInline(Address Dest, Address Src, uint64_t Size) {
+    return CreateMemCpyInline(
+        Dest.getPointer(), Dest.getAlignment().getAsAlign(), Src.getPointer(),
+        Src.getAlignment().getAsAlign(), getInt64(Size));
+  }
+
   using CGBuilderBaseTy::CreateMemMove;
   llvm::CallInst *CreateMemMove(Address Dest, Address Src, llvm::Value *Size,
                                 bool IsVolatile = false) {
     return CreateMemMove(Dest.getPointer(), Dest.getAlignment().getAsAlign(),
                          Src.getPointer(), Src.getAlignment().getAsAlign(),
                          Size, IsVolatile);
   }
 

clang/lib/CodeGen/CGBuiltin.cpp

case Builtin::BI__builtin_mempcpy: {
     Builder.CreateMemCpy(Dest, Src, SizeVal, false);
     if (BuiltinID == Builtin::BImempcpy ||
         BuiltinID == Builtin::BI__builtin_mempcpy)
       return RValue::get(Builder.CreateInBoundsGEP(Dest.getPointer(), SizeVal));
     else
       return RValue::get(Dest.getPointer());
   }
 
+  case Builtin::BI__builtin_memcpy_inline: {
+    Address Dest = EmitPointerWithAlignment(E->getArg(0));
+    Address Src = EmitPointerWithAlignment(E->getArg(1));
+    uint64_t Size =
+        E->getArg(2)->EvaluateKnownConstInt(getContext()).getZExtValue();
+    EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
+                        E->getArg(0)->getExprLoc(), FD, 0);
+    EmitNonNullArgCheck(RValue::get(Src.getPointer()), E->getArg(1)->getType(),
+                        E->getArg(1)->getExprLoc(), FD, 1);
+    Builder.CreateMemCpyInline(Dest, Src, Size);
+    return RValue::get(nullptr);
+  }
+
   case Builtin::BI__builtin_char_memchr:
     BuiltinID = Builtin::BI__builtin_memchr;
     break;
 
   case Builtin::BI__builtin___memcpy_chk: {
     // fold __builtin_memcpy_chk(x, y, cst1, cst2) to memcpy iff cst1<=cst2.
     Expr::EvalResult SizeResult, DstSizeResult;
     if (!E->getArg(2)->EvaluateAsInt(SizeResult, CGM.getContext()) ||

clang/lib/Sema/SemaChecking.cpp

llvm::Triple::ArchType CurArch =
       S.getASTContext().getTargetInfo().getTriple().getArch();
   if (llvm::is_contained(SupportedArchs, CurArch))
     return false;
   S.Diag(TheCall->getBeginLoc(), diag::err_builtin_target_unsupported)
       << TheCall->getSourceRange();
   return true;
 }
 
+static void CheckNonNullArgument(Sema &S, const Expr *ArgExpr,
+                                 SourceLocation CallSiteLoc);
+
 ExprResult
 Sema::CheckBuiltinFunctionCall(FunctionDecl *FDecl, unsigned BuiltinID,
                                CallExpr *TheCall) {
   ExprResult TheCallResult(TheCall);
 
   // Find out if any arguments are required to be integer constant expressions.
   unsigned ICEArguments = 0;
   ASTContext::GetBuiltinTypeError Error;
case Builtin::BI__sync_swap_16:
     return SemaBuiltinAtomicOverloaded(TheCallResult);
   case Builtin::BI__sync_synchronize:
     Diag(TheCall->getBeginLoc(), diag::warn_atomic_implicit_seq_cst)
         << TheCall->getCallee()->getSourceRange();
     break;
   case Builtin::BI__builtin_nontemporal_load:
   case Builtin::BI__builtin_nontemporal_store:
     return SemaBuiltinNontemporalOverloaded(TheCallResult);
+  case Builtin::BI__builtin_memcpy_inline: {
+    llvm::APSInt I(64);
+    if (!TheCall->getArg(2)->isIntegerConstantExpr(I, Context))
+      Diag(TheCall->getBeginLoc(), diag::err_expr_not_ice)
+          << LangOpts.CPlusPlus << TheCall->getArg(2)->getSourceRange();
+    if (I.isNullValue())
+      break;
+    CheckNonNullArgument(*this, TheCall->getArg(0), TheCall->getExprLoc());
+    CheckNonNullArgument(*this, TheCall->getArg(1), TheCall->getExprLoc());
+    break;
+  }
 #define BUILTIN(ID, TYPE, ATTRS)
 #define ATOMIC_BUILTIN(ID, TYPE, ATTRS) \
   case Builtin::BI##ID: \
     return SemaAtomicOpsOverloaded(TheCallResult, AtomicExpr::AO##ID);
 #include "clang/Basic/Builtins.def"
   case Builtin::BI__annotation:
     if (SemaBuiltinMSVCAnnotation(*this, TheCall))
       return ExprError();

clang/test/CodeGen/builtins-memcpy-inline.c

+// REQUIRES: x86-registered-target
+// RUN: %clang_cc1 -triple x86_64-unknown-linux -emit-llvm %s -o - | FileCheck %s
+
+// CHECK-LABEL: define void @test_memcpy_inline_0(i8* %dst, i8* %src)
+void test_memcpy_inline_0(void *dst, const void *src) {
+  // CHECK:   call void @llvm.memcpy.inline.p0i8.p0i8.i64(i8* align 1 %0, i8* align 1 %1, i64 0, i1 false)
+  __builtin_memcpy_inline(dst, src, 0);
+}
+
+// CHECK-LABEL: define void @test_memcpy_inline_1(i8* %dst, i8* %src)
+void test_memcpy_inline_1(void *dst, const void *src) {
+  // CHECK:   call void @llvm.memcpy.inline.p0i8.p0i8.i64(i8* align 1 %0, i8* align 1 %1, i64 1, i1 false)
+  __builtin_memcpy_inline(dst, src, 1);
+}
+
+// CHECK-LABEL: define void @test_memcpy_inline_4(i8* %dst, i8* %src)
+void test_memcpy_inline_4(void *dst, const void *src) {
+  // CHECK:   call void @llvm.memcpy.inline.p0i8.p0i8.i64(i8* align 1 %0, i8* align 1 %1, i64 4, i1 false)
+  __builtin_memcpy_inline(dst, src, 4);
+}
+
+// CHECK-LABEL: define void @test_memcpy_inline_aligned_buffers(i64* %dst, i64* %src)
+void test_memcpy_inline_aligned_buffers(unsigned long long *dst, const unsigned long long *src) {
+  // CHECK:   call void @llvm.memcpy.inline.p0i8.p0i8.i64(i8* align 8 %2, i8* align 8 %3, i64 4, i1 false)
+  __builtin_memcpy_inline(dst, src, 4);
+}

clang/test/Sema/builtins-memcpy-inline.c

+// RUN: %clang_cc1 -fsyntax-only -verify %s
+
+#define NULL ((char *)0)
+
+#if __has_feature(__builtin_memcpy_inline)
+#warning defined as expected
+// expected-warning@-1 {{defined as expected}}
+#endif
+
+void test_memcpy_inline_null_src(void *ptr) {
+  __builtin_memcpy_inline(ptr, NULL, 4); // expected-warning {{null passed to a callee that requires a non-null argument}}
+}
+
+void test_memcpy_inline_null_dst(void *ptr) {
+  __builtin_memcpy_inline(NULL, ptr, 4); // expected-warning {{null passed to a callee that requires a non-null argument}}
+}
+
+void test_memcpy_inline_null_buffers() {
+  __builtin_memcpy_inline(NULL, NULL, 4);
+  // expected-warning@-1 {{null passed to a callee that requires a non-null argument}}
+  // expected-warning@-2 {{null passed to a callee that requires a non-null argument}}
+}
+
+void test_memcpy_inline_null_buffer_is_ok_if_size_is_zero(void *ptr) {
+  __builtin_memcpy_inline(ptr, NULL, /*size */ 0);
+  __builtin_memcpy_inline(NULL, ptr, /*size */ 0);
+  __builtin_memcpy_inline(NULL, NULL, /*size */ 0);
+}
+
+void test_memcpy_inline_non_constant_size(void *dst, const void *src, unsigned size) {
+  __builtin_memcpy_inline(dst, src, size); // expected-error {{expression is not an integer constant expression}}
+}

llvm/include/llvm/IR/IRBuilder.h

LLVM_ATTRIBUTE_DEPRECATED(
       "Use the version that takes MaybeAlign instead");
   CallInst *CreateMemCpy(Value *Dst, MaybeAlign DstAlign, Value *Src,
                          MaybeAlign SrcAlign, Value *Size,
                          bool isVolatile = false, MDNode *TBAATag = nullptr,
                          MDNode *TBAAStructTag = nullptr,
                          MDNode *ScopeTag = nullptr,
                          MDNode *NoAliasTag = nullptr);
 
+  CallInst *CreateMemCpyInline(Value *Dst, MaybeAlign DstAlign, Value *Src,
+                               MaybeAlign SrcAlign, Value *Size);
+
   /// Create and insert an element unordered-atomic memcpy between the
   /// specified pointers.
   ///
   /// DstAlign/SrcAlign are the alignments of the Dst/Src pointers, respectively.
   ///
   /// If the pointers aren't i8*, they will be converted.  If a TBAA tag is
   /// specified, it will be added to the instruction. Likewise with alias.scope
   /// and noalias tags.

llvm/lib/IR/IRBuilder.cpp

if (ScopeTag)
     CI->setMetadata(LLVMContext::MD_alias_scope, ScopeTag);
 
   if (NoAliasTag)
     CI->setMetadata(LLVMContext::MD_noalias, NoAliasTag);
 
   return CI;
 }
 
+CallInst *IRBuilderBase::CreateMemCpyInline(Value *Dst, MaybeAlign DstAlign,
+                                            Value *Src, MaybeAlign SrcAlign,
+                                            Value *Size) {
+  Dst = getCastedInt8PtrValue(Dst);
+  Src = getCastedInt8PtrValue(Src);
+  Value *IsVolatile = getInt1(false);
+
+  Value *Ops[] = {Dst, Src, Size, IsVolatile};
+  Type *Tys[] = {Dst->getType(), Src->getType(), Size->getType()};
+  Function *F = BB->getParent();
+  Module *M = F->getParent();
+  Function *TheFn = Intrinsic::getDeclaration(M, Intrinsic::memcpy_inline, Tys);
+
+  CallInst *CI = createCallHelper(TheFn, Ops, this);
+
+  auto *MCI = cast<MemCpyInlineInst>(CI);
+  if (DstAlign)
+    MCI->setDestAlignment(*DstAlign);
+  if (SrcAlign)
+    MCI->setSourceAlignment(*SrcAlign);
+
+  return CI;
+}
+
 CallInst *IRBuilderBase::CreateElementUnorderedAtomicMemCpy(
     Value *Dst, Align DstAlign, Value *Src, Align SrcAlign, Value *Size,
     uint32_t ElementSize, MDNode *TBAATag, MDNode *TBAAStructTag,
     MDNode *ScopeTag, MDNode *NoAliasTag) {
   assert(DstAlign >= ElementSize &&
          "Pointer alignment must be at least element size");
   assert(SrcAlign >= ElementSize &&
          "Pointer alignment must be at least element size");