[InstCombine] Add support for strlcpy folding

[msebor]

Summary

The middle end support for strlcpy consists of transforming calls to __strlcpy_chk to the function itself if the buffer overflow checking overhead can be avoided. This change extends the support to also optimize a subset of calls to the function itself, similar to snprintf.

The strlcpy and strlcat functions have both recently been added to the next revision of POSIX (see issue 986). If/when this proposed enhancement is accepted I'll look into adding minimal support for strlcat as well.

[msebor]

Ping: Does anyone have any comments on this enhancement?

For additional motivation behind this change besides the recent inclusion of the functions in POSIX see also the 1999 paper strlcpy and strlcat — Consistent, Safe, String Copy and Concatenation.

[efriedma]

llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp

699

strlcpy accesses the second argument no matter what the size is (so it can compute the return value).

726

>=?

[msebor]

Revision 1 of the patch with the following changes:

• Annotate source argument dereferenceable regardless of the bound.
• Use getConstantStringInfo instead of GetStringLength to detect unterminated arrays.
• Add tests.

[msebor]

llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp

699

You're absolutely right, thanks!

726

That would work; at the same time, I was uneasy about GetStringLength not differentiating between an unterminated array and a string of the same length. I'd prefer to avoid assuming the source array is nul terminated, even though it's required to be. So I've replaced the call to GetStringLength with getConstantStringInfo and handled that case.

[efriedma]

LGTM

[thakis]

This doesn't build on Mac: http://45.33.8.238/macm1/42588/step_4.txt

Please take a look and revert for now if it takes a while to fix.

[msebor]

In D130666#3727473, @thakis wrote:

This doesn't build on Mac: http://45.33.8.238/macm1/42588/step_4.txt

Whoops! It's a template argument deduction failure due to a mismatch between size_t and uint64_t in ILP32. Sorry about that! It's fixed in rGa7a1be11e69d.

[joanahalili]

This might break the following opensource lib https://git.kernel.org/pub/scm/network/iproute2/iproute2.git

In https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/tree/configure#n445

The compilation for the following code no longer fails when setting -Wno-error-implicit-function-declaration

#include <string.h>
int main(int argc, char **argv) {
	char dst[10];
	strlcpy(dst, "test", sizeof(dst));
	return 0;
}

The behaviour difference between clang 14.0, clang trunk and gcc: https://godbolt.org/z/hKETMoEd6

[msebor]

In D130666#3760810, @joanahalili wrote:

This might break the following opensource lib https://git.kernel.org/pub/scm/network/iproute2/iproute2.git

In https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/tree/configure#n445

The compilation for the following code no longer fails when setting -Wno-error-implicit-function-declaration

#include <string.h>
int main(int argc, char **argv) {
	char dst[10];
	strlcpy(dst, "test", sizeof(dst));
	return 0;
}

The behaviour difference between clang 14.0, clang trunk and gcc: https://godbolt.org/z/hKETMoEd6

The configuration test assumes the function call isn't optimized when it's compiled with optimization enabled. A safer way to test for the presence of a function in a library than calling it without relying on the effect of the call in any way is to use its address in some nontrivial way, e.g., by printing it, or assigning it to a volatile-quialified pointer.

llvm/include/llvm/Transforms/Utils/SimplifyLibCalls.h

private:
   Value *optimizeStrNCat(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrChr(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrRChr(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrCmp(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrNCmp(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrNDup(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrCpy(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStpCpy(CallInst *CI, IRBuilderBase &B);
+  Value *optimizeStrLCpy(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrNCpy(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrLen(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrNLen(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrPBrk(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrTo(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrSpn(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrCSpn(CallInst *CI, IRBuilderBase &B);
   Value *optimizeStrStr(CallInst *CI, IRBuilderBase &B);

llvm/lib/Transforms/Utils/SimplifyLibCalls.cpp

Value *LibCallSimplifier::optimizeStpCpy(CallInst *CI, IRBuilderBase &B) {
   // copy for us.  Make a memcpy to copy the nul byte with align = 1.
   CallInst *NewCI = B.CreateMemCpy(Dst, Align(1), Src, Align(1), LenV);
   NewCI->setAttributes(CI->getAttributes());
   NewCI->removeRetAttrs(AttributeFuncs::typeIncompatible(NewCI->getType()));
   copyFlags(*CI, NewCI);
   return DstEnd;
 }
 
+// Optimize a call to size_t strlcpy(char*, const char*, size_t).
+
+Value *LibCallSimplifier::optimizeStrLCpy(CallInst *CI, IRBuilderBase &B) {
+  Value *Size = CI->getArgOperand(2);
+  if (isKnownNonZero(Size, DL))
+    // Like snprintf, the function stores into the destination only when
+    // the size argument is nonzero.
+    annotateNonNullNoUndefBasedOnAccess(CI, 0);
+  // The function reads the source argument regardless of Size (it returns
+  // its length).
+  annotateNonNullNoUndefBasedOnAccess(CI, 1);
+
+  uint64_t NBytes;
+  if (ConstantInt *SizeC = dyn_cast<ConstantInt>(Size))
+    NBytes = SizeC->getZExtValue();
+  else
+    return nullptr;
+
+  Value *Dst = CI->getArgOperand(0);
+  Value *Src = CI->getArgOperand(1);
+  if (NBytes <= 1) {
+    if (NBytes == 1)
+      // For a call to strlcpy(D, S, 1) first store a nul in *D.
+      B.CreateStore(B.getInt8(0), Dst);
+
+    // Transform strlcpy(D, S, 0) to a call to strlen(S).
+    return copyFlags(*CI, emitStrLen(Src, B, DL, TLI));
+  }
+
+  // Try to determine the length of the source, substituting its size
+  // when it's not nul-terminated (as it's required to be) to avoid
+  // reading past its end.
+  StringRef Str;
+  if (!getConstantStringInfo(Src, Str, 0, /*TrimAtNul=*/false))
+    return nullptr;
+
+  uint64_t SrcLen = Str.find('\0');
+  // Set if the terminating nul should be copied by the call to memcpy
+  // below.
+  bool NulTerm = SrcLen < NBytes;
+
+  if (NulTerm)
+    // Overwrite NBytes with the number of bytes to copy, including
+    // the terminating nul.
+    NBytes = SrcLen + 1;
+  else {
+    // Set the length of the source for the function to return to its
+    // size, and cap NBytes at the same.
+    SrcLen = std::min(SrcLen, Str.size());
+    NBytes = std::min(NBytes - 1, SrcLen);
+  }
+
+  if (SrcLen == 0) {
+    // Transform strlcpy(D, "", N) to (*D = '\0, 0).
+    B.CreateStore(B.getInt8(0), Dst);
+    return ConstantInt::get(CI->getType(), 0);
+  }
+
+  Function *Callee = CI->getCalledFunction();
+  Type *PT = Callee->getFunctionType()->getParamType(0);
+  // Transform strlcpy(D, S, N) to memcpy(D, S, N') where N' is the lower
+  // bound on strlen(S) + 1 and N, optionally followed by a nul store to
+  // D[N' - 1] if necessary.
+  CallInst *NewCI = B.CreateMemCpy(Dst, Align(1), Src, Align(1),
+                        ConstantInt::get(DL.getIntPtrType(PT), NBytes));
+  NewCI->setAttributes(CI->getAttributes());
+  NewCI->removeRetAttrs(AttributeFuncs::typeIncompatible(NewCI->getType()));
+  copyFlags(*CI, NewCI);
+
+  if (!NulTerm) {
+    Value *EndOff = ConstantInt::get(CI->getType(), NBytes);
+    Value *EndPtr = B.CreateInBoundsGEP(B.getInt8Ty(), Dst, EndOff);
+    B.CreateStore(B.getInt8(0), EndPtr);
+  }
+
+  // Like snprintf, strlcpy returns the number of nonzero bytes that would
+  // have been copied if the bound had been sufficiently big (which in this
+  // case is strlen(Src)).
+  return ConstantInt::get(CI->getType(), SrcLen);
+}
+
+// Optimize a call to strncpy.
+
 Value *LibCallSimplifier::optimizeStrNCpy(CallInst *CI, IRBuilderBase &B) {
   Function *Callee = CI->getCalledFunction();
   Value *Dst = CI->getArgOperand(0);
   Value *Src = CI->getArgOperand(1);
   Value *Size = CI->getArgOperand(2);
   annotateNonNullNoUndefBasedOnAccess(CI, 0);
   if (isKnownNonZero(Size, DL))
     annotateNonNullNoUndefBasedOnAccess(CI, 1);
if (TLI->getLibFunc(*Callee, Func) && isLibFuncEmittable(M, TLI, Func)) {
     case LibFunc_strcmp:
       return optimizeStrCmp(CI, Builder);
     case LibFunc_strncmp:
       return optimizeStrNCmp(CI, Builder);
     case LibFunc_strcpy:
       return optimizeStrCpy(CI, Builder);
     case LibFunc_stpcpy:
       return optimizeStpCpy(CI, Builder);
+    case LibFunc_strlcpy:
+      return optimizeStrLCpy(CI, Builder);
     case LibFunc_strncpy:
       return optimizeStrNCpy(CI, Builder);
     case LibFunc_strlen:
       return optimizeStrLen(CI, Builder);
     case LibFunc_strnlen:
       return optimizeStrNLen(CI, Builder);
     case LibFunc_strpbrk:
       return optimizeStrPBrk(CI, Builder);

llvm/test/Transforms/InstCombine/fortify-folding.ll

;
   %dst = getelementptr inbounds [60 x i8], [60 x i8]* @a, i32 0, i32 0
   %src = getelementptr inbounds [60 x i8], [60 x i8]* @b, i32 0, i32 0
   %ret = tail call i8* @__strncat_chk(i8* %dst, i8* %src, i64 22, i64 -1)
   ret i8* %ret
 }
 
 define i64 @test_strlcpy() {
 ; CHECK-LABEL: @test_strlcpy(
-; CHECK-NEXT:    [[STRLCPY:%.*]] = call i64 @strlcpy(i8* getelementptr inbounds ([60 x i8], [60 x i8]* @a, i64 0, i64 0), i8* getelementptr inbounds ([60 x i8], [60 x i8]* @b, i64 0, i64 0), i64 22)
+; CHECK-NEXT:    [[STRLCPY:%.*]] = call i64 @strlcpy(i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([60 x i8], [60 x i8]* @a, i64 0, i64 0), i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([60 x i8], [60 x i8]* @b, i64 0, i64 0), i64 22)
 ; CHECK-NEXT:    ret i64 [[STRLCPY]]
 ;
   %dst = getelementptr inbounds [60 x i8], [60 x i8]* @a, i32 0, i32 0
   %src = getelementptr inbounds [60 x i8], [60 x i8]* @b, i32 0, i32 0
   %ret = call i64 @__strlcpy_chk(i8* %dst, i8* %src, i64 22, i64 -1)
   ret i64 %ret
 }
 
 define i64 @test_not_strlcpy() {
 ; CHECK-LABEL: @test_not_strlcpy(
 ; CHECK-NEXT:    [[RET:%.*]] = call i64 @__strlcpy_chk(i8* getelementptr inbounds ([60 x i8], [60 x i8]* @a, i64 0, i64 0), i8* getelementptr inbounds ([60 x i8], [60 x i8]* @b, i64 0, i64 0), i64 22, i64 2)
 ; CHECK-NEXT:    ret i64 [[RET]]
 ;
   %dst = getelementptr inbounds [60 x i8], [60 x i8]* @a, i32 0, i32 0
   %src = getelementptr inbounds [60 x i8], [60 x i8]* @b, i32 0, i32 0
   %ret = call i64 @__strlcpy_chk(i8* %dst, i8* %src, i64 22, i64 2)
   ret i64 %ret
 }
 
 define i64 @test_strlcpy_tail() {
 ; CHECK-LABEL: @test_strlcpy_tail(
-; CHECK-NEXT:    [[STRLCPY:%.*]] = tail call i64 @strlcpy(i8* getelementptr inbounds ([60 x i8], [60 x i8]* @a, i64 0, i64 0), i8* getelementptr inbounds ([60 x i8], [60 x i8]* @b, i64 0, i64 0), i64 22)
+; CHECK-NEXT:    [[STRLCPY:%.*]] = tail call i64 @strlcpy(i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([60 x i8], [60 x i8]* @a, i64 0, i64 0), i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([60 x i8], [60 x i8]* @b, i64 0, i64 0), i64 22)
 ; CHECK-NEXT:    ret i64 [[STRLCPY]]
 ;
   %dst = getelementptr inbounds [60 x i8], [60 x i8]* @a, i32 0, i32 0
   %src = getelementptr inbounds [60 x i8], [60 x i8]* @b, i32 0, i32 0
   %ret = tail call i64 @__strlcpy_chk(i8* %dst, i8* %src, i64 22, i64 -1)
   ret i64 %ret
 }
 

llvm/test/Transforms/InstCombine/strlcpy-1.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; Assertions have been autogenerated by utils/update_test_checks.py
+;
+; RUN: opt < %s -data-layout="E" -passes=instcombine -S | FileCheck %s --check-prefixes=ANY,BE
+; RUN: opt < %s -data-layout="e" -passes=instcombine -S | FileCheck %s --check-prefixes=ANY,LE
+;
+; Test that the strncpy library call simplifier works correctly.
+
+declare i64 @strlcpy(i8*, i8*, i64)
+
+declare void @sink(i8*, i64)
+
+
+@s4 = constant [5 x i8] c"1234\00"
+
+
+; Verify that strlcpy(D, "", N) calls are transformed to a nul store
+; to *D for nonzero N and folded to zero for all values of N.
+
+define void @fold_strlcpy_s0(i8* %dst) {
+; ANY-LABEL: @fold_strlcpy_s0(
+; ANY-NEXT:    call void @sink(i8* [[DST:%.*]], i64 0)
+; ANY-NEXT:    store i8 0, i8* [[DST]], align 1
+; ANY-NEXT:    call void @sink(i8* nonnull [[DST]], i64 0)
+; ANY-NEXT:    store i8 0, i8* [[DST]], align 1
+; ANY-NEXT:    call void @sink(i8* nonnull [[DST]], i64 0)
+; ANY-NEXT:    ret void
+;
+  %ps0 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 4
+
+; Fold strlcpy(D, "", 0) to just 0.
+  %ns0_0 = call i64 @strlcpy(i8* %dst, i8* %ps0, i64 0)
+  call void @sink(i8* %dst, i64 %ns0_0)
+
+; Transform strlcpy(D, "", 1) to *D = '\0, 0.
+  %ns0_1 = call i64 @strlcpy(i8* %dst, i8* %ps0, i64 1)
+  call void @sink(i8* %dst, i64 %ns0_1)
+
+; Transform strlcpy(D, "", SIZE_MAX) to *D = '\0, 0.
+  %ns0_m1 = call i64 @strlcpy(i8* %dst, i8* %ps0, i64 -1)
+  call void @sink(i8* %dst, i64 %ns0_m1)
+
+  ret void
+}
+
+
+; Verify that strlcpy(D, "4", N) calls are transformed to a store to
+; D[0] for nonzero N (and a nul store to D[1] for N greater than 1)
+; and folded to 1 for all values of N.
+
+define void @fold_strlcpy_s1(i8* %dst) {
+; BE-LABEL: @fold_strlcpy_s1(
+; BE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 1)
+; BE-NEXT:    store i8 0, i8* [[DST]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; BE-NEXT:    [[TMP1:%.*]] = bitcast i8* [[DST]] to i16*
+; BE-NEXT:    store i16 13312, i16* [[TMP1]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; BE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i16*
+; BE-NEXT:    store i16 13312, i16* [[TMP2]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; BE-NEXT:    [[TMP3:%.*]] = bitcast i8* [[DST]] to i16*
+; BE-NEXT:    store i16 13312, i16* [[TMP3]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; BE-NEXT:    ret void
+;
+; LE-LABEL: @fold_strlcpy_s1(
+; LE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 1)
+; LE-NEXT:    store i8 0, i8* [[DST]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; LE-NEXT:    [[TMP1:%.*]] = bitcast i8* [[DST]] to i16*
+; LE-NEXT:    store i16 52, i16* [[TMP1]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; LE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i16*
+; LE-NEXT:    store i16 52, i16* [[TMP2]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; LE-NEXT:    [[TMP3:%.*]] = bitcast i8* [[DST]] to i16*
+; LE-NEXT:    store i16 52, i16* [[TMP3]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 1)
+; LE-NEXT:    ret void
+;
+  %ps1 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 3
+
+; Fold strlcpy(D, "4", 0) to 1.
+  %ns1_0 = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 0)
+  call void @sink(i8* %dst, i64 %ns1_0)
+
+; Transform strlcpy(D, "4", 1) to *D = '\0', 1.
+  %ns1_1 = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 1)
+  call void @sink(i8* %dst, i64 %ns1_1)
+
+; Transform strlcpy(D, "4", 2) to D[0] = '\4, D[1] = '\0', 1.
+  %ns1_2 = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 2)
+  call void @sink(i8* %dst, i64 %ns1_2)
+
+; Transform strlcpy(D, "4", 3) to D[0] = '\4, D[1] = '\0', 1..
+  %ns1_3 = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 3)
+  call void @sink(i8* %dst, i64 %ns1_3)
+
+; Transform strlcpy(D, "4", SIZE_MAX) to D[0] = '\4, D[1] = '\0', 1.
+  %ns1_m1 = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 -1)
+  call void @sink(i8* %dst, i64 %ns1_m1)
+
+  ret void
+}
+
+
+; Verify that strlcpy(D, "1234", N) calls are transformed to a copy of
+; the N - 1 leading characters of the string to D and folded to 4 for
+; all values of N.
+
+define void @fold_strlcpy_s5(i8* %dst) {
+; BE-LABEL: @fold_strlcpy_s5(
+; BE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 4)
+; BE-NEXT:    store i8 0, i8* [[DST]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    store i8 49, i8* [[DST]], align 1
+; BE-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 1
+; BE-NEXT:    store i8 0, i8* [[TMP1]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i16*
+; BE-NEXT:    store i16 12594, i16* [[TMP2]], align 1
+; BE-NEXT:    [[TMP3:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 2
+; BE-NEXT:    store i8 0, i8* [[TMP3]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(3) [[DST]], i8* noundef nonnull align 1 dereferenceable(3) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 3, i1 false)
+; BE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 3
+; BE-NEXT:    store i8 0, i8* [[TMP4]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; BE-NEXT:    ret void
+;
+; LE-LABEL: @fold_strlcpy_s5(
+; LE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 4)
+; LE-NEXT:    store i8 0, i8* [[DST]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    store i8 49, i8* [[DST]], align 1
+; LE-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 1
+; LE-NEXT:    store i8 0, i8* [[TMP1]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i16*
+; LE-NEXT:    store i16 12849, i16* [[TMP2]], align 1
+; LE-NEXT:    [[TMP3:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 2
+; LE-NEXT:    store i8 0, i8* [[TMP3]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(3) [[DST]], i8* noundef nonnull align 1 dereferenceable(3) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 3, i1 false)
+; LE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 3
+; LE-NEXT:    store i8 0, i8* [[TMP4]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 5, i1 false)
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 4)
+; LE-NEXT:    ret void
+;
+  %ps4 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 0
+
+; Fold strlcpy(D, "1234", 0) to 4.
+  %ns4_0 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 0)
+  call void @sink(i8* %dst, i64 %ns4_0)
+
+; Transform strlcpy(D, "1234", 1) to *D = '\0', 4.
+  %ns4_1 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 1)
+  call void @sink(i8* %dst, i64 %ns4_1)
+
+; Transform strlcpy(D, "1234", 2) to D[0] = '1', D[1] = '\0', 4.
+  %ns4_2 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 2)
+  call void @sink(i8* %dst, i64 %ns4_2)
+
+; Transform strlcpy(D, S="1234", 3) to memcpy(D, S, 2), D[2] = '\0', 4.
+  %ns4_3 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 3)
+  call void @sink(i8* %dst, i64 %ns4_3)
+
+; Transform strlcpy(D, S="1234", 4) to memcpy(D, S, 3), D[3] = '\0', 4.
+  %ns4_4 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 4)
+  call void @sink(i8* %dst, i64 %ns4_4)
+
+; Transform strlcpy(D, S="1234", 5) to memcpy(D, S, 5), 4.
+  %ns4_5 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 5)
+  call void @sink(i8* %dst, i64 %ns4_5)
+
+; Transform strlcpy(D, S="1234", 9) to memcpy(D, S, 5), 4.
+  %ns4_9 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 5)
+  call void @sink(i8* %dst, i64 %ns4_9)
+
+; Transform strlcpy(D, S="1234", SIZE_MAX) to memcpy(D, S, 5), 4.
+  %ns4_m1 = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 -1)
+  call void @sink(i8* %dst, i64 %ns4_m1)
+
+  ret void
+}
+
+; Verify that strlcpy(D, S, 1) calls are transformed into a nul store
+; to *D, strlcpy(D, S, 0) to a no-op, and the result of both folded
+; to strlen(S).
+
+define void @fold_strlcpy_s_0(i8* %dst, i8* %s, i64 %n) {
+; ANY-LABEL: @fold_strlcpy_s_0(
+; ANY-NEXT:    store i8 0, i8* [[DST:%.*]], align 1
+; ANY-NEXT:    [[STRLEN:%.*]] = call i64 @strlen(i8* noundef nonnull dereferenceable(1) [[S:%.*]])
+; ANY-NEXT:    call void @sink(i8* nonnull [[DST]], i64 [[STRLEN]])
+; ANY-NEXT:    [[STRLEN1:%.*]] = call i64 @strlen(i8* noundef nonnull dereferenceable(1) [[S]])
+; ANY-NEXT:    call void @sink(i8* nonnull [[DST]], i64 [[STRLEN1]])
+; ANY-NEXT:    [[STRLEN2:%.*]] = call i64 @strlen(i8* noundef nonnull dereferenceable(1) [[S]])
+; ANY-NEXT:    call void @sink(i8* nonnull [[DST]], i64 [[STRLEN2]])
+; ANY-NEXT:    ret void
+;
+; Transform strlcpy(D, S, 1) to *D = '\0', strlen(S).
+  %ns_1 = call i64 @strlcpy(i8* %dst, i8* %s, i64 1)
+  call void @sink(i8* %dst, i64 %ns_1)
+
+; For strlcpy(D, S, 0) to strlen(S).
+  %ns_0 = call i64 @strlcpy(i8* %dst, i8* %s, i64 0)
+  call void @sink(i8* %dst, i64 %ns_0)
+
+  ; Verify that calling strlcpy with a null destination is also folded
+  ; (to match a possible extension of some implementations that emulate
+  ; snprintf(0, 0, "%s", S)).
+  %n0_s_0 = call i64 @strlcpy(i8* null, i8* %s, i64 0)
+  call void @sink(i8* %dst, i64 %n0_s_0)
+
+  ret void
+}
+
+
+; Verify that strlcpy(D, S, N) calls are left alone when S and/or N are
+; not known (except for the cases handled above).  Also verify that they
+; annotate the destination argument with the dereferenceable attribute
+; only with nonzero N.
+
+define void @call_strlcpy_s0_n(i8* %dst, i8* %s, i64 %n) {
+; ANY-LABEL: @call_strlcpy_s0_n(
+; ANY-NEXT:    [[NS_2:%.*]] = call i64 @strlcpy(i8* noundef nonnull dereferenceable(1) [[DST:%.*]], i8* noundef nonnull dereferenceable(1) [[S:%.*]], i64 2)
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS_2]])
+; ANY-NEXT:    [[NS_N:%.*]] = call i64 @strlcpy(i8* [[DST]], i8* noundef nonnull dereferenceable(1) [[S]], i64 [[N:%.*]])
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS_N]])
+; ANY-NEXT:    [[NZ:%.*]] = or i64 [[N]], 1
+; ANY-NEXT:    [[NS_NZ:%.*]] = call i64 @strlcpy(i8* noundef nonnull dereferenceable(1) [[DST]], i8* noundef nonnull dereferenceable(1) [[S]], i64 [[NZ]])
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS_NZ]])
+; ANY-NEXT:    [[NS0_N:%.*]] = call i64 @strlcpy(i8* [[DST]], i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 4), i64 [[N]])
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS0_N]])
+; ANY-NEXT:    [[NS1_N:%.*]] = call i64 @strlcpy(i8* [[DST]], i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 3), i64 [[N]])
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS1_N]])
+; ANY-NEXT:    [[NS4_N:%.*]] = call i64 @strlcpy(i8* [[DST]], i8* noundef nonnull dereferenceable(1) getelementptr inbounds ([5 x i8], [5 x i8]* @s4, i64 0, i64 0), i64 [[N]])
+; ANY-NEXT:    call void @sink(i8* [[DST]], i64 [[NS4_N]])
+; ANY-NEXT:    ret void
+;
+  %ns_2 = call i64 @strlcpy(i8* %dst, i8* %s, i64 2)
+  call void @sink(i8* %dst, i64 %ns_2)
+
+  %ns_n = call i64 @strlcpy(i8* %dst, i8* %s, i64 %n)
+  call void @sink(i8* %dst, i64 %ns_n)
+
+  %nz = or i64 %n, 1
+  %ns_nz = call i64 @strlcpy(i8* %dst, i8* %s, i64 %nz)
+  call void @sink(i8* %dst, i64 %ns_nz)
+
+
+  %ps0 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 4
+  %ns0_n = call i64 @strlcpy(i8* %dst, i8* %ps0, i64 %n)
+  call void @sink(i8* %dst, i64 %ns0_n)
+
+  %ps1 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 3
+  %ns1_n = call i64 @strlcpy(i8* %dst, i8* %ps1, i64 %n)
+  call void @sink(i8* %dst, i64 %ns1_n)
+
+  %ps4 = getelementptr [5 x i8], [5 x i8]* @s4, i32 0, i32 0
+  %ns4_n = call i64 @strlcpy(i8* %dst, i8* %ps4, i64 %n)
+  call void @sink(i8* %dst, i64 %ns4_n)
+
+  ret void
+}
+
+
+@a5 = constant [5 x i8] c"12345"
+
+; Verify that the transformation behaves reasonably even when the source
+; array is not a nul-terminated string as it's required to be (and doesn't
+; for example attempt to read past its end).  All the calls below are
+; undefined so technically reading past the end would be fine but it's
+; easy to avoid.
+
+define void @fold_strlcpy_a5(i8* %dst, i64 %n) {
+; BE-LABEL: @fold_strlcpy_a5(
+; BE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 5)
+; BE-NEXT:    store i8 0, i8* [[DST]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; BE-NEXT:    store i8 49, i8* [[DST]], align 1
+; BE-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 1
+; BE-NEXT:    store i8 0, i8* [[TMP1]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; BE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i32*
+; BE-NEXT:    store i32 825373492, i32* [[TMP2]], align 1
+; BE-NEXT:    [[TMP3:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 4
+; BE-NEXT:    store i8 0, i8* [[TMP3]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @a5, i64 0, i64 0), i64 5, i1 false)
+; BE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 5
+; BE-NEXT:    store i8 0, i8* [[TMP4]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; BE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @a5, i64 0, i64 0), i64 5, i1 false)
+; BE-NEXT:    [[TMP5:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 5
+; BE-NEXT:    store i8 0, i8* [[TMP5]], align 1
+; BE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; BE-NEXT:    ret void
+;
+; LE-LABEL: @fold_strlcpy_a5(
+; LE-NEXT:    call void @sink(i8* [[DST:%.*]], i64 5)
+; LE-NEXT:    store i8 0, i8* [[DST]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; LE-NEXT:    store i8 49, i8* [[DST]], align 1
+; LE-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 1
+; LE-NEXT:    store i8 0, i8* [[TMP1]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; LE-NEXT:    [[TMP2:%.*]] = bitcast i8* [[DST]] to i32*
+; LE-NEXT:    store i32 875770417, i32* [[TMP2]], align 1
+; LE-NEXT:    [[TMP3:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 4
+; LE-NEXT:    store i8 0, i8* [[TMP3]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @a5, i64 0, i64 0), i64 5, i1 false)
+; LE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 5
+; LE-NEXT:    store i8 0, i8* [[TMP4]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; LE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(5) [[DST]], i8* noundef nonnull align 1 dereferenceable(5) getelementptr inbounds ([5 x i8], [5 x i8]* @a5, i64 0, i64 0), i64 5, i1 false)
+; LE-NEXT:    [[TMP5:%.*]] = getelementptr inbounds i8, i8* [[DST]], i64 5
+; LE-NEXT:    store i8 0, i8* [[TMP5]], align 1
+; LE-NEXT:    call void @sink(i8* nonnull [[DST]], i64 5)
+; LE-NEXT:    ret void
+;
+  %pa5 = getelementptr [5 x i8], [5 x i8]* @a5, i32 0, i32 0
+  %na5_0 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 0)
+  call void @sink(i8* %dst, i64 %na5_0)
+
+  %na5_1 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 1)
+  call void @sink(i8* %dst, i64 %na5_1)
+
+  %na5_2 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 2)
+  call void @sink(i8* %dst, i64 %na5_2)
+
+  %na5_5 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 5)
+  call void @sink(i8* %dst, i64 %na5_5)
+
+  %na5_6 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 6)
+  call void @sink(i8* %dst, i64 %na5_6)
+
+  %na5_9 = call i64 @strlcpy(i8* %dst, i8* %pa5, i64 9)
+  call void @sink(i8* %dst, i64 %na5_9)
+
+  ret void
+}