[libc] Minimal Darwin support

[tschuett]

Summary

This is the minimal copy and paste version to get something running on Darwin.

Note the file platfrom_defs.h.inc. The section name has a ',' because of Mach-O. The section attribute length can be at most 16 characters. Thus, I had to remove the llvm part of the name.
The objcopy changes are beyond my me.

It stills fails the liniting. It claims that stddef.h and stdint.h are system headers. stddef.h and stdint.h are provided by the libc and the resource directory of clang ...

[tschuett]

copy and paste api.td from linux ...

[tschuett]

some cmake improvements

[sivachandra]

Thanks a lot for working on this. I have a few first reaction questions, but I want to play with this patch before I ask them. So, it will take some time before I can comeback. In the meanwhile, @abrachet might share his opinions/ideas. Also, we are heading into a US holiday week during which I will be away the whole week.

[tschuett]

add missing include

[tschuett]

Notes:

• llvm-objcopy cannot do aliases for Mach-O. It can only rename symbols.
• ld64 supports an alias_list option to create aliases. It would probably require the mangled names.
• clang does not support the alias attribute for Darwin.
• use llvm-readobj to extract the mangled names from the entry point sections.

[abrachet]

Would you mind looking if it is possible for a program not to be linked against LibSystem.dylib which contains Apple's libc? I thought it was the case that it wasn't possible, but I might be wrong.

It stills fails the liniting. It claims that stddef.h and stdint.h are system headers. stddef.h and stdint.h are provided by the libc and the resource directory of clang ...

@PaulkaToast Might have some insight here, or maybe ideas on what we can do. But it does make sense that the clang function used in the linter is saying these are system headers, they are provided by Apple also. /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stddef.h This one isn't provided by clang, right?

[tschuett]

I don't know whether linking without systemb is supported at all, but LLVM libc is designed to be layered on top of the system libc. It seems to be a problem of the include order.

This is the stddef.h from clang. The path includes clang's version and something like lib/clang/version ...
/Applications/Xcode.app/ContentsDeveloper/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/stddef.h
This is the stddef.h from the libc. It is similar to your example: SDKs/MacOSX10.15.sdk/usr/include/stddef.h
/Applications/Xcode.app/ContentsDeveloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stddef.h
Mine is from the Xcode installation. Yours is from the command line tools.

[sivachandra]

In D82404#2118254, @abrachet wrote:

Would you mind looking if it is possible for a program not to be linked against LibSystem.dylib which contains Apple's libc? I thought it was the case that it wasn't possible, but I might be wrong.

It stills fails the liniting. It claims that stddef.h and stdint.h are system headers. stddef.h and stdint.h are provided by the libc and the resource directory of clang ...

@PaulkaToast Might have some insight here, or maybe ideas on what we can do. But it does make sense that the clang function used in the linter is saying these are system headers, they are provided by Apple also. /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stddef.h This one isn't provided by clang, right?

On Darwin/BSD, I think the OS provides the libc. Not sure if -ffreestanding has any implications. So, for Darwin/BSD, we probably have to use a whitelist approach?

A more important question on Darwin/BSD is, what exactly does a libc mean? Since the OS provides the libc (including the headers), should we just implement redirectors for everything? May be not. May be LLVM-libc will be an alternate for users wanting to use it and so we should be like any normal library. There can still be questions like, for how much of the OS provided libc do we provide an alternate?

However we evolve for Darwin/BSD, I do not think we need to block on concrete answers to the above questions at this point in time. At the same time, my notes here are probably not helping this patch in anyway. I did see @tschuett pointing out other valid problems. Unfortunately, I will be away until 6th of July so I will not be picking this patch up until then. If others have any further thoughts/opinions, feel free to share.

[tschuett]

Maybe the linting problem is not unique to Darwin, but highlights a more general problem with this approach. So far development was limited to Linux.

Does clang-tidy use -ffreestanding?

Supporting Darwin might help to accelerate the development of LLVM-libc. The development of the string and math library can be done on any platform. My first thought was adding redirectors for malloc and free to support the strdup implementation.

[tschuett]

/Users/XXX/Work/XXX/llvm-project/libc/src/string/memcpy.h:13:1: error: system include stddef.h not allowed, transitively included from /Users/XXX/Work/XXX/llvm-project/libc/src/string/memcpy.h (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/stddef.h) [llvmlibc-restrict-system-libc-headers,-warnings-as-errors]
#include <stddef.h> // size_t

Despite -freestanding, it gets the header from the C++ STL?!?

-nostdinc++ solved the linting problems.

[tschuett]

• fixed linting
• minimal entry point

[sivachandra]

libc/config/darwin/platfrom_defs.h.inc

12

This is probably OK as a compromise. Few points around this:

1. If we are actually OK with this in general and not just for Darwin, then we don' t need to use aliases at all. We can use an arrangement like this:

// entrypoint.h

#define ENTRYPOINT_DECL extern "C"

namespace __llvm_libc {

ENTRYPOINT_DECL <ret_type> entrypoint(<argl list>);

} // namespace __llvm_libc

// entrypoint.cpp
#include "entrypoint.h"

namespace __llvm_libc {

<ret_type> entrypoint(<arg list>) {
 ...
}

} // namespace __llvm_libc

This gives you a C symbol but requires scope qualification in the callers from within LLVM libc. Exactly what we want.

2. But, LLVM libc aims to be intermixed with other libcs In which case, the above mechanism or even using aliases, does not give us a way to ensure that calls from within LLVM libc to other LLVM libc entrypoints resolve to symbols from LLVM libc itself. This was the reason why we went with the LLVM_LIBC_ENTRYPOINT macro as it is today for linux.

[tschuett]

I see, but this solution seems to be a very Linux/ELF way of doing it. I could not find anything similar for Mach-O.

[sivachandra]

In D82404#2144863, @tschuett wrote:

I see, but this solution seems to be a very Linux/ELF way of doing it. I could not find anything similar for Mach-O.

I agree, but also do not have a good answer yet. I am still doing my homework on this. Personally, I strongly believe we need to solve this problem so I am on it. I will share as and when I have something concrete to share.

[tschuett]

Now, I realized that I cannot use/develop malloc/free redirectors on Darwin.

[tschuett]

Mach-O is my toy project. The real one will be COFF/Windows.

libc/config/CMakeLists.txt

 add_subdirectory(linux)
+add_subdirectory(darwin)

libc/config/darwin/CMakeLists.txt

+

libc/config/darwin/api.td

+include "config/public_api.td"
+
+include "spec/llvm_libc_ext.td"
+include "spec/stdc.td"
+
+def SizeT : TypeDecl<"size_t"> {
+  let Decl = [{
+    #define __need_size_t
+    #include <stddef.h>
+  }];
+}
+
+def NullMacro : MacroDef<"NULL"> {
+  let Defn = [{
+    #define __need_NULL
+    #include <stddef.h>
+  }];
+}
+
+def ErrnoMacro : MacroDef<"errno"> {
+  let Defn = [{
+    #ifdef __cplusplus
+    extern "C"
+    #endif
+    int *__errno_location();
+    #define errno (*__errno_location())
+  }];
+}
+
+def MathErrHandlingMacro : MacroDef<"math_errhandling"> {
+  let Defn = [{
+    #ifndef math_errhandling
+    #ifdef __FAST_MATH__
+    #define math_errhandling 0
+    #elif defined __NO_MATH_ERRNO__
+    #define math_errhandling (MATH_ERREXCEPT)
+    #else
+    #define math_errhandling (MATH_ERRNO | MATH_ERREXCEPT)
+    #endif
+    #endif // math_errhandling not defined
+  }];
+}
+
+def IsFiniteMacro : MacroDef<"isfinite"> {
+  let Defn = [{
+    #define isfinite(x) __builtin_isfinite(x)
+  }];
+}
+
+def IsInfMacro : MacroDef<"isinf"> {
+  let Defn = [{
+    #define isinf(x) __builtin_isinf(x)
+  }];
+}
+
+def IsNanMacro : MacroDef<"isnan"> {
+  let Defn = [{
+    #define isnan(x) __builtin_isnan(x)
+  }];
+}
+
+def FloatT : TypeDecl<"float_t"> {
+  let Decl = [{
+    #define __need_float_t
+    #include <__llvm-libc-stdc-types.h>
+  }];
+}
+
+def DoubleT : TypeDecl<"double_t"> {
+  let Decl = [{
+    #define __need_double_t
+    #include <__llvm-libc-stdc-types.h>
+  }];
+}
+
+def MathAPI : PublicAPI<"math.h"> {
+  let Macros = [
+    SimpleMacroDef<"MATH_ERRNO", "1">,
+    SimpleMacroDef<"MATH_ERREXCEPT", "2">,
+    MathErrHandlingMacro,
+
+    SimpleMacroDef<"INFINITY", "__builtin_inff()">,
+    SimpleMacroDef<"NAN", "__builtin_nanf(\"\")">,
+
+    IsFiniteMacro,
+    IsInfMacro,
+    IsNanMacro,
+  ];
+  let TypeDeclarations = [
+    DoubleT,
+    FloatT,
+  ];
+  let Functions = [
+   "copysign",
+   "copysignf",
+   "ceil",
+   "ceilf",
+   "cosf",
+   "fabs",
+   "fabsf",
+   "fabsl",
+   "floor",
+   "floorf",
+   "frexp",
+   "frexpf",
+   "logb",
+   "logbf",
+   "modf",
+   "modff",
+   "expf",
+   "exp2f",
+   "round",
+   "roundf",
+   "sincosf",
+   "sinf",
+   "trunc",
+   "truncf",
+   "truncl",
+  ];
+}
+
+def StringAPI : PublicAPI<"string.h"> {
+  let Functions = [
+    "bzero",
+    "memcpy",
+    "memmove",
+    "memcmp",
+    "memchr",
+    "memset",
+    "strcpy",
+    "strncpy",
+    "strcat",
+    "strncat",
+    "strcmp",
+    "strcoll",
+    "strncmp",
+    "strxfrm",
+    "strchr",
+    "strcspn",
+    "strpbrk",
+    "strrchr",
+    "strspn",
+    "strstr",
+    "strtok",
+    "strerror",
+    "strlen",
+  ];
+
+  let TypeDeclarations = [
+    SizeT,
+  ];
+
+  let Macros = [
+    NullMacro,
+  ];
+}

libc/config/darwin/platfrom_defs.h.inc

+//===----- Definition of platform specific macros for Mach-O platforms -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+%%begin()
+
+#define ENTRYPOINT_SECTION_ATTRIBUTE(name) \
+    __attribute__((section(".libc.entrypoint,"#name)))
+#define LLVM_LIBC_ENTRYPOINT(name) ENTRYPOINT_SECTION_ATTRIBUTE(name) name
+
+// Note the comma in the section name!!!

libc/config/darwin/x86_64/entrypoints.txt

+set(TARGET_LIBC_ENTRYPOINTS
+    # errno.h entrypoints
+    libc.src.errno.__errno_location
+
+    # string.h entrypoints
+    libc.src.string.bzero
+    libc.src.string.memcpy
+    libc.src.string.memset
+    libc.src.string.strcpy
+    libc.src.string.strcat
+    libc.src.string.strlen
+)
+
+set(TARGET_LIBM_ENTRYPOINTS
+    # math.h entrypoints
+    libc.src.math.copysign
+    libc.src.math.copysignf
+    libc.src.math.ceil
+    libc.src.math.ceilf
+    libc.src.math.cosf
+    libc.src.math.expf
+    libc.src.math.exp2f
+    libc.src.math.fabs
+    libc.src.math.fabsf
+    libc.src.math.fabsl
+    libc.src.math.floor
+    libc.src.math.floorf
+    libc.src.math.frexp
+    libc.src.math.frexpf
+    libc.src.math.logb
+    libc.src.math.logbf
+    libc.src.math.modf
+    libc.src.math.modff
+    libc.src.math.round
+    libc.src.math.roundf
+    libc.src.math.sincosf
+    libc.src.math.sinf
+    libc.src.math.trunc
+    libc.src.math.truncf
+    libc.src.math.truncl
+)

libc/config/darwin/x86_64/headers.txt

+set(PUBLIC_HEADERS
+    libc.include.errno
+    libc.include.math
+    libc.include.sys_syscall
+)
+

libc/config/linux/platfrom_defs.h.inc

-//===----- Definition of platform specific macros for ELF paltforms -------===//
+//===----- Definition of platform specific macros for ELF platforms -------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 %%begin()
 
 #define ENTRYPOINT_SECTION_ATTRIBUTE(name) \
     __attribute__((section(".llvm.libc.entrypoint."#name)))
 #define LLVM_LIBC_ENTRYPOINT(name) ENTRYPOINT_SECTION_ATTRIBUTE(name) name

libc/src/string/bzero.h

 //===-- Implementation header for bzero -------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIBC_SRC_STRING_BZERO_H
 #define LLVM_LIBC_SRC_STRING_BZERO_H
 
 #include "include/string.h"
+#include <stddef.h> // size_t
 
 namespace __llvm_libc {
 
 void bzero(void *ptr, size_t count);
 
 } // namespace __llvm_libc
 
 #endif // LLVM_LIBC_SRC_STRING_BZERO_H

libc/src/string/memset.h

 //===-- Implementation header for memset ------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIBC_SRC_STRING_MEMSET_H
 #define LLVM_LIBC_SRC_STRING_MEMSET_H
 
 #include "include/string.h"
+#include <stddef.h> // size_t
 
 namespace __llvm_libc {
 
 void *memset(void *ptr, int value, size_t count);
 
 } // namespace __llvm_libc
 
 #endif // LLVM_LIBC_SRC_STRING_MEMSET_H

libc/src/string/strlen.h

 //===-- Implementation header for strlen ------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIBC_SRC_STRING_STRLEN_H
 #define LLVM_LIBC_SRC_STRING_STRLEN_H
 
 #include "include/string.h"
+#include <stddef.h> // size_t
 
 namespace __llvm_libc {
 
 size_t strlen(const char *src);
 
 } // namespace __llvm_libc
 
 #endif // LLVM_LIBC_SRC_STRING_STRLEN_H

libc/src/string/strlen.cpp

 //===-- Implementation of strlen ------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "src/string/strlen.h"
 
 #include "src/__support/common.h"
+#include <stddef.h> // size_t
 
 namespace __llvm_libc {
 
 // TODO: investigate the performance of this function.
 // There might be potential for compiler optimization.
 size_t LLVM_LIBC_ENTRYPOINT(strlen)(const char *src) {
   const char *end = src;
   while (*end != '\0')
     ++end;
   return end - src;
 }
 
 } // namespace __llvm_libc

libc/test/config/darwin/CMakeLists.txt

+add_libc_testsuite(libc_darwin_tests)
+
+if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${LIBC_TARGET_MACHINE})
+  add_subdirectory(${LIBC_TARGET_MACHINE})
+endif()

libc/test/config/darwin/x86_64/CMakeLists.txt

+add_libc_unittest(
+  libc_darwin_x86_64_strlen_test
+  SUITE
+    libc_darwin_tests
+  SRCS
+    darwin.cpp
+  DEPENDS
+    libc.src.string.strlen
+)

libc/test/config/darwin/x86_64/darwin.cpp

+//===-- Unittests for strlen          -------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "src/string/strlen.h"
+#include "utils/UnitTest/Test.h"
+
+TEST(StrLenTest, EmptyString) {
+  const char *empty = "";
+
+  size_t result = __llvm_libc::strlen(empty);
+  ASSERT_EQ((size_t)0, result);
+}
+
+TEST(StrLenTest, AnyString) {
+  const char *any = "Hello World!";
+
+  size_t result = __llvm_libc::strlen(any);
+  ASSERT_EQ((size_t)12, result);
+}