This is an archive of the discontinued LLVM Phabricator instance.

[libc++]Use __libcpp_malloc/__libcpp_calloc in places of malloc/calloc
AbandonedPublic

Authored by xingxue on Oct 20 2021, 12:10 PM.

Details

Reviewers
ldionne
mstorsjo
daltenty
hubert.reinterpretcast
cebowleratibm
Group Reviewers
Restricted Project
Restricted Project
Summary

AIX system header <stdlib.h> maps system calls malloc and calloc to vec_malloc and vec_calloc under macro __VEC__ which is pre-defined by compiler when the target architecture supports Altivec. vec_malloc/vec_calloc give 16-byte aligned allocation and as a result, greatly increase the heap memory consumption if an application makes a large number of small heap allocations. This causes existing applications built with certain heap size to run out of memory unexpectedly when running against a newer version of libc++/libc++abi built with __VEC__ defined (on AIX, linker option -bmaxdata is used to specify the maximum size of user data area). For example, an application usually malloc()s 1.5GB heap memory through the new operator out of 40M allocations but it allocates more than 2GB when running against the newer libc++abi.

To mitigate this issue, this patch defines __libcpp_malloc()/__libcpp_calloc() that are true malloc()/calloc() and use them in libc++ and libc++abi implementations in places of the plain malloc()/calloc(} where a specific alignment is not required. For AIX, __libcpp_malloc()/__libcpp_calloc() map to assembly label malloc/calloc. For other platforms, they are function macros mapping to malloc/calloc. Functions std::malloc()/std::calloc() are not changed to maintain the original AIX semantic, i.e., they are mapped to vec_malloc()/vec_calloc() if macro __VEC__ is defined.

Calls to malloc()/calloc() in libunwind are used for DWARF based unwinding. Since AIX uses the traceback table based unwinding and the code for DWARF based unwinding is compiled out, the code in libunwind is not changed.

Diff Detail

Event Timeline

xingxue requested review of this revision.Oct 20 2021, 12:10 PM
xingxue created this revision.
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptOct 20 2021, 12:10 PM
Herald added a reviewer: Restricted Project. · View Herald Transcript
Herald added a reviewer: Restricted Project. · View Herald Transcript
xingxue updated this revision to Diff 381285.Oct 21 2021, 9:01 AM

Addressed issues in pre-merge check.

ldionne requested changes to this revision.Oct 21 2021, 2:20 PM

I don't understand the problem statement. Doesn't malloc and calloc always align stuff at 16 bytes? https://godbolt.org/z/oor5KaedP

This revision now requires changes to proceed.Oct 21 2021, 2:20 PM

I don't understand the problem statement. Doesn't malloc and calloc always align stuff at 16 bytes? https://godbolt.org/z/oor5KaedP

The malloc and calloc of the AIX libc do not always align at 16 bytes.

$ cat malloc.c
#include <stdlib.h>
#include <stdio.h>

int main()
{
  char *ptr=malloc(8);
  char *vec_ptr=vec_malloc(8);
  printf("ptr=0x%p, vec_ptr=0x%p\n", ptr, vec_ptr);
}
$ xlc malloc.c
$ a.out
ptr=0x200005b8, vec_ptr=0x200005d0
xingxue abandoned this revision.Oct 26 2021, 1:30 PM

The code block mapping malloc/calloc to vec_malloc/vec_calloc in AIX header stdlib.h is enabled by macro _ALL_SOURCE which is defined for building the libc++/libc++abi on AIX. As per discussion, we think it is a good idea not to define it. So, this patch is no longer needed.