[libc][WIP] move realloc into alloc_checker

[michaelrj]

Summary

By putting all allocation through alloc_checker it's easier to check
when allocation is being done. I didn't find an easy way to use realloc
through new, so currently I just call the function manually.

[michaelrj]

cleanup some experimental code

[mcgrathr]

What I meant about a custom operator new to do realloc was something like this (C++20 example and without the AllocChecker arg, which would be added here too):

#include <span>                                                                 
#include <cstdlib>                                                              
                                                                                
template<typename T>                                                            
[[nodiscard]] inline void* operator new[] (size_t size, std::span<T> old) noexcept {                                                                           
  void* ptr = ::realloc(old.data(), old.size_bytes() + size);        
  if (!ptr) { return nullptr; }                                                 
  return static_cast<char*>(ptr) + old.size_bytes();                            
}                                                                               
                                                                                
struct S { int x = 23; };                                                       
                                                                                
S* extend(S* old, size_t count) {                                               
  return new (std::span{old,count}) S[count];                                   
}                                                                               
                                                                                
int* extend(int* old, size_t count) {                                           
  return new (std::span{old,count}) int[count];                                 
}

Instead of the local span equivalent you might want to use a custom type just so the name makes it more obvious what's being done, e.g. new (Realloc{old, count}, ac) T[n];.

That example shows how for a type where default-initialization is not just uninitialized, it does what you want like new for the original allocation does, while for a type that can be uninitialized, that's still an option with no extra overhead.
And of course you can use it with constructor arguments in a situation where that makes sense.

Note that realloc is also sometimes used to trim an allocation to shorter than it was, and I'm not sure there's a way to do an analogous trick for that case such that destructors get run naturally.

libc/src/__support/CPP/new.h

59

You might consider making this templated so realloc<T> takes and returns a T* and takes a count that it multiplies by sizeof(T). That's also a good opportunity to static_assert(alignof(T) <= __STDCPP_DEFAULT_NEW_ALIGNMENT__);.
That makes it more similar to new in terms of the invariants around static types and alignment constraints, though it's still importantly different for any type that doesn't have trivial (i.e. uninitialized) default construction.

libc/src/__support/char_vector.h

37

Superfluous parens aren't usually used with delete.

[sivachandra]

In D157653#4580926, @mcgrathr wrote:

What I meant about a custom operator new to do realloc was something like this (C++20 example and without the AllocChecker arg, which would be added here too):

#include <span>                                                                 
#include <cstdlib>                                                              
                                                                                
template<typename T>                                                            
[[nodiscard]] inline void* operator new[] (size_t size, std::span<T> old) noexcept {                                                                           
  void* ptr = ::realloc(old.data(), old.size_bytes() + size);        
  if (!ptr) { return nullptr; }                                                 
  return static_cast<char*>(ptr) + old.size_bytes();                            
}                                                                               
                                                                                
struct S { int x = 23; };                                                       
                                                                                
S* extend(S* old, size_t count) {                                               
  return new (std::span{old,count}) S[count];                                   
}                                                                               
                                                                                
int* extend(int* old, size_t count) {                                           
  return new (std::span{old,count}) int[count];                                 
}

Instead of the local span equivalent you might want to use a custom type just so the name makes it more obvious what's being done, e.g. new (Realloc{old, count}, ac) T[n];.

That example shows how for a type where default-initialization is not just uninitialized, it does what you want like new for the original allocation does, while for a type that can be uninitialized, that's still an option with no extra overhead.
And of course you can use it with constructor arguments in a situation where that makes sense.

Note that realloc is also sometimes used to trim an allocation to shorter than it was, and I'm not sure there's a way to do an analogous trick for that case such that destructors get run naturally.

Should we restrict this to trivially destructible types?

[mcgrathr]

In D157653#4580990, @sivachandra wrote:

Should we restrict this to trivially destructible types?

realloc should be restricted to trivially copyable types, because realloc is going to copy them byte-by-byte for the existing elements in the array.
It doesn't really need to be restricted to trivially destructible types, because eventual delete[] should still work as it would with any array from new[].
Likewise, if we had a front-end for the trimming case, then that could explicitly call destructors on the tail elements if need be.

That said, it seems unlikely we'll ever use this for things that aren't trivially destructible, or even anything for which we're not doing uninitialized default initialization, so having conservative static_assert for now and revisiting when use cases arise seems sensible.

[sivachandra]

In D157653#4581526, @mcgrathr wrote:

In D157653#4580990, @sivachandra wrote:

Should we restrict this to trivially destructible types?

realloc should be restricted to trivially copyable types, because realloc is going to copy them byte-by-byte for the existing elements in the array.

Ah yes, trivially copyable also.

It doesn't really need to be restricted to trivially destructible types, because eventual delete[] should still work as it would with any array from new[].

Discussing for academic knowledge:
If realloc allocates a whole new array, existing objects are copied over and so we need them to be trivially copyable. But, since the old array is free-d, we will need the trivially destructible property also?
Further, after the call to the reallocating new, wouldn't the compiler try to initialize the copied over objects also? If yes, we will need the trivially constructible property also?

Likewise, if we had a front-end for the trimming case, then that could explicitly call destructors on the tail elements if need be.

By front-end, do you mean the caller of the reallocating new?

libc/src/__support/CPP/new.h

 //===-- Libc specific custom operator new and delete ------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIBC_SRC_SUPPORT_CPP_NEW_H
 #define LLVM_LIBC_SRC_SUPPORT_CPP_NEW_H
 
+#ifdef LIBC_COPT_DISABLE_NEW
+#warning "new included when LIBC_COPT_DISABLE_NEW is set."
+#endif
+
 #include "src/__support/common.h"
 
 #include <stddef.h> // For size_t
 #include <stdlib.h> // For malloc, free etc.
 
 // Defining members in the std namespace is not preferred. But, we do it here
 // so that we can use it to define the operator new which takes std::align_val_t
 // argument.
public:
   }
 
   LIBC_INLINE static void *aligned_alloc(size_t s, std::align_val_t align,
                                          AllocChecker &ac) {
     void *mem = ::aligned_alloc(static_cast<size_t>(align), s);
     ac = (mem != nullptr);
     return mem;
   }
+
+  LIBC_INLINE static void *realloc(void *ptr, size_t s, AllocChecker &ac) {
+    void *mem = ::realloc(ptr, s);
+    ac = (mem != nullptr);
+    return mem;
+  }
 };
 
 } // namespace __llvm_libc
 
 LIBC_INLINE void *operator new(size_t size,
                                __llvm_libc::AllocChecker &ac) noexcept {
   return __llvm_libc::AllocChecker::alloc(size, ac);
 }

libc/src/__support/File/file.cpp

case _IONBF:
     break;
   default:
     return EINVAL;
   }
 
   if (buffer == nullptr && size != 0 && buffer_mode != _IONBF) {
     // We exclude the case of buffer_mode == _IONBF in this branch
     // because we don't need to allocate buffer in such a case.
+    AllocChecker ac;
     if (own_buf) {
-      // This is one of the places where use a C allocation functon
-      // as C++ does not have an equivalent of realloc.
-      buf = reinterpret_cast<uint8_t *>(realloc(buf, size));
-      if (buf == nullptr)
+      buf = reinterpret_cast<uint8_t *>(AllocChecker::realloc(buf, size, ac));
+      if (!ac)
         return ENOMEM;
     } else {
-      AllocChecker ac;
       buf = new (ac) uint8_t[size];
       if (!ac)
         return ENOMEM;
       own_buf = true;
     }
     bufsize = size;
     // TODO: Handle allocation failures.
   } else {

libc/src/__support/char_vector.h

 //===-- Standalone implementation of a char vector --------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIBC_SRC_SUPPORT_CHARVECTOR_H
 #define LLVM_LIBC_SRC_SUPPORT_CHARVECTOR_H
 
+#include "src/__support/CPP/new.h"
 #include "src/__support/common.h"
 
 #include <stddef.h>
 #include <stdlib.h> // For allocation.
 
 namespace __llvm_libc {
 
 // This is very simple alternate of the std::string class. There is no
 // bounds check performed in any of the methods. The callers are expected to
 // do the checks before invoking the methods.
 //
 // This class will be extended as needed in future.
 
 class CharVector {
   static constexpr size_t INIT_BUFF_SIZE = 64;
   char local_buffer[INIT_BUFF_SIZE];
   char *cur_str = local_buffer;
   size_t cur_buff_size = INIT_BUFF_SIZE;
   size_t index = 0;
 
 public:
   CharVector() = default;
   LIBC_INLINE ~CharVector() {
     if (cur_str != local_buffer)
-      free(cur_str);
+      delete (cur_str);
   }
 
   // append returns true on success and false on allocation failure.
   LIBC_INLINE bool append(char new_char) {
     // Subtract 1 for index starting at 0 and another for the null terminator.
     if (index >= cur_buff_size - 2) {
       // If the new character would cause the string to be longer than the
       // buffer's size, attempt to allocate a new buffer.
+      AllocChecker ac;
       cur_buff_size = cur_buff_size * 2;
       if (cur_str == local_buffer) {
         char *new_str;
-        new_str = reinterpret_cast<char *>(malloc(cur_buff_size));
-        if (new_str == NULL) {
+        new_str = new (ac) char[cur_buff_size];
+        if (!ac) {
           return false;
         }
         // TODO: replace with inline memcpy
         for (size_t i = 0; i < index; ++i)
           new_str[i] = cur_str[i];
         cur_str = new_str;
       } else {
-        cur_str = reinterpret_cast<char *>(realloc(cur_str, cur_buff_size));
-        if (cur_str == NULL) {
+        cur_str = reinterpret_cast<char *>(
+            AllocChecker::realloc(cur_str, cur_buff_size, ac));
+        if (!ac) {
           return false;
         }
       }
     }
     cur_str[index] = new_char;
     ++index;
     return true;
   }