[scudo] Provide allocator declaration

[ddcc]

Summary

Ensure that extern allocator declaration is visible before definition

[vitalybuka]

Can you please split #include cleanup and the rest?

compiler-rt/lib/scudo/standalone/string_utils.cpp

13

there is memset on :50

compiler-rt/lib/scudo/standalone/vector.h

14

it's used on :46

compiler-rt/lib/scudo/standalone/wrappers_c.h

22

Why do we need this?

[ddcc]

Do you split want the non-#include changes split out?

compiler-rt/lib/scudo/standalone/string_utils.cpp

13

Same thing, common.h already provides it.

compiler-rt/lib/scudo/standalone/vector.h

14

common.h includes string.h already, so this additional include is unnecessary. We have some custom library paths in our build system, so I needed to go through each include.

compiler-rt/lib/scudo/standalone/wrappers_c.h

22

-Wmissing-variable-declarations complains about no previous extern declaration for Allocator in wrappers_c.cpp, so the intention was to centralize the duplicate extern declarations between wrappers_c.cpp and wrapper_cpp.cpp, but I think that got lost while splitting the commits, and I missed that there's a SCUDO_PREFIX macro here, so I'll just add a declaration to wrappers_c.cpp instead.

[ddcc]

Fix extern declaration of SCUDO_ALLOCATOR

[vitalybuka]

compiler-rt/lib/scudo/standalone/vector.h

14

Unless there is a strong reason to do so, I am for IWYU.

[ddcc]

Revert header file removal

compiler-rt/lib/scudo/standalone/vector.h

14

Ok, I've dropped the header removals from this patch

[vitalybuka]

compiler-rt/lib/scudo/standalone/wrappers_c.h

22

Cab we move extern to the patch where actually it's necessary?

[ddcc]

compiler-rt/lib/scudo/standalone/wrappers_c.cpp

25 ↗

(On Diff #416584)

@vitalybuka Are you saying to split this change off into another patch?

[vitalybuka]

compiler-rt/lib/scudo/standalone/common.h

120

There is the same in cpp file.
I believe the point was to hide declaration from the header to avoid use by mistake.

compiler-rt/lib/scudo/standalone/wrappers_c.cpp

31–32 ↗

(On Diff #416584)

if we need extern, I assumed there is a some new user outside this cpp file.
But I don't see benefits of having extern just before the actual var declaration.

I believe correct approach is how it's done in wrappers_c_bionic.cpp.

Separate patch for unrelated stuff will be nice. Actually each of this fixes better to have as a separate patch with "why" in description.

[ddcc]

compiler-rt/lib/scudo/standalone/wrappers_c.cpp

31–32 ↗

(On Diff #416584)

The comment above this line seems to suggest that it's not feasible to have separate allocators for C and C++. I did try it and it seemed to work fine locally, but I suspect there may be other issues that haven't yet appeared. For example, what if a heap allocation from the C++ scudo allocator is freed by C code or vice versa?

[vitalybuka]

compiler-rt/lib/scudo/standalone/wrappers_c.cpp

31–32 ↗

(On Diff #416584)

I see, I missed cpp file

then probably in wrappers_c.h
#if !SCUDO_ANDROID || !_BIONIC
extern HIDDEN scudo::Allocator<scudo::Config, malloc_postinit> Allocator;
#endif

and remove extern in cpp file

[ddcc]

Move extern declarations to common header

compiler-rt/lib/scudo/standalone/checksum.cpp

 //===-- checksum.cpp --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "checksum.h"
 #include "atomic_helpers.h"
+#include "chunk.h"
 
 #if defined(__x86_64__) || defined(__i386__)
 #include <cpuid.h>
 #elif defined(__arm__) || defined(__aarch64__)
 #if SCUDO_FUCHSIA
 #include <zircon/features.h>
 #include <zircon/syscalls.h>
 #else

compiler-rt/lib/scudo/standalone/common.h

for (u8 I = 0; I < Count; I++)
     __asm__ __volatile__("yield");
 #endif
   __asm__ __volatile__("" ::: "memory");
 }
 
 // Platform specific functions.
 
 extern uptr PageSizeCached;
+uptr getPageSize();
 uptr getPageSizeSlow();
 inline uptr getPageSizeCached() {
   // Bionic uses a hardcoded value.
   if (SCUDO_ANDROID)
     return 4096U;
   if (LIKELY(PageSizeCached))
     return PageSizeCached;
   return getPageSizeSlow();

compiler-rt/lib/scudo/standalone/flags_parser.cpp

 //===-- flags_parser.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "flags_parser.h"
 #include "common.h"
 #include "report.h"
 
 #include <stdlib.h>
-#include <string.h>
 
 namespace scudo {
 
 class UnknownFlagsRegistry {
   static const u32 MaxUnknownFlags = 16;
   const char *UnknownFlagsNames[MaxUnknownFlags];
   u32 NumberOfUnknownFlags;
 

compiler-rt/lib/scudo/standalone/mutex.h

 //===-- mutex.h -------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_MUTEX_H_
 #define SCUDO_MUTEX_H_
 
 #include "atomic_helpers.h"
 #include "common.h"
 
-#include <string.h>
-
 #if SCUDO_FUCHSIA
 #include <lib/sync/mutex.h> // for sync_mutex_t
 #endif
 
 namespace scudo {
 
 class HybridMutex {
 public:

compiler-rt/lib/scudo/standalone/secondary.h

template <typename F> void iterateOverBlocks(F Callback) const {
     for (const auto &H : InUseBlocks) {
       uptr Ptr = reinterpret_cast<uptr>(&H) + LargeBlock::getHeaderSize();
       if (allocatorSupportsMemoryTagging<Config>())
         Ptr = untagPointer(Ptr);
       Callback(Ptr);
     }
   }
 
-  uptr canCache(uptr Size) { return Cache.canCache(Size); }
+  bool canCache(uptr Size) { return Cache.canCache(Size); }
 
   bool setOption(Option O, sptr Value) { return Cache.setOption(O, Value); }
 
   void releaseToOS() { Cache.releaseToOS(); }
 
   void disableMemoryTagging() { Cache.disableMemoryTagging(); }
 
   void unmapTestOnly() { Cache.unmapTestOnly(); }

compiler-rt/lib/scudo/standalone/string_utils.cpp

 //===-- string_utils.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "string_utils.h"
 #include "common.h"
 
 #include <stdarg.h>
-#include <string.h>
 
 namespace scudo {
 
 static int appendChar(char **Buffer, const char *BufferEnd, char C) {
   if (*Buffer < BufferEnd) {
     **Buffer = C;
     (*Buffer)++;
   }

compiler-rt/lib/scudo/standalone/tests/bytemap_test.cpp

 //===-- bytemap_test.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
 #include "bytemap.h"
 
-#include <pthread.h>
-#include <string.h>
-
 template <typename T> void testMap(T &Map, scudo::uptr Size) {
   Map.init();
   for (scudo::uptr I = 0; I < Size; I += 7)
     Map.set(I, (I % 100) + 1);
   for (scudo::uptr J = 0; J < Size; J++) {
     if (J % 7)
       EXPECT_EQ(Map[J], 0);
     else

compiler-rt/lib/scudo/standalone/tests/flags_test.cpp

 //===-- flags_test.cpp ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
 #include "flags.h"
 #include "flags_parser.h"
 
-#include <string.h>
-
 static const char FlagName[] = "flag_name";
 static const char FlagDesc[] = "flag description";
 
 template <typename T>
 static void testFlag(scudo::FlagType Type, T StartValue, const char *Env,
                      T FinalValue) {
   scudo::FlagParser Parser;
   T Flag = StartValue;

compiler-rt/lib/scudo/standalone/tests/map_test.cpp

 //===-- map_test.cpp --------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
 #include "common.h"
 
-#include <string.h>
 #include <unistd.h>
 
 static const char *MappingName = "scudo:test";
 
 TEST(ScudoMapTest, PageSize) {
   EXPECT_EQ(scudo::getPageSizeCached(),
             static_cast<scudo::uptr>(getpagesize()));
 }

compiler-rt/lib/scudo/standalone/tests/mutex_test.cpp

 //===-- mutex_test.cpp ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
 #include "mutex.h"
 
 #include <pthread.h>
-#include <string.h>
 
 class TestData {
 public:
   explicit TestData(scudo::HybridMutex &M) : Mutex(M) {
     for (scudo::u32 I = 0; I < Size; I++)
       Data[I] = 0;
   }
 

compiler-rt/lib/scudo/standalone/tests/release_test.cpp

 //===-- release_test.cpp ----------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "tests/scudo_unit_test.h"
 
 #include "list.h"
 #include "release.h"
 #include "size_class_map.h"
 
-#include <string.h>
-
 #include <algorithm>
 #include <random>
 #include <set>
 
 TEST(ScudoReleaseTest, PackedCounterArray) {
   for (scudo::uptr I = 0; I < SCUDO_WORDSIZE; I++) {
     // Various valid counter's max values packed into one word.
     scudo::PackedCounterArray Counters2N(1U, 1U, 1UL << I);

compiler-rt/lib/scudo/standalone/vector.h

 //===-- vector.h ------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef SCUDO_VECTOR_H_
 #define SCUDO_VECTOR_H_
 
 #include "common.h"
 
-#include <string.h>
-
 namespace scudo {
 
 // A low-level vector based on map. May incur a significant memory overhead for
 // small vectors. The current implementation supports only POD types.
 template <typename T> class VectorNoCtor {
 public:
   constexpr void init(uptr InitialCapacity = 0) {
     Data = &LocalData[0];

compiler-rt/lib/scudo/standalone/wrappers_c.h

 
 // Bionic's struct mallinfo consists of size_t (mallinfo(3) uses int).
 #if SCUDO_ANDROID
 typedef size_t __scudo_mallinfo_data_t;
 #else
 typedef int __scudo_mallinfo_data_t;
 #endif
 
+extern "C" void malloc_postinit();
+extern scudo::Allocator<scudo::Config, malloc_postinit> Allocator;
+
 struct __scudo_mallinfo {
   __scudo_mallinfo_data_t arena;
   __scudo_mallinfo_data_t ordblks;
   __scudo_mallinfo_data_t smblks;
   __scudo_mallinfo_data_t hblks;
   __scudo_mallinfo_data_t hblkhd;
   __scudo_mallinfo_data_t usmblks;
   __scudo_mallinfo_data_t fsmblks;