This is an archive of the discontinued LLVM Phabricator instance.

[OpenMP] Introduced a bump-like allocator into the target memory management
AbandonedPublic

Authored by tianshilei1992 on Aug 4 2020, 9:32 PM.

Details

Summary

In this patch, a bump-like allocator is introduced to improve the efficiency of
target memory allocation and deallocation. The reason it is called a bump-like
allocator is it supports deallocation. Everytime a memory allocation request is
from the memory manager, it first checks whether there is a fitted slab. If not,
create a new slab. Bump the pointer in the slab and return to the memory manager,
like a regular bump allocator. We set a reference count for each slab to track
the number of memory requests from the slab.

Of course it is possible that we may run out of target memory. Then the memory
manager will try to release all its buffered memory in its FreeLists. For each
deallocation, the allocator first finds the slab it is from. If it cannot find
a slab, then it must be allocated directly on device so it will be deallocated
directly from the device. Otherwise, just decrease the reference count of the
slab. If the reference count is zero, which means all memory allocated are
returned. So we will call the device routine to deallocate the slab.

It is worth noting that the deallocation might only be called in three cases:

  1. Destructor of the memory manager;
  2. We are out of memory. The memory manager wants to free all its hold free memory;
  3. A large block of memory which is not allocated from a slab. For this case,

we might optimize it such that if the size is not too large, we may take it as
another slab when it is gonna be released by the memory manager.

Tests, comments, and debug output have not been added yet, but will soon.

Diff Detail

Event Timeline

tianshilei1992 created this revision.Aug 4 2020, 9:32 PM
Herald added a project: Restricted Project. · View Herald TranscriptAug 4 2020, 9:32 PM
tianshilei1992 requested review of this revision.Aug 4 2020, 9:32 PM

The reference counting to free is interesting. I don't think I've seen that on a bump allocator before. It doesn't allow memory reuse within a slab. I wonder whether there is usually some outstanding reference into the slab for most of the execution.

A common allocator pattern is a variant on N allocators, each for a fixed size. That makes alloc/free a stack operation, where you free the whole block if the stack becomes empty. Alloc as cheap as here, incremental free possible. Cost is the unused parts of the stack.

Go much beyond that, into allocating from the next size up etc, and one ends up with a heap. It's possible that's the end point here - do we actually just want malloc?

grokos added a subscriber: grokos.Oct 5 2020, 4:45 AM

I just revisited this patch. It seems it's based on a very early implementation of the base memory manager patch (D81054). Can you rebase the patch so that we review it? Thanks!

I just revisited this patch. It seems it's based on a very early implementation of the base memory manager patch (D81054). Can you rebase the patch so that we review it? Thanks!

Sure, no problem. I'm currently working on another patches, and will rebase it later.

simoll added a subscriber: simoll.Dec 10 2020, 12:00 AM
tianshilei1992 abandoned this revision.Jan 10 2021, 4:36 PM