[ADT] add FixedConcurrentHashTable class.

[avl]

Summary

This review is extracted from D96035.

It adds a hash table that could be filled asynchronously.
A possible use case for that kind of hashtable is keeping strings
(with possible additional data) and comparing them by
pointers comparisons. f.e. it might be used instead of
NonRelocatableStringpool which cannot be used in parallel.

The implementation of the hashmap is copied(with modifications)
from lld/COFF/DebugTypes.cpp.

[rnk]

@MaskRay, can you review this? I think you have been thinking the most about how to parallelize linkers, and what fundamental data structures are most important.

[MaskRay]

In D125979#3528140, @rnk wrote:

@MaskRay, can you review this? I think you have been thinking the most about how to parallelize linkers, and what fundamental data structures are most important.

Will try to do my best. I'll also need to spend some time on reading D96035...

[MaskRay]

LockFreeDataPool

The name is not descriptive. It misses the important words about "hash map". ConcurrentHashMap may be a better name.

The slab system appears to be the modification to lld/COFF/DebugTypes.cpp.
It provides limited growing capability when the capacity is hit.
There is a hard limit of 6 slabs, and report_fatal_error may eventually be called. Then why is reportWarning needed?
I think we should either implement a proper growing mechanism or just crash.

Does your application have a reasonable way estimating the capacity? Does it help to use HyperLogLog?
(There is a poor way to not implement a growing version: If we want to prevent a crash in a pathological case where the estimated capacity is insufficient and causes the concurrent hash map to crash, we can use something like lld::safeLldMain (llvm::CrashRecoveryContext).)

llvm/include/llvm/ADT/LockFreeDataPool.h

26 ↗

(On Diff #430692)

I think mentioning the origin in the summary is sufficient, no need to keep it in the comment.

58 ↗

(On Diff #430692)

Why so large?

159 ↗

(On Diff #430692)

relaxed load

229 ↗

(On Diff #430692)

MaxBucketLength needs an explanation.

236 ↗

(On Diff #430692)

Is this useful? See my main comment.

[MaskRay]

llvm/include/llvm/ADT/LockFreeDataPool.h

113 ↗

(On Diff #430692)

relaxed load on NextSlab and save it into a local variable.

[dblaikie]

(any chance we can avoid duplicating this, and instead refactor the lld one into LLVM and have lld use that one instead of using its own local copy?)

[avl]

The name is not descriptive. It misses the important words about "hash map". ConcurrentHashMap may be a better name.

The data structure which I need is a concurrent data pool of unique objects.
That pool should keep objects and matching objects should have the same entry.

The hash map is an implementation detail. The pool uses hash map to control the data entries. There might be used another(other than hash map) structure controlling pool entries.

Probably, it is possible to separate this LockFreeDataPool into two classes data pool and hash map.

The slab system appears to be the modification to lld/COFF/DebugTypes.cpp.
It provides limited growing capability when the capacity is hit.
There is a hard limit of 6 slabs, and report_fatal_error may eventually be called. Then why is reportWarning needed?
I think we should either implement a proper growing mechanism or just crash.

The best use case for that hash map is when it is allocated once and no growings are necessary. But, there are use cases when it is hard to estimate initial size. In such cases, without resizing, application will crash or need to waste memory(to allocate a huge amount of memory covering exceptional cases).

Slab system in this patch allows to resize hash map by the price of slower search(that is the source of restriction by the number of resizes). This allows to allocate smaller hash map initially and reallocate it for the exceptionally large cases. The purpose of the warning is to inform user that an exceptional case is hit.

Does your application have a reasonable way estimating the capacity?

I use an coefficient for incoming DWARF to calculate the size of hash map. In most cases it is quite accurate. But there are some exceptional cases when estimation is not accurate and then possibility to do resize helps.

Does it help to use HyperLogLog?

Let me examine this, please.

(There is a poor way to not implement a growing version: If we want to prevent a crash in a pathological case where the estimated capacity is insufficient and causes the concurrent hash map to crash, we can use something like lld::safeLldMain (llvm::CrashRecoveryContext).)

I will check it, thanks. Do you think it would be better than possibility to resize?

llvm/include/llvm/ADT/LockFreeDataPool.h

58 ↗

(On Diff #430692)

to avoid re-sizings for small sizes. It does not look to be too big value, but will do working with small sizes more smooth.

113 ↗

(On Diff #430692)

something like this ?:

Slab *Slab = CurSlab->NextSlab.load(std::memory_order_relaxed);

if (Slab == nullptr) {
      const std::lock_guard<std::mutex> lock(SlabsMutex);

      if (Slab  == nullptr)   <<<<<<<<<<<
        CurSlab->NextSlab = allocateNextSlab(Allocator);
    }
}

It looks like we could not save NextSlab into the local variable because CurSlab->NextSlab can change the value when we arrive at pointed line..

[avl]

In D125979#3529745, @dblaikie wrote:

(any chance we can avoid duplicating this, and instead refactor the lld one into LLVM and have lld use that one instead of using its own local copy?)

I thought about that. But the implementation from lld looks too specific for general implementation.
It uses knowledge about stored data:

1. It keeps data together with hash inside hash table. This is OK for 32-bit integer but it would not be OK for larger type.

2. It compares data to select which version should be stored. We need a way to do it or avoid it without introducing more checks.

if (!oldCell.isEmpty() >>>&& oldCell < newCell<<<)
  return idx;

I do not have effective way on how to put that behaviour in general implementation at the moment.
Probably, that might be done by inheritance and redefinition in child implementations.

[dblaikie]

In D125979#3532163, @avl wrote:

In D125979#3529745, @dblaikie wrote:

(any chance we can avoid duplicating this, and instead refactor the lld one into LLVM and have lld use that one instead of using its own local copy?)

I thought about that. But the implementation from lld looks too specific for general implementation.
It uses knowledge about stored data:

1. It keeps data together with hash inside hash table. This is OK for 32-bit integer but it would not be OK for larger type.

Other LLVM data structures make different choices based on the size of the content (eg: the default size of SmallVector varies depending on the size of the elements) - using traits or other template techniques seems suitable.

2. It compares data to select which version should be stored. We need a way to do it or avoid it without introducing more checks.

if (!oldCell.isEmpty() >>>&& oldCell < newCell<<<)
  return idx;

Yeah, some sort of trait could be used to implement that (but maybe there are other broader abstractions) - like maybe that functionality should stay out in lld, even if the data structure's moved into LLVM - lld could "insert, check that the thing was already present, then do the comparison/decide whether to overwrite" - doesn't sound like it'd be a performance issue.

[avl]

Ok, thanks. Will change review to use a general implementation in lld and DWARFLinker.

[avl]

addressed comments:

1. renamed LockFreeDataPool -> ConcurrentHashTable.
2. implemented support for both lld and DWARFLinker implementations.

[dblaikie]

llvm/include/llvm/ADT/ConcurrentHashTable.h

83 ↗

(On Diff #432913)

(similarly below/for the other ctor here)

117–118 ↗

(On Diff #432913)

It's pretty weird to have warnings coming out of data structures. @MaskRay would it be reasonable to remove these/replace them with some other tool if we're promoting this to a more general purpose data structure rather than something more special-case in lld?

315–316 ↗

(On Diff #432913)

This should probably be provided via specialization, rather than as a separately named type?

I guess it has different requirements for the type, by the looks of it? That's curious/maybe enough to warrant a separate name, I guess, though I'm not 100% sure.

324 ↗

(On Diff #432913)

generally operator== should be implemented as a non-member (so that implicit conversions are equally applied to the left and right hand operands) - maybe this whole specification area should be written less in terms of specific declarations that should be provided, and more in terms of code snippets that should be valid/have certain semantics? (that's how the C++ standard documents requirements on things like value types, iterator types, etc)

llvm/unittests/ADT/ConcurrentHashTableTest.cpp

44–46 ↗

(On Diff #432913)

[avl]

llvm/include/llvm/ADT/ConcurrentHashTable.h

117–118 ↗

(On Diff #432913)

Probably, instead of warning we can introduce method:

bool IsInGoodState () const {
  return CurSlabNum <= MaxNumberOfSlabs && !hasFullSlab;
}

the consumer could call this method somewhere and inform the user:

if (!HashTable.IsInGoodState()) {
    outs() << "Hash table is in suboptimal state";
}

?

315–316 ↗

(On Diff #432913)

They have different interface:

std::pair<KeyDataTy, bool> insert(KeyDataTy NewData)

and

std::pair<KeyDataTy *, bool> insert(AllocatorTy &Allocator, KeyTy Key)

Implementing this through the specialization looks to be more complex and it seems does not have benefits comparing to the variant with separately named types?

[dblaikie]

llvm/include/llvm/ADT/ConcurrentHashTable.h

117–118 ↗

(On Diff #432913)

Maybe? I'd be curious to hear from @MaskRay here, since I guess he wrote the original/had the original motivation for the warning.

315–316 ↗

(On Diff #432913)

Fair enough. I wonder if it'd be acceptable to have the hash table take the allocator as a ctor parameter (& then keep the same API otherwise) and store it, or if there are too many of these that the extra pointer member would be too costly.

Though coming back to the original comment - why is external allocation usage/recommendation related to the size of the keys/size of a pointer?

[avl]

llvm/include/llvm/ADT/ConcurrentHashTable.h

117–118 ↗

(On Diff #432913)

that is me, who introduced the warning. Original lld implementation does not have it.

My motivation is that it should be known somehow that hashtable is not in good state and it is necessary to revise initial size.

315–316 ↗

(On Diff #432913)

Fair enough. I wonder if it'd be acceptable to have the hash table take the allocator as a ctor parameter (& then keep the same API otherwise) and store it, or if there are too many of these that the extra pointer member would be too costly.

The reason is necessity to synchronize allocator.

The allocator may be passed as the ctor parameter if it would be able to work in multy-thread mode. This implementation assumes that allocator is not able to work in multi-thread mode. To properly working it requires that the same allocator is used from the same thread. In that case no need to synchronize allocator.

1. std::pair<KeyDataTy*, bool> insert(KeyTy Key)

The insert method might be called from different threads. if It would call to allocator stored in the constructor then we need to use mutex or to use special multi-thread implementation for the allocator.

2. std::pair<KeyDataTy *, bool> insert(AllocatorTy &Allocator, KeyTy Key)

This solution assumes that the same thread uses the same allocator. It allows not to synchronize allocators.

Though coming back to the original comment - why is external allocation usage/recommendation related to the size of the keys/size of a pointer?

entry size for the solution with external allocation could not be less than pointer(since each entry is a pointer to the externally allocated data).

entry size for the solution with allocating inside hash table can be less and then the table may have smaller size. It also does not need special allocator, since data are kept inside hash table.

We could not always keep data inside hash table: If data size is greater then pointer then each entry(even null) would occupy bigger size. That would result in a bigger hash table. It would also result in copying extra data, since data entries are passed by value in that case.

So if entry size is less than pointer then it would require less size and no additional allocator required when ConcurrentHashTable is used. If entry size is greater than pointer then less space is used(no need to allocate full data for null entries) and data copying is faster when ConcurrentHashTableExtAlloc is used.

[MaskRay]

llvm/include/llvm/ADT/ConcurrentHashTable.h

117–118 ↗

(On Diff #432913)

The root issue is that the patch does not implement proper grow/resize, so setting a capacity and having oversize warnings here look like hacks. For many general-purpose usages it's difficult to estimate the size beforehand, or without heavily refactoring the usage pattern. So I wish that we implement a proper resizable hashtable with amortized O(1) time complexity, not the current version which will be slow if the actual size is much more larger than the reserved capacity.

[avl]

llvm/include/llvm/ADT/ConcurrentHashTable.h

117–118 ↗

(On Diff #432913)

This is correct. This implementation is not a good choice for the case when it is difficult to estimate the size beforehand.

But this might be a good choice for the cases when such estimation is possible. F.e. lld/COFF already uses
this implementation even without the resizing feature. I used it for DWARF files. All clang/llvm binaries are processed
without warning(and far from the resizing limits). That means that estimations are quite accurate. Still, resizing feature might be helpful for exceptional cases.

The implementation from this patch has several advantages: it is fast and lightweight, it does not require extra memory for buckets, does not require locking table or buckets for most cases, and does not require any additional memory management for the most cases.

For the task when it is not possible to estimate the size beforehand - there should be used another implementation.

[avl]

addressed comments: removed warningHandler(added IsInSuboptimalState instead),
changed specification comments.

[avl]

removed resizing(as I am going to submit another resizeable solution),
refactored to have a single "insert" method.

[avl]

the patch with resizeable hashtable D132455

[avl]

Performance results for this patch https://reviews.llvm.org/file/data/a36xmbx43mp35xx5rznf/PHID-FILE-lncmoijytnb3gn4gjrii/Performance.pdf (Collected by utility from : D132548)

[avl]

closed in favor of D132455

llvm/include/llvm/ADT/FixedConcurrentHashTable.h

+//===- FixedConcurrentHashTable.h -------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_ADT_FIXEDCONCURRENTHASHTABLE_H
+#define LLVM_ADT_FIXEDCONCURRENTHASHTABLE_H
+
+#include "llvm/ADT/Hashing.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/NativeFormatting.h"
+#include "llvm/Support/WithColor.h"
+#include <atomic>
+
+namespace llvm {
+
+/// FixedConcurrentHashTable - is a non resizeable concurrent lock-free
+/// hashtable.
+///
+/// It is based on this paper:
+/// Concurrent Hash Tables: Fast and General(?)!
+/// https://dl.acm.org/doi/10.1145/3309206
+///
+/// This hash table is meant to be used in two phases:
+/// 1. concurrent insertions
+/// 2. concurrent reads
+/// It does not support lookup, deletion, or rehashing. It uses linear probing.
+///
+/// The paper describes storing a key-value pair in two machine words.
+///
+/// FixedConcurrentHashTable: keeps a KeyDataTy(which is a key-value pair).
+/// The state when all data bits of KeyDataTy are zero is reserved as
+/// a hashtable tombstone value.
+///
+/// FixedConcurrentHashTableExternalAlloc: keeps a pointer to the KeyDataTy
+/// class (which is a key-value pair), whose data are allocated and kept by the
+/// external allocator.
+
+template <typename KeyTy, typename KeyDataTy, typename AllocatorTy,
+          bool KeepDataInsideTable, typename Info>
+class FixedConcurrentHashTableBase {
+public:
+  using MapEntryTy = typename std::conditional<KeepDataInsideTable, KeyDataTy,
+                                               KeyDataTy *>::type;
+  using InsertionTy =
+      typename std::conditional<KeepDataInsideTable, KeyDataTy, KeyTy>::type;
+
+  static_assert((sizeof(MapEntryTy) == 1 || sizeof(MapEntryTy) == 2 ||
+                 sizeof(MapEntryTy) == 4 || sizeof(MapEntryTy) == 8),
+                "sizeof(MapEntryTy) should be of 1,2,4 or 8");
+  static_assert((sizeof(MapEntryTy) == std::alignment_of<MapEntryTy>::value),
+                "MapEntryTy should be naturally aligned");
+
+  FixedConcurrentHashTableBase() = delete;
+  FixedConcurrentHashTableBase(size_t InitialSize, AllocatorTy &Allocator)
+      : Allocator(Allocator) {
+    /// Initialize the table with the given size.
+    TableSize = NextPowerOf2(InitialSize);
+    HashTable = Allocator.template Allocate<MapEntryTy>(TableSize);
+    memset(HashTable, 0, TableSize * sizeof(MapEntryTy));
+  }
+  ~FixedConcurrentHashTableBase() {
+    Allocator.template Deallocate<MapEntryTy>(HashTable);
+  }
+
+  /// Print information about current state of hash table structures.
+  void printStatistic(raw_ostream &OS) {
+    OS << "\n--- HashTable statistic:\n";
+    OS << "\n HashTable size: " << TableSize;
+    size_t NumberOfEntries = getNumberOfEntriesForTable();
+    OS << "\n Number of entries: " << NumberOfEntries;
+    OS << "\n Load factor: "
+       << formatv("{0}", static_cast<float>(NumberOfEntries) /
+                             static_cast<float>(TableSize));
+
+    OS << "\n Overall size(in bytes): ";
+    if constexpr (KeepDataInsideTable)
+      OS << ((TableSize * sizeof(MapEntryTy)) + sizeof(*this));
+    else
+      OS << (TableSize * sizeof(MapEntryTy) + sizeof(*this) +
+             NumberOfEntries * sizeof(KeyDataTy));
+
+    OS << "\n\n";
+  }
+
+protected:
+  // Returns number of used elements inside hashtable.
+  size_t getNumberOfEntriesForTable() {
+    size_t NumberOfEntries = 0;
+    for (size_t CurIdx = 0; CurIdx < TableSize; CurIdx++) {
+      if (!isNull(HashTable[CurIdx]))
+        NumberOfEntries++;
+    }
+
+    return NumberOfEntries;
+  }
+
+  template <typename... Types>
+  std::pair<MapEntryTy, bool> insertImpl(InsertionTy NewData,
+                                         Types &... Alloc) {
+    // Do a linear probe starting at startIdx.
+    uint32_t StartIdx = getHashCode(NewData) & getHashMask();
+    uint32_t Idx = StartIdx;
+    MapEntryTy DataToInsert;
+    setToNull(&DataToInsert);
+
+    while (true) {
+      // Run a compare and swap loop. There are three cases:
+      // - cell is empty: CAS into place and return.
+      // - cell has matching key, return cell`s data.
+      // - cell has non-matching key: hash collision, probe next cell.
+      std::atomic<MapEntryTy> &EntryRef =
+          reinterpret_cast<std::atomic<MapEntryTy> &>(HashTable[Idx]);
+      MapEntryTy Candidate = EntryRef.load(std::memory_order_acquire);
+      if (isNull(Candidate)) {
+        // Create new entry.
+        if constexpr (KeepDataInsideTable)
+          DataToInsert = NewData;
+        else if (isNull(DataToInsert))
+          DataToInsert = Info::create(NewData, Alloc...);
+
+        // Try to put new entry into the table.
+        if (EntryRef.compare_exchange_weak(Candidate, DataToInsert,
+                                           std::memory_order_release))
+          return std::make_pair(DataToInsert, true);
+        continue;
+      }
+
+      if (isEqual(Candidate, NewData)) {
+        // Check order and overwrite entry if necessary.
+        if constexpr (KeepDataInsideTable) {
+          if constexpr (is_detected<haslessThan, Info>::value) {
+            // Check order and overwrite entry if necessary.
+            if (Info::lessThan(NewData, Candidate)) {
+              if (EntryRef.compare_exchange_weak(Candidate, NewData,
+                                                 std::memory_order_release))
+                return std::make_pair(NewData, true);
+
+              continue;
+            }
+          }
+        }
+
+        // Return already existing entry.
+        return std::make_pair(Candidate, false);
+      }
+
+      // Advance the probe. Wrap around to the beginning if we run off the end.
+      ++Idx;
+
+      Idx = Idx == this->TableSize ? 0 : Idx;
+      if (Idx == StartIdx) {
+        // If this becomes an issue, we could mark failure and rehash from the
+        // beginning with a bigger table. There is no difference between
+        // rehashing internally and starting over.
+        report_fatal_error("hash table is full");
+      }
+    }
+    llvm_unreachable("left infloop");
+  }
+
+  template <typename MapEntryTy,
+            typename std::enable_if<(std::is_pointer<MapEntryTy>::value),
+                                    bool>::type = true>
+  bool isNull(MapEntryTy Data) {
+    return Data == nullptr;
+  }
+
+  template <typename MapEntryTy,
+            typename std::enable_if<(!std::is_pointer<MapEntryTy>::value &&
+                                     sizeof(MapEntryTy) == sizeof(uint8_t)),
+                                    bool>::type = true>
+  bool isNull(MapEntryTy Data) {
+    return *reinterpret_cast<uint8_t *>(&Data) == static_cast<uint8_t>(0);
+  }
+
+  template <typename MapEntryTy,
+            typename std::enable_if<(!std::is_pointer<MapEntryTy>::value &&
+                                     sizeof(MapEntryTy) == sizeof(uint16_t)),
+                                    bool>::type = true>
+  bool isNull(MapEntryTy Data) {
+    return *reinterpret_cast<uint16_t *>(&Data) == static_cast<uint16_t>(0);
+  }
+
+  template <typename MapEntryTy,
+            typename std::enable_if<(!std::is_pointer<MapEntryTy>::value &&
+                                     sizeof(MapEntryTy) == sizeof(uint32_t)),
+                                    bool>::type = true>
+  bool isNull(MapEntryTy Data) {
+    return *reinterpret_cast<uint32_t *>(&Data) == static_cast<uint32_t>(0);
+  }
+
+  template <typename MapEntryTy,
+            typename std::enable_if<(!std::is_pointer<MapEntryTy>::value &&
+                                     sizeof(MapEntryTy) == sizeof(uint64_t)),
+                                    bool>::type = true>
+  bool isNull(MapEntryTy Data) {
+    return *reinterpret_cast<uint64_t *>(&Data) == static_cast<uint64_t>(0);
+  }
+
+  static inline bool setToNull(MapEntryTy *Data) {
+    return memset(Data, 0, sizeof(MapEntryTy));
+  }
+
+  bool isEqual(MapEntryTy LHS, const KeyTy &RHS) {
+    return Info::isEqual(Info::getKey(*LHS), RHS);
+  }
+
+  bool isEqual(MapEntryTy LHS, const KeyDataTy &RHS) {
+    return Info::isEqual(Info::getKey(LHS), Info::getKey(RHS));
+  }
+
+  hash_code getHashCode(const KeyTy &Data) { return Info::getHashValue(Data); }
+
+  hash_code getHashCode(const KeyDataTy &Data) {
+    return Info::getHashValue(Info::getKey(Data));
+  }
+
+  template <typename T>
+  using haslessThan = decltype(std::declval<T &>().lessThan(
+      std::declval<const KeyDataTy &>(), std::declval<const KeyDataTy &>()));
+
+  size_t getHashMask() { return TableSize - 1; }
+
+  // Pointer to the Hashtable.
+  MapEntryTy *HashTable = nullptr;
+
+  // Size of Hashtable.
+  size_t TableSize = 0;
+
+  // This allocator is used to allocate and deallocate hashtable.
+  AllocatorTy &Allocator;
+};
+
+/// FixedConcurrentHashTable:
+///
+
+template <typename KeyTy, class KeyDataTy> class FixedConcurrentHashTableInfo {
+public:
+  static inline hash_code getHashValue(KeyTy Key) {
+    return std::hash<KeyTy>()(Key);
+  }
+  static inline bool isEqual(KeyTy LHS, KeyTy RHS) { return LHS == RHS; }
+  static inline KeyTy getKey(const KeyDataTy &Data) { return Data.key(); }
+
+  // Optional:
+  // static inline bool lessThan(const KeyDataTy& LHS, const KeyDataTy& RHS) {
+  // return LHS < RHS; }
+};
+
+template <typename KeyTy, typename KeyDataTy,
+          typename AllocatorTy = BumpPtrAllocator,
+          typename Info = FixedConcurrentHashTableInfo<KeyTy, KeyDataTy>>
+class FixedConcurrentHashTable
+    : public FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, true,
+                                          Info> {
+public:
+  FixedConcurrentHashTable(size_t InitialSize, AllocatorTy &Allocator)
+      : FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, true, Info>(
+            InitialSize, Allocator) {}
+
+  /// Insert new entry \p NewData or return already existing entry.
+  ///
+  /// \returns entry and "true" if an entry is just inserted or
+  /// "false" if an entry already exists.
+  std::pair<KeyDataTy, bool> insert(KeyDataTy NewData) {
+    return FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, true,
+                                        Info>::insertImpl(NewData);
+  }
+};
+
+/// FixedConcurrentHashTableExternalAlloc:
+///
+/// Insertions into the internal hash table are lock-free, and
+/// allocations are done using an external non-mt-safe data pool.
+
+template <typename KeyTy, class KeyDataTy, class AllocatorTy>
+class FixedConcurrentHashTableInfoExternalAlloc {
+public:
+  static inline hash_code getHashValue(const KeyTy &Key) {
+    return std::hash<KeyTy>()(Key);
+  }
+  static inline bool isEqual(const KeyTy &LHS, const KeyTy &RHS) {
+    return LHS == RHS;
+  }
+  static inline KeyTy getKey(const KeyDataTy &Data) { return Data.key(); }
+  static inline KeyDataTy *create(const KeyTy &Key, AllocatorTy &Allocator) {
+    return KeyDataTy::create(Key, Allocator);
+  }
+};
+
+template <class KeyTy, class KeyDataTy, class AllocatorTy = BumpPtrAllocator,
+          typename Info = FixedConcurrentHashTableInfoExternalAlloc<
+              KeyTy, KeyDataTy, AllocatorTy>>
+class FixedConcurrentHashTableExternalAlloc
+    : public FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, false,
+                                          Info> {
+public:
+  FixedConcurrentHashTableExternalAlloc(size_t InitialSize,
+                                        AllocatorTy &Allocator)
+      : FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, false,
+                                     Info>(InitialSize, Allocator) {}
+
+  /// Insert new entry for \p Key or return already existing entry.
+  ///
+  /// \returns entry and "true" if an entry is just inserted or
+  /// "false" if an entry already exists.
+  std::pair<KeyDataTy *, bool> insert(const KeyTy &Key,
+                                      AllocatorTy &ThreadLocalAllocator) {
+    return FixedConcurrentHashTableBase<KeyTy, KeyDataTy, AllocatorTy, false,
+                                        Info>::insertImpl(Key,
+                                                          ThreadLocalAllocator);
+  }
+};
+
+} // end namespace llvm
+
+#endif // LLVM_ADT_FIXEDCONCURRENTHASHTABLE_H

llvm/unittests/ADT/CMakeLists.txt

add_llvm_unittest(ADTTests
   DenseMapTest.cpp
   DenseSetTest.cpp
   DepthFirstIteratorTest.cpp
   DirectedGraphTest.cpp
   EditDistanceTest.cpp
   EnumeratedArrayTest.cpp
   EquivalenceClassesTest.cpp
   FallibleIteratorTest.cpp
+  FixedConcurrentHashtableTest.cpp
   FloatingPointMode.cpp
   FoldingSet.cpp
   FunctionExtrasTest.cpp
   FunctionRefTest.cpp
   HashingTest.cpp
   IListBaseTest.cpp
   IListIteratorTest.cpp
   IListNodeBaseTest.cpp

llvm/unittests/ADT/FixedConcurrentHashtableTest.cpp

+//===- FixedConcurrentHashtableTest.cpp -----------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/FixedConcurrentHashTable.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/Parallel.h"
+#include "gtest/gtest.h"
+#include <limits>
+#include <vector>
+using namespace llvm;
+
+namespace {
+
+class Int {
+public:
+  Int() : Data(0x0) {}
+  uint32_t key() const { return Data & 0x7FFFFFFF; }
+
+  static Int create(uint32_t Data) { return Int(Data | 0x80000000); }
+
+  friend bool operator==(const Int &LHS, const Int &RHS) {
+    return LHS.Data == RHS.Data;
+  }
+
+protected:
+  Int(uint32_t Data) : Data(Data) {}
+
+  uint32_t Data;
+};
+
+class alignas(4) OrderedInt {
+public:
+  OrderedInt() : Data(0x0), Order(0) {}
+
+  uint16_t key() const { return Data & 0x7FFF; }
+
+  friend bool operator==(const OrderedInt &LHS, const OrderedInt &RHS) {
+    return std::tie(LHS.Data, LHS.Order) == std::tie(RHS.Data, RHS.Order);
+  }
+
+  static OrderedInt create(uint16_t Data, uint16_t Order) {
+    return OrderedInt(Data | 0x8000, Order);
+  }
+
+  bool lessThan(const OrderedInt &Other) const { return Order < Other.Order; }
+
+protected:
+  OrderedInt(uint16_t Data, uint16_t Order)
+      : Data(Data | 0x8000), Order(Order) {}
+
+  uint16_t Data;
+  uint16_t Order;
+};
+
+class String {
+public:
+  static String *create(const std::string &Key, BumpPtrAllocator &Allocator) {
+    return new (Allocator) String(Key);
+  }
+
+  const std::string &key() const { return Data; }
+
+protected:
+  String(const std::string &Data) : Data(Data) {}
+  std::string Data;
+};
+
+TEST(FixedConcurrentHashTableTest, AddIntEntries) {
+  BumpPtrAllocator Allocator;
+  FixedConcurrentHashTable<uint32_t, Int> HashTable(10, Allocator);
+
+  std::pair<Int, bool> res1 = HashTable.insert(Int::create(1));
+  // Check entry is inserted.
+  EXPECT_TRUE(res1.first.key() == 1);
+  EXPECT_TRUE(res1.second);
+
+  std::pair<Int, bool> res2 = HashTable.insert(Int::create(2));
+  // Check old entry is still valid.
+  EXPECT_TRUE(res1.first.key() == 1);
+  // Check new entry is inserted.
+  EXPECT_TRUE(res2.first.key() == 2);
+  EXPECT_TRUE(res2.second);
+  // Check new and old entries are not equal.
+  EXPECT_FALSE(res1.first == res2.first);
+
+  std::pair<Int, bool> res3 = HashTable.insert(Int::create(3));
+  // Check one more entry is inserted.
+  EXPECT_TRUE(res3.first.key() == 3);
+  EXPECT_TRUE(res3.second);
+
+  std::pair<Int, bool> res4 = HashTable.insert(Int::create(1));
+  // Check duplicated entry is inserted.
+  EXPECT_TRUE(res4.first.key() == 1);
+  EXPECT_FALSE(res4.second);
+  // Check duplicated entry matches with the first one.
+  EXPECT_TRUE(res1.first == res4.first);
+
+  // Check first entry is still valid.
+  EXPECT_TRUE(res1.first.key() == 1);
+
+  // Check statistic.
+  std::string StatisticString;
+  raw_string_ostream StatisticStream(StatisticString);
+  HashTable.printStatistic(StatisticStream);
+
+  EXPECT_TRUE(StatisticString.find("HashTable statistic:") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("HashTable size: 16") != std::string::npos);
+  EXPECT_TRUE(StatisticString.find("Load factor: 0.19") != std::string::npos);
+}
+
+TEST(FixedConcurrentHashTableTest, AddOrderedIntEntries) {
+  class HashTableInfo {
+  public:
+    static inline hash_code getHashValue(uint32_t Key) {
+      return llvm::hash_value(Key);
+    }
+    static inline bool isEqual(uint32_t LHS, uint32_t RHS) {
+      return LHS == RHS;
+    }
+    static inline uint32_t getKey(const OrderedInt &Data) { return Data.key(); }
+    static inline bool lessThan(const OrderedInt &LHS, const OrderedInt &RHS) {
+      return LHS.lessThan(RHS);
+    }
+  };
+
+  BumpPtrAllocator Allocator;
+  FixedConcurrentHashTable<uint32_t, OrderedInt, BumpPtrAllocator,
+                           HashTableInfo>
+      HashTable(10, Allocator);
+
+  std::pair<OrderedInt, bool> res = HashTable.insert(OrderedInt::create(1, 2));
+  // Check entry is inserted.
+  EXPECT_TRUE(res.first == OrderedInt::create(1, 2));
+  EXPECT_TRUE(res.second);
+
+  res = HashTable.insert(OrderedInt::create(1, 3));
+  // Check entry is unchanged.
+  EXPECT_TRUE(res.first == OrderedInt::create(1, 2));
+  EXPECT_FALSE(res.second);
+
+  res = HashTable.insert(OrderedInt::create(1, 2));
+  // Check entry is unchanged.
+  EXPECT_TRUE(res.first == OrderedInt::create(1, 2));
+  EXPECT_FALSE(res.second);
+
+  res = HashTable.insert(OrderedInt::create(1, 1));
+  // Check entry is overwritten.
+  EXPECT_TRUE(res.first == OrderedInt::create(1, 1));
+  EXPECT_TRUE(res.second);
+}
+
+TEST(FixedConcurrentHashTableTest, AddIntEntriesParallel) {
+  BumpPtrAllocator Allocator;
+  size_t NumElements = 10000;
+  FixedConcurrentHashTable<uint32_t, Int> HashTable(NumElements, Allocator);
+
+  // Check parallel insertion.
+  parallelFor(0, NumElements, [&](size_t I) {
+    std::pair<Int, bool> Entry = HashTable.insert(Int::create(I));
+    EXPECT_TRUE(Entry.second);
+    EXPECT_TRUE(Entry.first.key() == I);
+  });
+
+  // Check parallel insertion of duplicates.
+  parallelFor(0, NumElements, [&](size_t I) {
+    std::pair<Int, bool> Entry = HashTable.insert(Int::create(I));
+    EXPECT_FALSE(Entry.second);
+    EXPECT_TRUE(Entry.first.key() == I);
+  });
+
+  // Check statistic.
+  std::string StatisticString;
+  raw_string_ostream StatisticStream(StatisticString);
+  HashTable.printStatistic(StatisticStream);
+
+  EXPECT_TRUE(StatisticString.find("HashTable statistic:") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("HashTable size: 16384") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("Load factor: 0.61") != std::string::npos);
+}
+
+TEST(FixedConcurrentHashTableTest, AddStringEntries) {
+  BumpPtrAllocator Allocator;
+  FixedConcurrentHashTableExternalAlloc<std::string, String, BumpPtrAllocator>
+      HashTable(10, Allocator);
+
+  std::pair<String *, bool> res1 = HashTable.insert("1", Allocator);
+  // Check entry is inserted.
+  EXPECT_TRUE(res1.first->key() == "1");
+  EXPECT_TRUE(res1.second);
+
+  std::pair<String *, bool> res2 = HashTable.insert("2", Allocator);
+  // Check old entry is still valid.
+  EXPECT_TRUE(res1.first->key() == "1");
+  // Check new entry is inserted.
+  EXPECT_TRUE(res2.first->key() == "2");
+  EXPECT_TRUE(res2.second);
+  // Check new and old entries use different memory.
+  EXPECT_TRUE(res1.first != res2.first);
+
+  std::pair<String *, bool> res3 = HashTable.insert("3", Allocator);
+  // Check one more entry is inserted.
+  EXPECT_TRUE(res3.first->key() == "3");
+  EXPECT_TRUE(res3.second);
+
+  std::pair<String *, bool> res4 = HashTable.insert("1", Allocator);
+  // Check duplicated entry is inserted.
+  EXPECT_TRUE(res4.first->key() == "1");
+  EXPECT_FALSE(res4.second);
+  // Check duplicated entry matches with the first one.
+  EXPECT_TRUE(res1.first == res4.first);
+
+  // Check first entry is still valid.
+  EXPECT_TRUE(res1.first->key() == "1");
+
+  // Check data was allocated by allocator.
+  EXPECT_TRUE(Allocator.getBytesAllocated() > 0);
+
+  // Check statistic.
+  std::string StatisticString;
+  raw_string_ostream StatisticStream(StatisticString);
+  HashTable.printStatistic(StatisticStream);
+
+  EXPECT_TRUE(StatisticString.find("HashTable statistic:") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("HashTable size: 16") != std::string::npos);
+  EXPECT_TRUE(StatisticString.find("Load factor: 0.19") != std::string::npos);
+}
+
+TEST(FixedConcurrentHashTableTest, AddStringEntriesParallel) {
+  const size_t NumElements = 10000;
+  static LLVM_THREAD_LOCAL BumpPtrAllocator Allocator;
+  FixedConcurrentHashTableExternalAlloc<std::string, String, BumpPtrAllocator>
+      HashTable(NumElements, Allocator);
+
+  // Check parallel insertion.
+  parallelFor(0, NumElements, [&](size_t I) {
+    std::string StringForElement = formatv("{0}", I);
+    std::pair<String *, bool> Entry =
+        HashTable.insert(StringForElement, Allocator);
+    EXPECT_TRUE(Entry.second);
+    EXPECT_TRUE(Entry.first->key() == StringForElement);
+    EXPECT_TRUE(Allocator.getBytesAllocated() > 0);
+  });
+
+  // Check parallel insertion of duplicates.
+  parallelFor(0, NumElements, [&](size_t I) {
+    size_t BytesAllocated = Allocator.getBytesAllocated();
+    std::string StringForElement = formatv("{0}", I);
+    std::pair<String *, bool> Entry =
+        HashTable.insert(StringForElement, Allocator);
+    EXPECT_FALSE(Entry.second);
+    EXPECT_TRUE(Entry.first->key() == StringForElement);
+    // Check no additional bytes were allocated for duplicate.
+    EXPECT_TRUE(Allocator.getBytesAllocated() == BytesAllocated);
+  });
+
+  // Check statistic.
+  std::string StatisticString;
+  raw_string_ostream StatisticStream(StatisticString);
+  HashTable.printStatistic(StatisticStream);
+
+  EXPECT_TRUE(StatisticString.find("HashTable statistic:") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("HashTable size: 16384") !=
+              std::string::npos);
+  EXPECT_TRUE(StatisticString.find("Load factor: 0.61") != std::string::npos);
+}
+
+} // end anonymous namespace