ClusterManager is using a SmallPtrSet to store the objects in it. We always only add every object once so using a set is not necessary.
Furthermore having a set means that iterating over it is nondeterministic (at least with more than 16 objects in it), so the order in
which the destructors for the managed objects are called is currently also non-deterministic.
This just replaces the SmallPtrSet with a SmallVector.