I'm not 100% confident in this patch yet but because of the time crunch with the 3.6 release I'm posting it early.
The patch works by using static_cast on fancy class-like pointers and reinterpret_cast on raw pointers. It also changes it so that when the base node class is needed at a call site then it gets the base pointer directly instead of the derived pointer.
I'll do more work on this and actually convince myself of its correctness.
It seems strange to use static_cast here and reinterpret_cast below. Even assuming the reinterpret_cast approach works, don't we still have the same problem if the user uses a fancy pointer type?