We really wanted to avoid special handling of references but it looks like this is inevitable.
The main difference is the following, if you copy a gsl::Pointer, like an iterator, the result will dangle if and only if the original instance would dangle as well. Basically the copy just propagates the points-to set. But this is not the case for references. If we create an object from a reference it is often more similar to a dereference operation, or an LValueToRValue cast. This patch attempts to address this difference.