Like D88794, the Restorer depends on copy elision to happen. Without copy elision, the function ScopedSet calls the move constructor before its dtor. The dtor will prematurely restore the reference to the original value.
Inhibit recreating the restorer object completely to avoid the implicit creation of a second restorer object that would try to move the same object. The factory class ScopedSet is changed to use unnamed return value optimization which is mandatory in C++17.