For the design in question, overloads seem to be a much simpler and less subtle solution.
This removes ODR issues, and errors of the kind where code that uses the specialization in question will accidentally and erroneously specialize the primary template. This only "works" by accident; the program is ill-formed NDR.
(Found with -Wundefined-func-template.)
Should probably move the comment below up here and explain more what this is doing. IE: that this template is explicitly specialized for specific types only, etc.