Function specialization was crashing at i) poison values and 2) constexpr values.
The problem is that these values are not added to the solver, so when a lookup is performed the crash happens. This fixes that by not specialising on these values. For poison that is obvious, but for constexpr this is a change in behaviour. Thus, in one way this is a bit of a stopgap, but specialising on constexpr values wasn't done very intentionally, and need some more work and tests if we wanted to support this, which is why not allowing constexprs seems fine to me.
This fixes: https://bugs.llvm.org/show_bug.cgi?id=51600