The AVX/AVX512 check were based on the command line features. But the function may have a target attribute that gives more features than the command line. We shouldn't fail for 512 operand if the command line doesn't support it, but the target attribute does. gcc supports this.
I don't know how to get to the target attribute here, or if we even can without creating a layering issue. So I've just removed the check entirely and always allow 512 bits. In some simple testing, the backend seems to produce an error like "couldn't allocate output register for constraint 'x'" when the type is larger than supported by the subtarget features. So I think we can handle this gracefully without crashing.
I think the same issues apply to this SSELevel check. I suspect no one complains about these checks because everyone enables SSE2 in one way or another.