As discussed in D100691 and based on D100889.
I removed the ModeChecks cache which provides little value. Reduced
from three loops to two. Used ArrayRef to pass the Predicate to
AppendPattern to avoid needing to construct a vector for single
mode. Used SmallVector to avoid heap allocation constructing
DefaultCheck for the in tree targets the use it.