Currently InstCombine tends to recognize patterns like
%cmp = icmp slt iA %x, 0 %select = select i1 %cmp, iB %A, iB %B
as bit-test against highest bit, and convert it into one of multiple shift-based patterns
which may look differently depending on types iA and iB and values of %A and %B.
It only happens if %A and %B are constants. We observe at least three possible sutiatons:
Case 1:
%c2 = icmp slt %a, 0 %select1 = select %c2, -1, %Greater
converts into
(%a s>> 31) | %Greater
Case 2:
%c2 = icmp slt i32 %a, 0 %select1 = select i1 %c2, i32 %Less, i32 %Greater
converts into
(%a s>> 31) - (%Less - %Greater) + %Greater
or, if it is the equivalent, to
(%a s>> 31) - (%Less - %Greater) | %Greater
Case 3:
%c2 = icmp slt i64 %a, 0 %select1 = select i1 %c2, i32 %Less, i32 %Greater
converts to
(trunc (%a s>> 63) to i32) - (%Less - %Greater) + %Greater
There are possibly more variatons of this. The general idea is that select by comparison with 0 or -1
gets transformed into multiple different patterns, and very small changes in conditions and constants
may change this pattern. There is no clear benefits of that transforms, but the obvious downside of them
is decanonicalization. For example, we have a matcher of three-ways comparison that expects to see the
pattern of the following form:
%c1 = icmp eq %a, 0 %c2 = icmp slt %a, 0 %select1 = select %c2, %Less, %Greater %select2 = select %c1, %Equal, %select2
This pattern is widely seen across the code: it is a canonical representation of three-ways comparators
that return -1, 0 or 1 if %a is less, equal or greater than zero. In the result of transforms described in
cases 1, 2, 3, the pair of %c2 and %select1 gets transformed into something different (with shifts, casts
and different bit operations). So the three-way comparison pattern cannot be recognized anymore.
This patch removes the transforms that make us replace select by comparison against zero with all this bit
magic. We want to prefer selects over random bit magic as a more canonical form of doing that: select
has more transparent semantics and other parts of InstCombine make use it rather than try to recognize
that some sequence of non-trivial bit instructions actually represents a select by comparison against zero.
It is codegen's work to turn selects into bit operations in the end of pipeline to calculate it efficiently if
shifts are faster than cmoves, but doing so too early has no benifits and has obvious pessimization of other
pattern matchers within instcombine.
In this patch we only stop harmful transforms from happening. The code that turns all this bit magic back
to selects will go in a separate patch.
Not sure exactly where the logic hole is that caused this, but I moved a fold to InstSimplify in:
rL347896
...and now we get this case and the similar 'compare_against_arbitrary_value_type_mismatch' test.