[InstCombine] Add oneuse checks to shr + cmp constant folds.

[aemerson]

Summary

This change has virtually no code size regressions on the llvm test suite (+ SPECs)
while having these improvements (measured with -Os on Darwin arm64):

External/S.../CFP2006/450.soplex/450.soplex    214024.00      213920.00     -0.0%
External/S...7speed/641.leela_s/641.leela_s     93412.00       93348.00     -0.1%
External/S...17rate/541.leela_r/541.leela_r     93412.00       93348.00     -0.1%
MultiSourc.../Applications/JM/lencod/lencod    426044.00      425748.00     -0.1%
MultiSourc...rks/mediabench/gsm/toast/toast     20436.00       20416.00     -0.1%
MultiSourc...ench/telecomm-gsm/telecomm-gsm     20436.00       20416.00     -0.1%
MultiSourc...Prolangs-C/assembler/assembler     16172.00       16156.00     -0.1%
MultiSourc...nch/mpeg2/mpeg2dec/mpeg2decode     35332.00       35256.00     -0.2%
SingleSour...Adobe-C++/stepanov_abstraction      6904.00        6888.00     -0.2%
External/SPEC/CINT2000/254.gap/254.gap         366060.00      365132.00     -0.3%
MultiSourc...-ProxyApps-C++/PENNANT/PENNANT     79688.00       79484.00     -0.3%
External/S...NT2006/464.h264ref/464.h264ref    352044.00      351132.00     -0.3%
SingleSour...arks/Adobe-C++/functionobjects     15524.00       15480.00     -0.3%
SingleSour...arks/Adobe-C++/stepanov_vector     10728.00       10696.00     -0.3%
SingleSour...ks/Misc-C++/stepanov_container     16900.00       16848.00     -0.3%
MultiSource/Applications/oggenc/oggenc         124184.00      123780.00     -0.3%
SingleSour...tout-C++/Shootout-C++-wordfreq      7060.00        7036.00     -0.3%
MultiSourc...ity-rijndael/security-rijndael      8976.00        8936.00     -0.4%
MultiSource/Benchmarks/McCat/18-imp/imp          9816.00        9772.00     -0.4%
SingleSour...chmarks/Misc-C++/stepanov_v1p2      1772.00        1764.00     -0.5%
MultiSourc...iabench/g721/g721encode/encode      5492.00        5464.00     -0.5%
MultiSourc...rks/McCat/03-testtrie/testtrie      1364.00        1344.00     -1.5%
SingleSour.../execute/GCC-C-execute-pr42833       400.00         364.00     -9.0%

Doing so also prevents a regression described in https://reviews.llvm.org/D143624

[goldstein.w.n]

llvm/test/Transforms/InstCombine/icmp-shr-lt-gt.ll

900

Can you split the tests to a seperate patch so we can see the dif?

[aemerson]

llvm/test/Transforms/InstCombine/icmp-shr-lt-gt.ll

900

You can just look at the test immediately before this one to see what the original codegen is. The difference is just that the code is this case isn’t combined.

[goldstein.w.n]

Where are the codegen improvements coming from?
Better imm values or something? Personally this feels a bit of a hack at the expense of potential ILP.

[aemerson]

In D149918#4320934, @goldstein.w.n wrote:

Where are the codegen improvements coming from?
Better imm values or something? Personally this feels a bit of a hack at the expense of potential ILP.

I'll look into exactly what changed in the codegen tomorrow. Why does this feel like a hack as opposed to all the other hasOneUse() checks in the optimizers?

[goldstein.w.n]

In D149918#4321150, @aemerson wrote:

In D149918#4320934, @goldstein.w.n wrote:

Where are the codegen improvements coming from?
Better imm values or something? Personally this feels a bit of a hack at the expense of potential ILP.

I'll look into exactly what changed in the codegen tomorrow. Why does this feel like a hack as opposed to all the other hasOneUse() checks in the optimizers?

The hasOneUse checks are there to prevent creating more instructions. Even with multi-use on ashr this combine
doesn't create more instructions and does shrink the critical path so it passes muster for an InstCombine.

If it is only darwin arm64 that cares about this change, why not handle it in the backend?

[aemerson]

In D149918#4322154, @goldstein.w.n wrote:

In D149918#4321150, @aemerson wrote:

In D149918#4320934, @goldstein.w.n wrote:

Where are the codegen improvements coming from?
Better imm values or something? Personally this feels a bit of a hack at the expense of potential ILP.

I'll look into exactly what changed in the codegen tomorrow. Why does this feel like a hack as opposed to all the other hasOneUse() checks in the optimizers?

The hasOneUse checks are there to prevent creating more instructions. Even with multi-use on ashr this combine
doesn't create more instructions and does shrink the critical path so it passes muster for an InstCombine.

Ok, that's fair.

If it is only darwin arm64 that cares about this change, why not handle it in the backend?

First of all, I didn't say that only Darwin arm64 cares about this, it's just the system that I happened to run the test suite build on. Secondly, these combines aren't easy to undo (at least one of them involves combing two constants into one). If anything, I think these combines make more sense to do at the codegen level than here but I could be wrong.

So looking at two cases, in oggenc there are quite a few occurrences of select patterns being broken by this combine (because the compare immediate is being munged into something not useful). These result in fewer simplifications which will be both larger and slower.

In MiBench/security-rijndael, we have as you say unfriendly immediate to compares being generated which causes us to have to materialize to registers. This is both a performance and code size hit.

[dmgreen]

I can verify that this fixes the issues, including the other one from https://reviews.llvm.org/D143624#4315468 that looks more difficult to reverse. It may be worth adding some tests like https://godbolt.org/z/cxKrfM4TK. But otherwise it LGTM. Getting more ILP sounds like something the backend should handle, if it is profitable given the constants.

[goldstein.w.n]

In D149918#4325473, @dmgreen wrote:

I can verify that this fixes the issues, including the other one from https://reviews.llvm.org/D143624#4315468 that looks more difficult to reverse. It may be worth adding some tests like https://godbolt.org/z/cxKrfM4TK. But otherwise it LGTM. Getting more ILP sounds like something the backend should handle, if it is profitable given the constants.

I think the generic statement "more ILP is better than less ILP" is fair for instcombine to make. Choosing which constants are profitable (what this patch is trying to approximate), however, still doesn't make sense to me.

[aemerson]

In D149918#4325535, @goldstein.w.n wrote:

In D149918#4325473, @dmgreen wrote:

I can verify that this fixes the issues, including the other one from https://reviews.llvm.org/D143624#4315468 that looks more difficult to reverse. It may be worth adding some tests like https://godbolt.org/z/cxKrfM4TK. But otherwise it LGTM. Getting more ILP sounds like something the backend should handle, if it is profitable given the constants.

I think the generic statement "more ILP is better than less ILP" is fair for instcombine to make.

Do you have an alternative suggestion given that we can’t really undo some of these transforms? ILP is not the only consideration that instcombine has, code size (which we can demonstrate is improved by this change) is an equally important goal.

Choosing which constants are profitable (what this patch is trying to approximate), however, still doesn't make sense to me.

We’re not trying to approximate choosing constants. We’re just adding a one use check.

[nikic]

Please add tests as @dmgreen suggested (and also for whatever pattern you saw in oggenc, if it's a different one) and precommit them. See https://llvm.org/docs/TestingGuide.html#precommit-workflow-for-tests if you are unfamiliar with the precommit workflow.

I'm open to considering this, as ultimately what we care about in InstCombine is what form is most beneficial for further transforms. Blocking this in the multi-use case will lose us some folds that could have made use of the simplified condition, but your data seems to suggest that in practice we lose more by failing to match select patterns. The alternative is to try harder when matching those select patterns.

However, if we want to do this change, you will first have to implement LVI support to effectively undo this when reasoning about condition ranges. That is, LVI needs to know how to determine the range of X given a condition of the form (ashr X, C) pred C2. This works automatically if such comparisons get folded, but requires special support if they don't.

[aemerson]

In D149918#4327542, @nikic wrote:

Please add tests as @dmgreen suggested (and also for whatever pattern you saw in oggenc, if it's a different one) and precommit them. See https://llvm.org/docs/TestingGuide.html#precommit-workflow-for-tests if you are unfamiliar with the precommit workflow.

I'm open to considering this, as ultimately what we care about in InstCombine is what form is most beneficial for further transforms. Blocking this in the multi-use case will lose us some folds that could have made use of the simplified condition, but your data seems to suggest that in practice we lose more by failing to match select patterns. The alternative is to try harder when matching those select patterns.

However, if we want to do this change, you will first have to implement LVI support to effectively undo this when reasoning about condition ranges. That is, LVI needs to know how to determine the range of X given a condition of the form (ashr X, C) pred C2. This works automatically if such comparisons get folded, but requires special support if they don't.

I'm not familiar with LVI, but I plugged in:

define i1 @ashrsgt_01_00(i4 %x, ptr %p) {
  %s = ashr i4 %x, 1
  %c = icmp sgt i4 %s, 0
  ret i1 %c
}

into opt and even after instcombine folds that to a single icmp, LVI doesn't seem to know anything about %x, just that it's overdefined. I assume you mean something else?

[nikic]

@aemerson You need to use the comparison as a branch condition, something like this:

define i1 @test(i4 %x) {
  %s = ashr i4 %x, 1
  %c = icmp sgt i4 %s, 0
  br i1 %c, label %if, label %else
if:
  %c2 = icmp sgt i4 %x, 1
  ret i1 %c2
else:
  ret i1 false
}

CVP will not be able to fold this anymore if the comparison isn't moved to %x in a multi-use scenario (for that matter, InstCombine won't either, which is something it does handle right now).

[aemerson]

Rebase on top of pre-committed tests.

[aemerson]

@nikic Now that the LVI patch has went in, is this ok?

[nikic]

LGTM

llvm/test/Transforms/InstCombine/ashr-icmp-minmax-idiom-break.ll

3–4

Adjust comment.

llvm/lib/Transforms/InstCombine/InstCombineCompares.cpp

if (ShAmtVal >= TypeBits || ShAmtVal == 0)
     return nullptr;
 
   bool IsExact = Shr->isExact();
   Type *ShrTy = Shr->getType();
   // TODO: If we could guarantee that InstSimplify would handle all of the
   // constant-value-based preconditions in the folds below, then we could assert
   // those conditions rather than checking them. This is difficult because of
   // undef/poison (PR34838).
-  if (IsAShr) {
+  if (IsAShr && Shr->hasOneUse()) {
     if (IsExact || Pred == CmpInst::ICMP_SLT || Pred == CmpInst::ICMP_ULT) {
       // When ShAmtC can be shifted losslessly:
       // icmp PRED (ashr exact X, ShAmtC), C --> icmp PRED X, (C << ShAmtC)
       // icmp slt/ult (ashr X, ShAmtC), C --> icmp slt/ult X, (C << ShAmtC)
       APInt ShiftedC = C.shl(ShAmtVal);
       if (ShiftedC.ashr(ShAmtVal) == C)
         return new ICmpInst(Pred, X, ConstantInt::get(ShrTy, ShiftedC));
     }

llvm/test/Transforms/InstCombine/ashr-icmp-minmax-idiom-break.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 2
 ; RUN: opt < %s -passes=instcombine -S | FileCheck %s
 
-; This test is pre-committed to show sub-optimal codegen due to
-; min/max idiom breakage. On AArch64, these constants are also expensive to materialize,
+; Check we don't have sub-optimal codegen due to min/max idiom breakage.
+; On AArch64, these constants are also expensive to materialize,
 ; and therefore generate poor code vs maintaining the min/max idiom.
 
 define i64 @dont_break_minmax_i64(i64 %conv, i64 %conv2) {
 ; CHECK-LABEL: define i64 @dont_break_minmax_i64
 ; CHECK-SAME: (i64 [[CONV:%.*]], i64 [[CONV2:%.*]]) {
 ; CHECK-NEXT:    [[MUL:%.*]] = mul nsw i64 [[CONV]], [[CONV2]]
 ; CHECK-NEXT:    [[SHR:%.*]] = ashr i64 [[MUL]], 4
-; CHECK-NEXT:    [[CMP4_I:%.*]] = icmp slt i64 [[MUL]], 5579712
-; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = select i1 [[CMP4_I]], i64 [[SHR]], i64 348731
+; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = call i64 @llvm.smin.i64(i64 [[SHR]], i64 348731)
 ; CHECK-NEXT:    ret i64 [[SPEC_SELECT_I]]
 ;
   %mul = mul nsw i64 %conv, %conv2
   %shr = ashr i64 %mul, 4
   %cmp4.i = icmp sgt i64 %shr, 348731
   %switch.i = icmp ult i1 %cmp4.i, true
   %spec.select.i = select i1 %switch.i, i64 %shr, i64 348731
   ret i64 %spec.select.i
 }

llvm/test/Transforms/InstCombine/icmp-shr-lt-gt.ll

 ; CHECK-NEXT:    [[C:%.*]] = icmp sgt i4 [[X:%.*]], 1
 ; CHECK-NEXT:    ret i1 [[C]]
 ;
   %s = ashr i4 %x, 1
   %c = icmp sgt i4 %s, 0
   ret i1 %c
 }
 
 define i1 @ashrsgt_01_00_multiuse(i4 %x, ptr %p) {
 ; CHECK-LABEL: @ashrsgt_01_00_multiuse(
 ; CHECK-NEXT:    [[S:%.*]] = ashr i4 [[X:%.*]], 1
-; CHECK-NEXT:    [[C:%.*]] = icmp sgt i4 [[X]], 1
+; CHECK-NEXT:    [[C:%.*]] = icmp sgt i4 [[S]], 0
 ; CHECK-NEXT:    store i4 [[S]], ptr [[P:%.*]], align 1
 ; CHECK-NEXT:    ret i1 [[C]]
 ;
   %s = ashr i4 %x, 1
   %c = icmp sgt i4 %s, 0
   store i4 %s, ptr %p
   ret i1 %c
 }

llvm/test/Transforms/PhaseOrdering/icmp-ashr-breaking-select-idiom.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 3
 ; RUN: opt -O1 -S < %s  | FileCheck %s
 
 define i32 @testa(i32 %mul) {
 ; CHECK-LABEL: define i32 @testa(
 ; CHECK-SAME: i32 [[MUL:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
 ; CHECK-NEXT:    [[SHR:%.*]] = ashr i32 [[MUL]], 15
-; CHECK-NEXT:    [[CMP4_I:%.*]] = icmp slt i32 [[MUL]], 1073741824
-; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = select i1 [[CMP4_I]], i32 [[SHR]], i32 32767
+; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = tail call i32 @llvm.smin.i32(i32 [[SHR]], i32 32767)
 ; CHECK-NEXT:    ret i32 [[SPEC_SELECT_I]]
 ;
   %shr = ashr i32 %mul, 15
   %cmp4.i = icmp sgt i32 %shr, 32767
   %switch.i = icmp ult i1 %cmp4.i, true
   %spec.select.i = select i1 %switch.i, i32 %shr, i32 32767
   ret i32 %spec.select.i
 }
 
 define i32 @testb(i32 %mul) {
 ; CHECK-LABEL: define i32 @testb(
 ; CHECK-SAME: i32 [[MUL:%.*]]) local_unnamed_addr #[[ATTR0]] {
 ; CHECK-NEXT:    [[SHR102:%.*]] = ashr i32 [[MUL]], 7
-; CHECK-NEXT:    [[CMP4_I:%.*]] = icmp sgt i32 [[MUL]], 16383
-; CHECK-NEXT:    [[RETVAL_0_I:%.*]] = select i1 [[CMP4_I]], i32 127, i32 -128
-; CHECK-NEXT:    [[TMP1:%.*]] = add i32 [[MUL]], 16384
-; CHECK-NEXT:    [[CLEANUP_DEST_SLOT_0_I:%.*]] = icmp ult i32 [[TMP1]], 32768
-; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = select i1 [[CLEANUP_DEST_SLOT_0_I]], i32 [[SHR102]], i32 [[RETVAL_0_I]]
+; CHECK-NEXT:    [[TMP1:%.*]] = tail call i32 @llvm.smax.i32(i32 [[SHR102]], i32 -128)
+; CHECK-NEXT:    [[SPEC_SELECT_I:%.*]] = tail call i32 @llvm.smin.i32(i32 [[TMP1]], i32 127)
 ; CHECK-NEXT:    ret i32 [[SPEC_SELECT_I]]
 ;
   %shr102 = ashr i32 %mul, 7
   %cmp4.i = icmp sgt i32 %shr102, 127
   %cmp6.i = icmp slt i32 %shr102, -128
   %retval.0.i = select i1 %cmp4.i, i32 127, i32 -128
   %cleanup.dest.slot.0.i = select i1 %cmp4.i, i1 true, i1 %cmp6.i
   %switch.i = icmp ult i1 %cleanup.dest.slot.0.i, true
   %spec.select.i = select i1 %switch.i, i32 %shr102, i32 %retval.0.i
   ret i32 %spec.select.i
 }