[BreakFalseDeps] ignore function with minsize attribute

[spatel]

Summary

This came up in the x86-specific:
https://bugs.llvm.org/show_bug.cgi?id=43239
...but it seems like a general problem for the BreakFalseDeps pass. Dependencies are always broken by adding some other instruction, so that should be avoided if the overall goal is to minimize size.

[RKSimon]

This would make it policy for -Oz builds to not bother to break dependencies but -Os/-O0+ builds would still do.

Does anything else use BreakFalseDeps?

llvm/lib/CodeGen/BreakFalseDeps.cpp

15–16

NFC

[spatel]

In D67363#1663974, @RKSimon wrote:

This would make it policy for -Oz builds to not bother to break dependencies but -Os/-O0+ builds would still do.

Correct - that matches my interpretation of the vague clang docs:

-Os Like -O2 with extra optimizations to reduce code size.
-Oz Like -Os (and thus -O2), but reduces code size further.

In my experience, -Oz ("minsize") is used to ensure that code fits in some constrained space, so any extras that can be eliminated should be eliminated. Perf is a secondary concern. This is different than -Os ("optsize") where we are trying to balance speed and size.

Does anything else use BreakFalseDeps?

x86 and ARM are the only targets in trunk that use it from what I can see. Both use it the same way: insert dummy (not required for correctness) instructions to avoid known uarch hazards.

[spatel]

Patch updated:
Pre-committed NFC code comment diffs in rL371516.

[spatel]

Adding some ARM folks since ARM is (the only) other in-tree target that would be affected by this change.
It's not strictly stated that dependency-breaking always means adding an instruction, but that’s the practical definition based on x86 and ARM's implementation of this pass's hooks.

[craig.topper]

For VEX instructions don’t we just use the other input register to break the dependency without adding an instruction?

[spatel]

In D67363#1664689, @craig.topper wrote:

For VEX instructions don’t we just use the other input register to break the dependency without adding an instruction?

That sounds right, but it's not controlled by this pass. That's part of memory op folding?
If I'm seeing it correctly, that's already more aggressively optimizing for size than what we're doing here:

// Avoid partial and undef register update stalls unless optimizing for size.
if (!MF.getFunction().hasOptSize() &&
    (hasPartialRegUpdate(MI.getOpcode(), Subtarget, /*ForLoadFold*/true) ||
     shouldPreventUndefRegUpdateMemFold(MF, MI)))
  return nullptr;

[spatel]

Patch updated:
Rebased with improved test coverage for ARM and x86:
rL371526
rL371525

[spatel]

llvm/test/CodeGen/X86/sqrt-partial.ll

78

The AVX form of sqrtsd doesn't have a partial reg update, so no difference here.

[craig.topper]

In D67363#1664746, @spatel wrote:

In D67363#1664689, @craig.topper wrote:

For VEX instructions don’t we just use the other input register to break the dependency without adding an instruction?

That sounds right, but it's not controlled by this pass. That's part of memory op folding?
If I'm seeing it correctly, that's already more aggressively optimizing for size than what we're doing here:

// Avoid partial and undef register update stalls unless optimizing for size.
if (!MF.getFunction().hasOptSize() &&
    (hasPartialRegUpdate(MI.getOpcode(), Subtarget, /*ForLoadFold*/true) ||
     shouldPreventUndefRegUpdateMemFold(MF, MI)))
  return nullptr;

We disable folding if it takes away our opportunity to use the other input register. But something still needs to pick a register for the implicit_def operand. I though that was this pass? I think in absense of this pass it may just be set to the xmm0 by the register allocator. Do we have tests where xmm0 isn't the right choice?

[craig.topper]

I believe this test case compiled with avx needs this pass.

define double @minsize(double %x, double %y) minsize {

%t6 = tail call fast double @llvm.sqrt.f64(double %y)
%t= fadd fast double %t6, %y
ret double %t6

}
declare double @llvm.sqrt.f64(double)

[spatel]

In D67363#1664788, @craig.topper wrote:

I believe this test case compiled with avx needs this pass.

define double @minsize(double %x, double %y) minsize {

%t6 = tail call fast double @llvm.sqrt.f64(double %y)
%t= fadd fast double %t6, %y
ret double %t6

}
declare double @llvm.sqrt.f64(double)

Thanks! I didn't understand where that happened - it's in here (and no existing regression tests appear to capture that).
So yes, to limit this pass, we need to use a slightly finer check to distinguish between transforms that are only changing registers vs. adding instructions.
Added more tests with rL371528 and rL371551.

[spatel]

Patch updated:
Refine the places where minsize comes into play (where the target is called to add an instruction via TII->breakPartialRegDependency()).

[spatel]

llvm/test/CodeGen/X86/sqrt-partial.ll

99

This is the test showing that the pass is still capable of changing a reg.

[craig.topper]

X86 looks good to me.

[spatel]

Ping - ARM ok?

[efriedma]

llvm/test/CodeGen/ARM/a15-partial-update.ll

59

There's a missed optimization here: we could avoid both the false dependence and the extra instruction by modifying the vld1 to a splat load. But generally looks fine.

[spatel]

Patch updated:
Added a TODO comment about using a splat load on the ARM tests.

[spatel]

Marking patch as accepted based on "looks good" and "looks fine". I'll hold off on commit for a ~day in case there are any more comments.

[RKSimon]

LGTM - maybe raise a bug about the missing splat loads in a15-partial-update.ll ?

[spatel]

In D67363#1678217, @RKSimon wrote:

LGTM - maybe raise a bug about the missing splat loads in a15-partial-update.ll ?

Sure - https://bugs.llvm.org/show_bug.cgi?id=43410

llvm/lib/CodeGen/BreakFalseDeps.cpp

 //==- llvm/CodeGen/BreakFalseDeps.cpp - Break False Dependency Fix -*- C++ -*==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file Break False Dependency pass.
 ///
 /// Some instructions have false dependencies which cause unnecessary stalls.
 /// For example, instructions may write part of a register and implicitly
 /// need to read the other parts of the register. This may cause unwanted
 /// stalls preventing otherwise unrelated instructions from executing in
 /// parallel in an out-of-order CPU.
 /// This pass is aimed at identifying and avoiding these dependencies.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/CodeGen/LivePhysRegs.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/ReachingDefAnalysis.h"
 #include "llvm/CodeGen/RegisterClassInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
bool BreakFalseDeps::shouldBreakDependence(MachineInstr *MI, unsigned OpIdx,
   LLVM_DEBUG(dbgs() << ": OK .\n");
   return false;
 }
 
 void BreakFalseDeps::processDefs(MachineInstr *MI) {
   assert(!MI->isDebugInstr() && "Won't process debug values");
 
   // Break dependence on undef uses. Do this before updating LiveRegs below.
+  // This can remove a false dependence with no additional instructions.
   unsigned OpNum;
   unsigned Pref = TII->getUndefRegClearance(*MI, OpNum, TRI);
   if (Pref) {
     bool HadTrueDependency = pickBestRegisterForUndef(MI, OpNum, Pref);
     // We don't need to bother trying to break a dependency if this
     // instruction has a true dependency on that register through another
     // operand - we'll have to wait for it to be available regardless.
     if (!HadTrueDependency && shouldBreakDependence(MI, OpNum, Pref))
       UndefReads.push_back(std::make_pair(MI, OpNum));
   }
 
+  // The code below allows the target to create a new instruction to break the
+  // dependence. That opposes the goal of minimizing size, so bail out now.
+  if (MF->getFunction().hasMinSize())
+    return;
+
   const MCInstrDesc &MCID = MI->getDesc();
   for (unsigned i = 0,
     e = MI->isVariadic() ? MI->getNumOperands() : MCID.getNumDefs();
     i != e; ++i) {
     MachineOperand &MO = MI->getOperand(i);
     if (!MO.isReg() || !MO.getReg())
       continue;
     if (MO.isUse())
       continue;
     // Check clearance before partial register updates.
     unsigned Pref = TII->getPartialRegUpdateClearance(*MI, i, TRI);
     if (Pref && shouldBreakDependence(MI, i, Pref))
       TII->breakPartialRegDependency(*MI, i, TRI);
   }
 }
 
 void BreakFalseDeps::processUndefReads(MachineBasicBlock *MBB) {
   if (UndefReads.empty())
     return;
 
+  // The code below allows the target to create a new instruction to break the
+  // dependence. That opposes the goal of minimizing size, so bail out now.
+  if (MF->getFunction().hasMinSize())
+    return;
+
   // Collect this block's live out register units.
   LiveRegSet.init(*TRI);
   // We do not need to care about pristine registers as they are just preserved
   // but not actually used in the function.
   LiveRegSet.addLiveOutsNoPristines(*MBB);
 
   MachineInstr *UndefMI = UndefReads.back().first;
   unsigned OpIdx = UndefReads.back().second;

llvm/test/CodeGen/ARM/a15-partial-update.ll

loop:
   %tmp1 = load <4 x i8> , <4 x i8> *%p1, align 4
   store <4 x i8> %tmp1, <4 x i8> *%p2
   %cmp = icmp eq i32 %newcount, %n
   br i1 %cmp, label %loop, label %ret
 ret:
   ret void
 }
 
+; If minimizing size, that overrides perf, so no extra vmov.f64 here.
+
 define void @t2_minsize(<4 x i8> *%in, <4 x i8> *%out, i32 %n) minsize {
 ; CHECK-LABEL: t2_minsize:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    add r0, r0, #4
 ; CHECK-NEXT:    add r1, r1, #4
 ; CHECK-NEXT:  .LBB2_1: @ %loop
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vmov.f64 d16, #5.000000e-01
 ; CHECK-NEXT:    vld1.32 {d16[0]}, [r0:32]
 ; CHECK-NEXT:    vmovl.u8 q8, d16
 ; CHECK-NEXT:    vuzp.8 d16, d18
 ; CHECK-NEXT:    vst1.32 {d16[0]}, [r1:32]!
 ; CHECK-NEXT:    add r0, r0, #4
 ; CHECK-NEXT:    subs r2, r2, #1
 ; CHECK-NEXT:    beq .LBB2_1
 ; CHECK-NEXT:  @ %bb.2: @ %ret

llvm/test/CodeGen/X86/sqrt-partial.ll

 }
 
 define double @minsize(double %x, double %y) minsize {
 ; SSE-LABEL: minsize:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    mulsd %xmm0, %xmm0
 ; SSE-NEXT:    mulsd %xmm1, %xmm1
 ; SSE-NEXT:    addsd %xmm0, %xmm1
-; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    sqrtsd %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: minsize:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vmulsd %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    vmulsd %xmm1, %xmm1, %xmm1
 ; AVX-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; AVX-NEXT:    vsqrtsd %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %t3 = fmul fast double %x, %x
   %t4 = fmul fast double %y, %y
   %t5 = fadd fast double %t3, %t4
   %t6 = tail call fast double @llvm.sqrt.f64(double %t5)
   ret double %t6
 }
 
 ; Partial reg avoidance may involve register allocation
 ; rather than adding an instruction.
 
 define double @partial_dep_minsize(double %x, double %y) minsize {
 ; SSE-LABEL: partial_dep_minsize:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    sqrtsd %xmm1, %xmm0
 ; SSE-NEXT:    addsd %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: partial_dep_minsize:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vsqrtsd %xmm1, %xmm1, %xmm0
 ; AVX-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %t6 = tail call fast double @llvm.sqrt.f64(double %y)
   %t = fadd fast double %t6, %y
   ret double %t
 }
 
 declare float @sqrtf(float)
 declare double @sqrt(double)
 declare double @llvm.sqrt.f64(double)

llvm/test/CodeGen/X86/stack-folding-fp-avx1.ll

 
 define double @stack_fold_roundsd_minsize(double %a0) minsize {
 ; CHECK-LABEL: stack_fold_roundsd_minsize:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    vmovsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
-; CHECK-NEXT:    vxorps %xmm0, %xmm0, %xmm0
 ; CHECK-NEXT:    vroundsd $9, {{[-0-9]+}}(%r{{[sb]}}p), %xmm0, %xmm0 # 8-byte Folded Reload
 ; CHECK-NEXT:    retq
   %1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
   %2 = call double @llvm.floor.f64(double %a0)
   ret double %2
 }
 declare double @llvm.floor.f64(double) nounwind readnone
 

llvm/test/CodeGen/X86/stack-folding-fp-sse42.ll

 
 define float @stack_fold_cvtsd2ss(double %a0) minsize {
 ; CHECK-LABEL: stack_fold_cvtsd2ss:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movsd %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
-; CHECK-NEXT:    xorps %xmm0, %xmm0
 ; CHECK-NEXT:    cvtsd2ss {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 8-byte Folded Reload
 ; CHECK-NEXT:    retq
   %1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
   %2 = fptrunc double %a0 to float
   ret float %2
 }
 
 define <4 x float> @stack_fold_cvtsd2ss_int(<2 x double> %a0) optsize {

 
 define double @stack_fold_cvtss2sd(float %a0) minsize {
 ; CHECK-LABEL: stack_fold_cvtss2sd:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movss %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
-; CHECK-NEXT:    xorps %xmm0, %xmm0
 ; CHECK-NEXT:    cvtss2sd {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 4-byte Folded Reload
 ; CHECK-NEXT:    retq
   %1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
   %2 = fpext float %a0 to double
   ret double %2
 }
 
 define <2 x double> @stack_fold_cvtss2sd_int(<4 x float> %a0) optsize {

 
 define float @stack_fold_roundss(float %a0) minsize {
 ; CHECK-LABEL: stack_fold_roundss:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movss %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
-; CHECK-NEXT:    xorps %xmm0, %xmm0
 ; CHECK-NEXT:    roundss $9, {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 4-byte Folded Reload
 ; CHECK-NEXT:    retq
   %1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
   %2 = call float @llvm.floor.f32(float %a0)
   ret float %2
 }
 declare float @llvm.floor.f32(float) nounwind readnone
 

 
 define float @stack_fold_sqrtss(float %a0) minsize {
 ; CHECK-LABEL: stack_fold_sqrtss:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movss %xmm0, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
-; CHECK-NEXT:    xorps %xmm0, %xmm0
 ; CHECK-NEXT:    sqrtss {{[-0-9]+}}(%r{{[sb]}}p), %xmm0 # 4-byte Folded Reload
 ; CHECK-NEXT:    retq
   %1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
   %2 = call float @llvm.sqrt.f32(float %a0)
   ret float %2
 }
 declare float @llvm.sqrt.f32(float) nounwind readnone