[AArch64] Re-run load/store optimizer after aggressive tail duplication

[labrinea]

Summary

The Load/Store Optimizer runs before Machine Block Placement. At O3 the Tail Duplication Threshold is set to 4 instructions and this can create new opportunities for the Load/Store Optimizer. It seems worthwhile to run it once again.

[labrinea]

Ping

[dmgreen]

Do you have any compile time numbers for this?

[labrinea]

I've tested the patch with native builds of the llvm-test-suite on an AArch64 Cortex-A72 and couldn't spot anything interesting in terms of compilation time.

[dmgreen]

This looks find to me then, especially at aggressive opt level. IIRC, there is a test for the pass pipeline I would expect needs updating.

Other than that, LGTM

test/CodeGen/AArch64/ldst-opt-after-block-placement.ll

31

%2 is never used here.

[labrinea]

In D55108#1330014, @dmgreen wrote:

IIRC, there is a test for the pass pipeline I would expect needs updating.

Indeed that is test/CodeGen/AArch64/O3-pipeline.ll. Fixed now.

test/CodeGen/AArch64/ldst-opt-after-block-placement.ll

31

Good catch. Removed now.

[labrinea]

Committed as https://reviews.llvm.org/rL349338

lib/Target/AArch64/AArch64TargetMachine.cpp

if (TM->getOptLevel() != CodeGenOpt::None) {
     if (EnableLoadStoreOpt)
       addPass(createAArch64LoadStoreOptimizationPass());
     if (EnableFalkorHWPFFix)
       addPass(createFalkorHWPFFixPass());
   }
 }
 
 void AArch64PassConfig::addPreEmitPass() {
+  // Machine Block Placement might have created new opportunities when run
+  // at O3, where the Tail Duplication Threshold is set to 4 instructions.
+  // Run the load/store optimizer once more.
+  if (TM->getOptLevel() >= CodeGenOpt::Aggressive && EnableLoadStoreOpt)
+    addPass(createAArch64LoadStoreOptimizationPass());
+
   if (EnableA53Fix835769)
     addPass(createAArch64A53Fix835769());
   // Relax conditional branch instructions if they're otherwise out of
   // range of their destination.
   if (BranchRelaxation)
     addPass(&BranchRelaxationPassID);
 
   if (EnableBranchTargets)

test/CodeGen/AArch64/ldst-opt-after-block-placement.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O3 -mtriple=aarch64-arm < %s | FileCheck %s
 
 ; Run in -O3 to make sure Aggressive Tail Duplication is enabled and we managed
 ; optimize load/store instructions after Machine Block Placement took place.
 
 define void @foo(i1 %cond, i64* %ptr) {
 ; CHECK-LABEL: foo:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    tbz w0, #0, .LBB0_2
 ; CHECK-NEXT:  // %bb.1: // %if.then
-; CHECK-NEXT:    ldr x8, [x1, #16]
+; CHECK-NEXT:    ldp x9, x8, [x1, #8]
 ; CHECK-NEXT:    str xzr, [x1, #16]
-; CHECK-NEXT:    ldr x9, [x1, #8]
 ; CHECK-NEXT:    cmp x8, x9
 ; CHECK-NEXT:    b.lt .LBB0_3
 ; CHECK-NEXT:    b .LBB0_4
 ; CHECK-NEXT:  .LBB0_2: // %if.else
-; CHECK-NEXT:    ldr x8, [x1]
-; CHECK-NEXT:    ldr x9, [x1, #8]
+; CHECK-NEXT:    ldp x8, x9, [x1]
 ; CHECK-NEXT:    cmp x8, x9
 ; CHECK-NEXT:    b.ge .LBB0_4
 ; CHECK-NEXT:  .LBB0_3: // %exit1
 ; CHECK-NEXT:    str xzr, [x1, #8]
 ; CHECK-NEXT:  .LBB0_4: // %exit2
 ; CHECK-NEXT:    ret
 entry:
   br i1 %cond, label %if.then, label %if.else
 
 if.then:
   %0 = getelementptr inbounds i64, i64* %ptr, i64 2
   %1 = load i64, i64* %0, align 8
   %2 = getelementptr inbounds i64, i64* %ptr, i64 3
   store i64 0, i64* %0, align 8
   br label %if.end
 
 if.else:
   %3 = load i64, i64* %ptr, align 8
   br label %if.end
 
 if.end: