[argprom] Assessing impact of magic value MaxElements promoted on compiler performance

[teamiceberg]

Summary

Our initial hypothesis is that changes to the magic value of (unsigned) MaxElements variable in ArgumentPromotion.cpp/ArgumentPromotion.h file can lead to changes in both compile time and run time compiler performance. To assess such changes, we extern declare a global variable (unsigned MaxElemsToPromote) in the IPO.h header file (default defined to 3 in ArgumentPromotion.h). Then, we supply different values for this global variable through an OPT command line flag (-argpromotion-max-elements-to-promote) bound to this global variable via the cl::opt interface. Initial analysis of compiler performance using the built-in test suite demonstrate some significant changes in compile time as well as run time, including beneficial changes(reductions in CT/RT) for some test cases.We are in the process of validating if these changes are real or just noise induced by other means. More experiments and sample runs need to be run, to be sure.
However, we are setting up this initial patch as a way to learn how to setup a pipeline to enable the use of global variables to simulate magic values. Also, we are designing a LIT test in llvm/tests folder to ensure that we submit a patch that demonstrates that the OPT flag works.
Regression Test Setup: /llvm/test/Transforms/ArgumentPromotion has 1 test written by me (fourmember-struct-promote.ll) which demonstrates the flag works successfully.
Regression Test success criteria: For aggregate arguments, such as structs, by design only a maximum of 3 members are promoted. However by introducing a custom OPT flag, we can set this number to different values to promote any number of an aggregate's arguments. For instance, the regression test file "fourmember-struct-promote.ll" has a four member struct passed by reference to a simple callee that sums the values of the members. The flag '-argpromotion-max-elements-to-promote' value has to be 0 or 4 (or higher) for all 4 members in the struct to be promoted. '0' defaults to 'always promote'. Supplying a value less than the number of members in the aggregate (in this case 4) halts promotion. The regression tests are designed to PASS when the FILECHECK tool pattern matches against an output that reflects all members have been promoted to "by value" for -argpromotion-max-elements-to-promote=[0 , 3, INT_MAX] cases respectively.

RESULTS:
fourmember-struct-promote.ll test PASSES according to success criteria defined above.
Therefore, we conclude that the --argpromotion-max-elements-to-promote user flag works correctly.

Now that we have this flag working correctly, users can test out the impact of argument promotion of elements of aggregates on compile time and run time performance.

[jdoerfert]

llvm/include/llvm/Transforms/IPO/ArgumentPromotion.h

19

The comment describes something unrelated.
Please rename the variable according to the naming convention.
A static variable definition in a header is problematic if the header is included more than once. Please use an extern variable declaration here and put the definition in the cpp file.

llvm/lib/Transforms/IPO/ArgumentPromotion.cpp

94

use a more expressive command line option name, e.g., argpromotion-max-argument-increase-per-promotion. Please update the description to describe what this actually controls. No need for value_desc.

1123

Please do not simply ignore the argument MaxElements that is passed here by the user of this function. If you want to modify the default for the old pass manager, which is using this function, please update the call site where there is a 3.

llvm/test/Transforms/ArgumentPromotion/fourmember-struct-promote-DEFAULT.ll

91 ↗

(On Diff #278044)

No need for the metadata and attributes at the end of this file, please remove.

llvm/test/Transforms/ArgumentPromotion/fourmember-struct-promote-INTMAX.ll

1 ↗

(On Diff #278044)

Please merge all three test files into one with 3 RUN lines and different check labels.

[jdoerfert]

llvm/test/Transforms/ArgumentPromotion/fourmember-struct-promote-DEFAULT.ll

1 ↗

(On Diff #278044)

run mem2reg on the file *before* making it a test, not here.

[teamiceberg]

This comment was removed by teamiceberg.

[teamiceberg]

Diff Update V1 - All changes requested except consolidation of 3 RUNS in one file

[teamiceberg]

Diff Update V2 - Consolidated 3 RUNS in one test file. Individual files deleted.

[teamiceberg]

Re-basing against LLVM master

[jdoerfert]

llvm/include/llvm/Transforms/IPO.h

34

No need to explain it is a global variable. Explain what the meaning is, e.g., what does the number indicate.

llvm/lib/Transforms/IPO/ArgumentPromotion.cpp

97

The comment is above is not necessary. The first sentence of the description is sufficient. The option should be hidden, compare other command line options please. The name of the variable is not proper.

101

Explain the meaning not where an extern definition is and that the value is by default three.

1123

Where is the change in the pass manager?

[teamiceberg]

All change requirements comments except Pass Manager change requirements addressed.I need more information to address Pass Manager CR.

[teamiceberg]

Written text changes to patch's subject message. No source code changes.

llvm/include/llvm/Transforms/IPO.h

 //===- llvm/Transforms/IPO.h - Interprocedural Transformations --*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 class StringRef;
 class ModuleSummaryIndex;
 class ModulePass;
 class Pass;
 class BasicBlock;
 class GlobalValue;
 class raw_ostream;
 
+// optional OPT flag 'argpromotion-max-elements-to-promote' populates
+// NumAggregateElemsPromoted variable used in the Argument Promotion pass.
+// Currently, argument promotion pass promotes upto three elements in an
+// aggregeate argument if these elements are safe to promote.
+// If users need to promote more than 3 elements in an aggregate, that needs
+// to be specified through the  'argpromotion-max-elements-to-promote' flag
+// on the OPT command line, which then populates the NumAggregateElemsPromoted
+// variable. see ArgumentPromotion{.cpp,.h} for how the
+// NumAggregateElemsPromoted variable is used.
+extern unsigned NumAggregateElemsPromoted;
+
 //===----------------------------------------------------------------------===//
 //
 // These functions removes symbols from functions and modules.  If OnlyDebugInfo
 // is true, only debugging information is removed from the module.
 //
 ModulePass *createStripSymbolsPass(bool OnlyDebugInfo = false);
 
 //===----------------------------------------------------------------------===//

 /// bugpoint.
 ModulePass *createDeadArgHackingPass();
 
 //===----------------------------------------------------------------------===//
 /// createArgumentPromotionPass - This pass promotes "by reference" arguments to
 /// be passed by value if the number of elements passed is smaller or
 /// equal to maxElements (maxElements == 0 means always promote).
 ///
-Pass *createArgumentPromotionPass(unsigned maxElements = 3);
+Pass *
+createArgumentPromotionPass(unsigned maxElements = NumAggregateElemsPromoted);
 
 //===----------------------------------------------------------------------===//
 /// createOpenMPOptLegacyPass - OpenMP specific optimizations.
 Pass *createOpenMPOptLegacyPass();
 
 //===----------------------------------------------------------------------===//
 /// createIPConstantPropagationPass - This pass propagates constants from call
 /// sites into the bodies of functions.

llvm/include/llvm/Transforms/IPO/ArgumentPromotion.h

 //===- ArgumentPromotion.h - Promote by-reference arguments -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TRANSFORMS_IPO_ARGUMENTPROMOTION_H
 #define LLVM_TRANSFORMS_IPO_ARGUMENTPROMOTION_H
 
 #include "llvm/Analysis/CGSCCPassManager.h"
 #include "llvm/Analysis/LazyCallGraph.h"
 #include "llvm/IR/PassManager.h"
 
 namespace llvm {
+
 class TargetTransformInfo;
 
 /// Argument promotion pass.
 ///
 /// This pass walks the functions in each SCC and for each one tries to
 /// transform it and all of its callers to replace indirect arguments with
 /// direct (by-value) arguments.
 class ArgumentPromotionPass : public PassInfoMixin<ArgumentPromotionPass> {
   unsigned MaxElements;
 

llvm/lib/Transforms/IPO/ArgumentPromotion.cpp

 //===- ArgumentPromotion.cpp - Promote by-reference arguments -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 #include <string>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "argpromotion"
 
+// opt flag to capture user supplied value for global variable
+// NumAggregateElemsPromoted
+cl::opt<unsigned, true> numelems(
+    "argpromotion-max-elements-to-promote",
+    cl::desc("Specify maximum number of elements of an aggregate to promote.\
+      At present, only aggregates with a maximum of three elements\
+      arepromoted."),
+    cl::Hidden, cl::location(llvm::NumAggregateElemsPromoted));
+
+unsigned llvm::NumAggregateElemsPromoted = 3;
+
 STATISTIC(NumArgumentsPromoted, "Number of pointer arguments promoted");
 STATISTIC(NumAggregatesPromoted, "Number of aggregate arguments promoted");
 STATISTIC(NumByValArgsPromoted, "Number of byval arguments promoted");
 STATISTIC(NumArgumentsDead, "Number of dead pointer args eliminated");
 
 /// A vector used to hold the indices of a single GEP instruction
 using IndicesVector = std::vector<uint64_t>;
 

 
 namespace {
 
 /// ArgPromotion - The 'by reference' to 'by value' argument promotion pass.
 struct ArgPromotion : public CallGraphSCCPass {
   // Pass identification, replacement for typeid
   static char ID;
 
-  explicit ArgPromotion(unsigned MaxElements = 3)
+  explicit ArgPromotion(unsigned MaxElements = NumAggregateElemsPromoted)
       : CallGraphSCCPass(ID), MaxElements(MaxElements) {
     initializeArgPromotionPass(*PassRegistry::getPassRegistry());
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<AssumptionCacheTracker>();
     AU.addRequired<TargetLibraryInfoWrapperPass>();
     AU.addRequired<TargetTransformInfoWrapperPass>();

 INITIALIZE_PASS_DEPENDENCY(CallGraphWrapperPass)
 INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
 INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
 INITIALIZE_PASS_END(ArgPromotion, "argpromotion",
                     "Promote 'by reference' arguments to scalars", false, false)
 
 Pass *llvm::createArgumentPromotionPass(unsigned MaxElements) {
   return new ArgPromotion(MaxElements);
 }
 
 bool ArgPromotion::runOnSCC(CallGraphSCC &SCC) {
   if (skipSCC(SCC))
     return false;
 
   // Get the callgraph information that we need to update to reflect our
   // changes.
   CallGraph &CG = getAnalysis<CallGraphWrapperPass>().getCallGraph();

llvm/test/Transforms/ArgumentPromotion/fourmember-struct-promote.ll

+; RUN: opt < %s -argpromotion -argpromotion-max-elements-to-promote=0 -S | FileCheck %s --check-prefix=PROMOTED
+; RUN: opt < %s -argpromotion -argpromotion-max-elements-to-promote=2147483647 -S | FileCheck %s --check-prefix=PROMOTED
+; RUN: opt < %s -argpromotion -S | FileCheck %s --check-prefix=NOTPROMOTED
+
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+
+%struct.fourmem = type { i32, i32, i32, i32 }
+
+; Function Attrs: noinline norecurse ssp uwtable
+define i32 @main() #0 {
+entry:
+  %four = alloca %struct.fourmem, align 4
+  %a = getelementptr inbounds %struct.fourmem, %struct.fourmem* %four, i32 0, i32 0
+  store i32 1, i32* %a, align 4
+  %b = getelementptr inbounds %struct.fourmem, %struct.fourmem* %four, i32 0, i32 1
+  store i32 2, i32* %b, align 4
+  %c = getelementptr inbounds %struct.fourmem, %struct.fourmem* %four, i32 0, i32 2
+  store i32 3, i32* %c, align 4
+  %d = getelementptr inbounds %struct.fourmem, %struct.fourmem* %four, i32 0, i32 3
+  store i32 4, i32* %d, align 4
+  %call = call i32 @_ZL6calleeP7fourmem(%struct.fourmem* %four)
+  ret i32 0
+}
+
+; 'PROMOTED' tag is to check that flag works correctly to promote elements within aggregates
+; PROMOTED-LABEL: define {{[^@]+}}@main()
+; PROMOTED-NEXT: entry:
+; PROMOTED-NEXT: [[F:%.*]] = alloca {{.*}}
+; PROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; PROMOTED-NEXT: store {{.*}}
+; PROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; PROMOTED-NEXT: store {{.*}}
+; PROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; PROMOTED-NEXT: store {{.*}}
+; PROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; PROMOTED-NEXT: store {{.*}}
+; PROMOTED-NEXT: [[F]].idx{{.*}} = getelementptr %struct.fourmem,  %struct.fourmem* [[F]], i64 0, i32 0
+; PROMOTED-NEXT: [[F]].idx{{.*}}.val = load i32, i32* [[F]].idx{{.*}}, align 4
+; PROMOTED-NEXT: [[F]].idx{{.*}} = getelementptr %struct.fourmem,  %struct.fourmem* [[F]], i64 0, i32 1
+; PROMOTED-NEXT: [[F]].idx{{.*}}.val = load i32, i32* [[F]].idx{{.*}}, align 4
+; PROMOTED-NEXT: [[F]].idx{{.*}} = getelementptr %struct.fourmem,  %struct.fourmem* [[F]], i64 0, i32 2
+; PROMOTED-NEXT: [[F]].idx{{.*}}.val = load i32, i32* [[F]].idx{{.*}}, align 4
+; PROMOTED-NEXT: [[F]].idx{{.*}} = getelementptr %struct.fourmem,  %struct.fourmem* [[F]], i64 0, i32 3
+; PROMOTED-NEXT: [[F]].idx{{.*}}.val = load i32, i32* [[F]].idx{{.*}}, align 4
+; PROMOTED-NEXT: %{{.*}} = call i32 @{{.*}}fourmem(i32 [[F]].idx{{.*}}.val, i32 [[F]].idx{{.*}}.val, i32 [[F]].idx{{.*}}.val, i32 [[F]].idx{{.*}}.val)
+; PROMOTED-NEXT: ret i32 0
+;
+
+; 'NOTPROMOTED' tag is to check that flag works correctly to NOT promote elements within aggregates
+; NOTPROMOTED-LABEL: define {{[^@]+}}@main()
+; NOTPROMOTED-NEXT: entry:
+; NOTPROMOTED-NEXT: [[F:%.*]] = alloca {{.*}}
+; NOTPROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; NOTPROMOTED-NEXT: store {{.*}}
+; NOTPROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; NOTPROMOTED-NEXT: store {{.*}}
+; NOTPROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; NOTPROMOTED-NEXT: store {{.*}}
+; NOTPROMOTED-NEXT: {{%.*}} = getelementptr {{.*}}
+; NOTPROMOTED-NEXT: store {{.*}}
+; NOTPROMOTED-NEXT: %{{.*}} = call i32 @{{.*}}fourmem(%struct.fourmem* [[F]])
+; NOTPROMOTED-NEXT: ret i32 0
+;
+
+; Function Attrs: noinline nounwind ssp uwtable
+define internal i32 @_ZL6calleeP7fourmem(%struct.fourmem* %t) #1 {
+entry:
+  %a = getelementptr inbounds %struct.fourmem, %struct.fourmem* %t, i32 0, i32 0
+  %0 = load i32, i32* %a, align 4
+  %b = getelementptr inbounds %struct.fourmem, %struct.fourmem* %t, i32 0, i32 1
+  %1 = load i32, i32* %b, align 4
+  %add = add nsw i32 %0, %1
+  %c = getelementptr inbounds %struct.fourmem, %struct.fourmem* %t, i32 0, i32 2
+  %2 = load i32, i32* %c, align 4
+  %add1 = add nsw i32 %add, %2
+  %d = getelementptr inbounds %struct.fourmem, %struct.fourmem* %t, i32 0, i32 3
+  %3 = load i32, i32* %d, align 4
+  %add2 = add nsw i32 %add1, %3
+  ret i32 %add2
+}
+
+; 'PROMOTED' tag is to check that flag works correctly to promote elements within aggregates
+; PROMOTED-LABEL: define {{[^@]+}}@{{.*}}fourmem
+; PROMOTED-SAME: (i32 %t.0.{{.*}}.val, i32 %t.0.{{.*}}.val, i32 %t.0.{{.*}}.val, i32 %t.0.{{.*}}.val)
+; PROMOTED-NEXT: entry:
+; PROMOTED-NEXT: %add{{.*}} = add nsw i32 {{.*}}, {{.*}}
+; PROMOTED-NEXT: %add{{.*}} = add nsw i32 {{.*}}, {{.*}}
+; PROMOTED-NEXT: %add{{.*}} = add nsw i32 {{.*}}, {{.*}}
+; PROMOTED-NEXT: ret i32 %{{.*}}
+;
+
+; 'NOTPROMOTED' tag is to check that flag works correctly to NOT promote elements within aggregates
+; NOTPROMOTED: define {{[^@]+}}@{{.*}}fourmem(%struct.fourmem* %t)
+; NOTPROMOTED-NEXT: entry:
+; NOTPROMOTED-NEXT: %{{.*}} = getelementptr inbounds {{.*}}, {{%.*}}, i32 0, i32 0
+; NOTPROMOTED-NEXT: %{{.*}} = load i32, i32* {{%.*}}, align 4
+; NOTPROMOTED-NEXT: %{{.*}} = getelementptr inbounds {{.*}}, {{%.*}}, i32 0, i32 1
+; NOTPROMOTED-NEXT: %{{.*}} = load i32, i32* {{%.*}}, align 4
+; NOTPROMOTED-NEXT: %add{{.*}} = add nsw i32 {{%.*}}, {{%.*}}
+; NOTPROMOTED-NEXT: %{{.*}} = getelementptr inbounds {{.*}}, {{%.*}}, i32 0, i32 2
+; NOTPROMOTED-NEXT: %{{.*}} = load i32, i32* {{%.*}}, align 4
+; NOTPROMOTED-NEXT: %add{{.*}} = add nsw i32 {{%.*}}, {{%.*}}
+; NOTPROMOTED-NEXT: %{{.*}} = getelementptr inbounds {{.*}}, {{%.*}}, i32 0, i32 3
+; NOTPROMOTED-NEXT: %{{.*}} = load i32, i32* {{%.*}}, align 4
+; NOTPROMOTED-NEXT: %add{{.*}} = add nsw i32 {{%.*}}, {{%.*}}
+; NOTPROMOTED-NEXT: ret i32 %{{.*}}
+;