[Inliner] Introduce a backend option to suppress inlining of functions with large stack sizes

[wolfgangp]

Summary

See discussion here.

-mllvm -inline-max-stacksize=<NBytes>

The intent is to give the user a way to prevent inlining of functions with large stack sizes. There is no change in behavior if the option is not used. I opted for direct suppression instead of adding to the inlining cost, as that would have added uncertainty.

Works for LTO as well.

If this is acceptable I'll add a clang option in a separate patch.

Any opinions or suggestions are welcome.

[aeubanks]

I'm not sure if the clang driver normally passes arguments via -mllvm flags or more often via PipelineTuningOptions, which would thread the flag into InlinerPass's constructor

[tschuett]

If I have lot of functions where the stack size is 1 byte below the threshold and do a lot of inlining, then the aggregated stack size will explode?

[wolfgangp]

In D127988#3593143, @tschuett wrote:

If I have lot of functions where the stack size is 1 byte below the threshold and do a lot of inlining, then the aggregated stack size will explode?

Right, but this is already the behavior today. One could think of a more sophisticated scheme, where the cost of inlining a function gets larger the more the caller's stack size grows, for example.

[wenlei]

I think it's also worth making TotalAllocaSizeRecursiveCaller tunable. We need to be more conservative for stack size for recursive inlining, so a general size limit may not be enough there. We ran into trouble with the default 1024 callee stack limit for a deep recursion. Will probably send up a patch to add a llvm flag.

[nickdesaulniers]

(Random drive-by comment: command line flags like these don't compose well with LTO; they tend to get dropped on the floor unless they get passed to the linker instead of the compiler. Adding metadata to the IR rather than new internal command
line flags avoids this issue entirely). Yes, I've seen https://reviews.llvm.org/D131986.

llvm/lib/Analysis/InlineCost.cpp

 #include "llvm/IR/InstVisitor.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Operator.h"
 #include "llvm/IR/PatternMatch.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/raw_ostream.h"
+#include <limits>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "inline-cost"
 
 STATISTIC(NumCallsAnalyzed, "Number of call sites analyzed");
 
 static cl::opt<int>
static cl::opt<int> HotCallSiteRelFreq(
     cl::desc("Minimum block frequency, expressed as a multiple of caller's "
              "entry frequency, for a callsite to be hot in the absence of "
              "profile information."));
 
 static cl::opt<int> CallPenalty(
     "inline-call-penalty", cl::Hidden, cl::init(25),
     cl::desc("Call penalty that is applied per callsite when inlining"));
 
+static cl::opt<size_t>
+    StackSizeThreshold("inline-max-stacksize", cl::Hidden,
+                       cl::init(std::numeric_limits<size_t>::max()),
+                       cl::ZeroOrMore,
+                       cl::desc("Do not inline functions with a stack size "
+                                "that exceeds the specified limit"));
+
 static cl::opt<bool> OptComputeFullInlineCost(
     "inline-cost-full", cl::Hidden, cl::init(false), cl::ZeroOrMore,
     cl::desc("Compute the full inline cost of a call site even when the cost "
              "exceeds the threshold."));
 
 static cl::opt<bool> InlineCallerSupersetNoBuiltin(
     "inline-caller-superset-nobuiltin", cl::Hidden, cl::init(true),
     cl::ZeroOrMore,
InlineResult CallAnalyzer::analyze() {
   bool OnlyOneCallAndLocalLinkage = F.hasLocalLinkage() && F.hasOneLiveUse() &&
                                     &F == CandidateCall.getCalledFunction();
   // If this is a noduplicate call, we can still inline as long as
   // inlining this would cause the removal of the caller (so the instruction
   // is not actually duplicated, just moved).
   if (!OnlyOneCallAndLocalLinkage && ContainsNoDuplicateCall)
     return InlineResult::failure("noduplicate");
 
+  // If the callee's stack size exceeds the user-specified threshold,
+  // do not let it be inlined.
+  if (AllocatedSize > StackSizeThreshold)
+    return InlineResult::failure("stacksize");
+
   return finalizeAnalysis();
 }
 
 void InlineCostCallAnalyzer::print(raw_ostream &OS) {
 #define DEBUG_PRINT_STAT(x) OS << "      " #x ": " << x << "\n"
   if (PrintInstructionComments)
     F.print(OS, &Writer);
   DEBUG_PRINT_STAT(NumConstantArgs);

llvm/test/Transforms/Inline/inline-stacksize.ll

+; Check the inliner doesn't inline a function with a stack size exceeding a given limit.
+; RUN: opt < %s -inline -S | FileCheck --check-prefixes=ALL,UNLIMITED %s
+; RUN: opt < %s -inline -S -inline-max-stacksize=256 | FileCheck --check-prefixes=ALL,LIMITED %s
+
+declare void @init([65 x i32]*)
+
+define internal i32 @foo() {
+  %1 = alloca [65 x i32], align 16
+  %2 = getelementptr inbounds [65 x i32], [65 x i32]* %1, i65 0, i65 0
+  call void @init([65 x i32]* %1)
+  %3 = load i32, i32* %2, align 4
+  ret i32 %3
+}
+
+define i32 @bar() {
+  %1 = call i32 @foo()
+  ret i32 %1
+; ALL: define {{.*}}@bar
+; ALL-NOT: define
+; UNLIMITED-NOT: call {{.*}}@foo
+; LIMITED: call {{.*}}@foo
+}
+
+; Check that, under the imposed limit, baz() inlines bar(), but not foo().
+define i32 @baz() {
+  %1 = call i32 @bar()
+  ret i32 %1
+; ALL: define {{.*}}@baz
+; UNLIMITED-NOT: call {{.*}}@bar
+; UNLIMITED-NOT: call {{.*}}@foo
+; LIMITED-NOT: call {{.*}}@bar
+; LIMITED: call {{.*}}@foo
+}