[FuncSpec] Make the Function Specializer part of the IPSCCP pass.

[labrinea]

Summary

The aim of this patch is to minimize the compilation time overhead of running Function Specialization. It is about 40% slower to run as a standalone pass (IPSCCP + FuncSpec vs IPSCCP with FuncSpec) according to my measurements. I compiled the llvm testsuite with NewPM-O3 + LTO and measured single threaded [user + system] time of IPSCCP and FuncSpec by passing the '-time-passes' option to lld. Then I compared the two configurations in terms of Instruction Count of the total compilation (not of the individual passes) as in https://llvm-compile-time-tracker.com. Geomean for non-LTO builds is -0.25% and LTO is -0.5% approximately.

You can find more info below:
https://discourse.llvm.org/t/rfc-should-we-enable-function-specialization/61518

[labrinea]

This is a proof of concept for RFC: Should we enable Function Specialization?, not ready for review.

[labrinea]

rebased + fixed tests

[sinan]

tryToReplaceWithConstant method in SCCP does not update the lattice value map at SCCPSolver, and it might lead to a problem that

if

%arg = getelementptr %struct, %struct* @Global, i32 0, i32 3
%tmp0 = call i64 @func2(i64* %arg)

is folded into

%tmp0 = call i64 @func2(i64* getelementptr inbounds %struct, %struct* @Global, i32 0, i32 3)

a new callbase argument appears, but it is not recorded at SCCPSolver, and this leads to problems such as D128822.

Suggestion:
FunctionSpecializer::tryToReplaceWithConstant use the code piece below to update the propagated argument, and maybe we need such a change for tryToReplaceWithConstant as well.

for (auto *I : UseInsts)
  Solver.visit(I);

[labrinea]

In D126455#3644307, @sinan wrote:

tryToReplaceWithConstant method in SCCP does not update the lattice value map at SCCPSolver, and it might lead to a problem that

if

%arg = getelementptr %struct, %struct* @Global, i32 0, i32 3
%tmp0 = call i64 @func2(i64* %arg)

is folded into

%tmp0 = call i64 @func2(i64* getelementptr inbounds %struct, %struct* @Global, i32 0, i32 3)

a new callbase argument appears, but it is not recorded at SCCPSolver, and this leads to problems such as D128822.

Suggestion:
FunctionSpecializer::tryToReplaceWithConstant use the code piece below to update the propagated argument, and maybe we need such a change for tryToReplaceWithConstant as well.

for (auto *I : UseInsts)
  Solver.visit(I);

I am doing this in a later patch (see D126456) as I wanted to keep this one as close as possible to the original implementation.

[labrinea]

ping

[fhahn]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

301

can you fix the indentation here in a NFC?

610

IIUC this is done during in between solver runs, right? Is this needed? Isn't it sufficient to continue with the constant value in the value mapping? This would probably remove the need to tell the solver to forget instructions/values.

[labrinea]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

301

How? It seems ok.

610

This is not the only invocation of tryToReplaceWithConstant in FuncSpec. On this instance we try to replace the arguments of cloned functions. There's another invocation inside the functor RunSCCPSolver. On that instance we try to replace the instructions of cloned functions. Both calls occur as many times as FuncSpecializationMaxIters is set to.

Moreover, the SCCP pass itself does the same thing on arguments of tracked functions and on instructions of executable blocks (with tryToReplaceWithConstant and simplifyInstsInBlock accordingly). This happens after the Solver runs and before the Function Specializer is invoked. Therefore, I think we still need to tell the Solver to forget instructions/values if we want to merge the two passes.

[chill]

llvm/lib/Transforms/Scalar/SCCP.cpp

637–638

IMHO, the invocation of the FunctionSpecialization pass ought to happen in this place. The general flow would be like:

1. Initialise solver
2. Run solver once (Solver.solve() + resolvedUndefsIn loop)
3. Run function specialisation
4. Run solver again
5. Optionally go to 2.
6. Do replacements (from line 512 on)

At no point before the last step the passes ought to replace or delete anything (well, except called function operand for cloned functions). If an operand/argument is determined to be a constant, it does not need to be replaced right away, because the passes should consult its lattice value.

Yeah, the devil is in the details, but this is the approach to merging the tow passes, as I see it.

[labrinea]

I have moved the invocation of the specializer earlier in the ipsccp pass, such that no instructions get deleted until all the solving is done. This essentially makes all of D128822, D128823, D128824, D128825, D126456 and D128827 obsolete. There is one thing I couldn't get working, which is to update the lattice value of the callsites to specialized functions. Unfortunately the semantics of the solver do not allow lattices to move from a generic to a more specific state (i.e. from a wider to a narrower constant range). That said the zapping of returned values won't work on specialized functions, neither we can propagate a constant returned value to the function body where the callsite resides. On another note, the function analysis information which is provided to the pass (predication info, dominator tree) cannot be used on the specialized functions; not sure if that's a problem though.

[labrinea]

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

Just found that we need to do the same inside replaceSignedInst() too. I will move this code a function.

[chill]

llvm/lib/Transforms/Scalar/CMakeLists.txt

97

Why add IPO here ?

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

Would it be possible to call markUsersAsChanged here ?

229–231

Is there a specific reason to remove the instruction from the block?
If not, I'd suggest doing deletion in a single place, as opposed to spreading parts of it all over.

247

Likewise.

[labrinea]

llvm/lib/Transforms/Scalar/CMakeLists.txt

97

I vaguely remember a link time error without this change. See also llvm/utils/gn/secondary/llvm/lib/Transforms/Scalar/BUILD.gn at the bottom of this diff. The IPSCCP pass now depends on the FunctionSpecializer whose cpp file is under the IPO directory.

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

I think we can't because if we replace the uses first then the users of the old value will be empty. Can we markUsersAsChanged before we replaceAllUsesWith the new value? Btw markUsersAsChanged is private for the SCCPInstVisitor, but I suppose I could make it public if need be.

229–231

I am not entirely sure. I wanted to avoid revisiting this instruction accidentally in either of simplifyInstsInBlock(), solve(), or resolvedUndefsIn(). For simplifyInstsInBlock() I could skip the instruction if it's present in ToDelete. For the others I don't know what the consequences of revisitng would be. I need to run some tests first.

[labrinea]

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

Actually I could call markUsersAsChanged on the new Instrcution after replacing the uses of the old Instruction with it.

[chill]

llvm/lib/Transforms/Scalar/CMakeLists.txt

97

IPO already depends on Scalar, i.e. in IPO/CMakeLists.txt we have

...

  COMPONENT_NAME
  IPO

  LINK_COMPONENTS
...
  Scalar
...

Looks like a circular dependency.

Perhaps FunctionSpecialization needs to go to Utils (alongside SCCPSolver).
Or runIPSCCP needs to go to IPO/SCCP.cpp.
Or both.

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

OK, let's leave it hanging for now, until I can take a look on top of the latest trunk.

Ideally, we are trying to avoid changing code until the Solver is done.
Here we have found that an instruction has constant lattice value - we should not replace the users' operands
right away, but notify the Solver. The Solver in turn would add the instructions that need reexamining to the instructions worklist and update their lattice values the next time we invoke Solvet.solve().

Most likely SCCPSolver::visit should become private, the Solver (and the SCCP algorithm in general) is
driven by its worklists, we should stick to this design: want something done - add it to the worklist.

229–231

I can't see why would anything go wrong if the instruction is revisited.
Do we know if the instruction is safe to remove? It could be SDiv/SRem with a zero divisor.

[chill]

llvm/lib/Transforms/Scalar/SCCP.cpp

229–231

Actually, never mind, we're not replacing the instruction with a constant but with another instruction.

[labrinea]

llvm/lib/Transforms/Scalar/SCCP.cpp

167–178

Update: I tried this. It works for 'some' cases. Instead of replacing values with constants I create mappings from the old to the new value and only after all the solving is done then I replace the uses. The specialization of recursive functions doesn't work because it relies on finding allocas of constant integers. Also the rewriting of callsites doesn't work either if the actual arguments have been constant propagated prior to specialization, but the old value hasn't been replaced yet. In theory I could pass on the mappings from sccp to the specializer but it seems overly complicated to do so.

[labrinea]

Changes from last revision:

• rebased on top of main; that required adjusting a couple of pass-manager tests as the LoopInfo analysis is now used in the default pipelines by the sccp pass
• lazily eraseFromParent instructions which have been replaced instead of removeFromParent and deleteValue later as suggested by @chill
• used markUsersAsChanged instead of visit as suggested by @chill (required exposing it to the public interface)
• moved/renamed solveWhileResolvedUndefsIn to the solver as it is required from the specializer too (fixes a bug I have added a new testcase for)
• changed createSpecialization to return a pointer to the cloned function

[chill]

llvm/lib/Transforms/IPO/SCCP.cpp

46

This part was added for the FunctionSpecialization, if func spec is disabled maybe not pass along the LoopAnalysis?

[chill]

llvm/lib/Transforms/IPO/SCCP.cpp

53

Now that we added LoopAnalysis we may well preserve it too. (I should have included it in the patch which introduced the LoopAnalysis here)

[labrinea]

Good point. Running the LoopInfo analysis when we do not specialize adds unnecessary compile time overhead. After some investigation I found that notifying the users after replacing a value (instead of notifying only the old users), as well as not removing replaced instructions, both add significant compile time overhead. I am inclined to revert the latter and rework on the former so that markUsersAsChanged can accept a list of users.

[labrinea]

Changes from last revision:

• Predicated the LoopInfo analysis on the cmd-line option that enables function specialization. As a result we no longer need to adjust the pass manager unit-tests.
• Reverted the eraseFromParent to removeFromParent/deleteValue for replaced instructions to save compilation time.
• Modified markUsersAsChanged to accept a UserList so that we avoid revisiting irrelevant users after replacing a value (saves compilation time).

Note: The instruction count delta for CTMark with NewPM-O3+LTO between baseline (da5ded4fc9d8c8edfd4a79fa0e75c2ac9165fa7b) and this patch (with funcspec disabled) is about 0.05 % geomean.

[labrinea]

llvm/lib/Transforms/IPO/SCCP.cpp

53

I tried this but the compiler crashes. Probably because SCCP deletes dead basic blocks.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

239

WorkList is a too generic name for a parameter and it's not a worklist per se anyway.

275

This is a little bit fragile in the sense the caller may forget to clear the list.
It would be nicer if this function itself clears the list first thing when it starts execution.

WorkList could also be made a return, taking advantage of move semantics, which looks nicer on paper, but
may cause a few allocations/deallocations if we iterate.

llvm/lib/Transforms/Scalar/SCCP.cpp

116

Wouldn't it work without the temporary vector?

markUsersAsChanged would go over each user, look at the user's operands (including Old),
and find the New (which is some constant) form the lattice values map.

Thus we would maybe get just:

Solver.markUsersAsChanged(Old);
Old->replaceAllUsesWith(New);

167–178

Instead of replacing values with constants I create mappings from the old to the new value ..

But isn't this what the ValueState already contains?

Also the rewriting of callsites doesn't work either if the actual arguments have been constant propagated prior to specialization, but the old value hasn't been replaced yet.

Well, FunctionSpecializer::rewriteCallSites and everything else should lookup lattice values, not work directly with operands.

But OK, let's not make too many changes at once and revisit it later.

642

I would suggest not creating a vector of all the functions in the module as they could be quite a lot (e.g. in LTO)
and thus trigger several heap allocations for WorkList.

solveWhileResolvedUndefIn is quite small and could be overloaded for a Module * parameter.

I considered making this function a template along the lines of:

template<typename RangeT>
void printNames(RangeT &&R) {
    for (auto &F : R)
      llvm::dbgs() << magic(F)->getName();
}

std::vector<llvm::Function *> v;

llvm::Module *M;

int main() {
    printNames(M->functions());
    printNames(v);
}

but couldn't come up with magic.

As for propagateConstants it could be done with a few overloads as well:

static bool propagateConstants(SCCPSolver &Solver, Function *F,
                               SmallPtrSetImpl<Instruction *> &ToDelete);

static bool propagateConstants(SCCPSolver &Solver,
                               SmallVectorImpl<Function *> &WorkList,
                               SmallPtrSetImpl<Instruction *> &ToDelete) {
  for (Function *F : WorkList)
      propagateConstants(Solve, F, ToDelete);
}

static bool propagateConstants(SCCPSolver &Solver, Module *M,
                               SmallPtrSetImpl<Instruction *> &ToDelete) {
  for (auto &F : Module)
      propagateConstants(Solve, &F, ToDelete);
}

llvm/lib/Transforms/Utils/SCCPSolver.cpp

1585

All the functions here forward to the Visitor, this one should also just be forwarding.

(Not sure why this proxy class exists at all, but I guess we can address it later).

[labrinea]

This revision is somewhat different from the previous ones because we no longer replace instructions/arguments whilst in the main specialization loop of the SCCP pass. Instead we use the lattice value when rewritting callsites and when promoting constant stack values. Last time I checked this was even more lightweight from the previous revision. Builds successfully the llvm-test-suite with aggressive funcspec options; haven't tried clang bootstrap yet. Also improves one of the unit tests with recursive functions.

[labrinea]

llvm/include/llvm/Transforms/IPO/FunctionSpecialization.h

70

I've removed an unused typedef from here ;)

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

207–208

I am not sure whether this is necessary. The unit tests which exercise recursion are passing without it at least.

266

We now call this function once, not for every clone as it used to be.

658–662

There might be more call sites to rewrite than those in the CallSpecBinding that we have already found, therefore we need to repeat this look up of users here, but at least it now happens once for F compared to Clones.size() times which was the case before.

662

no need to examine lattices of arguments if it's the key of the CallSpecBinding

669–670

We are modifying the list whist traversing it, so we swap the current element with the last one and reduce the iteration range by one.

691

the condition was different before, but I think this is correct

llvm/lib/Transforms/Utils/SCCPSolver.cpp

439–457

I couldn't template these two. The main reason was that one iterates over Function * whereas the other over Function &.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

663–672

You can swap the order of the loops and get rid of CallSiteToRewrite.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

663–672

Maybe I'm getting a bit ahead of me, but you can change the function to rewrite just a single call site and factor out the iteration over call sites. The benefit is the function becomes more reusable in as you can independently choose the set of call sites it operates upon. (Incidently, I'm planning to use it that way, but it generally a good change, even if what I have in mind turns out non-working).

[labrinea]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

663–672

You mean to traverse F's users here instead? We are iterating over them while modifying them, which is the reason why CallSiteToRewrite existed in the first place I believe. Also the dynamic cast and other checks we do: CS->getCalledFunction() == F && Solver.isBlockExecutable(CS->getParent()) (note: this one is missing from the current revision) don't need to be repeated on every iteration of the outer loop which walks the specializaions.

663–672

If I change it the way you suggest it will regress the current behaviour. Qsort() from SPEC's mcf won't specialize (it's a recursive function) and it's been quite a drive for this work. What if you alter it once you refactor how we rewrite callsites?

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

663–672

What we have now is:

forall S : Specialisations {
  forall  C : CallSites {
   do_stuff(S, C);
 }
}

What I'm suggesting is to reorder the loops

forall C: call sites {
   forall S: specialisations {
    do_stuff(S, C);
  }
}

That'll avoid the swaps/pop_back. The vector itself stays, good point.

And then I'm suggesting to move the outer loop out of the function:

func foo() {
 ...
  forall C: CallSites {
    rewriteCallSite(C)
  }
...
}

func rewriteCallSite(C) {
    forall S : Specialisations {
     do_stuff(S, C);
   }
}

Both are NFC.

Also the dynamic cast and other checks we do: CS->getCalledFunction() == F && Solver.isBlockExecutable(CS->getParent()) (note: this one is missing from the current revision) don't need to be repeated on every iteration of the outer loop which walks the specializaions.

I don't understand this. We do nothing between loops, so interchanging them will execute exactly the same operations in the loop body.

If you add these checks somewhere, it's another argument to move the iteration over call sites to the outer loop.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

686–705

This is better placed outside of rewriteCallSites, perhaps just after the call to rewriteCallSites.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

238

I believe the LLVM convention for these kinds of classes/methods is run, e.g. Vectorizer::run(), EarlyCSE::run(), etc.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

686–705

Or a better idea: get the initial size of CallSitesToRewrite, decrement that number every time you update a call site. At the end if this number drops to zero mark the function unreachable.

[chill]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

658

Use braces around the for, since there are more than two levels of nesting.

[labrinea]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

686–705

that won't work for dead recursive functions

[labrinea]

Changes from last revision:

• renamed specialize() to run() as suggested
• renamed rewriteCallSites to updateCallSites
• interchanged the loops in updateCallSites as suggested
• update callsites of specializations before anything else (changes the test identical-specializations.ll)
• used braces for nested loop
• clang formated

[labrinea]

Found a bug in the last revision. The swapping idiom violates the expected order of traversing the call sites to update. They are supposed to be sorted by gain.

[labrinea]

Changes from last revison:

• Used a counter for updated callsites instead of revising them at the end to identify dead functions.

[labrinea]

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

639

I'll rename this and add a comment to explain what it is used for.

[labrinea]

Changes from last revision:

• rebased
• renamed the variable which determines whether we decrement the counter of left callsites to replace
• cached the cloned function pointer to the SpecializationInfo to avoid keeping the Clone vector in sync with the Specialization vector when replacing callsites

[labrinea]

Also removed the immediate replacement of known callsites. It's better to lookup the lattice value instead. This fixed a compile time regression possibly caused by otherwise dead duplicate specializations that had to be processed by later passes.

[chill]

Would you, please, mark as done the no longer relevant comments?
I think the only issue left is with the circular dependency between libraries.

[labrinea]

In D126455#3936611, @chill wrote:

Would you, please, mark as done the no longer relevant comments?
I think the only issue left is with the circular dependency between libraries.

Ack

[labrinea]

Removed the cyclic dependency between LLVMipo and LLVMScalarOpts and rebased on top of D138654.

[chill]

LGTM, but let's give a chance for other people to have a look too. @sinan @fhahn

[labrinea]

• rebase
• migrated all funcspec tests to use the -passes= cmdline option

[fhahn]

llvm/lib/Transforms/IPO/SCCP.cpp

27

move this to the the loop below, which uses it

llvm/lib/Transforms/Utils/SCCPSolver.cpp

60–61

I think DomTreeUpdater provides a constructor that doesn't take a DT which could be used unconditionally instead of having all those if (DTU) checks spread out across various functions.

[labrinea]

• Moved a flag close to its use.
• Used a DomTreeUpdater without DT/PDT for cloned functions.

llvm/include/llvm/InitializePasses.h

 void initializeFinalizeMachineBundlesPass(PassRegistry&);
 void initializeFixIrreduciblePass(PassRegistry &);
 void initializeFixupStatepointCallerSavedPass(PassRegistry&);
 void initializeFlattenCFGLegacyPassPass(PassRegistry &);
 void initializeFloat2IntLegacyPassPass(PassRegistry&);
 void initializeForceFunctionAttrsLegacyPassPass(PassRegistry&);
 void initializeForwardControlFlowIntegrityPass(PassRegistry&);
 void initializeFuncletLayoutPass(PassRegistry&);
-void initializeFunctionSpecializationLegacyPassPass(PassRegistry &);
 void initializeGCMachineCodeAnalysisPass(PassRegistry&);
 void initializeGCModuleInfoPass(PassRegistry&);
 void initializeGVNHoistLegacyPassPass(PassRegistry&);
 void initializeGVNLegacyPassPass(PassRegistry&);
 void initializeGVNSinkLegacyPassPass(PassRegistry&);
 void initializeGlobalDCELegacyPassPass(PassRegistry&);
 void initializeGlobalMergePass(PassRegistry&);
 void initializeGlobalOptLegacyPassPass(PassRegistry&);

llvm/include/llvm/LinkAllPasses.h

ForcePassLinking() {
       (void) llvm::createFloat2IntPass();
       (void) llvm::createEliminateAvailableExternallyPass();
       (void)llvm::createScalarizeMaskedMemIntrinLegacyPass();
       (void) llvm::createWarnMissedTransformationsPass();
       (void) llvm::createHardwareLoopsPass();
       (void) llvm::createInjectTLIMappingsLegacyPass();
       (void) llvm::createUnifyLoopExitsPass();
       (void) llvm::createFixIrreduciblePass();
-      (void)llvm::createFunctionSpecializationPass();
       (void)llvm::createSelectOptimizePass();
 
       (void)new llvm::IntervalPartition();
       (void)new llvm::ScalarEvolutionWrapperPass();
       llvm::Function::Create(nullptr, llvm::GlobalValue::ExternalLinkage)->viewCFGOnly();
       llvm::RGPassManager RGM;
       llvm::TargetLibraryInfoImpl TLII;
       llvm::TargetLibraryInfo TLI(TLII);

llvm/include/llvm/Transforms/IPO.h

 //===----------------------------------------------------------------------===//
 /// createIPSCCPPass - This pass propagates constants from call sites into the
 /// bodies of functions, and keeps track of whether basic blocks are executable
 /// in the process.
 ///
 ModulePass *createIPSCCPPass();
 
 //===----------------------------------------------------------------------===//
-/// createFunctionSpecializationPass - This pass propagates constants from call
-/// sites to the specialized version of the callee function.
-ModulePass *createFunctionSpecializationPass();
-
-//===----------------------------------------------------------------------===//
 //
 /// createLoopExtractorPass - This pass extracts all natural loops from the
 /// program into a function if it can.
 ///
 Pass *createLoopExtractorPass();
 
 /// createSingleLoopExtractorPass - This pass extracts one natural loop from the
 /// program into a function if it can.  This is used by bugpoint.

llvm/include/llvm/Transforms/IPO/FunctionSpecialization.h

+//===- FunctionSpecialization.h - Function Specialization -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This specialises functions with constant parameters. Constant parameters
+// like function pointers and constant globals are propagated to the callee by
+// specializing the function. The main benefit of this pass at the moment is
+// that indirect calls are transformed into direct calls, which provides inline
+// opportunities that the inliner would not have been able to achieve. That's
+// why function specialisation is run before the inliner in the optimisation
+// pipeline; that is by design. Otherwise, we would only benefit from constant
+// passing, which is a valid use-case too, but hasn't been explored much in
+// terms of performance uplifts, cost-model and compile-time impact.
+//
+// Current limitations:
+// - It does not yet handle integer ranges. We do support "literal constants",
+//   but that's off by default under an option.
+// - The cost-model could be further looked into (it mainly focuses on inlining
+//   benefits),
+//
+// Ideas:
+// - With a function specialization attribute for arguments, we could have
+//   a direct way to steer function specialization, avoiding the cost-model,
+//   and thus control compile-times / code-size.
+//
+// Todos:
+// - Specializing recursive functions relies on running the transformation a
+//   number of times, which is controlled by option
+//   `func-specialization-max-iters`. Thus, increasing this value and the
+//   number of iterations, will linearly increase the number of times recursive
+//   functions get specialized, see also the discussion in
+//   https://reviews.llvm.org/D106426 for details. Perhaps there is a
+//   compile-time friendlier way to control/limit the number of specialisations
+//   for recursive functions.
+// - Don't transform the function if function specialization does not trigger;
+//   the SCCPSolver may make IR changes.
+//
+// References:
+// - 2021 LLVM Dev Mtg “Introducing function specialisation, and can we enable
+//   it by default?”, https://www.youtube.com/watch?v=zJiCjeXgV5Q
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_IPO_FUNCTIONSPECIALIZATION_H
+#define LLVM_TRANSFORMS_IPO_FUNCTIONSPECIALIZATION_H
+
+#include "llvm/Analysis/CodeMetrics.h"
+#include "llvm/Analysis/InlineCost.h"
+#include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/TargetTransformInfo.h"
+#include "llvm/Transforms/Scalar/SCCP.h"
+#include "llvm/Transforms/Utils/Cloning.h"
+#include "llvm/Transforms/Utils/SCCPSolver.h"
+#include "llvm/Transforms/Utils/SizeOpts.h"
+
+using namespace llvm;
+
+namespace llvm {
+// Bookkeeping struct to pass data from the analysis and profitability phase
+// to the actual transform helper functions.
+struct SpecializationInfo {
+  SmallVector<ArgInfo, 8> Args; // Stores the {formal,actual} argument pairs.
+  InstructionCost Gain;         // Profitability: Gain = Bonus - Cost.
+};
+
+using CallArgBinding = std::pair<CallBase *, Constant *>;
+using CallSpecBinding = std::pair<CallBase *, SpecializationInfo>;
+// We are using MapVector because it guarantees deterministic iteration
+// order across executions.
+using SpecializationMap = SmallMapVector<CallBase *, SpecializationInfo, 8>;
+
+class FunctionSpecializer {
+
+  /// The IPSCCP Solver.
+  SCCPSolver &Solver;
+
+  Module &M;
+
+  /// Analyses used to help determine if a function should be specialized.
+  std::function<const TargetLibraryInfo &(Function &)> GetTLI;
+  std::function<TargetTransformInfo &(Function &)> GetTTI;
+  std::function<AssumptionCache &(Function &)> GetAC;
+
+  // The number of functions specialised, used for collecting statistics and
+  // also in the cost model.
+  unsigned NbFunctionsSpecialized = 0;
+
+  SmallPtrSet<Function *, 32> SpecializedFuncs;
+  SmallPtrSet<Function *, 32> FullySpecialized;
+  DenseMap<Function *, CodeMetrics> FunctionMetrics;
+
+public:
+  FunctionSpecializer(SCCPSolver &Solver, Module &M,
+      std::function<const TargetLibraryInfo &(Function &)> GetTLI,
+      std::function<TargetTransformInfo &(Function &)> GetTTI,
+      std::function<AssumptionCache &(Function &)> GetAC)
+    : Solver(Solver), M(M), GetTLI(GetTLI), GetTTI(GetTTI), GetAC(GetAC) {}
+
+  ~FunctionSpecializer() {
+    // Eliminate dead code.
+    removeDeadFunctions();
+    cleanUpSSA();
+  }
+
+  bool isClonedFunction(Function *F) { return SpecializedFuncs.count(F); }
+
+  bool specialize(SmallVectorImpl<Function *> &WorkList);
+
+  /// Iterate over the argument tracked functions see if there
+  /// are any new constant values for the call instruction via
+  /// stack variables.
+  void promoteConstantStackValues();
+
+private:
+  /// Clean up fully specialized functions.
+  void removeDeadFunctions();
+
+  /// Remove any ssa_copy intrinsics that may have been introduced.
+  void cleanUpSSA();
+
+  // Compute the code metrics for function \p F.
+  CodeMetrics &analyzeFunction(Function *F);
+
+  /// This function decides whether it's worthwhile to specialize function
+  /// \p F based on the known constant values its arguments can take on. It
+  /// only discovers potential specialization opportunities without actually
+  /// applying them.
+  ///
+  /// \returns true if any specializations have been found.
+  bool findSpecializations(Function *F, InstructionCost Cost,
+                           SmallVectorImpl<CallSpecBinding> &WorkList);
+
+  bool isCandidateFunction(Function *F);
+
+  Function *createSpecialization(Function *F, SpecializationInfo &SpecInfo);
+
+  /// Compute and return the cost of specializing function \p F.
+  InstructionCost getSpecializationCost(Function *F);
+
+  /// Compute a bonus for replacing argument \p A with constant \p C.
+  InstructionCost getSpecializationBonus(Argument *A, Constant *C,
+                                         const LoopInfo &LI);
+
+  /// Determine if it is possible to specialise the function for constant values
+  /// of the formal parameter \p A.
+  bool isArgumentInteresting(Argument *A);
+
+  /// Check if the value \p V  (an actual argument) is a constant or can only
+  /// have a constant value. Return that constant.
+  Constant *getCandidateConstant(Value *V);
+
+  /// Rewrite calls to function \p F to call function \p Clone instead.
+  ///
+  /// This function modifies calls to function \p F as long as the actual
+  /// arguments match those in \p Args. Note that for recursive calls we
+  /// need to compare against the cloned formal arguments.
+  ///
+  /// Callsites that have been marked with the MinSize function attribute won't
+  /// be specialized and rewritten.
+  void rewriteCallSites(Function *Clone, const SmallVectorImpl<ArgInfo> &Args,
+                        ValueToValueMapTy &Mappings);
+};
+} // namespace
+
+#endif // LLVM_TRANSFORMS_IPO_FUNCTIONSPECIALIZATION_H

llvm/include/llvm/Transforms/IPO/SCCP.h

 class Module;
 
 /// Pass to perform interprocedural constant propagation.
 class IPSCCPPass : public PassInfoMixin<IPSCCPPass> {
 public:
   PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
 };
 
-/// Pass to perform interprocedural constant propagation by specializing
-/// functions
-class FunctionSpecializationPass
-    : public PassInfoMixin<FunctionSpecializationPass> {
-public:
-  PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
-};
-
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_IPO_SCCP_H

llvm/include/llvm/Transforms/Scalar/SCCP.h

 /// This pass performs function-level constant propagation and merging.
 class SCCPPass : public PassInfoMixin<SCCPPass> {
 public:
   PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM);
 };
 
 bool runIPSCCP(Module &M, const DataLayout &DL,
                std::function<const TargetLibraryInfo &(Function &)> GetTLI,
-               function_ref<AnalysisResultsForFn(Function &)> getAnalysis);
-
-bool runFunctionSpecialization(
-    Module &M, const DataLayout &DL,
-    std::function<TargetLibraryInfo &(Function &)> GetTLI,
                std::function<TargetTransformInfo &(Function &)> GetTTI,
                std::function<AssumptionCache &(Function &)> GetAC,
-    function_ref<AnalysisResultsForFn(Function &)> GetAnalysis);
+               function_ref<AnalysisResultsForFn(Function &)> getAnalysis);
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_SCALAR_SCCP_H

llvm/include/llvm/Transforms/Utils/SCCPSolver.h

public:
 
   /// resolvedUndefsIn - While solving the dataflow for a function, we assume
   /// that branches on undef values cannot reach any of their successors.
   /// However, this is not a safe assumption.  After we solve dataflow, this
   /// method should be use to handle this.  If this returns true, the solver
   /// should be rerun.
   bool resolvedUndefsIn(Function &F);
 
+  void solveWhileResolvedUndefsIn(SmallVectorImpl<Function *> &WorkList);
+
   bool isBlockExecutable(BasicBlock *BB) const;
 
   // isEdgeFeasible - Return true if the control flow edge from the 'From' basic
   // block to the 'To' basic block is currently feasible.
   bool isEdgeFeasible(BasicBlock *From, BasicBlock *To) const;
 
   std::vector<ValueLatticeElement> getStructLatticeValueFor(Value *V) const;
 
public:
   void markArgInFuncSpecialization(Function *F,
                                    const SmallVectorImpl<ArgInfo> &Args);
 
   /// Mark all of the blocks in function \p F non-executable. Clients can used
   /// this method to erase a function from the module (e.g., if it has been
   /// completely specialized and is no longer needed).
   void markFunctionUnreachable(Function *F);
 
+  // Mark Users as changed, including I's AdditionalUsers.
+  void markUsersAsChanged(Value *I, const SmallVectorImpl<User *> &Users);
+
   void visit(Instruction *I);
   void visitCall(CallInst &I);
 };
 
 } // namespace llvm
 
 #endif // LLVM_TRANSFORMS_UTILS_SCCPSOLVER_H

llvm/lib/Passes/PassBuilderPipelines.cpp

PipelineTuningOptions::PipelineTuningOptions() {
   CallGraphProfile = true;
   MergeFunctions = EnableMergeFunctions;
   EagerlyInvalidateAnalyses = EnableEagerlyInvalidateAnalyses;
 }
 
 namespace llvm {
 extern cl::opt<unsigned> MaxDevirtIterations;
 extern cl::opt<bool> EnableConstraintElimination;
-extern cl::opt<bool> EnableFunctionSpecialization;
 extern cl::opt<bool> EnableGVNHoist;
 extern cl::opt<bool> EnableGVNSink;
 extern cl::opt<bool> EnableHotColdSplit;
 extern cl::opt<bool> EnableIROutliner;
 extern cl::opt<bool> EnableOrderFileInstrumentation;
 extern cl::opt<bool> EnableCHR;
 extern cl::opt<bool> EnableLoopInterchange;
 extern cl::opt<bool> EnableUnrollAndJam;
PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
   // post link pipeline after ICP. This is to enable usage of the type
   // tests in ICP sequences.
   if (Phase == ThinOrFullLTOPhase::ThinLTOPostLink)
     MPM.addPass(LowerTypeTestsPass(nullptr, nullptr, true));
 
   for (auto &C : PipelineEarlySimplificationEPCallbacks)
     C(MPM, Level);
 
-  // Specialize functions with IPSCCP.
-  if (EnableFunctionSpecialization && Level == OptimizationLevel::O3)
-    MPM.addPass(FunctionSpecializationPass());
-
   // Interprocedural constant propagation now that basic cleanup has occurred
   // and prior to optimizing globals.
   // FIXME: This position in the pipeline hasn't been carefully considered in
   // years, it should be re-analyzed.
   MPM.addPass(IPSCCPPass());
 
   // Attach metadata to indirect call sites indicating the set of functions
   // they may target at run-time. This should follow IPSCCP.
if (Level.getSpeedupLevel() > 1) {
 
     // Indirect call promotion. This should promote all the targets that are
     // left by the earlier promotion pass that promotes intra-module targets.
     // This two-step promotion is to save the compile time. For LTO, it should
     // produce the same result as if we only do promotion here.
     MPM.addPass(PGOIndirectCallPromotion(
         true /* InLTO */, PGOOpt && PGOOpt->Action == PGOOptions::SampleUse));
 
-    if (EnableFunctionSpecialization && Level == OptimizationLevel::O3)
-      MPM.addPass(FunctionSpecializationPass());
     // Propagate constants at call sites into the functions they call.  This
     // opens opportunities for globalopt (and inlining) by substituting function
     // pointers passed as arguments to direct uses of functions.
     MPM.addPass(IPSCCPPass());
 
     // Attach metadata to indirect call sites indicating the set of functions
     // they may target at run-time. This should follow IPSCCP.
     MPM.addPass(CalledValuePropagationPass());

llvm/lib/Passes/PassRegistry.def

 MODULE_PASS("cross-dso-cfi", CrossDSOCFIPass())
 MODULE_PASS("deadargelim", DeadArgumentEliminationPass())
 MODULE_PASS("debugify", NewPMDebugifyPass())
 MODULE_PASS("dot-callgraph", CallGraphDOTPrinterPass())
 MODULE_PASS("elim-avail-extern", EliminateAvailableExternallyPass())
 MODULE_PASS("extract-blocks", BlockExtractorPass())
 MODULE_PASS("forceattrs", ForceFunctionAttrsPass())
 MODULE_PASS("function-import", FunctionImportPass())
-MODULE_PASS("function-specialization", FunctionSpecializationPass())
 MODULE_PASS("globaldce", GlobalDCEPass())
 MODULE_PASS("globalopt", GlobalOptPass())
 MODULE_PASS("globalsplit", GlobalSplitPass())
 MODULE_PASS("hotcoldsplit", HotColdSplittingPass())
 MODULE_PASS("inferattrs", InferFunctionAttrsPass())
 MODULE_PASS("inliner-wrapper", ModuleInlinerWrapperPass())
 MODULE_PASS("inliner-ml-advisor-release", ModuleInlinerWrapperPass(getInlineParams(), true, {}, InliningAdvisorMode::Release, 0))
 MODULE_PASS("print<inline-advisor>", InlineAdvisorAnalysisPrinterPass(dbgs()))

llvm/lib/Transforms/IPO/FunctionSpecialization.cpp

 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/CodeMetrics.h"
 #include "llvm/Analysis/InlineCost.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Analysis/ValueLattice.h"
 #include "llvm/Analysis/ValueLatticeUtils.h"
 #include "llvm/IR/IntrinsicInst.h"
+#include "llvm/Transforms/IPO/FunctionSpecialization.h"
 #include "llvm/Transforms/Scalar/SCCP.h"
 #include "llvm/Transforms/Utils/Cloning.h"
 #include "llvm/Transforms/Utils/SCCPSolver.h"
 #include "llvm/Transforms/Utils/SizeOpts.h"
 #include <cmath>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "function-specialization"
 
 STATISTIC(NumFuncSpecialized, "Number of functions specialized");
 
 static cl::opt<bool> ForceFunctionSpecialization(
     "force-function-specialization", cl::init(false), cl::Hidden,
     cl::desc("Force function specialization for every call site with a "
              "constant argument"));
 
-static cl::opt<unsigned> FuncSpecializationMaxIters(
-    "func-specialization-max-iters", cl::Hidden,
-    cl::desc("The maximum number of iterations function specialization is run"),
-    cl::init(1));
-
 static cl::opt<unsigned> MaxClonesThreshold(
     "func-specialization-max-clones", cl::Hidden,
     cl::desc("The maximum number of clones allowed for a single function "
              "specialization"),
     cl::init(3));
 
 static cl::opt<unsigned> SmallFunctionThreshold(
     "func-specialization-size-threshold", cl::Hidden,

 //
 // https://llvm-compile-time-tracker.com
 // https://github.com/nikic/llvm-compile-time-tracker
 static cl::opt<bool> EnableSpecializationForLiteralConstant(
     "function-specialization-for-literal-constant", cl::init(false), cl::Hidden,
     cl::desc("Enable specialization of functions that take a literal constant "
              "as an argument."));
 
-namespace {
-// Bookkeeping struct to pass data from the analysis and profitability phase
-// to the actual transform helper functions.
-struct SpecializationInfo {
-  SmallVector<ArgInfo, 8> Args; // Stores the {formal,actual} argument pairs.
-  InstructionCost Gain;         // Profitability: Gain = Bonus - Cost.
-};
-} // Anonymous namespace
-
-using FuncList = SmallVectorImpl<Function *>;
-using CallArgBinding = std::pair<CallBase *, Constant *>;
-using CallSpecBinding = std::pair<CallBase *, SpecializationInfo>;
-// We are using MapVector because it guarantees deterministic iteration
-// order across executions.
-using SpecializationMap = SmallMapVector<CallBase *, SpecializationInfo, 8>;
-
-// Helper to check if \p LV is either a constant or a constant
-// range with a single element. This should cover exactly the same cases as the
-// old ValueLatticeElement::isConstant() and is intended to be used in the
-// transition to ValueLatticeElement.
-static bool isConstant(const ValueLatticeElement &LV) {
-  return LV.isConstant() ||
-         (LV.isConstantRange() && LV.getConstantRange().isSingleElement());
-}
-
-// Helper to check if \p LV is either overdefined or a constant int.
-static bool isOverdefined(const ValueLatticeElement &LV) {
-  return !LV.isUnknownOrUndef() && !isConstant(LV);
-}
-
 static Constant *getPromotableAlloca(AllocaInst *Alloca, CallInst *Call) {
   Value *StoreValue = nullptr;
   for (auto *User : Alloca->users()) {
     // We can't use llvm::isAllocaPromotable() as that would fail because of
     // the usage in the CallInst, which is what we check here.
     if (User == Call)
       continue;
     if (auto *Bitcast = dyn_cast<BitCastInst>(User)) {
for (auto *User : Alloca->users()) {
     return nullptr;
   }
   return dyn_cast_or_null<Constant>(StoreValue);
 }
 
 // A constant stack value is an AllocaInst that has a single constant
 // value stored to it. Return this constant if such an alloca stack value
 // is a function argument.
-static Constant *getConstantStackValue(CallInst *Call, Value *Val,
-                                       SCCPSolver &Solver) {
+static Constant *getConstantStackValue(CallInst *Call, Value *Val) {
   if (!Val)
     return nullptr;
   Val = Val->stripPointerCasts();
   if (auto *ConstVal = dyn_cast<ConstantInt>(Val))
     return ConstVal;
   auto *Alloca = dyn_cast<AllocaInst>(Val);
   if (!Alloca || !Alloca->getAllocatedType()->isIntegerTy())
     return nullptr;

 //
 //     @funcspec.arg = internal constant i32 2
 //
 //     define internal void @someFunc(i32* arg1) {
 //       call void @otherFunc(i32* nonnull @funcspec.arg)
 //       ret void
 //     }
 //
-static void constantArgPropagation(FuncList &WorkList, Module &M,
-                                   SCCPSolver &Solver) {
+void FunctionSpecializer::promoteConstantStackValues() {
   // Iterate over the argument tracked functions see if there
   // are any new constant values for the call instruction via
   // stack variables.
-  for (auto *F : WorkList) {
+  for (Function &F : M) {
+    if (!Solver.isArgumentTrackedFunction(&F))
+      continue;
 
-    for (auto *User : F->users()) {
+    for (auto *User : F.users()) {
 
       auto *Call = dyn_cast<CallInst>(User);
       if (!Call)
         continue;
 
       bool Changed = false;
       for (const Use &U : Call->args()) {
         unsigned Idx = Call->getArgOperandNo(&U);
         Value *ArgOp = Call->getArgOperand(Idx);
         Type *ArgOpType = ArgOp->getType();
 
         if (!Call->onlyReadsMemory(Idx) || !ArgOpType->isPointerTy())
           continue;
 
-        auto *ConstVal = getConstantStackValue(Call, ArgOp, Solver);
+        auto *ConstVal = getConstantStackValue(Call, ArgOp);
         if (!ConstVal)
           continue;
 
         Value *GV = new GlobalVariable(M, ConstVal->getType(), true,
                                        GlobalValue::InternalLinkage, ConstVal,
                                        "funcspec.arg");
         if (ArgOpType != ConstVal->getType())
           GV = ConstantExpr::getBitCast(cast<Constant>(GV), ArgOpType);
 
         Call->setArgOperand(Idx, GV);
         Changed = true;
       }
 
       // Add the changed CallInst to Solver Worklist
       if (Changed)
         Solver.visitCall(*Call);
     }
   }
 }
 
 // ssa_copy intrinsics are introduced by the SCCP solver. These intrinsics
-// interfere with the constantArgPropagation optimization.
+// interfere with the promoteConstantStackValues() optimization.
 static void removeSSACopy(Function &F) {
   for (BasicBlock &BB : F) {
     for (Instruction &Inst : llvm::make_early_inc_range(BB)) {
       auto *II = dyn_cast<IntrinsicInst>(&Inst);
       if (!II)
         continue;
       if (II->getIntrinsicID() != Intrinsic::ssa_copy)
         continue;
       Inst.replaceAllUsesWith(II->getOperand(0));
       Inst.eraseFromParent();
     }
   }
 }
 
-static void removeSSACopy(Module &M) {
-  for (Function &F : M)
-    removeSSACopy(F);
-}
-
-namespace {
-class FunctionSpecializer {
-
-  /// The IPSCCP Solver.
-  SCCPSolver &Solver;
-
-  /// Analyses used to help determine if a function should be specialized.
-  std::function<AssumptionCache &(Function &)> GetAC;
-  std::function<TargetTransformInfo &(Function &)> GetTTI;
-  std::function<TargetLibraryInfo &(Function &)> GetTLI;
-
-  SmallPtrSet<Function *, 4> SpecializedFuncs;
-  SmallPtrSet<Function *, 4> FullySpecialized;
-  SmallVector<Instruction *> ReplacedWithConstant;
-  DenseMap<Function *, CodeMetrics> FunctionMetrics;
-
-public:
-  FunctionSpecializer(SCCPSolver &Solver,
-                      std::function<AssumptionCache &(Function &)> GetAC,
-                      std::function<TargetTransformInfo &(Function &)> GetTTI,
-                      std::function<TargetLibraryInfo &(Function &)> GetTLI)
-      : Solver(Solver), GetAC(GetAC), GetTTI(GetTTI), GetTLI(GetTLI) {}
-
-  ~FunctionSpecializer() {
-    // Eliminate dead code.
-    removeDeadInstructions();
-    removeDeadFunctions();
-  }
+/// Remove any ssa_copy intrinsics that may have been introduced.
+void FunctionSpecializer::cleanUpSSA() {
+  for (Function *F : SpecializedFuncs)
+    removeSSACopy(*F);
 
-  /// Attempt to specialize functions in the module to enable constant
-  /// propagation across function boundaries.
-  ///
-  /// \returns true if at least one function is specialized.
-  bool specializeFunctions(FuncList &Candidates, FuncList &WorkList) {
-    bool Changed = false;
-    for (auto *F : Candidates) {
-      if (!isCandidateFunction(F))
-        continue;
+bool FunctionSpecializer::specialize(SmallVectorImpl<Function *> &WorkList) {
+  for (Function &F : M) {
+    if (!isCandidateFunction(&F))
 
-      auto Cost = getSpecializationCost(F);
-      if (!Cost.isValid()) {
-        LLVM_DEBUG(
-            dbgs() << "FnSpecialization: Invalid specialization cost.\n");
-        continue;
-      }
+    auto Cost = getSpecializationCost(&F);
 
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Specialization cost for "
-                        << F->getName() << " is " << Cost << "\n");
+                      << F.getName() << " is " << Cost << "\n");
 
-      SmallVector<CallSpecBinding, 8> Specializations;
-      if (!findSpecializations(F, Cost, Specializations)) {
-        LLVM_DEBUG(
-            dbgs() << "FnSpecialization: No possible specializations found\n");
-        continue;
-      }
+    if (!findSpecializations(&F, Cost, Specializations)) {
 
-      Changed = true;
-      for (auto &Entry : Specializations)
-        specializeFunction(F, Entry.second, WorkList);
-    }
+    SmallVector<Function *, 4> Clones;
+      Clones.push_back(createSpecialization(&F, Entry.second));
+
+    Solver.solveWhileResolvedUndefsIn(Clones);
+    WorkList.append(Clones);
 
-    updateSpecializedFuncs(Candidates, WorkList);
-    NumFuncSpecialized += NbFunctionsSpecialized;
-    return Changed;
-  }
+  LLVM_DEBUG(if (NbFunctionsSpecialized)
+               dbgs() << "FnSpecialization: Specialized "
+                      << NbFunctionsSpecialized << " functions in module "
+                      << M.getName() << "\n");
+
+  return !WorkList.empty();
 
-  void removeDeadInstructions() {
-    for (auto *I : ReplacedWithConstant) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Removing dead instruction " << *I
-                        << "\n");
-      I->eraseFromParent();
-    }
-    ReplacedWithConstant.clear();
-  }
 
-  void removeDeadFunctions() {
-    for (auto *F : FullySpecialized) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Removing dead function "
-                        << F->getName() << "\n");
-      F->eraseFromParent();
-    }
-    FullySpecialized.clear();
-  }
+void FunctionSpecializer::removeDeadFunctions() {
+  for (Function *F : FullySpecialized) {
 
-  bool tryToReplaceWithConstant(Value *V) {
-    if (!V->getType()->isSingleValueType() || isa<CallBase>(V) ||
-        V->user_empty())
-      return false;
-
-    const ValueLatticeElement &IV = Solver.getLatticeValueFor(V);
-    if (isOverdefined(IV))
-      return false;
-    auto *Const =
-        isConstant(IV) ? Solver.getConstant(IV) : UndefValue::get(V->getType());
-
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Replacing " << *V
-                      << "\nFnSpecialization: with " << *Const << "\n");
-
-    // Record uses of V to avoid visiting irrelevant uses of const later.
-    SmallVector<Instruction *> UseInsts;
-    for (auto *U : V->users())
-      if (auto *I = dyn_cast<Instruction>(U))
-        if (Solver.isBlockExecutable(I->getParent()))
-          UseInsts.push_back(I);
-
-    V->replaceAllUsesWith(Const);
-
-    for (auto *I : UseInsts)
-      Solver.visit(I);
-
-    // Remove the instruction from Block and Solver.
-    if (auto *I = dyn_cast<Instruction>(V)) {
-      if (I->isSafeToRemove()) {
-        ReplacedWithConstant.push_back(I);
-        Solver.removeLatticeValueFor(I);
-      }
-    }
-    return true;
-  }
-
-private:
-  // The number of functions specialised, used for collecting statistics and
-  // also in the cost model.
-  unsigned NbFunctionsSpecialized = 0;
-
-  // Compute the code metrics for function \p F.
-  CodeMetrics &analyzeFunction(Function *F) {
-    auto I = FunctionMetrics.insert({F, CodeMetrics()});
-    CodeMetrics &Metrics = I.first->second;
-    if (I.second) {
-      // The code metrics were not cached.
-      SmallPtrSet<const Value *, 32> EphValues;
-      CodeMetrics::collectEphemeralValues(F, &(GetAC)(*F), EphValues);
-      for (BasicBlock &BB : *F)
-        Metrics.analyzeBasicBlock(&BB, (GetTTI)(*F), EphValues);
+CodeMetrics &FunctionSpecializer::analyzeFunction(Function *F) {
 
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Code size of function "
-                        << F->getName() << " is " << Metrics.NumInsts
-                        << " instructions\n");
-    }
-    return Metrics;
-  }
 
-  /// Clone the function \p F and remove the ssa_copy intrinsics added by
-  /// the SCCPSolver in the cloned version.
-  Function *cloneCandidateFunction(Function *F, ValueToValueMapTy &Mappings) {
-    Function *Clone = CloneFunction(F, Mappings);
-    removeSSACopy(*Clone);
-    return Clone;
-  }
+static Function *cloneCandidateFunction(Function *F,
+                                        ValueToValueMapTy &Mappings) {
 
-  /// This function decides whether it's worthwhile to specialize function
-  /// \p F based on the known constant values its arguments can take on. It
-  /// only discovers potential specialization opportunities without actually
-  /// applying them.
-  ///
-  /// \returns true if any specializations have been found.
-  bool findSpecializations(Function *F, InstructionCost Cost,
+bool FunctionSpecializer::findSpecializations(Function *F, InstructionCost Cost,
                               SmallVectorImpl<CallSpecBinding> &WorkList) {
-    // Get a list of interesting arguments.
-    SmallVector<Argument *, 4> Args;
-    for (Argument &Arg : F->args())
-      if (isArgumentInteresting(&Arg))
-        Args.push_back(&Arg);
 
-    if (!Args.size())
-      return false;
 
-    // Find all the call sites for the function.
-    SpecializationMap Specializations;
-    for (User *U : F->users()) {
-      if (!isa<CallInst>(U) && !isa<InvokeInst>(U))
-        continue;
-      auto &CS = *cast<CallBase>(U);
-      // If the call site has attribute minsize set, that callsite won't be
-      // specialized.
-      if (CS.hasFnAttr(Attribute::MinSize))
-        continue;
 
-      // If the parent of the call site will never be executed, we don't need
-      // to worry about the passed value.
-      if (!Solver.isBlockExecutable(CS.getParent()))
-        continue;
 
-      // Examine arguments and create specialization candidates from call sites
-      // with constant arguments.
-      bool Added = false;
-      for (Argument *A : Args) {
-        Constant *C = getCandidateConstant(CS.getArgOperand(A->getArgNo()));
-        if (!C)
-          continue;
 
-        if (!Added) {
-          Specializations[&CS] = {{}, 0 - Cost};
-          Added = true;
-        }
 
-        SpecializationInfo &S = Specializations.back().second;
-        S.Gain += getSpecializationBonus(A, C, Solver.getLoopInfo(*F));
-        S.Args.push_back({A, C});
-      }
-      Added = false;
-    }
 
-    // Remove unprofitable specializations.
-    if (!ForceFunctionSpecialization)
-      Specializations.remove_if(
-          [](const auto &Entry) { return Entry.second.Gain <= 0; });
 
-    // Clear the MapVector and return the underlying vector.
-    WorkList = Specializations.takeVector();
 
-    // Sort the candidates in descending order.
-    llvm::stable_sort(WorkList, [](const auto &L, const auto &R) {
-      return L.second.Gain > R.second.Gain;
-    });
 
-    // Truncate the worklist to 'MaxClonesThreshold' candidates if necessary.
-    if (WorkList.size() > MaxClonesThreshold) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Number of candidates exceed "
-                        << "the maximum number of clones threshold.\n"
-                        << "FnSpecialization: Truncating worklist to "
-                        << MaxClonesThreshold << " candidates.\n");
-      WorkList.erase(WorkList.begin() + MaxClonesThreshold, WorkList.end());
-    }
 
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Specializations for function "
-                      << F->getName() << "\n";
-               for (const auto &Entry
-                    : WorkList) {
-                 dbgs() << "FnSpecialization:   Gain = " << Entry.second.Gain
-                        << "\n";
-                 for (const ArgInfo &Arg : Entry.second.Args)
-                   dbgs() << "FnSpecialization:   FormalArg = "
-                          << Arg.Formal->getNameOrAsOperand()
-                          << ", ActualArg = "
-                          << Arg.Actual->getNameOrAsOperand() << "\n";
-               });
 
-    return !WorkList.empty();
-  }
 
-  bool isCandidateFunction(Function *F) {
-    // Do not specialize the cloned function again.
-    if (SpecializedFuncs.contains(F))
-      return false;
+bool FunctionSpecializer::isCandidateFunction(Function *F) {
+  if (F->isDeclaration())
+    return false;
+
+  if (F->hasFnAttribute(Attribute::NoDuplicate))
+    return false;
+
+  if (!Solver.isArgumentTrackedFunction(F))
+    return false;
+
 
-    // If we're optimizing the function for size, we shouldn't specialize it.
-    if (F->hasOptSize() ||
-        shouldOptimizeForSize(F, nullptr, nullptr, PGSOQueryType::IRPass))
-      return false;
 
-    // Exit if the function is not executable. There's no point in specializing
-    // a dead function.
-    if (!Solver.isBlockExecutable(&F->getEntryBlock()))
-      return false;
 
-    // It wastes time to specialize a function which would get inlined finally.
-    if (F->hasFnAttribute(Attribute::AlwaysInline))
-      return false;
 
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Try function: " << F->getName()
-                      << "\n");
-    return true;
-  }
 
-  void specializeFunction(Function *F, SpecializationInfo &S,
-                          FuncList &WorkList) {
-    ValueToValueMapTy Mappings;
-    Function *Clone = cloneCandidateFunction(F, Mappings);
+Function *FunctionSpecializer::createSpecialization(Function *F,
+                                                    SpecializationInfo &S) {
 
-    // Rewrite calls to the function so that they call the clone instead.
-    rewriteCallSites(Clone, S.Args, Mappings);
 
-    // Initialize the lattice state of the arguments of the function clone,
-    // marking the argument on which we specialized the function constant
-    // with the given value.
-    Solver.markArgInFuncSpecialization(Clone, S.Args);
 
-    // Mark all the specialized functions
-    WorkList.push_back(Clone);
-    NbFunctionsSpecialized++;
+  Solver.addArgumentTrackedFunction(Clone);
+  Solver.markBlockExecutable(&Clone->front());
+
+  SpecializedFuncs.insert(Clone);
 
-    // If the function has been completely specialized, the original function
-    // is no longer needed. Mark it unreachable.
-    if (F->getNumUses() == 0 || all_of(F->users(), [F](User *U) {
-          if (auto *CS = dyn_cast<CallBase>(U))
-            return CS->getFunction() == F;
-          return false;
-        })) {
-      Solver.markFunctionUnreachable(F);
-      FullySpecialized.insert(F);
-    }
-  }
+  return Clone;
 
-  /// Compute and return the cost of specializing function \p F.
-  InstructionCost getSpecializationCost(Function *F) {
-    CodeMetrics &Metrics = analyzeFunction(F);
-    // If the code metrics reveal that we shouldn't duplicate the function, we
-    // shouldn't specialize it. Set the specialization cost to Invalid.
-    // Or if the lines of codes implies that this function is easy to get
-    // inlined so that we shouldn't specialize it.
-    if (Metrics.notDuplicatable || !Metrics.NumInsts.isValid() ||
-        (!ForceFunctionSpecialization &&
-         !F->hasFnAttribute(Attribute::NoInline) &&
-         Metrics.NumInsts < SmallFunctionThreshold))
-      return InstructionCost::getInvalid();
+InstructionCost FunctionSpecializer::getSpecializationCost(Function *F) {
 
-    // Otherwise, set the specialization cost to be the cost of all the
-    // instructions in the function and penalty for specializing more functions.
-    unsigned Penalty = NbFunctionsSpecialized + 1;
-    return Metrics.NumInsts * InlineConstants::getInstrCost() * Penalty;
-  }
 
-  InstructionCost getUserBonus(User *U, llvm::TargetTransformInfo &TTI,
+static InstructionCost getUserBonus(User *U, llvm::TargetTransformInfo &TTI,
                                     const LoopInfo &LI) {
-    auto *I = dyn_cast_or_null<Instruction>(U);
-    // If not an instruction we do not know how to evaluate.
-    // Keep minimum possible cost for now so that it doesnt affect
-    // specialization.
-    if (!I)
-      return std::numeric_limits<unsigned>::min();
 
-    InstructionCost Cost =
-        TTI.getInstructionCost(U, TargetTransformInfo::TCK_SizeAndLatency);
 
-    // Increase the cost if it is inside the loop.
-    unsigned LoopDepth = LI.getLoopDepth(I->getParent());
-    Cost *= std::pow((double)AvgLoopIterationCount, LoopDepth);
 
-    // Traverse recursively if there are more uses.
-    // TODO: Any other instructions to be added here?
-    if (I->mayReadFromMemory() || I->isCast())
-      for (auto *User : I->users())
-        Cost += getUserBonus(User, TTI, LI);
 
-    return Cost;
-  }
 
-  /// Compute a bonus for replacing argument \p A with constant \p C.
-  InstructionCost getSpecializationBonus(Argument *A, Constant *C,
-                                         const LoopInfo &LI) {
-    Function *F = A->getParent();
-    auto &TTI = (GetTTI)(*F);
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Analysing bonus for constant: "
-                      << C->getNameOrAsOperand() << "\n");
+InstructionCost FunctionSpecializer::getSpecializationBonus(Argument *A,
+                                         Constant *C, const LoopInfo &LI) {
 
-    InstructionCost TotalCost = 0;
-    for (auto *U : A->users()) {
-      TotalCost += getUserBonus(U, TTI, LI);
-      LLVM_DEBUG(dbgs() << "FnSpecialization:   User cost ";
-                 TotalCost.print(dbgs()); dbgs() << " for: " << *U << "\n");
-    }
 
-    // The below heuristic is only concerned with exposing inlining
-    // opportunities via indirect call promotion. If the argument is not a
-    // (potentially casted) function pointer, give up.
-    Function *CalledFunction = dyn_cast<Function>(C->stripPointerCasts());
-    if (!CalledFunction)
-      return TotalCost;
 
-    // Get TTI for the called function (used for the inline cost).
-    auto &CalleeTTI = (GetTTI)(*CalledFunction);
 
-    // Look at all the call sites whose called value is the argument.
-    // Specializing the function on the argument would allow these indirect
-    // calls to be promoted to direct calls. If the indirect call promotion
-    // would likely enable the called function to be inlined, specializing is a
-    // good idea.
-    int Bonus = 0;
-    for (User *U : A->users()) {
-      if (!isa<CallInst>(U) && !isa<InvokeInst>(U))
-        continue;
-      auto *CS = cast<CallBase>(U);
-      if (CS->getCalledOperand() != A)
-        continue;
 
-      // Get the cost of inlining the called function at this call site. Note
-      // that this is only an estimate. The called function may eventually
-      // change in a way that leads to it not being inlined here, even though
-      // inlining looks profitable now. For example, one of its called
-      // functions may be inlined into it, making the called function too large
-      // to be inlined into this call site.
-      //
-      // We apply a boost for performing indirect call promotion by increasing
-      // the default threshold by the threshold for indirect calls.
-      auto Params = getInlineParams();
-      Params.DefaultThreshold += InlineConstants::IndirectCallThreshold;
-      InlineCost IC =
-          getInlineCost(*CS, CalledFunction, Params, CalleeTTI, GetAC, GetTLI);
 
-      // We clamp the bonus for this call to be between zero and the default
-      // threshold.
-      if (IC.isAlways())
-        Bonus += Params.DefaultThreshold;
-      else if (IC.isVariable() && IC.getCostDelta() > 0)
-        Bonus += IC.getCostDelta();
 
-      LLVM_DEBUG(dbgs() << "FnSpecialization:   Inlining bonus " << Bonus
-                        << " for user " << *U << "\n");
-    }
 
-    return TotalCost + Bonus;
-  }
 
-  /// Determine if it is possible to specialise the function for constant values
-  /// of the formal parameter \p A.
-  bool isArgumentInteresting(Argument *A) {
-    // No point in specialization if the argument is unused.
-    if (A->user_empty())
-      return false;
+bool FunctionSpecializer::isArgumentInteresting(Argument *A) {
 
-    // For now, don't attempt to specialize functions based on the values of
-    // composite types.
-    Type *ArgTy = A->getType();
-    if (!ArgTy->isSingleValueType())
-      return false;
 
-    // Specialization of integer and floating point types needs to be explicitly
-    // enabled.
-    if (!EnableSpecializationForLiteralConstant &&
-        (ArgTy->isIntegerTy() || ArgTy->isFloatingPointTy()))
-      return false;
 
-    // SCCP solver does not record an argument that will be constructed on
-    // stack.
-    if (A->hasByValAttr() && !A->getParent()->onlyReadsMemory())
-      return false;
 
-    // Check the lattice value and decide if we should attemt to specialize,
-    // based on this argument. No point in specialization, if the lattice value
-    // is already a constant.
-    const ValueLatticeElement &LV = Solver.getLatticeValueFor(A);
-    if (LV.isUnknownOrUndef() || LV.isConstant() ||
-        (LV.isConstantRange() && LV.getConstantRange().isSingleElement())) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Nothing to do, argument "
-                        << A->getNameOrAsOperand() << " is already constant\n");
-      return false;
-    }
 
-    return true;
-  }
 
-  /// Check if the valuy \p V  (an actual argument) is a constant or can only
-  /// have a constant value. Return that constant.
-  Constant *getCandidateConstant(Value *V) {
-    if (isa<PoisonValue>(V))
-      return nullptr;
+Constant *FunctionSpecializer::getCandidateConstant(Value *V) {
 
-    // TrackValueOfGlobalVariable only tracks scalar global variables.
-    if (auto *GV = dyn_cast<GlobalVariable>(V)) {
-      // Check if we want to specialize on the address of non-constant
-      // global values.
-      if (!GV->isConstant() && !SpecializeOnAddresses)
-        return nullptr;
 
-      if (!GV->getValueType()->isSingleValueType())
-        return nullptr;
-    }
 
-    // Select for possible specialisation values that are constants or
-    // are deduced to be constants or constant ranges with a single element.
-    Constant *C = dyn_cast<Constant>(V);
-    if (!C) {
-      const ValueLatticeElement &LV = Solver.getLatticeValueFor(V);
-      if (LV.isConstant())
-        C = LV.getConstant();
-      else if (LV.isConstantRange() &&
-               LV.getConstantRange().isSingleElement()) {
-        assert(V->getType()->isIntegerTy() && "Non-integral constant range");
-        C = Constant::getIntegerValue(
-            V->getType(), *LV.getConstantRange().getSingleElement());
-      } else
-        return nullptr;
-    }
 
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Found interesting argument "
-                      << V->getNameOrAsOperand() << "\n");
 
-    return C;
-  }
 
-  /// Rewrite calls to function \p F to call function \p Clone instead.
-  ///
-  /// This function modifies calls to function \p F as long as the actual
-  /// arguments match those in \p Args. Note that for recursive calls we
-  /// need to compare against the cloned formal arguments.
-  ///
-  /// Callsites that have been marked with the MinSize function attribute won't
-  /// be specialized and rewritten.
-  void rewriteCallSites(Function *Clone, const SmallVectorImpl<ArgInfo> &Args,
+void FunctionSpecializer::rewriteCallSites(Function *Clone,
+                                           const SmallVectorImpl<ArgInfo> &Args,
                                            ValueToValueMapTy &Mappings) {
-    assert(!Args.empty() && "Specialization without arguments");
-    Function *F = Args[0].Formal->getParent();
 
-    SmallVector<CallBase *, 8> CallSitesToRewrite;
-    for (auto *U : F->users()) {
-      if (!isa<CallInst>(U) && !isa<InvokeInst>(U))
-        continue;
-      auto &CS = *cast<CallBase>(U);
-      if (!CS.getCalledFunction() || CS.getCalledFunction() != F)
-        continue;
-      CallSitesToRewrite.push_back(&CS);
-    }
 
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Replacing call sites of "
-                      << F->getName() << " with " << Clone->getName() << "\n");
 
-    for (auto *CS : CallSitesToRewrite) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization:   "
-                        << CS->getFunction()->getName() << " ->" << *CS
-                        << "\n");
-      if (/* recursive call */
-          (CS->getFunction() == Clone &&
-           all_of(Args,
-                  [CS, &Mappings](const ArgInfo &Arg) {
-                    unsigned ArgNo = Arg.Formal->getArgNo();
-                    return CS->getArgOperand(ArgNo) == Mappings[Arg.Formal];
-                  })) ||
-          /* normal call */
-          all_of(Args, [CS](const ArgInfo &Arg) {
-            unsigned ArgNo = Arg.Formal->getArgNo();
-            return CS->getArgOperand(ArgNo) == Arg.Actual;
-          })) {
-        CS->setCalledFunction(Clone);
-        Solver.markOverdefined(CS);
-      }
-    }
-  }
 
-  void updateSpecializedFuncs(FuncList &Candidates, FuncList &WorkList) {
-    for (auto *F : WorkList) {
-      SpecializedFuncs.insert(F);
-
-      // Initialize the state of the newly created functions, marking them
-      // argument-tracked and executable.
-      if (F->hasExactDefinition() && !F->hasFnAttribute(Attribute::Naked))
-        Solver.addTrackedFunction(F);
-
-      Solver.addArgumentTrackedFunction(F);
-      Candidates.push_back(F);
-      Solver.markBlockExecutable(&F->front());
-
-      // Replace the function arguments for the specialized functions.
-      for (Argument &Arg : F->args())
-        if (!Arg.use_empty() && tryToReplaceWithConstant(&Arg))
-          LLVM_DEBUG(dbgs() << "FnSpecialization: Replaced constant argument: "
-                            << Arg.getNameOrAsOperand() << "\n");
-    }
-  }
-};
-} // namespace
-
-bool llvm::runFunctionSpecialization(
-    Module &M, const DataLayout &DL,
-    std::function<TargetLibraryInfo &(Function &)> GetTLI,
-    std::function<TargetTransformInfo &(Function &)> GetTTI,
-    std::function<AssumptionCache &(Function &)> GetAC,
-    function_ref<AnalysisResultsForFn(Function &)> GetAnalysis) {
-  SCCPSolver Solver(DL, GetTLI, M.getContext());
-  FunctionSpecializer FS(Solver, GetAC, GetTTI, GetTLI);
-  bool Changed = false;
-
-  // Loop over all functions, marking arguments to those with their addresses
-  // taken or that are external as overdefined.
-  for (Function &F : M) {
-    if (F.isDeclaration())
-      continue;
-    if (F.hasFnAttribute(Attribute::NoDuplicate))
-      continue;
-
-    LLVM_DEBUG(dbgs() << "\nFnSpecialization: Analysing decl: " << F.getName()
-                      << "\n");
-    Solver.addAnalysis(F, GetAnalysis(F));
-
-    // Determine if we can track the function's arguments. If so, add the
-    // function to the solver's set of argument-tracked functions.
-    if (canTrackArgumentsInterprocedurally(&F)) {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Can track arguments\n");
-      Solver.addArgumentTrackedFunction(&F);
-      continue;
-    } else {
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Can't track arguments!\n"
-                        << "FnSpecialization: Doesn't have local linkage, or "
-                        << "has its address taken\n");
-    }
-
-    // Assume the function is called.
-    Solver.markBlockExecutable(&F.front());
-
-    // Assume nothing about the incoming arguments.
-    for (Argument &AI : F.args())
-      Solver.markOverdefined(&AI);
-  }
-
-  // Determine if we can track any of the module's global variables. If so, add
-  // the global variables we can track to the solver's set of tracked global
-  // variables.
-  for (GlobalVariable &G : M.globals()) {
-    G.removeDeadConstantUsers();
-    if (canTrackGlobalVariableInterprocedurally(&G))
-      Solver.trackValueOfGlobalVariable(&G);
-  }
-
-  auto &TrackedFuncs = Solver.getArgumentTrackedFunctions();
-  SmallVector<Function *, 16> FuncDecls(TrackedFuncs.begin(),
-                                        TrackedFuncs.end());
-
-  // No tracked functions, so nothing to do: don't run the solver and remove
-  // the ssa_copy intrinsics that may have been introduced.
-  if (TrackedFuncs.empty()) {
-    removeSSACopy(M);
-    return false;
-  }
-
-  // Solve for constants.
-  auto RunSCCPSolver = [&](auto &WorkList) {
-    bool ResolvedUndefs = true;
-
-    while (ResolvedUndefs) {
-      // Not running the solver unnecessary is checked in regression test
-      // nothing-to-do.ll, so if this debug message is changed, this regression
-      // test needs updating too.
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Running solver\n");
-
-      Solver.solve();
-      LLVM_DEBUG(dbgs() << "FnSpecialization: Resolving undefs\n");
-      ResolvedUndefs = false;
-      for (Function *F : WorkList)
-        if (Solver.resolvedUndefsIn(*F))
-          ResolvedUndefs = true;
-    }
-
-    for (auto *F : WorkList) {
-      for (BasicBlock &BB : *F) {
-        if (!Solver.isBlockExecutable(&BB))
-          continue;
-        // FIXME: The solver may make changes to the function here, so set
-        // Changed, even if later function specialization does not trigger.
-        for (auto &I : make_early_inc_range(BB))
-          Changed |= FS.tryToReplaceWithConstant(&I);
-      }
-    }
-  };
-
-#ifndef NDEBUG
-  LLVM_DEBUG(dbgs() << "FnSpecialization: Worklist fn decls:\n");
-  for (auto *F : FuncDecls)
-    LLVM_DEBUG(dbgs() << "FnSpecialization: *) " << F->getName() << "\n");
-#endif
-
-  // Initially resolve the constants in all the argument tracked functions.
-  RunSCCPSolver(FuncDecls);
-
-  SmallVector<Function *, 8> WorkList;
-  unsigned I = 0;
-  while (FuncSpecializationMaxIters != I++ &&
-         FS.specializeFunctions(FuncDecls, WorkList)) {
-    LLVM_DEBUG(dbgs() << "FnSpecialization: Finished iteration " << I << "\n");
-
-    // Run the solver for the specialized functions.
-    RunSCCPSolver(WorkList);
-
-    // Replace some unresolved constant arguments.
-    constantArgPropagation(FuncDecls, M, Solver);
-
-    WorkList.clear();
-    Changed = true;
-  }
-
-  LLVM_DEBUG(dbgs() << "FnSpecialization: Number of specializations = "
-                    << NumFuncSpecialized << "\n");
-
-  // Remove any ssa_copy intrinsics that may have been introduced.
-  removeSSACopy(M);
-  return Changed;
-}

llvm/lib/Transforms/IPO/IPO.cpp

void llvm::initializeIPO(PassRegistry &Registry) {
   initializeOpenMPOptCGSCCLegacyPassPass(Registry);
   initializeAnnotation2MetadataLegacyPass(Registry);
   initializeCalledValuePropagationLegacyPassPass(Registry);
   initializeConstantMergeLegacyPassPass(Registry);
   initializeCrossDSOCFIPass(Registry);
   initializeDAEPass(Registry);
   initializeDAHPass(Registry);
   initializeForceFunctionAttrsLegacyPassPass(Registry);
-  initializeFunctionSpecializationLegacyPassPass(Registry);
   initializeGlobalDCELegacyPassPass(Registry);
   initializeGlobalOptLegacyPassPass(Registry);
   initializeGlobalSplitPass(Registry);
   initializeHotColdSplittingLegacyPassPass(Registry);
   initializeIROutlinerLegacyPassPass(Registry);
   initializeAlwaysInlinerLegacyPassPass(Registry);
   initializeSimpleInlinerPass(Registry);
   initializeInferFunctionAttrsLegacyPassPass(Registry);

llvm/lib/Transforms/IPO/PassManagerBuilder.cpp

cl::opt<bool> EnableMatrix(
     "enable-matrix", cl::init(false), cl::Hidden,
     cl::desc("Enable lowering of the matrix intrinsics"));
 
 cl::opt<bool> EnableConstraintElimination(
     "enable-constraint-elimination", cl::init(false), cl::Hidden,
     cl::desc(
         "Enable pass to eliminate conditions based on linear constraints."));
 
-cl::opt<bool> EnableFunctionSpecialization(
-    "enable-function-specialization", cl::init(false), cl::Hidden,
-    cl::desc("Enable Function Specialization pass"));
-
 cl::opt<AttributorRunOption> AttributorRun(
     "attributor-enable", cl::Hidden, cl::init(AttributorRunOption::NONE),
     cl::desc("Enable the attributor inter-procedural deduction pass."),
     cl::values(clEnumValN(AttributorRunOption::ALL, "all",
                           "enable all attributor runs"),
                clEnumValN(AttributorRunOption::MODULE, "module",
                           "enable module-wide attributor runs"),
                clEnumValN(AttributorRunOption::CGSCC, "cgscc",
void PassManagerBuilder::populateModulePassManager(
   if (AttributorRun & AttributorRunOption::MODULE)
     MPM.add(createAttributorLegacyPass());
 
   addExtensionsToPM(EP_ModuleOptimizerEarly, MPM);
 
   if (OptLevel > 2)
     MPM.add(createCallSiteSplittingPass());
 
-  // Propage constant function arguments by specializing the functions.
-  if (OptLevel > 2 && EnableFunctionSpecialization)
-    MPM.add(createFunctionSpecializationPass());
-
   MPM.add(createIPSCCPPass());          // IP SCCP
   MPM.add(createCalledValuePropagationPass());
 
   MPM.add(createGlobalOptimizerPass()); // Optimize out global vars
   // Promote any localized global vars.
   MPM.add(createPromoteMemoryToRegisterPass());
 
   MPM.add(createDeadArgEliminationPass()); // Dead argument elimination

llvm/lib/Transforms/IPO/SCCP.cpp

 
 #include "llvm/Transforms/IPO/SCCP.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/PostDominators.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/InitializePasses.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/Scalar/SCCP.h"
 #include "llvm/Transforms/Utils/SCCPSolver.h"
 
 using namespace llvm;
 
+extern cl::opt<bool> SpecializeFunctions;
+
 PreservedAnalyses IPSCCPPass::run(Module &M, ModuleAnalysisManager &AM) {
   const DataLayout &DL = M.getDataLayout();
   auto &FAM = AM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
   auto GetTLI = [&FAM](Function &F) -> const TargetLibraryInfo & {
     return FAM.getResult<TargetLibraryAnalysis>(F);
   };
+  auto GetTTI = [&FAM](Function &F) -> TargetTransformInfo & {
+    return FAM.getResult<TargetIRAnalysis>(F);
+  };
+  auto GetAC = [&FAM](Function &F) -> AssumptionCache & {
+    return FAM.getResult<AssumptionAnalysis>(F);
+  };
   auto getAnalysis = [&FAM](Function &F) -> AnalysisResultsForFn {
     DominatorTree &DT = FAM.getResult<DominatorTreeAnalysis>(F);
     return {
         std::make_unique<PredicateInfo>(F, DT, FAM.getResult<AssumptionAnalysis>(F)),
         &DT, FAM.getCachedResult<PostDominatorTreeAnalysis>(F),
-        nullptr};
+        SpecializeFunctions ? &FAM.getResult<LoopAnalysis>(F) : nullptr };
   };
 
-  if (!runIPSCCP(M, DL, GetTLI, getAnalysis))
+  if (!runIPSCCP(M, DL, GetTLI, GetTTI, GetAC, getAnalysis))
     return PreservedAnalyses::all();
 
   PreservedAnalyses PA;
   PA.preserve<DominatorTreeAnalysis>();
   PA.preserve<PostDominatorTreeAnalysis>();
   PA.preserve<FunctionAnalysisManagerModuleProxy>();
   return PA;
 }
 
 namespace {
 
 //===--------------------------------------------------------------------===//
public:
 
   bool runOnModule(Module &M) override {
     if (skipModule(M))
       return false;
     const DataLayout &DL = M.getDataLayout();
     auto GetTLI = [this](Function &F) -> const TargetLibraryInfo & {
       return this->getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(F);
     };
+    auto GetTTI = [this](Function &F) -> TargetTransformInfo & {
+      return this->getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);
+    };
+    auto GetAC = [this](Function &F) -> AssumptionCache & {
+      return this->getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
+    };
     auto getAnalysis = [this](Function &F) -> AnalysisResultsForFn {
       DominatorTree &DT =
           this->getAnalysis<DominatorTreeWrapperPass>(F).getDomTree();
       return {
           std::make_unique<PredicateInfo>(
               F, DT,
               this->getAnalysis<AssumptionCacheTracker>().getAssumptionCache(
                   F)),
           nullptr,  // We cannot preserve the LI, DT or PDT with the legacy pass
           nullptr,  // manager, so set them to nullptr.
           nullptr};
     };
 
-    return runIPSCCP(M, DL, GetTLI, getAnalysis);
+    return runIPSCCP(M, DL, GetTLI, GetTTI, GetAC, getAnalysis);
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<AssumptionCacheTracker>();
     AU.addRequired<DominatorTreeWrapperPass>();
     AU.addRequired<TargetLibraryInfoWrapperPass>();
+    AU.addRequired<TargetTransformInfoWrapperPass>();
   }
 };
 
 } // end anonymous namespace
 
 char IPSCCPLegacyPass::ID = 0;
 
 INITIALIZE_PASS_BEGIN(IPSCCPLegacyPass, "ipsccp",
                       "Interprocedural Sparse Conditional Constant Propagation",
                       false, false)
 INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
 INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
 INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
 INITIALIZE_PASS_END(IPSCCPLegacyPass, "ipsccp",
                     "Interprocedural Sparse Conditional Constant Propagation",
                     false, false)
 
 // createIPSCCPPass - This is the public interface to this file.
 ModulePass *llvm::createIPSCCPPass() { return new IPSCCPLegacyPass(); }
-
-PreservedAnalyses FunctionSpecializationPass::run(Module &M,
-                                                  ModuleAnalysisManager &AM) {
-  const DataLayout &DL = M.getDataLayout();
-  auto &FAM = AM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
-  auto GetTLI = [&FAM](Function &F) -> TargetLibraryInfo & {
-    return FAM.getResult<TargetLibraryAnalysis>(F);
-  };
-  auto GetTTI = [&FAM](Function &F) -> TargetTransformInfo & {
-    return FAM.getResult<TargetIRAnalysis>(F);
-  };
-  auto GetAC = [&FAM](Function &F) -> AssumptionCache & {
-    return FAM.getResult<AssumptionAnalysis>(F);
-  };
-  auto GetAnalysis = [&FAM](Function &F) -> AnalysisResultsForFn {
-    DominatorTree &DT = FAM.getResult<DominatorTreeAnalysis>(F);
-    return {std::make_unique<PredicateInfo>(
-                F, DT, FAM.getResult<AssumptionAnalysis>(F)),
-            &DT, FAM.getCachedResult<PostDominatorTreeAnalysis>(F),
-            &FAM.getResult<LoopAnalysis>(F)};
-  };
-
-  if (!runFunctionSpecialization(M, DL, GetTLI, GetTTI, GetAC, GetAnalysis))
-    return PreservedAnalyses::all();
-
-  PreservedAnalyses PA;
-  PA.preserve<DominatorTreeAnalysis>();
-  PA.preserve<PostDominatorTreeAnalysis>();
-  PA.preserve<FunctionAnalysisManagerModuleProxy>();
-  return PA;
-}
-
-namespace {
-struct FunctionSpecializationLegacyPass : public ModulePass {
-  static char ID; // Pass identification, replacement for typeid
-  FunctionSpecializationLegacyPass() : ModulePass(ID) {}
-
-  void getAnalysisUsage(AnalysisUsage &AU) const override {
-    AU.addRequired<AssumptionCacheTracker>();
-    AU.addRequired<DominatorTreeWrapperPass>();
-    AU.addRequired<TargetLibraryInfoWrapperPass>();
-    AU.addRequired<TargetTransformInfoWrapperPass>();
-  }
-
-  bool runOnModule(Module &M) override {
-    if (skipModule(M))
-      return false;
-
-    const DataLayout &DL = M.getDataLayout();
-    auto GetTLI = [this](Function &F) -> TargetLibraryInfo & {
-      return this->getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(F);
-    };
-    auto GetTTI = [this](Function &F) -> TargetTransformInfo & {
-      return this->getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);
-    };
-    auto GetAC = [this](Function &F) -> AssumptionCache & {
-      return this->getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
-    };
-
-    auto GetAnalysis = [this](Function &F) -> AnalysisResultsForFn {
-      DominatorTree &DT =
-          this->getAnalysis<DominatorTreeWrapperPass>(F).getDomTree();
-      return {
-          std::make_unique<PredicateInfo>(
-              F, DT,
-              this->getAnalysis<AssumptionCacheTracker>().getAssumptionCache(
-                  F)),
-          nullptr, // We cannot preserve the LI, DT, or PDT with the legacy pass
-          nullptr, // manager, so set them to nullptr.
-          nullptr};
-    };
-    return runFunctionSpecialization(M, DL, GetTLI, GetTTI, GetAC, GetAnalysis);
-  }
-};
-} // namespace
-
-char FunctionSpecializationLegacyPass::ID = 0;
-
-INITIALIZE_PASS_BEGIN(
-    FunctionSpecializationLegacyPass, "function-specialization",
-    "Propagate constant arguments by specializing the function", false, false)
-
-INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
-INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
-INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
-INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
-INITIALIZE_PASS_END(FunctionSpecializationLegacyPass, "function-specialization",
-                    "Propagate constant arguments by specializing the function",
-                    false, false)
-
-ModulePass *llvm::createFunctionSpecializationPass() {
-  return new FunctionSpecializationLegacyPass();
-}

llvm/lib/Transforms/Scalar/CMakeLists.txt

add_llvm_component_library(LLVMScalarOpts
   COMPONENT_NAME
   Scalar
 
   LINK_COMPONENTS
   AggressiveInstCombine
   Analysis
   Core
   InstCombine
+  IPO
   Support
   TransformUtils
   )

llvm/lib/Transforms/Scalar/SCCP.cpp

 #include "llvm/IR/User.h"
 #include "llvm/IR/Value.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
+#include "llvm/Transforms/IPO/FunctionSpecialization.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/SCCPSolver.h"
 #include <cassert>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "sccp"
 
 STATISTIC(NumInstRemoved, "Number of instructions removed");
 STATISTIC(NumDeadBlocks , "Number of basic blocks unreachable");
 STATISTIC(NumInstReplaced,
           "Number of instructions replaced with (simpler) instruction");
 
 STATISTIC(IPNumInstRemoved, "Number of instructions removed by IPSCCP");
 STATISTIC(IPNumArgsElimed ,"Number of arguments constant propagated by IPSCCP");
 STATISTIC(IPNumGlobalConst, "Number of globals found to be constant by IPSCCP");
 STATISTIC(
     IPNumInstReplaced,
     "Number of instructions replaced with (simpler) instruction by IPSCCP");
 
+cl::opt<bool> SpecializeFunctions("specialize-functions", cl::init(false),
+    cl::Hidden, cl::desc("Enable function specialization"));
+
+static cl::opt<unsigned> FuncSpecializationMaxIters(
+    "func-specialization-max-iters", cl::Hidden,
+    cl::desc("The maximum number of iterations function specialization is run"),
+    cl::init(1));
+
 // Helper to check if \p LV is either a constant or a constant
 // range with a single element. This should cover exactly the same cases as the
 // old ValueLatticeElement::isConstant() and is intended to be used in the
 // transition to ValueLatticeElement.
 static bool isConstant(const ValueLatticeElement &LV) {
   return LV.isConstant() ||
          (LV.isConstantRange() && LV.getConstantRange().isSingleElement());
 }
static bool canRemoveInstruction(Instruction *I) {
   // Some instructions can be handled but are rejected above. Catch
   // those cases by falling through to here.
   // TODO: Mark globals as being constant earlier, so
   // TODO: wouldInstructionBeTriviallyDead() knows that atomic loads
   // TODO: are safe to remove.
   return isa<LoadInst>(I);
 }
 
+static void replaceUsesOfWith(SCCPSolver &Solver, Value *Old, Value *New) {
+  SmallVector<User *> Users(Old->users());
+  Old->replaceAllUsesWith(New);
+  Solver.markUsersAsChanged(Old, Users);
+}
+
 static bool tryToReplaceWithConstant(SCCPSolver &Solver, Value *V) {
   Constant *Const = nullptr;
   if (V->getType()->isStructTy()) {
     std::vector<ValueLatticeElement> IVs = Solver.getStructLatticeValueFor(V);
     if (llvm::any_of(IVs, isOverdefined))
       return false;
     std::vector<Constant *> ConstVals;
     auto *ST = cast<StructType>(V->getType());
if (CB && ((CB->isMustTailCall() &&
 
     LLVM_DEBUG(dbgs() << "  Can\'t treat the result of call " << *CB
                       << " as a constant\n");
     return false;
   }
 
   LLVM_DEBUG(dbgs() << "  Constant: " << *Const << " = " << *V << '\n');
 
-  // Replaces all of the uses of a variable with uses of the constant.
-  V->replaceAllUsesWith(Const);
+  replaceUsesOfWith(Solver, V, Const);
   return true;
 }
 
 /// Try to replace signed instructions with their unsigned equivalent.
 static bool replaceSignedInst(SCCPSolver &Solver,
+                              SmallPtrSetImpl<Instruction *> &ToDelete,
                               SmallPtrSetImpl<Value *> &InsertedValues,
                               Instruction &Inst) {
   // Determine if a signed value is known to be >= 0.
   auto isNonNegative = [&Solver](Value *V) {
     // If this value was constant-folded, it may not have a solver entry.
     // Handle integers. Otherwise, return false.
     if (auto *C = dyn_cast<Constant>(V)) {
       auto *CInt = dyn_cast<ConstantInt>(C);
       return CInt && !CInt->isNegative();
     }
     const ValueLatticeElement &IV = Solver.getLatticeValueFor(V);
     return IV.isConstantRange(/*UndefAllowed=*/false) &&
            IV.getConstantRange().isAllNonNegative();
static bool replaceSignedInst(SCCPSolver &Solver,
   default:
     return false;
   }
 
   // Wire up the new instruction and update state.
   assert(NewInst && "Expected replacement instruction");
   NewInst->takeName(&Inst);
   InsertedValues.insert(NewInst);
-  Inst.replaceAllUsesWith(NewInst);
-  Solver.removeLatticeValueFor(&Inst);
-  Inst.eraseFromParent();
+  replaceUsesOfWith(Solver, &Inst, NewInst);
+  Inst.removeFromParent();
+  ToDelete.insert(&Inst);
   return true;
 }
 
 static bool simplifyInstsInBlock(SCCPSolver &Solver, BasicBlock &BB,
+                                 SmallPtrSetImpl<Instruction *> &ToDelete,
                                  SmallPtrSetImpl<Value *> &InsertedValues,
                                  Statistic &InstRemovedStat,
                                  Statistic &InstReplacedStat) {
   bool MadeChanges = false;
   for (Instruction &Inst : make_early_inc_range(BB)) {
     if (Inst.getType()->isVoidTy())
       continue;
     if (tryToReplaceWithConstant(Solver, &Inst)) {
-      if (canRemoveInstruction(&Inst))
-        Inst.eraseFromParent();
+      if (canRemoveInstruction(&Inst)) {
+        Inst.removeFromParent();
+        ToDelete.insert(&Inst);
+      }
 
       MadeChanges = true;
       ++InstRemovedStat;
-    } else if (replaceSignedInst(Solver, InsertedValues, Inst)) {
+    } else if (replaceSignedInst(Solver, ToDelete, InsertedValues, Inst)) {
       MadeChanges = true;
       ++InstReplacedStat;
     }
   }
   return MadeChanges;
 }
 
 static bool removeNonFeasibleEdges(const SCCPSolver &Solver, BasicBlock *BB,
-                                   DomTreeUpdater &DTU,
+                                   DomTreeUpdater *DTU,
                                    BasicBlock *&NewUnreachableBB);
 
 // runSCCP() - Run the Sparse Conditional Constant Propagation algorithm,
 // and return true if the function was modified.
 static bool runSCCP(Function &F, const DataLayout &DL,
                     const TargetLibraryInfo *TLI, DomTreeUpdater &DTU) {
   LLVM_DEBUG(dbgs() << "SCCP on function '" << F.getName() << "'\n");
   SCCPSolver Solver(
static bool runSCCP(Function &F, const DataLayout &DL,
   }
 
   bool MadeChanges = false;
 
   // If we decided that there are basic blocks that are dead in this function,
   // delete their contents now.  Note that we cannot actually delete the blocks,
   // as we cannot modify the CFG of the function.
 
+  SmallPtrSet<Instruction *, 32> ToDelete;
   SmallPtrSet<Value *, 32> InsertedValues;
   SmallVector<BasicBlock *, 8> BlocksToErase;
   for (BasicBlock &BB : F) {
     if (!Solver.isBlockExecutable(&BB)) {
       LLVM_DEBUG(dbgs() << "  BasicBlock Dead:" << BB);
       ++NumDeadBlocks;
       BlocksToErase.push_back(&BB);
       MadeChanges = true;
       continue;
     }
 
-    MadeChanges |= simplifyInstsInBlock(Solver, BB, InsertedValues,
+    MadeChanges |= simplifyInstsInBlock(Solver, BB, ToDelete, InsertedValues,
                                         NumInstRemoved, NumInstReplaced);
   }
+  for (Instruction *DeadInst : ToDelete)
+    DeadInst->deleteValue();
 
   // Remove unreachable blocks and non-feasible edges.
   for (BasicBlock *DeadBB : BlocksToErase)
     NumInstRemoved += changeToUnreachable(DeadBB->getFirstNonPHI(),
                                           /*PreserveLCSSA=*/false, &DTU);
 
   BasicBlock *NewUnreachableBB = nullptr;
   for (BasicBlock &BB : F)
-    MadeChanges |= removeNonFeasibleEdges(Solver, &BB, DTU, NewUnreachableBB);
+    MadeChanges |= removeNonFeasibleEdges(Solver, &BB, &DTU, NewUnreachableBB);
 
   for (BasicBlock *DeadBB : BlocksToErase)
     if (!DeadBB->hasAddressTaken())
       DTU.deleteBB(DeadBB);
 
   return MadeChanges;
 }
 
for (BasicBlock &BB : F) {
 
     if (auto *RI = dyn_cast<ReturnInst>(BB.getTerminator()))
       if (!isa<UndefValue>(RI->getOperand(0)))
         ReturnsToZap.push_back(RI);
   }
 }
 
 static bool removeNonFeasibleEdges(const SCCPSolver &Solver, BasicBlock *BB,
-                                   DomTreeUpdater &DTU,
+                                   DomTreeUpdater *DTU,
                                    BasicBlock *&NewUnreachableBB) {
   SmallPtrSet<BasicBlock *, 8> FeasibleSuccessors;
   bool HasNonFeasibleEdges = false;
   for (BasicBlock *Succ : successors(BB)) {
     if (Solver.isEdgeFeasible(BB, Succ))
       FeasibleSuccessors.insert(Succ);
     else
       HasNonFeasibleEdges = true;
assert((isa<BranchInst>(TI) || isa<SwitchInst>(TI) ||
          "Terminator must be a br, switch or indirectbr");
 
   if (FeasibleSuccessors.size() == 0) {
     // Branch on undef/poison, replace with unreachable.
     SmallPtrSet<BasicBlock *, 8> SeenSuccs;
     SmallVector<DominatorTree::UpdateType, 8> Updates;
     for (BasicBlock *Succ : successors(BB)) {
       Succ->removePredecessor(BB);
-      if (SeenSuccs.insert(Succ).second)
+      if (DTU && SeenSuccs.insert(Succ).second)
         Updates.push_back({DominatorTree::Delete, BB, Succ});
     }
     TI->eraseFromParent();
     new UnreachableInst(BB->getContext(), BB);
-    DTU.applyUpdatesPermissive(Updates);
+    if (DTU)
+      DTU->applyUpdatesPermissive(Updates);
   } else if (FeasibleSuccessors.size() == 1) {
     // Replace with an unconditional branch to the only feasible successor.
     BasicBlock *OnlyFeasibleSuccessor = *FeasibleSuccessors.begin();
     SmallVector<DominatorTree::UpdateType, 8> Updates;
     bool HaveSeenOnlyFeasibleSuccessor = false;
     for (BasicBlock *Succ : successors(BB)) {
       if (Succ == OnlyFeasibleSuccessor && !HaveSeenOnlyFeasibleSuccessor) {
         // Don't remove the edge to the only feasible successor the first time
         // we see it. We still do need to remove any multi-edges to it though.
         HaveSeenOnlyFeasibleSuccessor = true;
         continue;
       }
 
       Succ->removePredecessor(BB);
+      if (DTU)
         Updates.push_back({DominatorTree::Delete, BB, Succ});
     }
 
     BranchInst::Create(OnlyFeasibleSuccessor, BB);
     TI->eraseFromParent();
-    DTU.applyUpdatesPermissive(Updates);
+    if (DTU)
+      DTU->applyUpdatesPermissive(Updates);
   } else if (FeasibleSuccessors.size() > 1) {
     SwitchInstProfUpdateWrapper SI(*cast<SwitchInst>(TI));
     SmallVector<DominatorTree::UpdateType, 8> Updates;
 
     // If the default destination is unfeasible it will never be taken. Replace
     // it with a new block with a single Unreachable instruction.
     BasicBlock *DefaultDest = SI->getDefaultDest();
     if (!FeasibleSuccessors.contains(DefaultDest)) {
       if (!NewUnreachableBB) {
         NewUnreachableBB =
             BasicBlock::Create(DefaultDest->getContext(), "default.unreachable",
                                DefaultDest->getParent(), DefaultDest);
         new UnreachableInst(DefaultDest->getContext(), NewUnreachableBB);
       }
 
       SI->setDefaultDest(NewUnreachableBB);
+      if (DTU) {
         Updates.push_back({DominatorTree::Delete, BB, DefaultDest});
         Updates.push_back({DominatorTree::Insert, BB, NewUnreachableBB});
       }
+    }
 
     for (auto CI = SI->case_begin(); CI != SI->case_end();) {
       if (FeasibleSuccessors.contains(CI->getCaseSuccessor())) {
         ++CI;
         continue;
       }
 
       BasicBlock *Succ = CI->getCaseSuccessor();
       Succ->removePredecessor(BB);
+      if (DTU)
         Updates.push_back({DominatorTree::Delete, BB, Succ});
       SI.removeCase(CI);
       // Don't increment CI, as we removed a case.
     }
 
-    DTU.applyUpdatesPermissive(Updates);
+    if (DTU)
+      DTU->applyUpdatesPermissive(Updates);
   } else {
     llvm_unreachable("Must have at least one feasible successor");
   }
   return true;
 }
 
+static bool propagateConstants(SCCPSolver &Solver,
+                               SmallVectorImpl<Function *> &WorkList,
+                               SmallPtrSetImpl<Instruction *> &ToDelete) {
+  bool MadeChanges = false;
+
+  for (Function *F : WorkList) {
+    if (F->isDeclaration())
+      continue;
+
+    if (Solver.isBlockExecutable(&F->front())) {
+      bool ReplacedPointerArg = false;
+      for (Argument &Arg : F->args()) {
+        if (!Arg.use_empty() && tryToReplaceWithConstant(Solver, &Arg)) {
+          ReplacedPointerArg |= Arg.getType()->isPointerTy();
+          ++IPNumArgsElimed;
+        }
+      }
+
+      // If we replaced an argument, the argmemonly and
+      // inaccessiblemem_or_argmemonly attributes do not hold any longer. Remove
+      // them from both the function and callsites.
+      if (ReplacedPointerArg) {
+        AttributeMask AttributesToRemove;
+        AttributesToRemove.addAttribute(Attribute::ArgMemOnly);
+        AttributesToRemove.addAttribute(Attribute::InaccessibleMemOrArgMemOnly);
+        F->removeFnAttrs(AttributesToRemove);
+
+        for (User *U : F->users()) {
+          auto *CB = dyn_cast<CallBase>(U);
+          if (!CB || CB->getCalledFunction() != F)
+            continue;
+
+          CB->removeFnAttrs(AttributesToRemove);
+        }
+      }
+      MadeChanges |= ReplacedPointerArg;
+    }
+
+    SmallPtrSet<Value *, 32> InsertedValues;
+    for (BasicBlock &BB : *F)
+      if (Solver.isBlockExecutable(&BB))
+        MadeChanges |= simplifyInstsInBlock(Solver, BB, ToDelete, InsertedValues,
+                                            IPNumInstRemoved, IPNumInstReplaced);
+  }
+
+  return MadeChanges;
+}
+
 bool llvm::runIPSCCP(
     Module &M, const DataLayout &DL,
     std::function<const TargetLibraryInfo &(Function &)> GetTLI,
+    std::function<TargetTransformInfo &(Function &)> GetTTI,
+    std::function<AssumptionCache &(Function &)> GetAC,
     function_ref<AnalysisResultsForFn(Function &)> getAnalysis) {
   SCCPSolver Solver(DL, GetTLI, M.getContext());
+  FunctionSpecializer Specializer(Solver, M, GetTLI, GetTTI, GetAC);
 
   // Loop over all functions, marking arguments to those with their addresses
   // taken or that are external as overdefined.
   for (Function &F : M) {
     if (F.isDeclaration())
       continue;
 
     Solver.addAnalysis(F, getAnalysis(F));
bool llvm::runIPSCCP(
   // Determine if we can track any of the module's global variables. If so, add
   // the global variables we can track to the solver's set of tracked global
   // variables.
   for (GlobalVariable &G : M.globals()) {
     G.removeDeadConstantUsers();
     if (canTrackGlobalVariableInterprocedurally(&G))
       Solver.trackValueOfGlobalVariable(&G);
   }
 
-  // Solve for constants.
-  bool ResolvedUndefs = true;
-  Solver.solve();
-  while (ResolvedUndefs) {
-    LLVM_DEBUG(dbgs() << "RESOLVING UNDEFS\n");
-    ResolvedUndefs = false;
-    for (Function &F : M) {
-      if (Solver.resolvedUndefsIn(F))
-        ResolvedUndefs = true;
-    }
-    if (ResolvedUndefs)
-      Solver.solve();
-  }
-
   bool MadeChanges = false;
+  SmallVector<Function *> WorkList;
+  SmallPtrSet<Instruction *, 32> ToDelete;
 
-  // Iterate over all of the instructions in the module, replacing them with
-  // constants if we have found them to be of constant values.
+  for (Function &F : M)
+    WorkList.push_back(&F);
 
-  for (Function &F : M) {
-    if (F.isDeclaration())
-      continue;
+  // Solve for constants.
+  Solver.solveWhileResolvedUndefsIn(WorkList);
 
-    SmallVector<BasicBlock *, 512> BlocksToErase;
+  // Iterate over all of the instructions in the module, replacing them with
+  // constants if we have found them to be of constant values.
+  MadeChanges |= propagateConstants(Solver, WorkList, ToDelete);
 
-    if (Solver.isBlockExecutable(&F.front())) {
-      bool ReplacedPointerArg = false;
-      for (Argument &Arg : F.args()) {
-        if (!Arg.use_empty() && tryToReplaceWithConstant(Solver, &Arg)) {
-          ReplacedPointerArg |= Arg.getType()->isPointerTy();
-          ++IPNumArgsElimed;
-        }
-      }
+  if (SpecializeFunctions) {
+    unsigned Iters = 0;
+    WorkList.clear();
+    while (Iters++ < FuncSpecializationMaxIters &&
+           Specializer.specialize(WorkList)) {
+      MadeChanges |= propagateConstants(Solver, WorkList, ToDelete);
+      Specializer.promoteConstantStackValues();
+      WorkList.clear();
 
-      // If we replaced an argument, the argmemonly and
-      // inaccessiblemem_or_argmemonly attributes do not hold any longer. Remove
-      // them from both the function and callsites.
-      if (ReplacedPointerArg) {
-        AttributeMask AttributesToRemove;
-        AttributesToRemove.addAttribute(Attribute::ArgMemOnly);
-        AttributesToRemove.addAttribute(Attribute::InaccessibleMemOrArgMemOnly);
-        F.removeFnAttrs(AttributesToRemove);
+  // Don't delete anything until solving is done.
+  for (Instruction *DeadInst : ToDelete)
+    DeadInst->deleteValue();
 
-        for (User *U : F.users()) {
-          auto *CB = dyn_cast<CallBase>(U);
-          if (!CB || CB->getCalledFunction() != &F)
-            continue;
+  for (Function &F : M) {
+    if (F.isDeclaration())
 
-          CB->removeFnAttrs(AttributesToRemove);
-        }
-      }
-      MadeChanges |= ReplacedPointerArg;
-    }
-
-    SmallPtrSet<Value *, 32> InsertedValues;
+    SmallVector<BasicBlock *, 512> BlocksToErase;
     for (BasicBlock &BB : F) {
       if (!Solver.isBlockExecutable(&BB)) {
         LLVM_DEBUG(dbgs() << "  BasicBlock Dead:" << BB);
         ++NumDeadBlocks;
 
         MadeChanges = true;
 
         if (&BB != &F.front())
           BlocksToErase.push_back(&BB);
-        continue;
       }
-
-      MadeChanges |= simplifyInstsInBlock(Solver, BB, InsertedValues,
-                                          IPNumInstRemoved, IPNumInstReplaced);
     }
 
-    DomTreeUpdater DTU = Solver.getDTU(F);
+    Optional<DomTreeUpdater> OptDTU =
+        SpecializeFunctions && Specializer.isClonedFunction(&F) ?
+        None : Optional<DomTreeUpdater>(Solver.getDTU(F));
+
+    DomTreeUpdater *DTU = OptDTU ? &*OptDTU : nullptr;
+
     // Change dead blocks to unreachable. We do it after replacing constants
     // in all executable blocks, because changeToUnreachable may remove PHI
     // nodes in executable blocks we found values for. The function's entry
     // block is not part of BlocksToErase, so we have to handle it separately.
     for (BasicBlock *BB : BlocksToErase) {
       NumInstRemoved += changeToUnreachable(BB->getFirstNonPHI(),
-                                            /*PreserveLCSSA=*/false, &DTU);
+                                            /*PreserveLCSSA=*/false, DTU);
     }
     if (!Solver.isBlockExecutable(&F.front()))
       NumInstRemoved += changeToUnreachable(F.front().getFirstNonPHI(),
-                                            /*PreserveLCSSA=*/false, &DTU);
+                                            /*PreserveLCSSA=*/false, DTU);
 
     BasicBlock *NewUnreachableBB = nullptr;
     for (BasicBlock &BB : F)
       MadeChanges |= removeNonFeasibleEdges(Solver, &BB, DTU, NewUnreachableBB);
 
-    for (BasicBlock *DeadBB : BlocksToErase)
-      if (!DeadBB->hasAddressTaken())
-        DTU.deleteBB(DeadBB);
+    for (BasicBlock *DeadBB : BlocksToErase) {
+      if (!DeadBB->hasAddressTaken()) {
+        if (DTU)
+          DTU->deleteBB(DeadBB);
+        else
+          DeadBB->eraseFromParent();
+      }
+    }
 
     for (BasicBlock &BB : F) {
       for (Instruction &Inst : llvm::make_early_inc_range(BB)) {
         if (Solver.getPredicateInfoFor(&Inst)) {
           if (auto *II = dyn_cast<IntrinsicInst>(&Inst)) {
             if (II->getIntrinsicID() == Intrinsic::ssa_copy) {
               Value *Op = II->getOperand(0);
               Inst.replaceAllUsesWith(Op);

llvm/lib/Transforms/Utils/SCCPSolver.cpp

 
 // Helper to check if \p LV is either overdefined or a constant range with more
 // than a single element. This should cover exactly the same cases as the old
 // ValueLatticeElement::isOverdefined() and is intended to be used in the
 // transition to ValueLatticeElement.
 bool isOverdefined(const ValueLatticeElement &LV) {
   return !LV.isUnknownOrUndef() && !isConstant(LV);
 }
 
 } // namespace
 
 namespace llvm {
 
 /// Helper class for SCCPSolver. This implements the instruction visitor and
 /// holds all the state.
 class SCCPInstVisitor : public InstVisitor<SCCPInstVisitor> {
   const DataLayout &DL;
   std::function<const TargetLibraryInfo &(Function &)> GetTLI;
private:
 
   // Add U as additional user of V.
   void addAdditionalUser(Value *V, User *U) {
     auto Iter = AdditionalUsers.insert({V, {}});
     Iter.first->second.insert(U);
   }
 
   // Mark I's users as changed, including AdditionalUsers.
-  void markUsersAsChanged(Value *I) {
-    // Functions include their arguments in the use-list. Changed function
-    // values mean that the result of the function changed. We only need to
-    // update the call sites with the new function result and do not have to
-    // propagate the call arguments.
-    if (isa<Function>(I)) {
-      for (User *U : I->users()) {
-        if (auto *CB = dyn_cast<CallBase>(U))
-          handleCallResult(*CB);
-      }
-    } else {
-      for (User *U : I->users())
-        if (auto *UI = dyn_cast<Instruction>(U))
-          operandChangedState(UI);
-    }
+  void markUsersAsChanged(Value *I) { markUsersAsChanged(I, I->users()); }
 
-    auto Iter = AdditionalUsers.find(I);
-    if (Iter != AdditionalUsers.end()) {
-      // Copy additional users before notifying them of changes, because new
-      // users may be added, potentially invalidating the iterator.
-      SmallVector<Instruction *, 2> ToNotify;
-      for (User *U : Iter->second)
-        if (auto *UI = dyn_cast<Instruction>(U))
-          ToNotify.push_back(UI);
-      for (Instruction *UI : ToNotify)
-        operandChangedState(UI);
-    }
-  }
   void handleCallOverdefined(CallBase &CB);
   void handleCallResult(CallBase &CB);
   void handleCallArguments(CallBase &CB);
 
 private:
   friend class InstVisitor<SCCPInstVisitor>;
 
   // visit implementations - Something changed in this instruction.  Either an
public:
 
   void markArgInFuncSpecialization(Function *F,
                                    const SmallVectorImpl<ArgInfo> &Args);
 
   void markFunctionUnreachable(Function *F) {
     for (auto &BB : *F)
       BBExecutable.erase(&BB);
   }
+
+  // Mark Users as changed, including I's AdditionalUsers.
+  //
+  // UserList can be either of:
+  //   (a) const SmallVectorImpl<User *> &&
+  //   (b) iterator_range<Value::user_iterator>
+  template<typename UserList>
+  void markUsersAsChanged(Value *I, UserList &&Users) {
+    // Functions include their arguments in the use-list. Changed function
+    // values mean that the result of the function changed. We only need to
+    // update the call sites with the new function result and do not have to
+    // propagate the call arguments.
+    if (isa<Function>(I)) {
+      for (User *U : Users) {
+        if (auto *CB = dyn_cast<CallBase>(U))
+          handleCallResult(*CB);
+      }
+    } else {
+      for (User *U : Users)
+        if (auto *UI = dyn_cast<Instruction>(U))
+          operandChangedState(UI);
+    }
+
+    auto Iter = AdditionalUsers.find(I);
+    if (Iter != AdditionalUsers.end()) {
+      // Copy additional users before notifying them of changes, because new
+      // users may be added, potentially invalidating the iterator.
+      SmallVector<Instruction *, 2> ToNotify;
+      for (User *U : Iter->second)
+        if (auto *UI = dyn_cast<Instruction>(U))
+          ToNotify.push_back(UI);
+      for (Instruction *UI : ToNotify)
+        operandChangedState(UI);
+    }
+  }
 };
 
 } // namespace llvm
 
 bool SCCPInstVisitor::markBlockExecutable(BasicBlock *BB) {
   if (!BBExecutable.insert(BB).second)
     return false;
   LLVM_DEBUG(dbgs() << "Marking Block Executable: " << BB->getName() << '\n');
for (Instruction &I : BB) {
         continue;
       }
 
       markOverdefined(&I);
       MadeChange = true;
     }
   }
 
+  LLVM_DEBUG(if (MadeChange)
+               dbgs() << "\nResolved undefs in " << F.getName() << '\n');
+
   return MadeChange;
 }
 
 //===----------------------------------------------------------------------===//
 //
 // SCCPSolver implementations
 //
 SCCPSolver::SCCPSolver(

 }
 
 void SCCPSolver::solve() { Visitor->solve(); }
 
 bool SCCPSolver::resolvedUndefsIn(Function &F) {
   return Visitor->resolvedUndefsIn(F);
 }
 
+void
+SCCPSolver::solveWhileResolvedUndefsIn(SmallVectorImpl<Function *> &WorkList) {
+  bool ResolvedUndefs = true;
+  while (ResolvedUndefs) {
+    Visitor->solve();
+    ResolvedUndefs = false;
+    for (Function *F : WorkList)
+      ResolvedUndefs |= Visitor->resolvedUndefsIn(*F);
+  }
+}
+
 bool SCCPSolver::isBlockExecutable(BasicBlock *BB) const {
   return Visitor->isBlockExecutable(BB);
 }
 
 bool SCCPSolver::isEdgeFeasible(BasicBlock *From, BasicBlock *To) const {
   return Visitor->isEdgeFeasible(From, To);
 }
 
void SCCPSolver::markArgInFuncSpecialization(
     Function *F, const SmallVectorImpl<ArgInfo> &Args) {
   Visitor->markArgInFuncSpecialization(F, Args);
 }
 
 void SCCPSolver::markFunctionUnreachable(Function *F) {
   Visitor->markFunctionUnreachable(F);
 }
 
+void SCCPSolver::markUsersAsChanged(Value *I,
+                                    const SmallVectorImpl<User *> &Users) {
+  Visitor->markUsersAsChanged(I, std::move(Users));
+}
+
 void SCCPSolver::visit(Instruction *I) { Visitor->visit(I); }
 
 void SCCPSolver::visitCall(CallInst &I) { Visitor->visitCall(I); }

llvm/test/Transforms/FunctionSpecialization/bug52821-use-after-free.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -S < %s | FileCheck %s
 
 %mystruct = type { i32, [2 x i64] }
 
 define internal %mystruct* @myfunc(%mystruct* %arg) {
 ; CHECK-LABEL: @myfunc(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[FOR_COND:%.*]]
 ; CHECK:       for.cond:
-; CHECK-NEXT:    br i1 true, label [[FOR_COND2:%.*]], label [[FOR_BODY:%.*]]
-; CHECK:       for.body:
-; CHECK-NEXT:    call void @callee(%mystruct* nonnull null)
-; CHECK-NEXT:    br label [[FOR_COND]]
+; CHECK-NEXT:    br label [[FOR_COND2:%.*]]
 ; CHECK:       for.cond2:
-; CHECK-NEXT:    br i1 false, label [[FOR_END:%.*]], label [[FOR_BODY2:%.*]]
+; CHECK-NEXT:    br label [[FOR_BODY2:%.*]]
 ; CHECK:       for.body2:
-; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [[MYSTRUCT:%.*]], %mystruct* null, i64 0, i32 1, i64 3
 ; CHECK-NEXT:    br label [[FOR_COND2]]
-; CHECK:       for.end:
-; CHECK-NEXT:    ret %mystruct* [[ARG:%.*]]
 ;
 entry:
   br label %for.cond
 
 for.cond:                                     ; preds = %for.body, %entry
   %phi = phi %mystruct* [ undef, %for.body ], [ null, %entry ]
   %cond = icmp eq %mystruct* %phi, null
   br i1 %cond, label %for.cond2, label %for.body

 for.end:                                      ; preds = %for.cond2
   ret %mystruct* %arg
 }
 
 define %mystruct* @caller() {
 ; CHECK-LABEL: @caller(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CALL:%.*]] = call %mystruct* @myfunc(%mystruct* undef)
-; CHECK-NEXT:    ret %mystruct* [[CALL]]
+; CHECK-NEXT:    ret %mystruct* undef
 ;
 entry:
   %call = call %mystruct* @myfunc(%mystruct* undef)
   ret %mystruct* %call
 }
 
 declare void @callee(%mystruct*)
+

llvm/test/Transforms/FunctionSpecialization/bug55000-read-uninitialized-value.ll

-; RUN: opt -function-specialization -func-specialization-max-iters=2 -func-specialization-size-threshold=20 -func-specialization-avg-iters-cost=20 -function-specialization-for-literal-constant=true -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=2 -func-specialization-max-clones=1 -function-specialization-for-literal-constant=true -S < %s | FileCheck %s
 
 declare hidden i1 @compare(ptr) align 2
 declare hidden { i8, ptr } @getType(ptr) align 2
 
 ; CHECK-LABEL: @foo
 ; CHECK-LABEL: @foo.1
 ; CHECK-LABEL: @foo.2
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-always-inline.ll

-; RUN: opt -function-specialization -func-specialization-avg-iters-cost=3 -func-specialization-size-threshold=10 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-avg-iters-cost=3 -func-specialization-size-threshold=10 -S < %s | FileCheck %s
 
 ; CHECK-NOT: foo.{{[0-9]+}}
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 @A = external dso_local constant i32, align 4
 @B = external dso_local constant i32, align 4
 @C = external dso_local constant i32, align 4

 
 if.else:
   %call1 = call i32 @foo(i32 %y, i32* @B, i32* @D)
   br label %return
 
 return:
   %retval.0 = phi i32 [ %call, %if.then ], [ %call1, %if.else ]
   ret i32 %retval.0
-}
+}
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-expression.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 
 ; Test function specialization wouldn't crash due to constant expression.
 ; Note that this test case shows that function specialization pass would
 ; transform the function even if no specialization happened.
 
-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 %struct = type { i8, i16, i32, i64, i64}
 @Global = internal constant %struct {i8 0, i16 1, i32 2, i64 3, i64 4}
 
 define internal i64 @func2(i64 *%x) {
 entry:
   %val = ptrtoint i64* %x to i64
   ret i64 %val
 }
 
 define internal i64 @func(i64 *%x, i64 (i64*)* %binop) {
 ; CHECK-LABEL: @func(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[TMP0:%.*]] = call i64 [[BINOP:%.*]](i64* [[X:%.*]])
-; CHECK-NEXT:    ret i64 [[TMP0]]
+; CHECK-NEXT:    unreachable
 ;
 entry:
   %tmp0 = call i64 %binop(i64* %x)
   ret i64 %tmp0
 }
 
 define internal i64 @zoo(i1 %flag) {
 ; CHECK-LABEL: @zoo(

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-expression2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; Check that we don't crash and specialise on a constant expression.
 
 %struct.pluto = type { %struct.spam }
 %struct.quux = type { i16 }
 %struct.spam = type { i16 }
 
 @global.5 = external dso_local global [4 x %struct.pluto], align 1

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-expression3.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 define i32 @main() {
 ; CHECK-LABEL: @main(
 ; CHECK-NEXT:  bb:
 ; CHECK-NEXT:    tail call void @wombat.1(i8* undef, i64 undef, i64 undef, i32 (i8*, i8*)* bitcast (i32 ()* @quux to i32 (i8*, i8*)*))
 ; CHECK-NEXT:    tail call void @wombat.2(i8* undef, i64 undef, i64 undef, i32 (i8*, i8*)* bitcast (i32 ()* @eggs to i32 (i8*, i8*)*))
 ; CHECK-NEXT:    ret i32 undef
 ;

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-expression4.ll

-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; Check that we don't crash and specialise on a function call with byval attribute.
 
 ; CHECK-NOT: wombat.{{[0-9]+}}
 
 declare i32* @quux()
 declare i32* @eggs()
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-expression5.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -force-function-specialization -func-specialization-on-address -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-on-address -S < %s | FileCheck %s
 
 ; Check that we don't crash and specialise on a scalar global variable with byval attribute.
 
 ; CHECK-NOT: wombat.{{[0-9]+}}
 
 %struct.pluto = type { %struct.spam }
 %struct.quux = type { i16 }
 %struct.spam = type { i16 }

llvm/test/Transforms/FunctionSpecialization/function-specialization-constant-integers.ll

-; RUN: opt -function-specialization -function-specialization-for-literal-constant=true -func-specialization-size-threshold=10 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -function-specialization-for-literal-constant=true -func-specialization-size-threshold=10 -S < %s | FileCheck %s
 
 ; Check that the literal constant parameter could be specialized.
 ; CHECK: @foo.1(
 ; CHECK: @foo.2(
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 declare i32 @getValue()

 }
 
 define dso_local i32 @bar(i32 %x, i32 %y) {
 entry:
   %retval.1 = call i32 @foo(i1 1)
   %retval.2 = call i32 @foo(i1 0)
   %retval = add nsw i32 %retval.1, %retval.2
   ret i32 %retval
-}
+}
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-loop.ll

-; RUN: opt -function-specialization -func-specialization-avg-iters-cost=5 -func-specialization-size-threshold=10 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-avg-iters-cost=5 -func-specialization-size-threshold=10 -S < %s | FileCheck %s
 
 ; Check that the loop depth results in a larger specialization bonus.
 ; CHECK: @foo.1(
 ; CHECK: @foo.2(
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 @A = external dso_local constant i32, align 4

llvm/test/Transforms/FunctionSpecialization/function-specialization-minsize.ll

-; RUN: opt -function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -S < %s | FileCheck %s
 
 ; CHECK-NOT: @compute.1
 ; CHECK-NOT: @compute.2
 
 define i64 @main(i64 %x, i1 %flag) {
 entry:
   br i1 %flag, label %plus, label %minus
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-minsize2.ll

-; RUN: opt -function-specialization -func-specialization-size-threshold=3 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-size-threshold=3 -S < %s | FileCheck %s
 
 ; Checks for callsites that have been annotated with MinSize. No specialisation
 ; expected here:
 ;
 ; CHECK-NOT: @compute.1
 ; CHECK-NOT: @compute.2
 
 define i64 @main(i64 %x, i1 %flag) {

llvm/test/Transforms/FunctionSpecialization/function-specialization-minsize3.ll

-; RUN: opt -function-specialization -func-specialization-size-threshold=3 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-size-threshold=3 -S < %s | FileCheck %s
 
 ; Checks for callsites that have been annotated with MinSize. We only expect
 ; specialisation for the call that does not have the attribute:
 ;
 ; CHECK: plus:
 ; CHECK:   %tmp0 = call i64 @compute.1(i64 %x, i64 (i64)* @plus)
 ; CHECK:   br label %merge
 ; CHECK: minus:

llvm/test/Transforms/FunctionSpecialization/function-specialization-nodup.ll

-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; Function @foo has function attribute 'noduplicate', so check that we don't
 ; specialize it:
 
 ; CHECK-NOT: @foo.1(
 ; CHECK-NOT: @foo.2(
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"

llvm/test/Transforms/FunctionSpecialization/function-specialization-nodup2.ll

-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; Check that function foo does not gets specialised as it contains an intrinsic
 ; that is marked as NoDuplicate.
 ; Please note that the use of the hardwareloop intrinsic is arbitrary; it's
 ; just an easy to use intrinsic that has NoDuplicate.
 
 ; CHECK-NOT: @foo.1(
 ; CHECK-NOT: @foo.2(

llvm/test/Transforms/FunctionSpecialization/function-specialization-noexec.ll

-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; The if.then block is not executed, so check that we don't specialise here.
 
 ; CHECK-NOT: @foo.1(
 ; CHECK-NOT: @foo.2(
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-nonconst-glob.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 
-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
-; RUN: opt -function-specialization -force-function-specialization -func-specialization-on-address=0 -S < %s | FileCheck %s
-; RUN: opt -function-specialization -force-function-specialization -func-specialization-on-address=1 -S < %s | FileCheck %s --check-prefix=ON-ADDRESS
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-on-address=0 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-on-address=1 -S < %s | FileCheck %s --check-prefix=ON-ADDRESS
 
 ; Global B is not constant. We do not specialise on addresses unless we
 ; enable that:
 
 ; ON-ADDRESS: call i32 @foo.1(i32 %x, i32* @A)
 ; ON-ADDRESS: call i32 @foo.2(i32 %y, i32* @B)
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"

llvm/test/Transforms/FunctionSpecialization/function-specialization-nothing-todo.ll

-; REQUIRES: asserts
-; RUN: opt -function-specialization -debug -S < %s 2>&1 | FileCheck %s
-
-; The purpose of this test is to check that we don't run the solver as there's
-; nothing to do here. For a test that doesn't trigger function specialisation,
-; it is intentionally 'big' because we also want to check that the ssa.copy
-; intrinsics that are introduced by the solver are cleaned up if we bail
-; early. Thus, first check the debug messages for the introduction of these
-; intrinsics:
-
-; CHECK: FnSpecialization: Analysing decl: foo
-; CHECK: Found replacement{{.*}} call i32 @llvm.ssa.copy.i32
-; CHECK: Found replacement{{.*}} call i32 @llvm.ssa.copy.i32
-
-; Then, make sure the solver didn't run:
-
-; CHECK-NOT: Running solver
-
-; Finally, check the absence and thus removal of these intrinsics:
-
-; CHECK-LABEL: @foo
-; CHECK-NOT:   call i32 @llvm.ssa.copy.i32
-
-@N = external dso_local global i32, align 4
-@B = external dso_local global i32*, align 8
-@A = external dso_local global i32*, align 8
-
-define dso_local i32 @foo() {
-entry:
-  br label %for.cond
-
-for.cond:
-  %i.0 = phi i32 [ 0, %entry ], [ %inc, %for.body ]
-  %0 = load i32, i32* @N, align 4
-  %cmp = icmp slt i32 %i.0, %0
-  br i1 %cmp, label %for.body, label %for.cond.cleanup
-
-for.cond.cleanup:
-  ret i32 undef
-
-for.body:
-  %1 = load i32*, i32** @B, align 8
-  %idxprom = sext i32 %i.0 to i64
-  %arrayidx = getelementptr inbounds i32, i32* %1, i64 %idxprom
-  %2 = load i32, i32* %arrayidx, align 4
-  %3 = load i32*, i32** @A, align 8
-  %arrayidx2 = getelementptr inbounds i32, i32* %3, i64 %idxprom
-  store i32 %2, i32* %arrayidx2, align 4
-  %inc = add nsw i32 %i.0, 1
-  br label %for.cond
-}

llvm/test/Transforms/FunctionSpecialization/function-specialization-poison.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; Check that we don't crash and specialise on a poison value.
 
 %struct.quux = type { i16 }
 %struct.spam = type { i16 }
 
 @global.12 = external global %struct.quux, align 1
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-recursive.ll

-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=2 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS2
-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=3 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS3
-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=4 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS4
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=2 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS2
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=3 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS3
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=4 -inline -instcombine -S < %s | FileCheck %s --check-prefix=ITERS4
 
 @low = internal constant i32 0, align 4
 @high = internal constant i32 6, align 4
 
 define internal void @recursiveFunc(i32* nocapture readonly %lo, i32 %step, i32* nocapture readonly %hi) {
   %lo.temp = alloca i32, align 4
   %hi.temp = alloca i32, align 4
   %lo.load = load i32, i32* %lo, align 4

llvm/test/Transforms/FunctionSpecialization/function-specialization-recursive2.ll

-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
 
 ; Volatile store preventing recursive specialisation:
 ;
 ; CHECK:     @recursiveFunc.1
 ; CHECK-NOT: @recursiveFunc.2
 
 @Global = internal constant i32 1, align 4
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-recursive3.ll

-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
 
 ; Duplicate store preventing recursive specialisation:
 ;
 ; CHECK:     @recursiveFunc.1
 ; CHECK-NOT: @recursiveFunc.2
 
 @Global = internal constant i32 1, align 4
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-recursive4.ll

-; RUN: opt -function-specialization -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -func-specialization-max-iters=2 -S < %s | FileCheck %s
 
 ; Alloca is not an integer type:
 ;
 ; CHECK:     @recursiveFunc.1
 ; CHECK-NOT: @recursiveFunc.2
 
 @Global = internal constant i32 1, align 4
 

llvm/test/Transforms/FunctionSpecialization/function-specialization-stats.ll

 ; REQUIRES: asserts
-; RUN: opt -stats -function-specialization -S -force-function-specialization < %s 2>&1 | FileCheck %s
+; RUN: opt -stats -ipsccp -specialize-functions -S -force-function-specialization < %s 2>&1 | FileCheck %s
 
 ; CHECK: 2 function-specialization - Number of functions specialized
 
 define i64 @main(i64 %x, i1 %flag) {
 entry:
   br i1 %flag, label %plus, label %minus
 
 plus:

llvm/test/Transforms/FunctionSpecialization/function-specialization.ll

-; RUN: opt -function-specialization -func-specialization-size-threshold=3 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-size-threshold=3 -S < %s | FileCheck %s
 
 define i64 @main(i64 %x, i1 %flag) {
 ;
 ; CHECK-LABEL: @main(i64 %x, i1 %flag) {
 ; CHECK:      entry:
 ; CHECK-NEXT:   br i1 %flag, label %plus, label %minus
 ; CHECK:      plus:
 ; CHECK-NEXT:   [[TMP0:%.+]] = call i64 @compute.1(i64 %x, i64 (i64)* @plus)

llvm/test/Transforms/FunctionSpecialization/function-specialization2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -deadargelim -force-function-specialization -S < %s | FileCheck %s
-; RUN: opt -function-specialization -func-specialization-max-iters=1 -deadargelim -force-function-specialization -S < %s | FileCheck %s
-; RUN: opt -function-specialization -func-specialization-max-iters=0 -deadargelim -force-function-specialization -S < %s | FileCheck %s --check-prefix=DISABLED
-; RUN: opt -function-specialization -func-specialization-avg-iters-cost=1 -deadargelim -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -deadargelim -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-iters=1 -deadargelim -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-iters=0 -deadargelim -force-function-specialization -S < %s | FileCheck %s --check-prefix=DISABLED
+; RUN: opt -ipsccp -specialize-functions -func-specialization-avg-iters-cost=1 -deadargelim -force-function-specialization -S < %s | FileCheck %s
 
 ; DISABLED-NOT: @func.1(
 ; DISABLED-NOT: @func.2(
 
 define internal i32 @func(i32* %0, i32 %1, void (i32*)* nocapture %2) {
   %4 = alloca i32, align 4
   store i32 %1, i32* %4, align 4
   %5 = load i32, i32* %4, align 4

 define internal void @decrement(i32* nocapture %0) {
   %2 = load i32, i32* %0, align 4
   %3 = add nsw i32 %2, -1
   store i32 %3, i32* %0, align 4
   ret void
 }
 
 define i32 @main(i32* %0, i32 %1) {
-; CHECK:    [[TMP3:%.*]] = call i32 @func.2(i32* [[TMP0:%.*]], i32 [[TMP1:%.*]])
+; CHECK:    call void @func.2(i32* [[TMP0:%.*]], i32 [[TMP1:%.*]])
   %3 = call i32 @func(i32* %0, i32 %1, void (i32*)* nonnull @increment)
-; CHECK:    [[TMP4:%.*]] = call i32 @func.1(i32* [[TMP0]], i32 [[TMP3]])
+; CHECK:    call void @func.1(i32* [[TMP0]], i32 0)
   %4 = call i32 @func(i32* %0, i32 %3, void (i32*)* nonnull @decrement)
+; CHECK:    ret i32 0
   ret i32 %4
 }
 
 ; CHECK: @func.1(
 ; CHECK:    [[TMP3:%.*]] = alloca i32, align 4
 ; CHECK:    store i32 [[TMP1:%.*]], i32* [[TMP3]], align 4
 ; CHECK:    [[TMP4:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP5:%.*]] = icmp slt i32 [[TMP4]], 1
 ; CHECK:    br i1 [[TMP5]], label [[TMP13:%.*]], label [[TMP6:%.*]]
 ; CHECK:       6:
 ; CHECK:    [[TMP7:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP8:%.*]] = sext i32 [[TMP7]] to i64
 ; CHECK:    [[TMP9:%.*]] = getelementptr inbounds i32, i32* [[TMP0:%.*]], i64 [[TMP8]]
 ; CHECK:    call void @decrement(i32* [[TMP9]])
 ; CHECK:    [[TMP10:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP11:%.*]] = add nsw i32 [[TMP10]], -1
-; CHECK:    [[TMP12:%.*]] = call i32 @func.1(i32* [[TMP0]], i32 [[TMP11]])
-; CHECK:    br label [[TMP13]]
-; CHECK:       13:
-; CHECK:    ret i32 0
+; CHECK:    call void @func.1(i32* [[TMP0]], i32 [[TMP11]])
+; CHECK:    br label [[TMP12:%.*]]
+; CHECK:       12:
+; CHECK:    ret void
 ;
 ;
 ; CHECK: @func.2(
 ; CHECK:    [[TMP3:%.*]] = alloca i32, align 4
 ; CHECK:    store i32 [[TMP1:%.*]], i32* [[TMP3]], align 4
 ; CHECK:    [[TMP4:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP5:%.*]] = icmp slt i32 [[TMP4]], 1
 ; CHECK:    br i1 [[TMP5]], label [[TMP13:%.*]], label [[TMP6:%.*]]
 ; CHECK:       6:
 ; CHECK:    [[TMP7:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP8:%.*]] = sext i32 [[TMP7]] to i64
 ; CHECK:    [[TMP9:%.*]] = getelementptr inbounds i32, i32* [[TMP0:%.*]], i64 [[TMP8]]
 ; CHECK:    call void @increment(i32* [[TMP9]])
 ; CHECK:    [[TMP10:%.*]] = load i32, i32* [[TMP3]], align 4
 ; CHECK:    [[TMP11:%.*]] = add nsw i32 [[TMP10]], -1
-; CHECK:    [[TMP12:%.*]] = call i32 @func.2(i32* [[TMP0]], i32 [[TMP11]])
-; CHECK:    br label [[TMP13]]
-; CHECK:    ret i32 0
+; CHECK:    call void @func.2(i32* [[TMP0]], i32 [[TMP11]])
+; CHECK:    br label [[TMP12:%.*]]
+; CHECK:       12:
+; CHECK:    ret void

llvm/test/Transforms/FunctionSpecialization/function-specialization3.ll

-; RUN: opt -function-specialization -func-specialization-avg-iters-cost=3 -S < %s | \
+; RUN: opt -ipsccp -specialize-functions -func-specialization-avg-iters-cost=3 -S < %s | \
 ; RUN:   FileCheck %s --check-prefixes=COMMON,DISABLED
-; RUN: opt -function-specialization -force-function-specialization -S < %s | \
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | \
 ; RUN:   FileCheck %s --check-prefixes=COMMON,FORCE
-; RUN: opt -function-specialization -func-specialization-avg-iters-cost=3 -force-function-specialization -S < %s | \
+; RUN: opt -ipsccp -specialize-functions -func-specialization-avg-iters-cost=3 -force-function-specialization -S < %s | \
 ; RUN:   FileCheck %s --check-prefixes=COMMON,FORCE
 
 ; Test for specializing a constant global.
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 @A = external dso_local constant i32, align 4
 @B = external dso_local constant i32, align 4

llvm/test/Transforms/FunctionSpecialization/function-specialization4.ll

-; RUN: opt -function-specialization -force-function-specialization \
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization \
 ; RUN:   -func-specialization-max-clones=2 -S < %s | FileCheck %s
 
-; RUN: opt -function-specialization -force-function-specialization \
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization \
 ; RUN:   -func-specialization-max-clones=1 -S < %s | FileCheck %s --check-prefix=CONST1
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 
 @A = external dso_local constant i32, align 4
 @B = external dso_local constant i32, align 4
 @C = external dso_local constant i32, align 4
 @D = external dso_local constant i32, align 4

llvm/test/Transforms/FunctionSpecialization/function-specialization5.ll

-; RUN: opt -function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 ; There's nothing to specialize here as both calls are the same, so check that:
 ;
 ; CHECK-NOT: define internal i32 @foo.1(
 ; CHECK-NOT: define internal i32 @foo.2(
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 

llvm/test/Transforms/FunctionSpecialization/get-possible-constants.ll

-; RUN: opt -S --passes=function-specialization < %s | FileCheck %s
+; RUN: opt -S --passes=ipsccp -specialize-functions < %s | FileCheck %s
 define dso_local i32 @p0(i32 noundef %x) {
 entry:
   %add = add nsw i32 %x, 1
   ret i32 %add
 }
 
 define dso_local i32 @p1(i32 noundef %x) {
 entry:
   %sub = add nsw i32 %x, -1
   ret i32 %sub
 }
 
+; CHECK-LABEL: define dso_local i32 @f0
+; CHECK:       tail call fastcc i32 @g.[[#A:]]({{.*}}@p0)
+;
+define dso_local i32 @f0(i32 noundef %x) {
+entry:
+  %call = tail call fastcc i32 @g(i32 noundef %x, ptr noundef nonnull @p0)
+  ret i32 %call
+}
+
+; CHECK-LABEL: define dso_local i32 @f1
+; CHECK:       tail call fastcc i32 @g.[[#B:]]({{.*}}@p1)
+;
+define dso_local i32 @f1(i32 noundef %x) {
+entry:
+  %call = tail call fastcc i32 @g(i32 noundef %x, ptr noundef nonnull @p1)
+  ret i32 %call
+}
+
+; @g gets fully specialized
+; CHECK-NOT: define internal fastcc i32 @g(
+
+define internal fastcc i32 @g(i32 noundef %x, ptr nocapture noundef readonly %p) noinline  {
+entry:
+  %pcall = tail call i32 %p(i32 noundef %x)
+  %fcall = tail call fastcc i32 @f(i32 noundef %pcall, ptr noundef nonnull %p)
+  ret i32 %fcall
+}
+
+; CHECK-LABEL: define dso_local i32 @g0
+; CHECK:       tail call fastcc i32 @f.[[#C:]]({{.*}}@p0)
+;
 define dso_local i32 @g0(i32 noundef %x) {
 entry:
   %call = tail call fastcc i32 @f(i32 noundef %x, ptr noundef nonnull @p0)
   ret i32 %call
 }
 
 define internal fastcc i32 @f(i32 noundef %x, ptr nocapture noundef readonly %p) noinline  {
 entry:
   %call = tail call i32 %p(i32 noundef %x)
   %add = add nsw i32 %call, %x
   ret i32 %add
 }
 
+; CHECK-LABEL: define dso_local i32 @g1
+; CHECK:       tail call fastcc i32 @f.[[#D:]]({{.*}}@p1)
+;
 define dso_local i32 @g1(i32 noundef %x) {
 entry:
   %call = tail call fastcc i32 @f(i32 noundef %x, ptr noundef nonnull @p1)
   ret i32 %call
 }
 
 define dso_local i32 @g2(i32 noundef %x) {
 entry:
   %call = tail call fastcc i32 @f(i32 noundef %x, ptr poison)
   ret i32 %call
 }
 
 ; Check that a single argument, that cannot be used for specialisation, does not
 ; prevent specialisation based on other arguments.
-; CHECK: @f.1
-; CHECK: @f.2
+;
+; Also check that for callsites which reside in the body of newly created
+; (specialized) functions, the lattice value of the arguments is known.
+;
+; CHECK-DAG: define internal fastcc i32 @g.[[#A]]
+; CHECK-DAG: define internal fastcc i32 @g.[[#B]]
+; CHECK-DAG: define internal fastcc i32 @f.[[#C]]
+; CHECK-DAG: define internal fastcc i32 @f.[[#D]]

llvm/test/Transforms/FunctionSpecialization/identical-specializations.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes=function-specialization -force-function-specialization -S < %s | FileCheck %s
+; RUN: opt -passes=ipsccp -specialize-functions -force-function-specialization -S < %s | FileCheck %s
 
 define i64 @main(i64 %x, i64 %y, i1 %flag) {
 ; CHECK-LABEL: @main(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br i1 [[FLAG:%.*]], label [[PLUS:%.*]], label [[MINUS:%.*]]
 ; CHECK:       plus:
 ; CHECK-NEXT:    [[CMP0:%.*]] = call i64 @compute.1(i64 [[X:%.*]], i64 [[Y:%.*]], i64 (i64, i64)* @plus, i64 (i64, i64)* @minus)
 ; CHECK-NEXT:    br label [[MERGE:%.*]]

llvm/test/Transforms/FunctionSpecialization/literal-const.ll

-; RUN: opt -S --passes=function-specialization \
+; RUN: opt -S --passes=ipsccp -specialize-functions \
 ; RUN:        -force-function-specialization < %s | FileCheck %s -check-prefix CHECK-NOLIT
-; RUN: opt -S --passes=function-specialization \
+; RUN: opt -S --passes=ipsccp -specialize-functions \
 ; RUN:        -function-specialization-for-literal-constant \
 ; RUN:        -force-function-specialization < %s | FileCheck %s -check-prefix CHECK-LIT
 
 define i32 @f0(i32 noundef %x) {
 entry:
   %call = tail call i32 @neg(i32 noundef %x, i1 noundef zeroext false)
   ret i32 %call
 }

llvm/test/Transforms/FunctionSpecialization/no-spec-unused-arg.ll

-; RUN: opt -S --passes=function-specialization -force-function-specialization -function-specialization-for-literal-constant < %s | FileCheck %s
+; RUN: opt -S --passes=ipsccp -specialize-functions -force-function-specialization -function-specialization-for-literal-constant < %s | FileCheck %s
 define internal i32 @f(i32 %x, i32 %y) noinline {
     ret i32 %x
 }
 
 define i32 @g0() {
     %r = call i32 @f(i32 1, i32 1)
     ret i32 %r
 }
 
 define i32 @g1() {
     %r = call i32 @f(i32 1, i32 2)
     ret i32 %r
 }
 
 ; Check that there are no specialisation of `f`: first parameter is deduced
 ; to be a constant without the need for function specialisation and
 ; the second parameter is unused.
 
-;  CHECK-NOT: @f.
+;  CHECK-NOT: @f.
 

llvm/test/Transforms/FunctionSpecialization/noinline.ll

-; RUN: opt -S --passes=function-specialization < %s | FileCheck %s
+; RUN: opt -S --passes=ipsccp -specialize-functions < %s | FileCheck %s
 define dso_local i32 @p0(i32 noundef %x) {
 entry:
   %add = add nsw i32 %x, 1
   ret i32 %add
 }
 
 define dso_local i32 @p1(i32 noundef %x) {
 entry:

llvm/test/Transforms/FunctionSpecialization/remove-dead-recursive-function.ll

-; RUN: opt -function-specialization -func-specialization-size-threshold=3 -S < %s | FileCheck %s
+; RUN: opt -ipsccp -specialize-functions -func-specialization-size-threshold=3 -S < %s | FileCheck %s
 
 define i64 @main(i64 %x, i1 %flag) {
 entry:
   br i1 %flag, label %plus, label %minus
 
 plus:
   %tmp0 = call i64 @compute(i64 %x, i64 (i64)* @plus)
   br label %merge

llvm/test/Transforms/FunctionSpecialization/specialization-order.ll

-; RUN: opt -S --passes=function-specialization,deadargelim -force-function-specialization < %s | FileCheck %s
+; RUN: opt -S --passes=ipsccp,deadargelim -specialize-functions -force-function-specialization < %s | FileCheck %s
 define dso_local i32 @add(i32 %x, i32 %y) {
 entry:
   %add = add nsw i32 %y, %x
   ret i32 %add
 }
 
 define dso_local i32 @sub(i32 %x, i32 %y) {
 entry:

llvm/test/Transforms/FunctionSpecialization/specialize-multiple-arguments.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -function-specialization -func-specialization-max-clones=0 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=NONE
-; RUN: opt -function-specialization -func-specialization-max-clones=1 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=ONE
-; RUN: opt -function-specialization -func-specialization-max-clones=2 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=TWO
-; RUN: opt -function-specialization -func-specialization-max-clones=3 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=THREE
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-clones=0 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=NONE
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-clones=1 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=ONE
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-clones=2 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=TWO
+; RUN: opt -ipsccp -specialize-functions -func-specialization-max-clones=3 -func-specialization-size-threshold=14 -S < %s | FileCheck %s --check-prefix=THREE
 
 ; Make sure that we iterate correctly after sorting the specializations:
 ; FnSpecialization: Specializations for function compute
 ; FnSpecialization:   Gain = 608
 ; FnSpecialization:   FormalArg = binop1, ActualArg = power
 ; FnSpecialization:   FormalArg = binop2, ActualArg = mul
 ; FnSpecialization:   Gain = 982
 ; FnSpecialization:   FormalArg = binop1, ActualArg = plus

llvm/utils/gn/secondary/llvm/lib/Transforms/Scalar/BUILD.gn

 static_library("Scalar") {
   output_name = "LLVMScalarOpts"
   deps = [
     "//llvm/include/llvm/Config:llvm-config",
     "//llvm/lib/Analysis",
     "//llvm/lib/IR",
     "//llvm/lib/Support",
     "//llvm/lib/Transforms/AggressiveInstCombine",
+    "//llvm/lib/Transforms/IPO",
     "//llvm/lib/Transforms/InstCombine",
     "//llvm/lib/Transforms/Utils",
   ]
   sources = [
     "ADCE.cpp",
     "AlignmentFromAssumptions.cpp",
     "AnnotationRemarks.cpp",
     "BDCE.cpp",