[lib/LTO] Add a way to run a custom pipeline

[davide]

Summary

We use this in lld, so it's kinda needed for parity with what's used there.
I might try to add an option for that in the gold-plugin as well.

[davide]

Phab doesn't seem to send e-mails today.

[mehdi_amini]

lib/LTO/LTOBackend.cpp

145

Please use better variable name.

(I know the rest of the file doesn't, but we should fix it and not spread this).

[• rafael]

Can you write a test with llvm-lto2?

[mehdi_amini]

lib/LTO/LTOBackend.cpp

145

Instead of taking a config, you can pass directly the string and the bool you need.
Then, it looks like this function is something that will be needed in multiple place in LLVM and could be implemented somewhere else.

165

We always verify the input.

[davide]

Sorry, I was distracted by other things. This should address part of the comments and provide feedback where I have a different opinion. Thanks!

[mehdi_amini]

lib/LTO/LTOBackend.cpp

125

It seems like this should be provided in LLVM somewhere, why would this be different between opt or clang for instance and LTO?

[davide]

lib/LTO/LTOBackend.cpp

145

I changed the arguments to the function. About centralizing somewhere, maybe.
This can probably be shared with opt but I'd like to discuss this with Chandler before exposing a new API. So I'll leave the static helper here for now.

145

I'd rather be consistent with what's there already, and change the whole naming convention in a second pass, if you don't mind.

165

Not in lld, but, OK, I think it's a decent idea, so I made the change.

[mehdi_amini]

lib/LTO/LTOBackend.cpp

177

About centralizing somewhere, maybe.
This can probably be shared with opt but I'd like to discuss this with Chandler before exposing a new API. So I'll leave the static helper here for now.

Sorry, it does not make sense to me, opt has to need the same thing.
(indeed I just checked and it is calling llvm::runPassPipeline. Why aren't you using it?)

I'd rather be consistent with what's there already, and change the whole naming convention in a second pass, if you don't mind.

If you don't want to break consistency, that's fine: first change the names in the whole file, and then update this patch.

[mehdi_amini]

LGTM

include/llvm/LTO/Config.h

struct Config {
   CodeModel::Model CodeModel = CodeModel::Default;
   CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
   unsigned OptLevel = 2;
   bool DisableVerify = false;
 
   /// Disable entirely the optimizer, including importing for ThinLTO
   bool CodeGenOnly = false;
 
+  /// If this field is set, the set of passes run in the middle-end optimizer
+  /// will be the one specified by the string. Only works with the new pass
+  /// manager as the old one doesn't have this ability.
+  std::string OptPipeline;
+
   /// Setting this field will replace target triples in input files with this
   /// triple.
   std::string OverrideTriple;
 
   /// Setting this field will replace unspecified target triples in input files
   /// with this triple.
   std::string DefaultTriple;
 
struct Config {
   Config() {}
   // FIXME: Remove once MSVC can synthesize move ops.
   Config(Config &&X)
       : CPU(std::move(X.CPU)), Features(std::move(X.Features)),
         Options(std::move(X.Options)), MAttrs(std::move(X.MAttrs)),
         RelocModel(std::move(X.RelocModel)), CodeModel(std::move(X.CodeModel)),
         CGOptLevel(std::move(X.CGOptLevel)), OptLevel(std::move(X.OptLevel)),
         DisableVerify(std::move(X.DisableVerify)),
+        OptPipeline(std::move(X.OptPipeline)),
         OverrideTriple(std::move(X.OverrideTriple)),
         DefaultTriple(std::move(X.DefaultTriple)),
         ShouldDiscardValueNames(std::move(X.ShouldDiscardValueNames)),
         DiagHandler(std::move(X.DiagHandler)),
         ResolutionFile(std::move(X.ResolutionFile)),
         PreOptModuleHook(std::move(X.PreOptModuleHook)),
         PostPromoteModuleHook(std::move(X.PostPromoteModuleHook)),
         PostInternalizeModuleHook(std::move(X.PostInternalizeModuleHook)),
         PostImportModuleHook(std::move(X.PostImportModuleHook)),
         PostOptModuleHook(std::move(X.PostOptModuleHook)),
         PreCodeGenModuleHook(std::move(X.PreCodeGenModuleHook)),
         CombinedIndexHook(std::move(X.CombinedIndexHook)) {}
   // FIXME: Remove once MSVC can synthesize move ops.
   Config &operator=(Config &&X) {
     CPU = std::move(X.CPU);
     Features = std::move(X.Features);
     Options = std::move(X.Options);
     MAttrs = std::move(X.MAttrs);
     RelocModel = std::move(X.RelocModel);
     CodeModel = std::move(X.CodeModel);
     CGOptLevel = std::move(X.CGOptLevel);
     OptLevel = std::move(X.OptLevel);
     DisableVerify = std::move(X.DisableVerify);
+    OptPipeline = std::move(X.OptPipeline);
     OverrideTriple = std::move(X.OverrideTriple);
     DefaultTriple = std::move(X.DefaultTriple);
     ShouldDiscardValueNames = std::move(X.ShouldDiscardValueNames);
     DiagHandler = std::move(X.DiagHandler);
     ResolutionFile = std::move(X.ResolutionFile);
     PreOptModuleHook = std::move(X.PreOptModuleHook);
     PostPromoteModuleHook = std::move(X.PostPromoteModuleHook);
     PostInternalizeModuleHook = std::move(X.PostInternalizeModuleHook);

lib/LTO/LLVMBuild.txt

  CodeGen
  Core
  IPO
  InstCombine
  Linker
  MC
  ObjCARC
  Object
+ Passes
  Scalar
  Support
  Target
  TransformUtils

lib/LTO/LTOBackend.cpp

 // This file implements the "backend" phase of LTO, i.e. it performs
 // optimization and code generation on a loaded module. It is generally used
 // internally by the LTO class but can also be used independently, for example
 // to implement a standalone ThinLTO backend.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTOBackend.h"
+#include "llvm/Analysis/AliasAnalysis.h"
+#include "llvm/Analysis/CGSCCPassManager.h"
+#include "llvm/Analysis/LoopPassManager.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/IR/Verifier.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Passes/PassBuilder.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
createTargetMachine(Config &C, StringRef TheTriple, const Target *TheTarget) {
 
   return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
       TheTriple, C.CPU, Features.getString(), C.Options, C.RelocModel,
       C.CodeModel, C.CGOptLevel));
 }
 
 static void runOldPMPasses(Config &C, Module &M, TargetMachine *TM,
                            bool IsThinLto) {
   legacy::PassManager passes;
   passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
 
   PassManagerBuilder PMB;
   PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
   PMB.Inliner = createFunctionInliningPass();
   // Unconditionally verify input since it is not verified before this
   // point and has unknown origin.
   PMB.VerifyInput = true;
   PMB.VerifyOutput = !C.DisableVerify;
   PMB.LoopVectorize = true;
   PMB.SLPVectorize = true;
   PMB.OptLevel = C.OptLevel;
   if (IsThinLto)
     PMB.populateThinLTOPassManager(passes);
   else
     PMB.populateLTOPassManager(passes);
   passes.run(M);
 }
 
+static void runNewPMCustomPasses(Config &C, Module &M, TargetMachine *TM) {
+  PassBuilder PB(TM);
+  AAManager AA;
+  LoopAnalysisManager LAM;
+  FunctionAnalysisManager FAM;
+  CGSCCAnalysisManager CGAM;
+  ModuleAnalysisManager MAM;
+
+  // Register the AA manager first so that our version is the one used.
+  FAM.registerPass([&] { return std::move(AA); });
+
+  // Register all the basic analyses with the managers.
+  PB.registerModuleAnalyses(MAM);
+  PB.registerCGSCCAnalyses(CGAM);
+  PB.registerFunctionAnalyses(FAM);
+  PB.registerLoopAnalyses(LAM);
+  PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
+
+  ModulePassManager MPM;
+  if (!C.DisableVerify)
+    MPM.addPass(VerifierPass());
+
+  // Now, add all the passes we've been requested to.
+  if (!PB.parsePassPipeline(MPM, C.OptPipeline))
+    report_fatal_error("unable to parse pass pipeline description: " +
+                       C.OptPipeline);
+
+  if (!C.DisableVerify)
+    MPM.addPass(VerifierPass());
+  MPM.run(M, MAM);
+}
+
 bool opt(Config &C, TargetMachine *TM, unsigned Task, Module &M,
          bool IsThinLto) {
   M.setDataLayout(TM->createDataLayout());
+  if (C.OptPipeline.empty())
     runOldPMPasses(C, M, TM, IsThinLto);
+  else
+    runNewPMCustomPasses(C, M, TM);
   if (C.PostOptModuleHook && !C.PostOptModuleHook(Task, M))
     return false;
 
   return true;
 }
 
 /// Monolithic LTO does not support caching (yet), this is a convenient wrapper
 /// around AddOutput to workaround this.