[lib/LTO] Add a way to run a custom pipeline

[davide]

Summary

We use this in lld, so it's kinda needed for parity with what's used there.
I might try to add an option for that in the gold-plugin as well.

[davide]

Phab doesn't seem to send e-mails today.

[mehdi_amini]

lib/LTO/LTOBackend.cpp

180

Please use better variable name.

(I know the rest of the file doesn't, but we should fix it and not spread this).

[• rafael]

Can you write a test with llvm-lto2?

[mehdi_amini]

lib/LTO/LTOBackend.cpp

180

Instead of taking a config, you can pass directly the string and the bool you need.
Then, it looks like this function is something that will be needed in multiple place in LLVM and could be implemented somewhere else.

200

We always verify the input.

[davide]

Sorry, I was distracted by other things. This should address part of the comments and provide feedback where I have a different opinion. Thanks!

[mehdi_amini]

lib/LTO/LTOBackend.cpp

125

It seems like this should be provided in LLVM somewhere, why would this be different between opt or clang for instance and LTO?

[davide]

lib/LTO/LTOBackend.cpp

180

I changed the arguments to the function. About centralizing somewhere, maybe.
This can probably be shared with opt but I'd like to discuss this with Chandler before exposing a new API. So I'll leave the static helper here for now.

180

I'd rather be consistent with what's there already, and change the whole naming convention in a second pass, if you don't mind.

200

Not in lld, but, OK, I think it's a decent idea, so I made the change.

[mehdi_amini]

lib/LTO/LTOBackend.cpp

180

About centralizing somewhere, maybe.
This can probably be shared with opt but I'd like to discuss this with Chandler before exposing a new API. So I'll leave the static helper here for now.

Sorry, it does not make sense to me, opt has to need the same thing.
(indeed I just checked and it is calling llvm::runPassPipeline. Why aren't you using it?)

I'd rather be consistent with what's there already, and change the whole naming convention in a second pass, if you don't mind.

If you don't want to break consistency, that's fine: first change the names in the whole file, and then update this patch.

[mehdi_amini]

LGTM

include/llvm/LTO/Config.h

struct Config {
   CodeModel::Model CodeModel = CodeModel::Default;
   CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
   unsigned OptLevel = 2;
   bool DisableVerify = false;
 
   /// Disable entirely the optimizer, including importing for ThinLTO
   bool CodeGenOnly = false;
 
+  /// If this field is set, the set of passes run in the middle-end optimizer
+  /// will be the one specified by the string. Only works with the new pass
+  /// manager as the old one doesn't have this ability.
+  std::string OptPipeline;
+
   /// Setting this field will replace target triples in input files with this
   /// triple.
   std::string OverrideTriple;
 
   /// Setting this field will replace unspecified target triples in input files
   /// with this triple.
   std::string DefaultTriple;
 
struct Config {
   Config() {}
   // FIXME: Remove once MSVC can synthesize move ops.
   Config(Config &&X)
       : CPU(std::move(X.CPU)), Features(std::move(X.Features)),
         Options(std::move(X.Options)), MAttrs(std::move(X.MAttrs)),
         RelocModel(std::move(X.RelocModel)), CodeModel(std::move(X.CodeModel)),
         CGOptLevel(std::move(X.CGOptLevel)), OptLevel(std::move(X.OptLevel)),
         DisableVerify(std::move(X.DisableVerify)),
+        OptPipeline(std::move(X.OptPipeline)),
         OverrideTriple(std::move(X.OverrideTriple)),
         DefaultTriple(std::move(X.DefaultTriple)),
         ShouldDiscardValueNames(std::move(X.ShouldDiscardValueNames)),
         DiagHandler(std::move(X.DiagHandler)),
         ResolutionFile(std::move(X.ResolutionFile)),
         PreOptModuleHook(std::move(X.PreOptModuleHook)),
         PostPromoteModuleHook(std::move(X.PostPromoteModuleHook)),
         PostInternalizeModuleHook(std::move(X.PostInternalizeModuleHook)),
         PostImportModuleHook(std::move(X.PostImportModuleHook)),
         PostOptModuleHook(std::move(X.PostOptModuleHook)),
         PreCodeGenModuleHook(std::move(X.PreCodeGenModuleHook)),
         CombinedIndexHook(std::move(X.CombinedIndexHook)) {}
   // FIXME: Remove once MSVC can synthesize move ops.
   Config &operator=(Config &&X) {
     CPU = std::move(X.CPU);
     Features = std::move(X.Features);
     Options = std::move(X.Options);
     MAttrs = std::move(X.MAttrs);
     RelocModel = std::move(X.RelocModel);
     CodeModel = std::move(X.CodeModel);
     CGOptLevel = std::move(X.CGOptLevel);
     OptLevel = std::move(X.OptLevel);
     DisableVerify = std::move(X.DisableVerify);
+    OptPipeline = std::move(X.OptPipeline);
     OverrideTriple = std::move(X.OverrideTriple);
     DefaultTriple = std::move(X.DefaultTriple);
     ShouldDiscardValueNames = std::move(X.ShouldDiscardValueNames);
     DiagHandler = std::move(X.DiagHandler);
     ResolutionFile = std::move(X.ResolutionFile);
     PreOptModuleHook = std::move(X.PreOptModuleHook);
     PostPromoteModuleHook = std::move(X.PostPromoteModuleHook);
     PostInternalizeModuleHook = std::move(X.PostInternalizeModuleHook);

lib/LTO/LLVMBuild.txt

  CodeGen
  Core
  IPO
  InstCombine
  Linker
  MC
  ObjCARC
  Object
+ Passes
  Scalar
  Support
  Target
  TransformUtils

lib/LTO/LTOBackend.cpp

 // This file implements the "backend" phase of LTO, i.e. it performs
 // optimization and code generation on a loaded module. It is generally used
 // internally by the LTO class but can also be used independently, for example
 // to implement a standalone ThinLTO backend.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/LTOBackend.h"
+#include "llvm/Analysis/AliasAnalysis.h"
+#include "llvm/Analysis/CGSCCPassManager.h"
+#include "llvm/Analysis/LoopPassManager.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/ReaderWriter.h"
 #include "llvm/IR/LegacyPassManager.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/IR/Verifier.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Passes/PassBuilder.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
createTargetMachine(Config &C, StringRef TheTriple, const Target *TheTarget) {
   for (const std::string &A : C.MAttrs)
     Features.AddFeature(A);
 
   return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
       TheTriple, C.CPU, Features.getString(), C.Options, C.RelocModel,
       C.CodeModel, C.CGOptLevel));
 }
 
+static void runNewPMCustomPasses(Module &M, TargetMachine *TM,
+                                 std::string PipelineDesc,
+                                 bool DisableVerify) {
+  PassBuilder PB(TM);
+  AAManager AA;
+  LoopAnalysisManager LAM;
+  FunctionAnalysisManager FAM;
+  CGSCCAnalysisManager CGAM;
+  ModuleAnalysisManager MAM;
+
+  // Register the AA manager first so that our version is the one used.
+  FAM.registerPass([&] { return std::move(AA); });
+
+  // Register all the basic analyses with the managers.
+  PB.registerModuleAnalyses(MAM);
+  PB.registerCGSCCAnalyses(CGAM);
+  PB.registerFunctionAnalyses(FAM);
+  PB.registerLoopAnalyses(LAM);
+  PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
+
+  ModulePassManager MPM;
+
+  // Always verify the input.
+  MPM.addPass(VerifierPass());
+
+  // Now, add all the passes we've been requested to.
+  if (!PB.parsePassPipeline(MPM, PipelineDesc))
+    report_fatal_error("unable to parse pass pipeline description: " +
+                       PipelineDesc);
+
+  if (!DisableVerify)
+    MPM.addPass(VerifierPass());
+  MPM.run(M, MAM);
+}
+
 static void runOldPMPasses(Config &C, Module &M, TargetMachine *TM,
                            bool IsThinLto) {
   legacy::PassManager passes;
   passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
 
   PassManagerBuilder PMB;
   PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
   PMB.Inliner = createFunctionInliningPass();
   // Unconditionally verify input since it is not verified before this
   // point and has unknown origin.
   PMB.VerifyInput = true;
   PMB.VerifyOutput = !C.DisableVerify;
   PMB.LoopVectorize = true;
   PMB.SLPVectorize = true;
   PMB.OptLevel = C.OptLevel;
   if (IsThinLto)
     PMB.populateThinLTOPassManager(passes);
   else
     PMB.populateLTOPassManager(passes);
   passes.run(M);
 }
 
 bool opt(Config &C, TargetMachine *TM, unsigned Task, Module &M,
          bool IsThinLto) {
   M.setDataLayout(TM->createDataLayout());
+  if (C.OptPipeline.empty())
     runOldPMPasses(C, M, TM, IsThinLto);
+  else
+    runNewPMCustomPasses(M, TM, C.OptPipeline, C.DisableVerify);
   return !C.PostOptModuleHook || C.PostOptModuleHook(Task, M);
 }
 
 /// Monolithic LTO does not support caching (yet), this is a convenient wrapper
 /// around AddOutput to workaround this.
 static AddOutputFn getUncachedOutputWrapper(AddOutputFn &AddOutput,
                                             unsigned Task) {
   return [Task, &AddOutput](unsigned TaskId) {
     auto Output = AddOutput(Task);
     if (Output->isCachingEnabled() && Output->tryLoadFromCache(""))
       report_fatal_error("Cache hit without a valid key?");
     assert(Task == TaskId && "Unexpexted TaskId mismatch");
     return Output;
   };
 }
 
 void codegen(Config &C, TargetMachine *TM, AddOutputFn AddOutput, unsigned Task,
              Module &M) {
   if (C.PreCodeGenModuleHook && !C.PreCodeGenModuleHook(Task, M))
     return;
 
   auto Output = AddOutput(Task);

test/tools/llvm-lto2/X86/pipeline.ll

+; RUN: llvm-as < %s > %t1.bc
+
+; Try a custom pipeline
+; RUN: llvm-lto2 %t1.bc -o %t.o -save-temps \
+; RUN:  -r %t1.bc,patatino,px -opt-pipeline loweratomic
+; RUN: llvm-dis < %t.o.0.4.opt.bc | FileCheck %s --check-prefix=CUSTOM
+
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @patatino() {
+  fence seq_cst
+  ret void
+}
+
+; CUSTOM: define void @patatino() {
+; CUSTOM-NEXT:   ret void
+; CUSTOM-NEXT: }
+
+; Check that invalid pipeline are caught as errors.
+; RUN: not llvm-lto2 %t1.bc -o %t.o -save-temps \
+; RUN:  -r %t1.bc,patatino,px -opt-pipeline foogoo 2>&1 | \
+; RUN:  FileCheck %s --check-prefix=ERR
+
+; ERR: LLVM ERROR: unable to parse pass pipeline description: foogoo

tools/llvm-lto2/llvm-lto2.cpp

 
 static cl::opt<std::string> OutputFilename("o", cl::Required,
                                            cl::desc("Output filename"),
                                            cl::value_desc("filename"));
 
 static cl::opt<std::string> CacheDir("cache-dir", cl::desc("Cache Directory"),
                                      cl::value_desc("directory"));
 
+static cl::opt<std::string> OptPipeline("opt-pipeline",
+                                        cl::desc("Optimizer Pipeline"),
+                                        cl::value_desc("pipeline"));
+
 static cl::opt<bool> SaveTemps("save-temps", cl::desc("Save temporary files"));
 
 static cl::opt<bool>
     ThinLTODistributedIndexes("thinlto-distributed-indexes", cl::init(false),
                               cl::desc("Write out individual index and "
                                        "import files for the "
                                        "distributed backend case"));
 
int main(int argc, char **argv) {
   Conf.DiagHandler = [](const DiagnosticInfo &) {
     exit(1);
   };
 
   if (SaveTemps)
     check(Conf.addSaveTemps(OutputFilename + "."),
           "Config::addSaveTemps failed");
 
+  // Run a custom pipeline, if asked for.
+  Conf.OptPipeline = OptPipeline;
+
   ThinBackend Backend;
   if (ThinLTODistributedIndexes)
     Backend = createWriteIndexesThinBackend("", "", true, "");
   else
     Backend = createInProcessThinBackend(Threads);
   LTO Lto(std::move(Conf), std::move(Backend));
 
   bool HasErrors = false;