[PM] Add some internal options for testing out the new PM.

[silvas]

Summary

These are internal developer options and so just use report_fatal_error
if an invalid pass pipeline is specified.

This is heavily derived from the code in LLD, but there are a couple
tiny differences that make it hard to share.
We can rip this out once we integrate the new PM into the main
compilation path of course.

I have been using this to build test-suite with the new PM using D21921.
I can successfully build and run test-suite with e.g. the following pipeline:

CMAKE_C_FLAGS:STRING=-Xclang "-middle-end-passes=module(function(sroa,instcombine,simplify-cfg),globaldce,globalopt,deadargelim,function(sroa,instcombine,simplify-cfg),require<profile-summary>,require<targetlibinfo>,cgscc(prune-eh,inline,function-attrs,argpromotion,function(instcombine,simplify-cfg,instcombine,jump-threading,simplify-cfg)))" -Xclang -middle-end-aa-pipeline=basic-aa,scoped-noalias-aa,type-based-aa,globals-aa

I encourage others to try out the new PM and report any bugs!
I've already been bitten by https://llvm.org/bugs/show_bug.cgi?id=28400 in a couple ways!

[davidxl]

include/clang/Driver/CC1Options.td

160

Why not removing 'new' ? Eventually newpm will become pm anyway.

162–163

same here

lib/CodeGen/BackendUtil.cpp

753

Outline this into its own helper method ?

[silvas]

Update for David's comments.

[silvas]

Fix a thinko that broke *all* the clang IRGen tests. Oops.

[silvas]

Slight cleanup.

[silvas]

Change variable name.

[silvas]

Submit a reply that I had left in an "unsubmitted" state for some reason.

include/clang/Driver/CC1Options.td

160

I was originally thinking that these would be deleted once we have the new PM hooked up, but after thinking about it this is generally useful functionality for developers.

[chandlerc]

Uh, so this was sent to llvm-commits, but it really should go to cfe-commits. ;] Not sure if you want to start a fresh thread or what.

[silvas]

Actually add cfe-commits. Oops.

Thanks to Chandler for pointing out I had added llvm-commits instead.

[silvas]

Ping.

[chandlerc]

I'd like someone more active on the Clang side to comment about this, but here is my 2 cents...

It seems like it'd be nice to not populate or run or otherwise build up the legacy pass managers when using this option. I guess I would expect something to more split early between the two pipelines, and then merge back before doing codegen.

I'm also somewhat hesitant with the '-middle-end-...' naming, but not sure what would work better. We don't have anything even comparable with the legacy PM, so maybe just "-llvm-custom-passes" or something similar?

[silvas]

Rebase after r275507.

include/clang/Driver/CC1Options.td

HelpText<"Save temporary labels in the symbol table. "
            "Note this may change .s semantics and shouldn't generally be used "
            "on compiler-generated code.">;
 def mrelocation_model : Separate<["-"], "mrelocation-model">,
   HelpText<"The relocation model to use">;
 def fno_math_builtin : Flag<["-"], "fno-math-builtin">,
   HelpText<"Disable implicit builtin knowledge of math functions">;
 }
 
+def newpm_passes_EQ : Joined<["-"], "newpm-passes=">,
+  HelpText<"Pass pipeline description for new PM">;
+def newpm_aa_pipeline_EQ : Joined<["-"], "newpm-aa-pipeline=">,
+  HelpText<"Alias analysis pipeline for the new PM">;
+
 def disable_llvm_optzns : Flag<["-"], "disable-llvm-optzns">,
   HelpText<"Don't run LLVM optimization passes">;
 def disable_llvm_verifier : Flag<["-"], "disable-llvm-verifier">,
   HelpText<"Don't run the LLVM IR verifier pass">;
 def disable_llvm_passes : Flag<["-"], "disable-llvm-passes">,
   HelpText<"Use together with -emit-llvm to get pristine LLVM IR from the "
            "frontend by not running any LLVM passes at all">;
 def disable_red_zone : Flag<["-"], "disable-red-zone">,

include/clang/Frontend/CodeGenOptions.h

public:
   std::string CoverageFile;
 
   /// The version string to put into coverage files.
   char CoverageVersion[4];
 
   /// Enable additional debugging information.
   std::string DebugPass;
 
+  /// Pass description string for the new pass manager.
+  /// This is primarily for experimentation while bringing up the new pass
+  /// manager. It will completely replace the middle-end pipeline and so
+  /// will interact with options that require modifying the optimization
+  /// pipeline (e.g. sanitizers).
+  std::string NewPMPasses;
+
+  /// Alias analysis pipeline for the new pass manager.
+  /// This is primarily for experimentation while bringing up the new pass
+  /// manager. It will completely replace the middle-end pipeline and so
+  /// will interact with options that require modifying the optimization
+  /// pipeline (e.g. sanitizers).
+  std::string NewPMAAPipeline;
+
   /// The string to embed in debug information as the current working directory.
   std::string DebugCompilationDir;
 
   /// The string to embed in the debug information for the compile unit, if
   /// non-empty.
   std::string DwarfDebugFlags;
 
   std::map<std::string, std::string> DebugPrefixMap;

lib/CodeGen/BackendUtil.cpp

 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Object/ModuleSummaryIndexObjectFile.h"
+#include "llvm/Passes/PassBuilder.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Target/TargetSubtargetInfo.h"
static void addSymbolRewriterPass(const CodeGenOptions &Opts,
 
   MPM->add(createRewriteSymbolsPass(DL));
 }
 
 void EmitAssemblyHelper::CreatePasses(ModuleSummaryIndex *ModuleSummary) {
   if (CodeGenOpts.DisableLLVMPasses)
     return;
 
+  // If we are going to parse a new PM pass pipeline, don't add any old PM
+  // passes here.
+  if (!CodeGenOpts.NewPMPasses.empty())
+    return;
+
   unsigned OptLevel = CodeGenOpts.OptimizationLevel;
   CodeGenOptions::InliningMethod Inlining = CodeGenOpts.getInlining();
 
   // Handle disabling of LLVM optimization, where we want to preserve the
   // internal module before any optimization.
   if (CodeGenOpts.DisableLLVMOpts) {
     OptLevel = 0;
     Inlining = CodeGenOpts.NoInlining;
void EmitAssemblyHelper::EmitAssembly(BackendAction Action,
   default:
     if (!AddEmitPasses(Action, *OS))
       return;
   }
 
   // Before executing passes, print the final values of the LLVM options.
   cl::PrintOptionValues();
 
+  if (!CodeGenOpts.NewPMPasses.empty()) {
+    // Use the new PM for the middle end.
+    // Note that we still run the old PM manager later because they contain
+    // e.g. the passes for printing out a BC or LL file (e.g. for -flto).
+    // But there is nothing else run except for the printing passes so this
+    // doesn't affect the actual optimization.
+
+    // This is a really low-level internal developer option.
+    // Use report_fatal_error if anything goes wrong.
+
+    PassBuilder PB(TM.get());
+    AAManager AA;
+    if (!PB.parseAAPipeline(AA, CodeGenOpts.NewPMAAPipeline))
+      report_fatal_error("Unable to parse AA pipeline description: " +
+                         CodeGenOpts.NewPMAAPipeline);
+
+    LoopAnalysisManager LAM;
+    FunctionAnalysisManager FAM;
+    CGSCCAnalysisManager CGAM;
+    ModuleAnalysisManager MAM;
+
+    // Register the AA manager first so that our version is the one used.
+    FAM.registerPass([&] { return std::move(AA); });
+
+    // Register all the basic analyses with the managers.
+    PB.registerModuleAnalyses(MAM);
+    PB.registerCGSCCAnalyses(CGAM);
+    PB.registerFunctionAnalyses(FAM);
+    PB.registerLoopAnalyses(LAM);
+    PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
+
+    ModulePassManager MPM;
+    MPM.addPass(VerifierPass());
+
+    // Now, add all the passes we've been requested to.
+    if (!PB.parsePassPipeline(MPM, CodeGenOpts.NewPMPasses))
+      report_fatal_error("unable to parse pass pipeline description: " +
+                         CodeGenOpts.NewPMPasses);
+
+    MPM.addPass(VerifierPass());
+    MPM.run(*TheModule, MAM);
+  }
+
   // Run passes. For now we do all passes at once, but eventually we
   // would like to have the option of streaming code generation.
 
   if (PerFunctionPasses) {
     PrettyStackTraceString CrashInfo("Per-function optimization");
 
     PerFunctionPasses->doInitialization();
     for (Function &F : *TheModule)

lib/CodeGen/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Analysis
   BitReader
   BitWriter
   Core
   Coverage
   IPO
   IRReader
   InstCombine
   Instrumentation
   Linker
   MC
   ObjCARCOpts
   Object
+  Passes
   ProfileData
   ScalarOpts
   Support
   Target
   TransformUtils
   )
 
 # In a standard Clang+LLVM build, we need to generate intrinsics before

lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   Opts.CudaGpuBinaryFileNames =
       Args.getAllArgValues(OPT_fcuda_include_gpubinary);
 
   Opts.Backchain = Args.hasArg(OPT_mbackchain);
 
   Opts.EmitCheckPathComponentsToStrip = getLastArgIntValue(
       Args, OPT_fsanitize_undefined_strip_path_components_EQ, 0, Diags);
 
+  if (Arg *A = Args.getLastArg(OPT_newpm_passes_EQ))
+    Opts.NewPMPasses = A->getValue();
+  if (Arg *A = Args.getLastArg(OPT_newpm_aa_pipeline_EQ))
+    Opts.NewPMAAPipeline = A->getValue();
+
   return Success;
 }
 
 static void ParseDependencyOutputArgs(DependencyOutputOptions &Opts,
                                       ArgList &Args) {
   using namespace options;
   Opts.OutputFile = Args.getLastArgValue(OPT_dependency_file);
   Opts.Targets = Args.getAllArgValues(OPT_MT);

test/CodeGen/newpm-passes.c

+// Check that passes get run.
+
+// RUN: %clang_cc1 %s -emit-llvm -o /dev/null -newpm-passes=mem2reg,aa-eval 2>&1 | FileCheck %s --check-prefix=NO-AA
+
+// NO-AA: ===== Alias Analysis Evaluator Report =====
+// NO-AA: 0 must alias responses
+
+// RUN: %clang_cc1 %s -emit-llvm -o /dev/null -newpm-passes=mem2reg,aa-eval -newpm-aa-pipeline=basic-aa 2>&1 | FileCheck %s --check-prefix=WITH-AA
+
+// WITH-AA: ===== Alias Analysis Evaluator Report =====
+// WITH-AA: 1 must alias responses
+
+
+// The actual output with -flto or -emit-llvm is generated by running a
+// pass in the old PM. Verify that that we still get output.
+
+// RUN: %clang -flto -c %s -o - -Xclang -newpm-passes=mem2reg 2>&1 | opt -S | FileCheck %s --check-prefix=OUTPUT-IS-GENERATED
+// RUN: %clang -emit-llvm -S %s -o - -Xclang -newpm-passes=mem2reg 2>&1 | FileCheck %s --check-prefix=OUTPUT-IS-GENERATED
+
+// OUTPUT-IS-GENERATED: @foo
+
+int foo(int *x) {
+  int *y = x + 1;
+  int *z = y - 1;
+  return *x + *z;
+}