[PM] Add some internal options for testing out the new PM.

[silvas]

Summary

These are internal developer options and so just use report_fatal_error
if an invalid pass pipeline is specified.

This is heavily derived from the code in LLD, but there are a couple
tiny differences that make it hard to share.
We can rip this out once we integrate the new PM into the main
compilation path of course.

I have been using this to build test-suite with the new PM using D21921.
I can successfully build and run test-suite with e.g. the following pipeline:

CMAKE_C_FLAGS:STRING=-Xclang "-middle-end-passes=module(function(sroa,instcombine,simplify-cfg),globaldce,globalopt,deadargelim,function(sroa,instcombine,simplify-cfg),require<profile-summary>,require<targetlibinfo>,cgscc(prune-eh,inline,function-attrs,argpromotion,function(instcombine,simplify-cfg,instcombine,jump-threading,simplify-cfg)))" -Xclang -middle-end-aa-pipeline=basic-aa,scoped-noalias-aa,type-based-aa,globals-aa

I encourage others to try out the new PM and report any bugs!
I've already been bitten by https://llvm.org/bugs/show_bug.cgi?id=28400 in a couple ways!

[davidxl]

include/clang/Driver/CC1Options.td

160

Why not removing 'new' ? Eventually newpm will become pm anyway.

162–163

same here

lib/CodeGen/BackendUtil.cpp

789

Outline this into its own helper method ?

[silvas]

Update for David's comments.

[silvas]

Fix a thinko that broke *all* the clang IRGen tests. Oops.

[silvas]

Slight cleanup.

[silvas]

Change variable name.

[silvas]

Submit a reply that I had left in an "unsubmitted" state for some reason.

include/clang/Driver/CC1Options.td

160

I was originally thinking that these would be deleted once we have the new PM hooked up, but after thinking about it this is generally useful functionality for developers.

[chandlerc]

Uh, so this was sent to llvm-commits, but it really should go to cfe-commits. ;] Not sure if you want to start a fresh thread or what.

[silvas]

Actually add cfe-commits. Oops.

Thanks to Chandler for pointing out I had added llvm-commits instead.

[silvas]

Ping.

[chandlerc]

I'd like someone more active on the Clang side to comment about this, but here is my 2 cents...

It seems like it'd be nice to not populate or run or otherwise build up the legacy pass managers when using this option. I guess I would expect something to more split early between the two pipelines, and then merge back before doing codegen.

I'm also somewhat hesitant with the '-middle-end-...' naming, but not sure what would work better. We don't have anything even comparable with the legacy PM, so maybe just "-llvm-custom-passes" or something similar?

[silvas]

Rebase after r275507.

include/clang/Driver/CC1Options.td

HelpText<"Save temporary labels in the symbol table. "
            "Note this may change .s semantics and shouldn't generally be used "
            "on compiler-generated code.">;
 def mrelocation_model : Separate<["-"], "mrelocation-model">,
   HelpText<"The relocation model to use">;
 def fno_math_builtin : Flag<["-"], "fno-math-builtin">,
   HelpText<"Disable implicit builtin knowledge of math functions">;
 }
 
+def middle_end_passes_EQ
+    : Joined<[ "-" ], "middle-end-passes=">,
+      HelpText<"Override the middle-end passes that are run">;
+def middle_end_aa_pipeline_EQ
+    : Joined<[ "-" ], "middle-end-aa-pipeline=">,
+      HelpText<"Alias analysis pipeline to use with -middle-end-passes=">;
+
 def disable_llvm_optzns : Flag<["-"], "disable-llvm-optzns">,
   HelpText<"Don't run LLVM optimization passes">;
 def disable_llvm_verifier : Flag<["-"], "disable-llvm-verifier">,
   HelpText<"Don't run the LLVM IR verifier pass">;
 def disable_llvm_passes : Flag<["-"], "disable-llvm-passes">,
   HelpText<"Use together with -emit-llvm to get pristine LLVM IR from the "
            "frontend by not running any LLVM passes at all">;
 def disable_red_zone : Flag<["-"], "disable-red-zone">,

include/clang/Frontend/CodeGenOptions.h

public:
   std::string CoverageFile;
 
   /// The version string to put into coverage files.
   char CoverageVersion[4];
 
   /// Enable additional debugging information.
   std::string DebugPass;
 
+  /// Pass description string to override the middle-end pass pipeline.
+  /// This is primarily for developers experimenting with different
+  /// optimization pipelines.
+  /// This will completely replace the middle-end pass pipeline, so
+  /// features that depend on changes to the pipeline (e.g. sanitizers) may
+  /// not work as expected in conjunction with this option.
+  /// The string has the same format as the string passed to opt's
+  /// `-passes=` flag.
+  std::string MiddleEndPasses;
+
+  /// Alias analysis description string to use with MiddleEndPasses.
+  /// This is primarily for developers experimenting with different
+  /// optimization pipelines.
+  /// This option only affects passes run by MiddleEndPasses.
+  /// The string has the same format as the string passed to opt's
+  /// `-aa-pipeline=` flag.
+  std::string MiddleEndAAPipeline;
+
   /// The string to embed in debug information as the current working directory.
   std::string DebugCompilationDir;
 
   /// The string to embed in the debug information for the compile unit, if
   /// non-empty.
   std::string DwarfDebugFlags;
 
   std::map<std::string, std::string> DebugPrefixMap;

lib/CodeGen/BackendUtil.cpp

 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/IRPrintingPasses.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Object/ModuleSummaryIndexObjectFile.h"
+#include "llvm/Passes/PassBuilder.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Target/TargetSubtargetInfo.h"
if (TM->addPassesToEmitFile(*PM, OS, CGFT,
                               /*DisableVerify=*/!CodeGenOpts.VerifyModule)) {
     Diags.Report(diag::err_fe_unable_to_interface_with_target);
     return false;
   }
 
   return true;
 }
 
+static void runMiddleEndPasses(Module &M, TargetMachine *TM, StringRef Passes,
+                               StringRef AAPipeline) {
+  // This is a really low-level internal developer option.
+  // Use report_fatal_error if anything goes wrong.
+
+  PassBuilder PB(TM);
+  AAManager AA;
+  if (!PB.parseAAPipeline(AA, AAPipeline))
+    report_fatal_error("Unable to parse AA pipeline description: " +
+                       AAPipeline);
+
+  LoopAnalysisManager LAM;
+  FunctionAnalysisManager FAM;
+  CGSCCAnalysisManager CGAM;
+  ModuleAnalysisManager MAM;
+
+  // Register the AA manager first so that our version is the one used.
+  FAM.registerPass([&] { return std::move(AA); });
+
+  // Register all the basic analyses with the managers.
+  PB.registerModuleAnalyses(MAM);
+  PB.registerCGSCCAnalyses(CGAM);
+  PB.registerFunctionAnalyses(FAM);
+  PB.registerLoopAnalyses(LAM);
+  PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
+
+  ModulePassManager MPM;
+  MPM.addPass(VerifierPass());
+
+  // Now, add all the passes we've been requested to.
+  if (!PB.parsePassPipeline(MPM, Passes))
+    report_fatal_error("Unable to parse pass pipeline description: " + Passes);
+
+  MPM.addPass(VerifierPass());
+  MPM.run(M, MAM);
+}
+
 void EmitAssemblyHelper::EmitAssembly(BackendAction Action,
                                       raw_pwrite_stream *OS) {
   TimeRegion Region(llvm::TimePassesIsEnabled ? &CodeGenerationTime : nullptr);
 
   setCommandLineOpts();
 
   bool UsesCodeGen = (Action != Backend_EmitNothing &&
                       Action != Backend_EmitBC &&
if (std::error_code EC = IndexOrErr.getError()) {
       errs() << "Error loading index file '" << CodeGenOpts.ThinLTOIndexFile
              << "': " << Error << "\n";
       return;
     }
     ModuleSummary = std::move(IndexOrErr.get());
     assert(ModuleSummary && "Expected non-empty module summary index");
   }
 
+
+  // If we are going to use a custom middle-end pass pipeline, don't add
+  // any of the usual passes.
+  if (!CodeGenOpts.MiddleEndPasses.empty())
     CreatePasses(ModuleSummary.get());
 
   switch (Action) {
   case Backend_EmitNothing:
     break;
 
   case Backend_EmitBC:
     getPerModulePasses()->add(createBitcodeWriterPass(
         *OS, CodeGenOpts.EmitLLVMUseLists, CodeGenOpts.EmitSummaryIndex,
         CodeGenOpts.EmitSummaryIndex));
     break;
 
   case Backend_EmitLL:
     getPerModulePasses()->add(
         createPrintModulePass(*OS, "", CodeGenOpts.EmitLLVMUseLists));
     break;
 
   default:
     if (!AddEmitPasses(Action, *OS))
       return;
   }
 
   // Before executing passes, print the final values of the LLVM options.
   cl::PrintOptionValues();
 
+  if (!CodeGenOpts.MiddleEndPasses.empty()) {
+    // Override the middle-end pass pipeline.
+    // Note that we still run the old PM managers later because they
+    // contain e.g. the passes for printing out a BC or LL file (e.g. for
+    // -flto).
+    // But these passes don't affect the actual optimization because there
+    // is nothing else run in the old PM except for the printing passes.
+
+    runMiddleEndPasses(*TheModule, TM.get(), CodeGenOpts.MiddleEndPasses,
+                       CodeGenOpts.MiddleEndAAPipeline);
+  }
+
   // Run passes. For now we do all passes at once, but eventually we
   // would like to have the option of streaming code generation.
 
   if (PerFunctionPasses) {
     PrettyStackTraceString CrashInfo("Per-function optimization");
 
     PerFunctionPasses->doInitialization();
     for (Function &F : *TheModule)

lib/CodeGen/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Analysis
   BitReader
   BitWriter
   Core
   Coverage
   IPO
   IRReader
   InstCombine
   Instrumentation
   Linker
   MC
   ObjCARCOpts
   Object
+  Passes
   ProfileData
   ScalarOpts
   Support
   Target
   TransformUtils
   )
 
 # In a standard Clang+LLVM build, we need to generate intrinsics before

lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   Opts.CudaGpuBinaryFileNames =
       Args.getAllArgValues(OPT_fcuda_include_gpubinary);
 
   Opts.Backchain = Args.hasArg(OPT_mbackchain);
 
   Opts.EmitCheckPathComponentsToStrip = getLastArgIntValue(
       Args, OPT_fsanitize_undefined_strip_path_components_EQ, 0, Diags);
 
+  if (Arg *A = Args.getLastArg(OPT_middle_end_passes_EQ))
+    Opts.MiddleEndPasses = A->getValue();
+  if (Arg *A = Args.getLastArg(OPT_middle_end_aa_pipeline_EQ))
+    Opts.MiddleEndAAPipeline = A->getValue();
+
   return Success;
 }
 
 static void ParseDependencyOutputArgs(DependencyOutputOptions &Opts,
                                       ArgList &Args) {
   using namespace options;
   Opts.OutputFile = Args.getLastArgValue(OPT_dependency_file);
   Opts.Targets = Args.getAllArgValues(OPT_MT);

test/CodeGen/middle-end-passes.c

+// Check that passes get run.
+
+// RUN: %clang_cc1 %s -emit-llvm -o /dev/null -middle-end-passes=mem2reg,aa-eval 2>&1 | FileCheck %s --check-prefix=NO-AA
+
+// NO-AA: ===== Alias Analysis Evaluator Report =====
+// NO-AA: 0 must alias responses
+
+// RUN: %clang_cc1 %s -emit-llvm -o /dev/null -middle-end-passes=mem2reg,aa-eval -middle-end-aa-pipeline=basic-aa 2>&1 | FileCheck %s --check-prefix=WITH-AA
+
+// WITH-AA: ===== Alias Analysis Evaluator Report =====
+// WITH-AA: 1 must alias responses
+
+
+// The actual output with -flto or -emit-llvm is generated by running a
+// pass in the old PM. Verify that that we still get output.
+
+// RUN: %clang -flto -c %s -o - -Xclang -middle-end-passes=mem2reg 2>&1 | opt -S | FileCheck %s --check-prefix=OUTPUT-IS-GENERATED
+// RUN: %clang -emit-llvm -S %s -o - -Xclang -middle-end-passes=mem2reg 2>&1 | FileCheck %s --check-prefix=OUTPUT-IS-GENERATED
+
+// OUTPUT-IS-GENERATED: @foo
+
+int foo(int *x) {
+  int *y = x + 1;
+  int *z = y - 1;
+  return *x + *z;
+}