[AArch64] Register passes so they can be run by llc

[rovka]

Summary

Initialize all AArch64-specific passes in the TargetMachine so they can be run
by llc. This can lead to conflicts in opt with some command line options that
share the same name as the pass, so I took this opportunity to do some cleanups:

• rename all relevant command line options from "aarch64-blah" to "aarch64-enable-blah" and update the tests accordingly
• run clang-format on their declarations
• move all these declarations to a common place (the TargetMachine) as opposed to having them scattered around (AArch64BranchRelaxation and AArch64AddressTypePromotion were the only offenders)

[t.p.northover]

Why are only some names conflicting? I'm not quite sure why registering a pass would add an option, at least not in all cases (it seems particularly inappropriate for llc, which doesn't work like that).

It would also be good to be consistent if possible rather than having some with "-aarch64-enable-whatever" and some just "-aarch64-whatever".

[rovka]

In D22868#497831, @t.p.northover wrote:

Why are only some names conflicting? I'm not quite sure why registering a pass would add an option, at least not in all cases (it seems particularly inappropriate for llc, which doesn't work like that).

Right, I wasn't very clear about this - the conflicts are in opt. I'm not particularly excited about the fact that llc and opt work in different ways, nor about the fact that you need to do *some stuff* to see your pass in print-before-all and *some other stuff* so you can use print-before=BlahPass, but I guess that's not the point here.
You only get conflicts in some cases because not all registered passes have an option defined in AArch64TargetMachine [1], or the options have a different name than the one the passes are registered with (e.g. "aarch64-stp-suppress" vs "aarch64-store-pair-suppress").

It would also be good to be consistent if possible rather than having some with "-aarch64-enable-whatever" and some just "-aarch64-whatever".

Now that you mention it, I guess I could get creative about the names we use to register the passes, and then we'd avoid the conflicts in the first place. Alternatively, we could be a bit more tidy and rename all the options to "-aarch64-enable-whatever", because that describes their effect better. Do you have any preference here?

Another inconsistency here is *where* the options are defined - a bunch of them are in AArch64TargetMachine, but that's not the case for all of the passes, e.g. AArch64BranchRelaxation defines the option in its own file. I'm slightly in favor of keeping the option in the same file as the pass. I'm volunteering to move all of them if you think it's a good idea.

Thanks for having a look.

[1] e.g. AFAICT the local-dynamic TLS cleanup pass doesn't have an option; although there is a flag that sounds like it in AArch64ISelLowering, it doesn't seem to affect it

[t.p.northover]

Ah, that makes sense.

Alternatively, we could be a bit more tidy and rename all the options to "-aarch64-enable-whatever", because that describes their effect better. Do you have any preference here?

I think this option is probably simplest.

Another inconsistency here is *where* the options are defined - a bunch of them are in AArch64TargetMachine, but that's not the case for all of the passes, e.g. AArch64BranchRelaxation defines the option in its own file.

I think we've noticed this in the past, but I can't find the thread now. It would be good to get everything consistent. Either way seems about the same to me, so you get to choose as the one doing the work!

Cheers.

Tim.

[rovka]

Went ahead and renamed the command line options and moved them all to TargetMachine. In the end I preferred this solution because there's no point in adding passes to the PassConfig if they've been disabled on the command line.

[t.p.northover]

Thanks Diana! Looks good to me.

[rovka]

r277322.

lib/Target/AArch64/AArch64.h

 FunctionPass *createAArch64AddressTypePromotionPass();
 FunctionPass *createAArch64A57FPLoadBalancing();
 FunctionPass *createAArch64A53Fix835769();
 
 FunctionPass *createAArch64CleanupLocalDynamicTLSPass();
 
 FunctionPass *createAArch64CollectLOHPass();
 
+void initializeAArch64A53Fix835769Pass(PassRegistry&);
+void initializeAArch64A57FPLoadBalancingPass(PassRegistry&);
+void initializeAArch64AddressTypePromotionPass(PassRegistry&);
+void initializeAArch64AdvSIMDScalarPass(PassRegistry&);
+void initializeAArch64BranchRelaxationPass(PassRegistry&);
+void initializeAArch64CollectLOHPass(PassRegistry&);
+void initializeAArch64ConditionalComparesPass(PassRegistry&);
+void initializeAArch64ConditionOptimizerPass(PassRegistry&);
+void initializeAArch64DeadRegisterDefinitionsPass(PassRegistry&);
 void initializeAArch64ExpandPseudoPass(PassRegistry&);
 void initializeAArch64LoadStoreOptPass(PassRegistry&);
+void initializeAArch64PromoteConstantPass(PassRegistry&);
+void initializeAArch64RedundantCopyEliminationPass(PassRegistry&);
+void initializeAArch64StorePairSuppressPass(PassRegistry&);
+void initializeLDTLSCleanupPass(PassRegistry&);
 } // end namespace llvm
 
 #endif

lib/Target/AArch64/AArch64A53Fix835769.cpp

 //===----------------------------------------------------------------------===//
 
 namespace {
 class AArch64A53Fix835769 : public MachineFunctionPass {
   const TargetInstrInfo *TII;
 
 public:
   static char ID;
-  explicit AArch64A53Fix835769() : MachineFunctionPass(ID) {}
+  explicit AArch64A53Fix835769() : MachineFunctionPass(ID) {
+    initializeAArch64A53Fix835769Pass(*PassRegistry::getPassRegistry());
+  }
 
   bool runOnMachineFunction(MachineFunction &F) override;
 
   MachineFunctionProperties getRequiredProperties() const override {
     return MachineFunctionProperties().set(
         MachineFunctionProperties::Property::AllVRegsAllocated);
   }
 
   const char *getPassName() const override {
     return "Workaround A53 erratum 835769 pass";
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesCFG();
     MachineFunctionPass::getAnalysisUsage(AU);
   }
 
 private:
   bool runOnBasicBlock(MachineBasicBlock &MBB);
 };
 char AArch64A53Fix835769::ID = 0;
 
 } // end anonymous namespace
 
+INITIALIZE_PASS(AArch64A53Fix835769, "aarch64-a53-fix-835769",
+                "AArch64 fix for A53 erratum 835769", false, false)
+
 //===----------------------------------------------------------------------===//
 
 bool
 AArch64A53Fix835769::runOnMachineFunction(MachineFunction &F) {
   DEBUG(dbgs() << "***** AArch64A53Fix835769 *****\n");
   bool Changed = false;
   TII = F.getSubtarget().getInstrInfo();
 

lib/Target/AArch64/AArch64A57FPLoadBalancing.cpp

static bool isMla(MachineInstr *MI) {
   case AArch64::FNMSUBDrrr:
   case AArch64::FNMADDDrrr:
     return true;
   default:
     return false;
   }
 }
 
-namespace llvm {
-static void initializeAArch64A57FPLoadBalancingPass(PassRegistry &);
-}
-
 //===----------------------------------------------------------------------===//
 
 namespace {
 /// A "color", which is either even or odd. Yes, these aren't really colors
 /// but the algorithm is conceptually doing two-color graph coloring.
 enum class Color { Even, Odd };
 #ifndef NDEBUG
 static const char *ColorNames[2] = { "Even", "Odd" };

lib/Target/AArch64/AArch64AddressTypePromotion.cpp

 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "aarch64-type-promotion"
 
 static cl::opt<bool>
-EnableAddressTypePromotion("aarch64-type-promotion", cl::Hidden,
+EnableAddressTypePromotion("aarch64-enable-type-promotion", cl::Hidden,
                            cl::desc("Enable the type promotion pass"),
                            cl::init(true));
 static cl::opt<bool>
 EnableMerge("aarch64-type-promotion-merge", cl::Hidden,
             cl::desc("Enable merging of redundant sexts when one is dominating"
                      " the other."),
             cl::init(true));
 
 #define AARCH64_TYPE_PROMO_NAME "AArch64 Address Type Promotion"
 
 //===----------------------------------------------------------------------===//
 //                       AArch64AddressTypePromotion
 //===----------------------------------------------------------------------===//
 
-namespace llvm {
-void initializeAArch64AddressTypePromotionPass(PassRegistry &);
-}
-
 namespace {
 class AArch64AddressTypePromotion : public FunctionPass {
 
 public:
   static char ID;
   AArch64AddressTypePromotion()
       : FunctionPass(ID), Func(nullptr), ConsideredSExtType(nullptr) {
     initializeAArch64AddressTypePromotionPass(*PassRegistry::getPassRegistry());

lib/Target/AArch64/AArch64AdvSIMDScalarPass.cpp

 TransformAll("aarch64-simd-scalar-force-all",
              cl::desc("Force use of AdvSIMD scalar instructions everywhere"),
              cl::init(false), cl::Hidden);
 
 STATISTIC(NumScalarInsnsUsed, "Number of scalar instructions used");
 STATISTIC(NumCopiesDeleted, "Number of cross-class copies deleted");
 STATISTIC(NumCopiesInserted, "Number of cross-class copies inserted");
 
-namespace llvm {
-void initializeAArch64AdvSIMDScalarPass(PassRegistry &);
-}
-
 #define AARCH64_ADVSIMD_NAME "AdvSIMD Scalar Operation Optimization"
 
 namespace {
 class AArch64AdvSIMDScalar : public MachineFunctionPass {
   MachineRegisterInfo *MRI;
   const TargetInstrInfo *TII;
 
 private:

lib/Target/AArch64/AArch64BranchRelaxation.cpp

 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "aarch64-branch-relax"
 
 static cl::opt<bool>
-BranchRelaxation("aarch64-branch-relax", cl::Hidden, cl::init(true),
+BranchRelaxation("aarch64-enable-branch-relax", cl::Hidden, cl::init(true),
                  cl::desc("Relax out of range conditional branches"));
 
 static cl::opt<unsigned>
 TBZDisplacementBits("aarch64-tbz-offset-bits", cl::Hidden, cl::init(14),
                     cl::desc("Restrict range of TB[N]Z instructions (DEBUG)"));
 
 static cl::opt<unsigned>
 CBZDisplacementBits("aarch64-cbz-offset-bits", cl::Hidden, cl::init(19),

lib/Target/AArch64/AArch64CleanupLocalDynamicTLSPass.cpp

 #include "AArch64TargetMachine.h"
 #include "llvm/CodeGen/MachineDominators.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 using namespace llvm;
 
+#define TLSCLEANUP_PASS_NAME "AArch64 Local Dynamic TLS Access Clean-up"
+
 namespace {
 struct LDTLSCleanup : public MachineFunctionPass {
   static char ID;
-  LDTLSCleanup() : MachineFunctionPass(ID) {}
+  LDTLSCleanup() : MachineFunctionPass(ID) {
+    initializeLDTLSCleanupPass(*PassRegistry::getPassRegistry());
+  }
 
   bool runOnMachineFunction(MachineFunction &MF) override {
     if (skipFunction(*MF.getFunction()))
       return false;
 
     AArch64FunctionInfo *AFI = MF.getInfo<AArch64FunctionInfo>();
     if (AFI->getNumLocalDynamicTLSAccesses() < 2) {
       // No point folding accesses if there isn't at least two.
MachineInstr *setRegister(MachineInstr &I, unsigned *TLSBaseAddrReg) {
     MachineInstr *Copy =
         BuildMI(*I.getParent(), ++I.getIterator(), I.getDebugLoc(),
                 TII->get(TargetOpcode::COPY), *TLSBaseAddrReg)
             .addReg(AArch64::X0);
 
     return Copy;
   }
 
-  const char *getPassName() const override {
-    return "Local Dynamic TLS Access Clean-up";
-  }
+  const char *getPassName() const override { return TLSCLEANUP_PASS_NAME; }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesCFG();
     AU.addRequired<MachineDominatorTree>();
     MachineFunctionPass::getAnalysisUsage(AU);
   }
 };
 }
 
+INITIALIZE_PASS(LDTLSCleanup, "aarch64-local-dynamic-tls-cleanup",
+                TLSCLEANUP_PASS_NAME, false, false)
+
 char LDTLSCleanup::ID = 0;
 FunctionPass *llvm::createAArch64CleanupLocalDynamicTLSPass() {
   return new LDTLSCleanup();
 }

lib/Target/AArch64/AArch64CollectLOH.cpp

 STATISTIC(NumADRPToLDR, "Number of simplifiable LDR reachable by ADRP");
 STATISTIC(NumCplxLvl1, "Number of complex case of level 1");
 STATISTIC(NumTooCplxLvl1, "Number of too complex case of level 1");
 STATISTIC(NumCplxLvl2, "Number of complex case of level 2");
 STATISTIC(NumTooCplxLvl2, "Number of too complex case of level 2");
 STATISTIC(NumADRSimpleCandidate, "Number of simplifiable ADRP + ADD");
 STATISTIC(NumADRComplexCandidate, "Number of too complex ADRP + ADD");
 
-namespace llvm {
-void initializeAArch64CollectLOHPass(PassRegistry &);
-}
-
 #define AARCH64_COLLECT_LOH_NAME "AArch64 Collect Linker Optimization Hint (LOH)"
 
 namespace {
 struct AArch64CollectLOH : public MachineFunctionPass {
   static char ID;
   AArch64CollectLOH() : MachineFunctionPass(ID) {
     initializeAArch64CollectLOHPass(*PassRegistry::getPassRegistry());
   }

lib/Target/AArch64/AArch64ConditionOptimizer.cpp

class AArch64ConditionOptimizer : public MachineFunctionPass {
   const MachineRegisterInfo *MRI;
 
 public:
   // Stores immediate, compare instruction opcode and branch condition (in this
   // order) of adjusted comparison.
   typedef std::tuple<int, unsigned, AArch64CC::CondCode> CmpInfo;
 
   static char ID;
-  AArch64ConditionOptimizer() : MachineFunctionPass(ID) {}
+  AArch64ConditionOptimizer() : MachineFunctionPass(ID) {
+    initializeAArch64ConditionOptimizerPass(*PassRegistry::getPassRegistry());
+  }
   void getAnalysisUsage(AnalysisUsage &AU) const override;
   MachineInstr *findSuitableCompare(MachineBasicBlock *MBB);
   CmpInfo adjustCmp(MachineInstr *CmpMI, AArch64CC::CondCode Cmp);
   void modifyCmp(MachineInstr *CmpMI, const CmpInfo &Info);
   bool adjustTo(MachineInstr *CmpMI, AArch64CC::CondCode Cmp, MachineInstr *To,
                 int ToImm);
   bool runOnMachineFunction(MachineFunction &MF) override;
   const char *getPassName() const override {
     return "AArch64 Condition Optimizer";
   }
 };
 } // end anonymous namespace
 
 char AArch64ConditionOptimizer::ID = 0;
 
-namespace llvm {
-void initializeAArch64ConditionOptimizerPass(PassRegistry &);
-}
-
 INITIALIZE_PASS_BEGIN(AArch64ConditionOptimizer, "aarch64-condopt",
                       "AArch64 CondOpt Pass", false, false)
 INITIALIZE_PASS_DEPENDENCY(MachineDominatorTree)
 INITIALIZE_PASS_END(AArch64ConditionOptimizer, "aarch64-condopt",
                     "AArch64 CondOpt Pass", false, false)
 
 FunctionPass *llvm::createAArch64ConditionOptimizerPass() {
   return new AArch64ConditionOptimizer();

lib/Target/AArch64/AArch64ConditionalCompares.cpp

class AArch64ConditionalCompares : public MachineFunctionPass {
   MachineDominatorTree *DomTree;
   MachineLoopInfo *Loops;
   MachineTraceMetrics *Traces;
   MachineTraceMetrics::Ensemble *MinInstr;
   SSACCmpConv CmpConv;
 
 public:
   static char ID;
-  AArch64ConditionalCompares() : MachineFunctionPass(ID) {}
+  AArch64ConditionalCompares() : MachineFunctionPass(ID) {
+    initializeAArch64ConditionalComparesPass(*PassRegistry::getPassRegistry());
+  }
   void getAnalysisUsage(AnalysisUsage &AU) const override;
   bool runOnMachineFunction(MachineFunction &MF) override;
   const char *getPassName() const override {
     return "AArch64 Conditional Compares";
   }
 
 private:
   bool tryConvert(MachineBasicBlock *);
   void updateDomTree(ArrayRef<MachineBasicBlock *> Removed);
   void updateLoops(ArrayRef<MachineBasicBlock *> Removed);
   void invalidateTraces();
   bool shouldConvert();
 };
 } // end anonymous namespace
 
 char AArch64ConditionalCompares::ID = 0;
 
-namespace llvm {
-void initializeAArch64ConditionalComparesPass(PassRegistry &);
-}
-
 INITIALIZE_PASS_BEGIN(AArch64ConditionalCompares, "aarch64-ccmp",
                       "AArch64 CCMP Pass", false, false)
 INITIALIZE_PASS_DEPENDENCY(MachineDominatorTree)
 INITIALIZE_PASS_DEPENDENCY(MachineTraceMetrics)
 INITIALIZE_PASS_END(AArch64ConditionalCompares, "aarch64-ccmp",
                     "AArch64 CCMP Pass", false, false)
 
 FunctionPass *llvm::createAArch64ConditionalCompares() {

lib/Target/AArch64/AArch64DeadRegisterDefinitionsPass.cpp

 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetSubtargetInfo.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "aarch64-dead-defs"
 
 STATISTIC(NumDeadDefsReplaced, "Number of dead definitions replaced");
 
-namespace llvm {
-void initializeAArch64DeadRegisterDefinitionsPass(PassRegistry &);
-}
-
 #define AARCH64_DEAD_REG_DEF_NAME "AArch64 Dead register definitions"
 
 namespace {
 class AArch64DeadRegisterDefinitions : public MachineFunctionPass {
 private:
   const TargetRegisterInfo *TRI;
   bool implicitlyDefinesOverlappingReg(unsigned Reg, const MachineInstr &MI);
   bool processMachineBasicBlock(MachineBasicBlock &MBB);

lib/Target/AArch64/AArch64PromoteConstant.cpp

struct UpdateRecord {
     Instruction *User;
     unsigned Op;
 
     UpdateRecord(Constant *C, Instruction *User, unsigned Op)
         : C(C), User(User), Op(Op) {}
   };
 
   static char ID;
-  AArch64PromoteConstant() : ModulePass(ID) {}
+  AArch64PromoteConstant() : ModulePass(ID) {
+    initializeAArch64PromoteConstantPass(*PassRegistry::getPassRegistry());
+  }
 
   const char *getPassName() const override { return "AArch64 Promote Constant"; }
 
   /// Iterate over the functions and promote the interesting constants into
   /// global variables with module scope.
   bool runOnModule(Module &M) override {
     DEBUG(dbgs() << getPassName() << '\n');
     if (skipModule(M))
static void appendAndTransferDominatedUses(Instruction *NewPt,
     // Erase IPI.
     InsertPts.erase(OldInstr);
   }
 };
 } // end anonymous namespace
 
 char AArch64PromoteConstant::ID = 0;
 
-namespace llvm {
-void initializeAArch64PromoteConstantPass(PassRegistry &);
-}
-
 INITIALIZE_PASS_BEGIN(AArch64PromoteConstant, "aarch64-promote-const",
                       "AArch64 Promote Constant Pass", false, false)
 INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
 INITIALIZE_PASS_END(AArch64PromoteConstant, "aarch64-promote-const",
                     "AArch64 Promote Constant Pass", false, false)
 
 ModulePass *llvm::createAArch64PromoteConstantPass() {
   return new AArch64PromoteConstant();

lib/Target/AArch64/AArch64RedundantCopyElimination.cpp

 #include "llvm/Support/Debug.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "aarch64-copyelim"
 
 STATISTIC(NumCopiesRemoved, "Number of copies removed.");
 
-namespace llvm {
-void initializeAArch64RedundantCopyEliminationPass(PassRegistry &);
-}
-
 namespace {
 class AArch64RedundantCopyElimination : public MachineFunctionPass {
   const MachineRegisterInfo *MRI;
   const TargetRegisterInfo *TRI;
 
 public:
   static char ID;
-  AArch64RedundantCopyElimination() : MachineFunctionPass(ID) {}
+  AArch64RedundantCopyElimination() : MachineFunctionPass(ID) {
+    initializeAArch64RedundantCopyEliminationPass(
+        *PassRegistry::getPassRegistry());
+  }
   bool optimizeCopy(MachineBasicBlock *MBB);
   bool runOnMachineFunction(MachineFunction &MF) override;
   MachineFunctionProperties getRequiredProperties() const override {
     return MachineFunctionProperties().set(
         MachineFunctionProperties::Property::AllVRegsAllocated);
   }
   const char *getPassName() const override {
     return "AArch64 Redundant Copy Elimination";

lib/Target/AArch64/AArch64StorePairSuppress.cpp

 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetInstrInfo.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "aarch64-stp-suppress"
 
+#define STPSUPPRESS_PASS_NAME "AArch64 Store Pair Suppression"
+
 namespace {
 class AArch64StorePairSuppress : public MachineFunctionPass {
   const AArch64InstrInfo *TII;
   const TargetRegisterInfo *TRI;
   const MachineRegisterInfo *MRI;
   TargetSchedModel SchedModel;
   MachineTraceMetrics *Traces;
   MachineTraceMetrics::Ensemble *MinInstr;
 
 public:
   static char ID;
-  AArch64StorePairSuppress() : MachineFunctionPass(ID) {}
-
-  const char *getPassName() const override {
-    return "AArch64 Store Pair Suppression";
+  AArch64StorePairSuppress() : MachineFunctionPass(ID) {
+    initializeAArch64StorePairSuppressPass(*PassRegistry::getPassRegistry());
   }
 
+  const char *getPassName() const override { return STPSUPPRESS_PASS_NAME; }
+
   bool runOnMachineFunction(MachineFunction &F) override;
 
 private:
   bool shouldAddSTPToBlock(const MachineBasicBlock *BB);
 
   bool isNarrowFPStore(const MachineInstr &MI);
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesCFG();
     AU.addRequired<MachineTraceMetrics>();
     AU.addPreserved<MachineTraceMetrics>();
     MachineFunctionPass::getAnalysisUsage(AU);
   }
 };
 char AArch64StorePairSuppress::ID = 0;
 } // anonymous
 
+INITIALIZE_PASS(AArch64StorePairSuppress, "aarch64-store-pair-suppress",
+                STPSUPPRESS_PASS_NAME, false, false)
+
 FunctionPass *llvm::createAArch64StorePairSuppressPass() {
   return new AArch64StorePairSuppress();
 }
 
 /// Return true if an STP can be added to this block without increasing the
 /// critical resource height. STP is good to form in Ld/St limited blocks and
 /// bad to form in float-point limited blocks. This is true independent of the
 /// critical path. If the critical path is longer than the resource height, the

lib/Target/AArch64/AArch64TargetMachine.cpp

 #include "llvm/InitializePasses.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Transforms/Scalar.h"
 using namespace llvm;
 
 static cl::opt<bool>
-EnableCCMP("aarch64-ccmp", cl::desc("Enable the CCMP formation pass"),
+EnableCCMP("aarch64-enable-ccmp", cl::desc("Enable the CCMP formation pass"),
            cl::init(true), cl::Hidden);
 
 static cl::opt<bool> EnableMCR("aarch64-mcr",
                                cl::desc("Enable the machine combiner pass"),
                                cl::init(true), cl::Hidden);
 
 static cl::opt<bool>
 EnableStPairSuppress("aarch64-stp-suppress", cl::desc("Suppress STP for AArch64"),
                      cl::init(true), cl::Hidden);
 
 static cl::opt<bool>
-EnableAdvSIMDScalar("aarch64-simd-scalar", cl::desc("Enable use of AdvSIMD scalar"
-                    " integer instructions"), cl::init(false), cl::Hidden);
+EnableAdvSIMDScalar("aarch64-enable-simd-scalar",
+                    cl::desc("Enable use of AdvSIMD scalar integer instructions"),
+                    cl::init(false), cl::Hidden);
 
 static cl::opt<bool>
-EnablePromoteConstant("aarch64-promote-const", cl::desc("Enable the promote "
-                      "constant pass"), cl::init(true), cl::Hidden);
+EnablePromoteConstant("aarch64-enable-promote-const",
+                      cl::desc("Enable the promote constant pass"),
+                      cl::init(true), cl::Hidden);
 
 static cl::opt<bool>
-EnableCollectLOH("aarch64-collect-loh", cl::desc("Enable the pass that emits the"
-                 " linker optimization hints (LOH)"), cl::init(true),
-                 cl::Hidden);
+EnableCollectLOH("aarch64-enable-collect-loh",
+                 cl::desc("Enable the pass that emits the linker optimization"
+                          "hints (LOH)"), cl::init(true), cl::Hidden);
 
 static cl::opt<bool>
 EnableDeadRegisterElimination("aarch64-dead-def-elimination", cl::Hidden,
                               cl::desc("Enable the pass that removes dead"
                                        " definitons and replaces stores to"
                                        " them with stores to the zero"
                                        " register"),
                               cl::init(true));
EnableAtomicTidy("aarch64-atomic-cfg-tidy", cl::Hidden,
                  cl::init(true));
 
 static cl::opt<bool>
 EnableEarlyIfConversion("aarch64-enable-early-ifcvt", cl::Hidden,
                         cl::desc("Run early if-conversion"),
                         cl::init(true));
 
 static cl::opt<bool>
-EnableCondOpt("aarch64-condopt",
+EnableCondOpt("aarch64-enable-condopt",
               cl::desc("Enable the condition optimizer pass"),
               cl::init(true), cl::Hidden);
 
 static cl::opt<bool>
 EnableA53Fix835769("aarch64-fix-cortex-a53-835769", cl::Hidden,
                 cl::desc("Work around Cortex-A53 erratum 835769"),
                 cl::init(false));
 

 
 extern "C" void LLVMInitializeAArch64Target() {
   // Register the target.
   RegisterTargetMachine<AArch64leTargetMachine> X(TheAArch64leTarget);
   RegisterTargetMachine<AArch64beTargetMachine> Y(TheAArch64beTarget);
   RegisterTargetMachine<AArch64leTargetMachine> Z(TheARM64Target);
   auto PR = PassRegistry::getPassRegistry();
   initializeGlobalISel(*PR);
+  initializeAArch64A53Fix835769Pass(*PR);
+  initializeAArch64A57FPLoadBalancingPass(*PR);
+  initializeAArch64AddressTypePromotionPass(*PR);
+  initializeAArch64AdvSIMDScalarPass(*PR);
+  initializeAArch64BranchRelaxationPass(*PR);
+  initializeAArch64CollectLOHPass(*PR);
+  initializeAArch64ConditionalComparesPass(*PR);
+  initializeAArch64ConditionOptimizerPass(*PR);
+  initializeAArch64DeadRegisterDefinitionsPass(*PR);
   initializeAArch64ExpandPseudoPass(*PR);
   initializeAArch64LoadStoreOptPass(*PR);
+  initializeAArch64PromoteConstantPass(*PR);
+  initializeAArch64RedundantCopyEliminationPass(*PR);
+  initializeAArch64StorePairSuppressPass(*PR);
+  initializeLDTLSCleanupPass(*PR);
 }
 
 //===----------------------------------------------------------------------===//
 // AArch64 Lowering public interface.
 //===----------------------------------------------------------------------===//
 static std::unique_ptr<TargetLoweringObjectFile> createTLOF(const Triple &TT) {
   if (TT.isOSBinFormatMachO())
     return make_unique<AArch64_MachoTargetObjectFile>();

test/CodeGen/AArch64/arm64-AdvSIMD-Scalar.ll

-; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-NOOPT
-; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-OPT
-; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-NOOPT
-; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-OPT
+; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-NOOPT
+; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-OPT
+; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-NOOPT
+; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-OPT
 
 define <2 x i64> @bar(<2 x i64> %a, <2 x i64> %b) nounwind readnone {
 ; CHECK-LABEL: bar:
 ; CHECK: add.2d v[[REG:[0-9]+]], v0, v1
 ; CHECK: add  d[[REG3:[0-9]+]], d[[REG]], d1
 ; CHECK: sub  d[[REG2:[0-9]+]], d[[REG]], d1
 ; Without advanced copy optimization, we end up with cross register
 ; banks copies that cannot be coalesced.

test/CodeGen/AArch64/arm64-addr-type-promotion.ll

-; RUN: llc < %s -mtriple=arm64-apple-ios3.0.0 -aarch64-collect-loh=false | FileCheck %s
+; RUN: llc < %s -mtriple=arm64-apple-ios3.0.0 -aarch64-enable-collect-loh=false \
+; RUN:     | FileCheck %s
 ; rdar://13452552
 ; Disable the collecting of LOH so that the labels do not get in the
 ; way of the NEXT patterns.
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-n32:64-S128"
 
 @block = common global i8* null, align 8
 
 define zeroext i8 @fullGtU(i32 %i1, i32 %i2) {

test/CodeGen/AArch64/arm64-ccmp-heuristics.ll

-; RUN: llc < %s -mcpu=cyclone -verify-machineinstrs -aarch64-ccmp | FileCheck %s
+; RUN: llc < %s -mcpu=cyclone -verify-machineinstrs -aarch64-enable-ccmp \
+; RUN:     | FileCheck %s
 target triple = "arm64-apple-ios7.0.0"
 
 @channelColumns = external global i64
 @channelTracks = external global i64
 @mazeRoute = external hidden unnamed_addr global i8*, align 8
 @TOP = external global i64*
 @BOT = external global i64*
 @netsAssign = external global i64*

test/CodeGen/AArch64/arm64-ccmp.ll

-; RUN: llc < %s -mcpu=cyclone -verify-machineinstrs -aarch64-ccmp -aarch64-stress-ccmp | FileCheck %s
+; RUN: llc < %s -mcpu=cyclone -verify-machineinstrs -aarch64-enable-ccmp \
+; RUN:     -aarch64-stress-ccmp | FileCheck %s
 target triple = "arm64-apple-ios"
 
 ; CHECK: single_same
 ; CHECK: cmp w0, #5
 ; CHECK-NEXT: ccmp w1, #17, #4, ne
 ; CHECK-NEXT: b.ne
 ; CHECK: %if.then
 ; CHECK: bl _foo

test/CodeGen/AArch64/arm64-collect-loh-garbage-crash.ll

-; RUN: llc -mtriple=arm64-apple-ios -O3 -aarch64-collect-loh -aarch64-collect-loh-bb-only=true -aarch64-collect-loh-pre-collect-register=false < %s -o - | FileCheck %s
+; RUN: llc -mtriple=arm64-apple-ios -O3 -aarch64-enable-collect-loh \
+; RUN:     -aarch64-collect-loh-bb-only=true \
+; RUN:     -aarch64-collect-loh-pre-collect-register=false < %s -o - \
+; RUN:     | FileCheck %s
 ; Check that the LOH analysis does not crash when the analysed chained
 ; contains instructions that are filtered out.
 ;
 ; Before the fix for <rdar://problem/16041712>, these cases were removed
 ; from the main container. Now, the deterministic container does not allow
 ; to remove arbitrary values, so we have to live with garbage values.
 ; <rdar://problem/16041712>
 

test/CodeGen/AArch64/arm64-collect-loh-str.ll

-; RUN: llc -mtriple=arm64-apple-ios -O2 -aarch64-collect-loh -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s
+; RUN: llc -mtriple=arm64-apple-ios -O2 -aarch64-enable-collect-loh \
+; RUN:     -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s
 ; Test case for <rdar://problem/15942912>.
 ; AdrpAddStr cannot be used when the store uses same
 ; register as address and value. Indeed, the related
 ; if applied, may completely remove the definition or
 ; at least provide a wrong one (with the offset folded
 ; into the definition).
 
 %struct.anon = type { i32*, i32** }

test/CodeGen/AArch64/arm64-collect-loh.ll

-; RUN: llc -mtriple=arm64-apple-ios -O2 -aarch64-collect-loh -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s
-; RUN: llc -mtriple=arm64-linux-gnu -O2 -aarch64-collect-loh -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s --check-prefix=CHECK-ELF
+; RUN: llc -mtriple=arm64-apple-ios -O2 -aarch64-enable-collect-loh \
+; RUN:     -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s
+; RUN: llc -mtriple=arm64-linux-gnu -O2 -aarch64-enable-collect-loh \
+; RUN:     -aarch64-collect-loh-bb-only=false < %s -o - | FileCheck %s --check-prefix=CHECK-ELF
 
 ; CHECK-ELF-NOT: .loh
 ; CHECK-ELF-NOT: AdrpAdrp
 ; CHECK-ELF-NOT: AdrpAdd
 ; CHECK-ELF-NOT: AdrpLdrGot
 
 @a = internal unnamed_addr global i32 0, align 4
 @b = external global i32

test/CodeGen/AArch64/arm64-neon-add-sub.ll

-; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-none-linux-gnu -mattr=+neon -aarch64-simd-scalar| FileCheck %s
+; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-none-linux-gnu \
+; RUN:     -mattr=+neon -aarch64-enable-simd-scalar| FileCheck %s
 
 define <8 x i8> @add8xi8(<8 x i8> %A, <8 x i8> %B) {
 ;CHECK: add {{v[0-9]+}}.8b, {{v[0-9]+}}.8b, {{v[0-9]+}}.8b
   %tmp3 = add <8 x i8> %A, %B;
   ret <8 x i8> %tmp3
 }
 
 define <16 x i8> @add16xi8(<16 x i8> %A, <16 x i8> %B) {

test/CodeGen/AArch64/arm64-promote-const.ll

 ; Disable machine cse to stress the different path of the algorithm.
 ; Otherwise, we always fall in the simple case, i.e., only one definition.
 ; RUN: llc < %s -mtriple=arm64-apple-ios7.0 -disable-machine-cse -aarch64-stress-promote-const -mcpu=cyclone | FileCheck -check-prefix=PROMOTED %s
 ; The REGULAR run just checks that the inputs passed to promote const expose
 ; the appropriate patterns.
-; RUN: llc < %s -mtriple=arm64-apple-ios7.0 -disable-machine-cse -aarch64-promote-const=false -mcpu=cyclone | FileCheck -check-prefix=REGULAR %s
+; RUN: llc < %s -mtriple=arm64-apple-ios7.0 -disable-machine-cse -aarch64-enable-promote-const=false -mcpu=cyclone | FileCheck -check-prefix=REGULAR %s
 
 %struct.uint8x16x4_t = type { [4 x <16 x i8>] }
 
 ; Constant is a structure
 define %struct.uint8x16x4_t @test1() {
 ; PROMOTED-LABEL: test1:
 ; Promote constant has created a big constant for the whole structure
 ; PROMOTED: adrp [[PAGEADDR:x[0-9]+]], __PromotedConst@PAGE

test/CodeGen/AArch64/global-merge-group-by-use.ll

-; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false -aarch64-collect-loh=false \
-; RUN:   -aarch64-global-merge -global-merge-group-by-use -global-merge-ignore-single-use=false \
-; RUN:   %s -o - | FileCheck %s
+; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false \
+; RUN:   -aarch64-enable-collect-loh=false -aarch64-global-merge \
+; RUN:   -global-merge-group-by-use -global-merge-ignore-single-use=false %s \
+; RUN:   -o - | FileCheck %s
 
 ; We assume that globals of the same size aren't reordered inside a set.
 
 ; Check that we create two MergedGlobal instances for two functions using
 ; disjoint sets of globals
 
 @m1 = internal global i32 0, align 4
 @n1 = internal global i32 0, align 4

test/CodeGen/AArch64/global-merge-ignore-single-use-minsize.ll

-; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false -aarch64-collect-loh=false \
-; RUN:   -O1 -global-merge-group-by-use -global-merge-ignore-single-use \
-; RUN:   %s -o - | FileCheck %s
+; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false \
+; RUN:   -aarch64-enable-collect-loh=false -O1 -global-merge-group-by-use \
+; RUN:   -global-merge-ignore-single-use %s -o - | FileCheck %s
 
 ; Check that, at -O1, we only merge globals used in minsize functions.
 ; We assume that globals of the same size aren't reordered inside a set.
 ; We use -global-merge-ignore-single-use, and thus only expect one merged set.
 
 @m1 = internal global i32 0, align 4
 @n1 = internal global i32 0, align 4
 

test/CodeGen/AArch64/global-merge-ignore-single-use.ll

-; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false -aarch64-collect-loh=false \
-; RUN:   -aarch64-global-merge -global-merge-group-by-use -global-merge-ignore-single-use \
-; RUN:   %s -o - | FileCheck %s
+; RUN: llc -mtriple=aarch64-apple-ios -asm-verbose=false \
+; RUN:   -aarch64-enable-collect-loh=false -aarch64-global-merge \
+; RUN:   -global-merge-group-by-use -global-merge-ignore-single-use %s -o - \
+; RUN:   | FileCheck %s
 
 ; We assume that globals of the same size aren't reordered inside a set.
 
 @m1 = internal global i32 0, align 4
 @n1 = internal global i32 0, align 4
 @o1 = internal global i32 0, align 4
 
 ; CHECK-LABEL: f1: