Add new MachineFunction property FailsVerification

[foad]

Summary

TargetPassConfig::addPass takes a "bool verifyAfter" argument which lets
you skip machine verification after a particular pass. Unfortunately
this is used in generic code in TargetPassConfig itself to skip
verification after a generic pass, only because some previous target-
specific pass damaged the MIR on that specific target. This is bad
because problems in one target cause lack of verification for all
targets.

This patch replaces that mechanism with a new MachineFunction property
called "FailsVerification" which can be set by (usually target-specific)
passes that are known to introduce problems. Later passes can reset it
again if they are known to clean up the previous problems.

[foad]

This is an RFC to see if people like the idea.

Known issues:

1. I'm not sure if the new "Verifies" property needs to be serialised in .mir files.
2. When you look at the TargetMachine class for a particular target, it's not very easy to see where the "Verifies" flag gets turned off and on -- you have to look into the individual passes. It would be easy to have it turned off at one point in the middle of the pass pipeline, and forget to turn it on again later.

[MatzeB]

As discussed in https://reviews.llvm.org/D111386 I have a feeling that some targets can likely be improved by moving code from addPreEmitPass to addPreEmitPass2. But we should probably discuss this independently of this patch, since I have a feeling that one way or another we won't end up with 100% of things fixed.

I am in favor of this change and just have some nitpicks.

llvm/include/llvm/CodeGen/MachineFunction.h

152–166

Can we use a flag where the default/desired state is not set?

llvm/lib/CodeGen/MachineFunction.cpp

160

See above (when switching to DisableVerification).

llvm/lib/CodeGen/MachineVerifier.cpp

296–297

FWIW: I don't have high hopes for this flag ever disappearing once we introduce it and would tend to drop the comment (but no strong opinion you can keep it if you want).

llvm/lib/Target/AMDGPU/AMDILCFGStructurizer.cpp

130

So, there's this thing I've seen other projects do, where they file tickets in the bugtracker and reference those in code. I know we don't usually do this in LLVM yet, but maybe we can set a new precedent here, what do you think?

llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp

790

As above.

llvm/lib/Target/Hexagon/HexagonVLIWPacketizer.cpp

206

as above

[MatzeB]

In D111397#3050710, @foad wrote:

This is an RFC to see if people like the idea.

Known issues:

1. I'm not sure if the new "Verifies" property needs to be serialised in .mir files.

Yes I think so.

2. When you look at the TargetMachine class for a particular target, it's not very easy to see where the "Verifies" flag gets turned off and on -- you have to look into the individual passes. It would be easy to have it turned off at one point in the middle of the pass pipeline, and forget to turn it on again later.

Well, I don't think we have a better solution. Regardless it's better than what we currently have.

[foad]

Invert the sense of the attribute. Split the patch into two.

[foad]

In D111397#3058959, @MatzeB wrote:

In D111397#3050710, @foad wrote:

Known issues:

1. I'm not sure if the new "Verifies" property needs to be serialised in .mir files.

Yes I think so.

Done.

llvm/lib/CodeGen/MachineVerifier.cpp

296–297

I'm an optimist!

llvm/lib/Target/AMDGPU/AMDILCFGStructurizer.cpp

130

I'm happy to do this in principle.

[MatzeB]

LGTM

[foad]

llvm/lib/Target/AMDGPU/AMDILCFGStructurizer.cpp

130

Done for the AMDGPU SILowerControlFlow case: a129932b0d45949a884cee90726bf90217c2e737
For the Hexagon packetizer case, it seems like there's more debate over whether there's really a "bug" and how it could be "fixed".

[arsenm]

llvm/include/llvm/CodeGen/MIRYamlMapping.h

726 ↗

(On Diff #380315)

This should not need explicit serialization. The verifier runs after mir parsing anyway.

It should also be illegal to clear this after it’s set

[foad]

llvm/include/llvm/CodeGen/MIRYamlMapping.h

726 ↗

(On Diff #380315)

This should not need explicit serialization.

Why? It means "this function has known problems that I haven't fixed yet so please don't run the verifier on it".

It should also be illegal to clear this after it’s set

Why?

[arsenm]

llvm/include/llvm/CodeGen/MIRYamlMapping.h

726 ↗

(On Diff #380315)

Oh, I misread the intent of this change. I assumed this was solving a problem I've run into where a pass introduces a compilation failure, and as a result ends up in some invalid state where the verifier crashes. For example this happens when the register allocator fails and emits an error, but the compilation still needs to continue.

I don't love the idea of allowing functions to fail the verifier, and then be fixed up later. Once the verifier fails a function should be dead.

[MatzeB]

llvm/include/llvm/CodeGen/MIRYamlMapping.h

726 ↗

(On Diff #380315)

Yeah, FailsVerifier is an intentional decision by pass authors, that they are aware that the pass produces MIR that will not pass the verifier.

I am pretty sure everyone would be happier to just force every machine function to very, but it's a practical concern to get to that state. Before this change we had verification completely disable for a number of passes, with this change at least we can enable verification for all passes in general and the instances where this isn't the case target authors can work around by setting the FailsVerification flag and fixing things on their own pace...

llvm/include/llvm/CodeGen/MachineFunction.h

public:
   //  generic instructions have been eliminated; i.e., all instructions are now
   //  target-specific or non-pre-isel generic instructions (e.g., COPY).
   //  Since only pre-isel generic instructions can have generic virtual register
   //  operands, this also means that all generic virtual registers have been
   //  constrained to virtual registers (assigned to register classes) and that
   //  all sizes attached to them have been eliminated.
   // TiedOpsRewritten: The twoaddressinstruction pass will set this flag, it
   //  means that tied-def have been rewritten to meet the RegConstraint.
+  // Verifies: Means that the function is expected to pass machine verification.
+  //  This can be cleared by passes that introduce known problems that have not
+  //  been fixed yet.
   enum class Property : unsigned {
     IsSSA,
     NoPHIs,
     TracksLiveness,
     NoVRegs,
     FailedISel,
     Legalized,
     RegBankSelected,
     Selected,
     TiedOpsRewritten,
-    LastProperty = TiedOpsRewritten,
+    Verifies,
+    LastProperty = Verifies,
   };
 
   bool hasProperty(Property P) const {
     return Properties[static_cast<unsigned>(P)];
   }
 
   MachineFunctionProperties &set(Property P) {
     Properties.set(static_cast<unsigned>(P));

llvm/include/llvm/CodeGen/TargetPassConfig.h

public:
   }
 
   /// Allow the target to override a specific pass without overriding the pass
   /// pipeline. When passes are added to the standard pipeline at the
   /// point where StandardID is expected, add TargetID in its place.
   void substitutePass(AnalysisID StandardID, IdentifyingPassPtr TargetID);
 
   /// Insert InsertedPassID pass after TargetPassID pass.
-  void insertPass(AnalysisID TargetPassID, IdentifyingPassPtr InsertedPassID,
-                  bool VerifyAfter = true);
+  void insertPass(AnalysisID TargetPassID, IdentifyingPassPtr InsertedPassID);
 
   /// Allow the target to enable a specific standard pass by default.
   void enablePass(AnalysisID PassID) { substitutePass(PassID, PassID); }
 
   /// Allow the target to disable a specific standard pass by default.
   void disablePass(AnalysisID PassID) {
     substitutePass(PassID, IdentifyingPassPtr());
   }
public:
   void addCheckDebugPass();
 
   /// Add standard passes before a pass that's about to be added. For example,
   /// the DebugifyMachineModulePass if it is enabled.
   void addMachinePrePasses(bool AllowDebugify = true);
 
   /// Add standard passes after a pass that has just been added. For example,
   /// the MachineVerifier if it is enabled.
-  void addMachinePostPasses(const std::string &Banner, bool AllowVerify = true,
-                            bool AllowStrip = true);
+  void addMachinePostPasses(const std::string &Banner);
 
   /// Check whether or not GlobalISel should abort on error.
   /// When this is disabled, GlobalISel will fall back on SDISel instead of
   /// erroring out.
   bool isGlobalISelAbortEnabled() const;
 
   /// Check whether or not a diagnostic should be emitted when GlobalISel
   /// uses the fallback path. In other words, it will emit a diagnostic
protected:
   // `addPreEmitPass` *should* be but in reality isn't.
   virtual void addPreEmitPass2() {}
 
   /// Utilities for targets to add passes to the pass manager.
   ///
 
   /// Add a CodeGen pass at this point in the pipeline after checking overrides.
   /// Return the pass that was added, or zero if no pass was added.
-  /// @p verifyAfter   if true and adding a machine function pass add an extra
-  ///                  machine verification pass afterwards.
-  AnalysisID addPass(AnalysisID PassID, bool verifyAfter = true);
+  AnalysisID addPass(AnalysisID PassID);
 
   /// Add a pass to the PassManager if that pass is supposed to be run, as
   /// determined by the StartAfter and StopAfter options. Takes ownership of the
   /// pass.
-  /// @p verifyAfter   if true and adding a machine function pass add an extra
-  ///                  machine verification pass afterwards.
-  void addPass(Pass *P, bool verifyAfter = true);
+  void addPass(Pass *P);
 
   /// addMachinePasses helper to create the target-selected or overriden
   /// regalloc pass.
   virtual FunctionPass *createRegAllocPass(bool Optimized);
 
   /// Add core register allocator passes which do the actual register assignment
   /// and rewriting. \returns true if any passes were added.
   virtual bool addRegAssignAndRewriteFast();

llvm/lib/CodeGen/MachineFunction.cpp

static const char *getPropertyName(MachineFunctionProperties::Property Prop) {
   case P::IsSSA: return "IsSSA";
   case P::Legalized: return "Legalized";
   case P::NoPHIs: return "NoPHIs";
   case P::NoVRegs: return "NoVRegs";
   case P::RegBankSelected: return "RegBankSelected";
   case P::Selected: return "Selected";
   case P::TracksLiveness: return "TracksLiveness";
   case P::TiedOpsRewritten: return "TiedOpsRewritten";
+  case P::Verifies: return "Verifies";
   }
   llvm_unreachable("Invalid machine function property");
 }
 
 // Pin the vtable to this file.
 void MachineFunction::Delegate::anchor() {}
 
 void MachineFunctionProperties::print(raw_ostream &OS) const {
void MachineFunction::handleRemoval(MachineInstr &MI) {
   if (TheDelegate)
     TheDelegate->MF_HandleRemoval(MI);
 }
 
 void MachineFunction::init() {
   // Assume the function starts in SSA form with correct liveness.
   Properties.set(MachineFunctionProperties::Property::IsSSA);
   Properties.set(MachineFunctionProperties::Property::TracksLiveness);
+  Properties.set(MachineFunctionProperties::Property::Verifies);
   if (STI->getRegisterInfo())
     RegInfo = new (Allocator) MachineRegisterInfo(this);
   else
     RegInfo = nullptr;
 
   MFInfo = nullptr;
   // We can realign the stack if the target supports it and the user hasn't
   // explicitly asked us not to.

llvm/lib/CodeGen/MachineVerifier.cpp

MachineVerifierPass(std::string banner = std::string())
       }
 
     void getAnalysisUsage(AnalysisUsage &AU) const override {
       AU.setPreservesAll();
       MachineFunctionPass::getAnalysisUsage(AU);
     }
 
     bool runOnMachineFunction(MachineFunction &MF) override {
+      // Skip functions that have known verification problems.
+      // FIXME: Remove this mechanism when all problematic passes have been
+      // fixed.
+      if (!MF.getProperties().hasProperty(
+              MachineFunctionProperties::Property::Verifies))
+        return false;
+
       unsigned FoundErrors = MachineVerifier(this, Banner.c_str()).verify(MF);
       if (FoundErrors)
         report_fatal_error("Found "+Twine(FoundErrors)+" machine code errors.");
       return false;
     }
   };
 
 } // end anonymous namespace

llvm/lib/CodeGen/TargetPassConfig.cpp

INITIALIZE_PASS(TargetPassConfig, "targetpassconfig",
                 "Target Pass Configuration", false, false)
 char TargetPassConfig::ID = 0;
 
 namespace {
 
 struct InsertedPass {
   AnalysisID TargetPassID;
   IdentifyingPassPtr InsertedPassID;
-  bool VerifyAfter;
 
-  InsertedPass(AnalysisID TargetPassID, IdentifyingPassPtr InsertedPassID,
-               bool VerifyAfter)
-      : TargetPassID(TargetPassID), InsertedPassID(InsertedPassID),
-        VerifyAfter(VerifyAfter) {}
+  InsertedPass(AnalysisID TargetPassID, IdentifyingPassPtr InsertedPassID)
+      : TargetPassID(TargetPassID), InsertedPassID(InsertedPassID) {}
 
   Pass *getInsertedPass() const {
     assert(InsertedPassID.isValid() && "Illegal Pass ID!");
     if (InsertedPassID.isInstance())
       return InsertedPassID.getInstance();
     Pass *NP = Pass::createPass(InsertedPassID.getID());
     assert(NP && "Pass ID not registered");
     return NP;

 }
 
 CodeGenOpt::Level TargetPassConfig::getOptLevel() const {
   return TM->getOptLevel();
 }
 
 /// Insert InsertedPassID pass after TargetPassID.
 void TargetPassConfig::insertPass(AnalysisID TargetPassID,
-                                  IdentifyingPassPtr InsertedPassID,
-                                  bool VerifyAfter) {
+                                  IdentifyingPassPtr InsertedPassID) {
   assert(((!InsertedPassID.isInstance() &&
            TargetPassID != InsertedPassID.getID()) ||
           (InsertedPassID.isInstance() &&
            TargetPassID != InsertedPassID.getInstance()->getPassID())) &&
          "Insert a pass after itself!");
-  Impl->InsertedPasses.emplace_back(TargetPassID, InsertedPassID, VerifyAfter);
+  Impl->InsertedPasses.emplace_back(TargetPassID, InsertedPassID);
 }
 
 /// createPassConfig - Create a pass configuration object to be used by
 /// addPassToEmitX methods for generating a pipeline of CodeGen passes.
 ///
 /// Targets may override this to extend TargetPassConfig.
 TargetPassConfig *LLVMTargetMachine::createPassConfig(PassManagerBase &PM) {
   return new TargetPassConfig(*this, PM);
return !FinalPtr.isValid() || FinalPtr.isInstance() ||
       FinalPtr.getID() != ID;
 }
 
 /// Add a pass to the PassManager if that pass is supposed to be run.  If the
 /// Started/Stopped flags indicate either that the compilation should start at
 /// a later pass or that it should stop after an earlier pass, then do not add
 /// the pass.  Finally, compare the current pass against the StartAfter
 /// and StopAfter options and change the Started/Stopped flags accordingly.
-void TargetPassConfig::addPass(Pass *P, bool verifyAfter) {
+void TargetPassConfig::addPass(Pass *P) {
   assert(!Initialized && "PassConfig is immutable");
 
   // Cache the Pass ID here in case the pass manager finds this pass is
   // redundant with ones already scheduled / available, and deletes it.
   // Fundamentally, once we add the pass to the manager, we no longer own it
   // and shouldn't reference it.
   AnalysisID PassID = P->getPassID();
 
   if (StartBefore == PassID && StartBeforeCount++ == StartBeforeInstanceNum)
     Started = true;
   if (StopBefore == PassID && StopBeforeCount++ == StopBeforeInstanceNum)
     Stopped = true;
   if (Started && !Stopped) {
     if (AddingMachinePasses)
       addMachinePrePasses();
     std::string Banner;
     // Construct banner message before PM->add() as that may delete the pass.
-    if (AddingMachinePasses && verifyAfter)
+    if (AddingMachinePasses)
       Banner = std::string("After ") + std::string(P->getPassName());
     PM->add(P);
     if (AddingMachinePasses)
-      addMachinePostPasses(Banner, /*AllowVerify*/ verifyAfter);
+      addMachinePostPasses(Banner);
 
     // Add the passes after the pass P if there is any.
     for (const auto &IP : Impl->InsertedPasses) {
       if (IP.TargetPassID == PassID)
-        addPass(IP.getInsertedPass(), IP.VerifyAfter);
+        addPass(IP.getInsertedPass());
     }
   } else {
     delete P;
   }
 
   if (StopAfter == PassID && StopAfterCount++ == StopAfterInstanceNum)
     Stopped = true;
 
   if (StartAfter == PassID && StartAfterCount++ == StartAfterInstanceNum)
     Started = true;
   if (Stopped && !Started)
     report_fatal_error("Cannot stop compilation after pass that is not run");
 }
 
 /// Add a CodeGen pass at this point in the pipeline after checking for target
 /// and command line overrides.
 ///
 /// addPass cannot return a pointer to the pass instance because is internal the
 /// PassManager and the instance we create here may already be freed.
-AnalysisID TargetPassConfig::addPass(AnalysisID PassID, bool verifyAfter) {
+AnalysisID TargetPassConfig::addPass(AnalysisID PassID) {
   IdentifyingPassPtr TargetID = getPassSubstitution(PassID);
   IdentifyingPassPtr FinalPtr = overridePass(PassID, TargetID);
   if (!FinalPtr.isValid())
     return nullptr;
 
   Pass *P;
   if (FinalPtr.isInstance())
     P = FinalPtr.getInstance();
   else {
     P = Pass::createPass(FinalPtr.getID());
     if (!P)
       llvm_unreachable("Pass ID not registered");
   }
   AnalysisID FinalID = P->getPassID();
-  addPass(P, verifyAfter); // Ends the lifetime of P.
+  addPass(P); // Ends the lifetime of P.
 
   return FinalID;
 }
 
 void TargetPassConfig::printAndVerify(const std::string &Banner) {
   addPrintPass(Banner);
   addVerifyPass(Banner);
 }

 
 void TargetPassConfig::addMachinePrePasses(bool AllowDebugify) {
   if (AllowDebugify && DebugifyIsSafe &&
       (DebugifyAndStripAll == cl::BOU_TRUE ||
        DebugifyCheckAndStripAll == cl::BOU_TRUE))
     addDebugifyPass();
 }
 
-void TargetPassConfig::addMachinePostPasses(const std::string &Banner,
-                                            bool AllowVerify, bool AllowStrip) {
+void TargetPassConfig::addMachinePostPasses(const std::string &Banner) {
   if (DebugifyIsSafe) {
     if (DebugifyCheckAndStripAll == cl::BOU_TRUE) {
       addCheckDebugPass();
       addStripDebugPass();
     } else if (DebugifyAndStripAll == cl::BOU_TRUE)
       addStripDebugPass();
   }
-  if (AllowVerify)
-    addVerifyPass(Banner);
 }
 
 /// Add common target configurable passes that perform LLVM IR to IR transforms
 /// following machine independent optimization.
 void TargetPassConfig::addIRPasses() {
   // Before running any passes, run the verifier to determine if the input
   // coming from the front-end and/or optimizer is valid.
   if (!DisableVerify)
void TargetPassConfig::addMachinePasses() {
 
   if (TM->Options.EnableIPRA)
     // Collect register usage information and produce a register mask of
     // clobbered registers, to be used to optimize call sites.
     addPass(createRegUsageInfoCollector());
 
   // FIXME: Some backends are incompatible with running the verifier after
   // addPreEmitPass.  Maybe only pass "false" here for those targets?
-  addPass(&FuncletLayoutID, false);
+  addPass(&FuncletLayoutID);
 
-  addPass(&StackMapLivenessID, false);
-  addPass(&LiveDebugValuesID, false);
+  addPass(&StackMapLivenessID);
+  addPass(&LiveDebugValuesID);
 
   if (TM->Options.EnableMachineOutliner && getOptLevel() != CodeGenOpt::None &&
       EnableMachineOutliner != RunOutliner::NeverOutline) {
     bool RunOnAllFunctions =
         (EnableMachineOutliner == RunOutliner::AlwaysOutline);
     bool AddOutliner =
         RunOnAllFunctions || TM->Options.SupportsDefaultOutlining;
     if (AddOutliner)

llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

 
 void GCNPassConfig::addFastRegAlloc() {
   // FIXME: We have to disable the verifier here because of PHIElimination +
   // TwoAddressInstructions disabling it.
 
   // This must be run immediately after phi elimination and before
   // TwoAddressInstructions, otherwise the processing of the tied operand of
   // SI_ELSE will introduce a copy of the tied operand source after the else.
-  insertPass(&PHIEliminationID, &SILowerControlFlowID, false);
+  insertPass(&PHIEliminationID, &SILowerControlFlowID);
 
   insertPass(&TwoAddressInstructionPassID, &SIWholeQuadModeID);
   insertPass(&TwoAddressInstructionPassID, &SIPreAllocateWWMRegsID);
 
   TargetPassConfig::addFastRegAlloc();
 }
 
 void GCNPassConfig::addOptimizedRegAlloc() {
void GCNPassConfig::addOptimizedRegAlloc() {
   if (TM->getOptLevel() > CodeGenOpt::Less)
     insertPass(&MachineSchedulerID, &SIFormMemoryClausesID);
 
   // FIXME: when an instruction has a Killed operand, and the instruction is
   // inside a bundle, seems only the BUNDLE instruction appears as the Kills of
   // the register in LiveVariables, this would trigger a failure in verifier,
   // we should fix it and enable the verifier.
   if (OptVGPRLiveRange)
-    insertPass(&LiveVariablesID, &SIOptimizeVGPRLiveRangeID, false);
+    insertPass(&LiveVariablesID, &SIOptimizeVGPRLiveRangeID);
   // This must be run immediately after phi elimination and before
   // TwoAddressInstructions, otherwise the processing of the tied operand of
   // SI_ELSE will introduce a copy of the tied operand source after the else.
-  insertPass(&PHIEliminationID, &SILowerControlFlowID, false);
+  insertPass(&PHIEliminationID, &SILowerControlFlowID);
 
   if (EnableDCEInRA)
     insertPass(&DetectDeadLanesID, &DeadMachineInstructionElimID);
 
   TargetPassConfig::addOptimizedRegAlloc();
 }
 
 bool GCNPassConfig::addPreRewrite() {

llvm/lib/Target/AMDGPU/AMDILCFGStructurizer.cpp

public:
   bool run();
 
   /// Perform the CFG preparation
   /// This step will remove every unconditionnal/dead jump instructions and make
   /// sure all loops have an exit block
   bool prepare();
 
   bool runOnMachineFunction(MachineFunction &MF) override {
+    // FIXME: This pass causes verification failures.
+    MF.getProperties().reset(MachineFunctionProperties::Property::Verifies);
+
     TII = MF.getSubtarget<R600Subtarget>().getInstrInfo();
     TRI = &TII->getRegisterInfo();
     LLVM_DEBUG(MF.dump(););
     OrderedBlks.clear();
     Visited.clear();
     FuncRep = &MF;
     MLI = &getAnalysis<MachineLoopInfo>();
     LLVM_DEBUG(dbgs() << "LoopInfo:\n"; PrintLoopinfo(*MLI););

llvm/lib/Target/AMDGPU/R600TargetMachine.cpp

 bool R600PassConfig::addInstSelector() {
   addPass(createR600ISelDag(&getAMDGPUTargetMachine(), getOptLevel()));
   return false;
 }
 
 void R600PassConfig::addPreRegAlloc() { addPass(createR600VectorRegMerger()); }
 
 void R600PassConfig::addPreSched2() {
-  addPass(createR600EmitClauseMarkers(), false);
+  addPass(createR600EmitClauseMarkers());
   if (EnableR600IfConvert)
-    addPass(&IfConverterID, false);
-  addPass(createR600ClauseMergePass(), false);
+    addPass(&IfConverterID);
+  addPass(createR600ClauseMergePass());
 }
 
 void R600PassConfig::addPreEmitPass() {
-  addPass(createAMDGPUCFGStructurizerPass(), false);
-  addPass(createR600ExpandSpecialInstrsPass(), false);
-  addPass(&FinalizeMachineBundlesID, false);
-  addPass(createR600Packetizer(), false);
-  addPass(createR600ControlFlowFinalizer(), false);
+  addPass(createAMDGPUCFGStructurizerPass());
+  addPass(createR600ExpandSpecialInstrsPass());
+  addPass(&FinalizeMachineBundlesID);
+  addPass(createR600Packetizer());
+  addPass(createR600ControlFlowFinalizer());
 }
 
 TargetPassConfig *R600TargetMachine::createPassConfig(PassManagerBase &PM) {
   return new R600PassConfig(*this, PM);
 }

llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp

if (!GetFallThroughSucc(Succ)) {
               FallThrough->findBranchDebugLoc(), TII->get(AMDGPU::S_BRANCH))
           .addMBB(Succ);
   }
 
   return true;
 }
 
 bool SILowerControlFlow::runOnMachineFunction(MachineFunction &MF) {
+  // FIXME: This pass causes verification failures.
+  MF.getProperties().reset(MachineFunctionProperties::Property::Verifies);
+
   const GCNSubtarget &ST = MF.getSubtarget<GCNSubtarget>();
   TII = ST.getInstrInfo();
   TRI = &TII->getRegisterInfo();
 
   // This doesn't actually need LiveIntervals, but we can preserve them.
   LIS = getAnalysisIfAvailable<LiveIntervals>();
   MDT = getAnalysisIfAvailable<MachineDominatorTree>();
   MRI = &MF.getRegInfo();

llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp

default:
       continue;
     }
     if (SplitPoint)
       splitBlock(MBB, SplitPoint);
   }
 }
 
 bool SIWholeQuadMode::runOnMachineFunction(MachineFunction &MF) {
+  // This pass is a convenient place to re-enable machine verification after the
+  // problems caused by SILowerControlFlow have been fixed.
+  MF.getProperties().set(MachineFunctionProperties::Property::Verifies);
+
   LLVM_DEBUG(dbgs() << "SI Whole Quad Mode on " << MF.getName()
                     << " ------------- \n");
   LLVM_DEBUG(MF.dump(););
 
   Instructions.clear();
   Blocks.clear();
   LiveMaskQueries.clear();
   LowerToCopyInstrs.clear();

llvm/lib/Target/Hexagon/HexagonTargetMachine.cpp

if (!NoOpt) {
     if (!DisableHardwareLoops)
       addPass(createHexagonFixupHwLoops());
     // Generate MUX from pairs of conditional transfers.
     if (EnableGenMux)
       addPass(createHexagonGenMux());
   }
 
   // Packetization is mandatory: it handles gather/scatter at all opt levels.
-  addPass(createHexagonPacketizer(NoOpt), false);
+  addPass(createHexagonPacketizer(NoOpt));
 
   if (EnableVectorPrint)
-    addPass(createHexagonVectorPrint(), false);
+    addPass(createHexagonVectorPrint());
 
   // Add CFI instructions if necessary.
-  addPass(createHexagonCallFrameInformation(), false);
+  addPass(createHexagonCallFrameInformation());
 }

llvm/lib/Target/Hexagon/HexagonVLIWPacketizer.cpp

static MachineBasicBlock::iterator moveInstrOut(MachineInstr &MI,
   MachineInstr &SingleI = *BundleIt->getNextNode();
   SingleI.unbundleFromPred();
   assert(!SingleI.isBundledWithSucc());
   BundleIt->eraseFromParent();
   return NextIt;
 }
 
 bool HexagonPacketizer::runOnMachineFunction(MachineFunction &MF) {
+  // FIXME: This pass causes verification failures.
+  MF.getProperties().reset(MachineFunctionProperties::Property::Verifies);
+
   auto &HST = MF.getSubtarget<HexagonSubtarget>();
   HII = HST.getInstrInfo();
   HRI = HST.getRegisterInfo();
   auto &MLI = getAnalysis<MachineLoopInfo>();
   auto *AA = &getAnalysis<AAResultsWrapperPass>().getAAResults();
   auto *MBPI = &getAnalysis<MachineBranchProbabilityInfo>();
 
   if (EnableGenAllInsnClass)

llvm/lib/Target/MSP430/MSP430TargetMachine.cpp

 bool MSP430PassConfig::addInstSelector() {
   // Install an instruction selector.
   addPass(createMSP430ISelDag(getMSP430TargetMachine(), getOptLevel()));
   return false;
 }
 
 void MSP430PassConfig::addPreEmitPass() {
   // Must run branch selection immediately preceding the asm printer.
-  addPass(createMSP430BranchSelectionPass(), false);
+  addPass(createMSP430BranchSelectionPass());
 }

llvm/lib/Target/NVPTX/NVPTXTargetMachine.cpp

 }
 
 void NVPTXPassConfig::addPreRegAlloc() {
   // Remove Proxy Register pseudo instructions used to keep `callseq_end` alive.
   addPass(createNVPTXProxyRegErasurePass());
 }
 
 void NVPTXPassConfig::addPostRegAlloc() {
-  addPass(createNVPTXPrologEpilogPass(), false);
+  addPass(createNVPTXPrologEpilogPass());
   if (getOptLevel() != CodeGenOpt::None) {
     // NVPTXPrologEpilogPass calculates frame object offset and replace frame
     // index with VRFrame register. NVPTXPeephole need to be run after that and
     // will replace VRFrame with VRFrameLocal when possible.
     addPass(createNVPTXPeephole());
   }
 }
 

llvm/lib/Target/SystemZ/SystemZTargetMachine.cpp

if (getOptLevel() != CodeGenOpt::None)
     addPass(&IfConverterID);
 }
 
 void SystemZPassConfig::addPreEmitPass() {
   // Do instruction shortening before compare elimination because some
   // vector instructions will be shortened into opcodes that compare
   // elimination recognizes.
   if (getOptLevel() != CodeGenOpt::None)
-    addPass(createSystemZShortenInstPass(getSystemZTargetMachine()), false);
+    addPass(createSystemZShortenInstPass(getSystemZTargetMachine()));
 
   // We eliminate comparisons here rather than earlier because some
   // transformations can change the set of available CC values and we
   // generally want those transformations to have priority.  This is
   // especially true in the commonest case where the result of the comparison
   // is used by a single in-range branch instruction, since we will then
   // be able to fuse the compare and the branch instead.
   //
void SystemZPassConfig::addPreEmitPass() {
   //
   // Another reason is that we only want to use BRANCH ON COUNT in cases
   // where we know that the count register is not going to be spilled.
   //
   // Doing it so late makes it more likely that a register will be reused
   // between the comparison and the branch, but it isn't clear whether
   // preventing that would be a win or not.
   if (getOptLevel() != CodeGenOpt::None)
-    addPass(createSystemZElimComparePass(getSystemZTargetMachine()), false);
+    addPass(createSystemZElimComparePass(getSystemZTargetMachine()));
   addPass(createSystemZLongBranchPass(getSystemZTargetMachine()));
 
   // Do final scheduling after all other optimizations, to get an
   // optimal input for the decoder (branch relaxation must happen
   // after block placement).
   if (getOptLevel() != CodeGenOpt::None)
     addPass(&PostMachineSchedulerID);
 }

llvm/lib/Target/XCore/XCoreTargetMachine.cpp

 }
 
 bool XCorePassConfig::addInstSelector() {
   addPass(createXCoreISelDag(getXCoreTargetMachine(), getOptLevel()));
   return false;
 }
 
 void XCorePassConfig::addPreEmitPass() {
-  addPass(createXCoreFrameToArgsOffsetEliminationPass(), false);
+  addPass(createXCoreFrameToArgsOffsetEliminationPass());
 }
 
 // Force static initialization.
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeXCoreTarget() {
   RegisterTargetMachine<XCoreTargetMachine> X(getTheXCoreTarget());
 }
 
 TargetTransformInfo
 XCoreTargetMachine::getTargetTransformInfo(const Function &F) {
   return TargetTransformInfo(XCoreTTIImpl(this, F));
 }