Add a TargetMachineVerifier that runs along with the existing MachineVerifier

[dsanders]

Summary

Targets often have constraints on their MIR that need to hold true throughout
certain regions of the backend but cannot be verified in the MachineVerifier.
Some examples include:

• Immediates must be in a certain range on a given instruction
• Certain instructions must not appear for certain subtargets
• Certain instructions must not appear in delay slots
• Fences need to occur before a some types of load but fence/load/load is preferable to fence/load/fence/load

and so on.

Sometimes, it's enough to roll verification into the pass that does the
relevant work as there's no chance of something else interfering but sometimes
there's also a risk of subsequent passes violating constraints on the MIR due
to assumptions that turn out to be untrue or just simply bugs.

This patch expands on the existing MachineVerifier mechanism by introducing
a TargetMachineVerifier that installs itself into -verify-machineinstrs and
MachineFunction::verify() and therefore runs after every pass or more
frequently if requested. The generic MachineVerifier runs first so it should be
safe for the TargetMachineVerifier to assume that the MIR is generally ok and
that defs dominate uses, etc, etc.

There's no specific mechanism for having certain verifications apply beyond (or
until) a certain point in the backend but I'd recommend using function
attributes set by a given pass (like GlobalISel) or the equivalent in
MachineFunctionInfo to achieve this.

There's a simple sample verification in AArch64 but my main focus is an out of
tree target so I haven't gone very far with verifying AArch64. I'm posting this
to see if there's general interest in having this mechanism upstream.

Long term, I foresee a VerifyMethod (to go with PrintMethod, DecodeMethod, etc.)
on Instructions and Operands in tablegen that populate this pass with sanity
checks to detect when C++ code violates constraints defined in tablegen. I'm
aware that some targets are exploiting the lack of checks to do things that
conflict with tablegen definitions so such a thing will have to be opt-in.

[dsanders]

Remove a static cast that's not necessary on trunk

[dsanders]

Missed the *& left over in the last update

[arsenm]

lib/Target/AArch64/AArch64MachineVerifier.cpp

80–93

While I do think there is a need for a target verifier pass, you don't need it to verify instruction level things like this. You can already do that in TargetInstrInfo::verifyInstruction

unittests/CodeGen/GlobalISel/PatternMatchTest.cpp

531

EXPECT_FALSE

[arsenm]

The current target verification problems I have are:

• Verifying legal register bank mappings, so I don't have to defend against them in InstructionSelect
• Verifying G_INTRINSIC operand types and numbers. There's missing API to do this.
• Verifying some target IR constraints, in an IR verifier
• Verifying some special block level properties for exec masking, which is the only category I would probably want to put in this pass

[dsanders]

In D64650#1583208, @arsenm wrote:

The current target verification problems I have are:

• Verifying legal register bank mappings, so I don't have to defend against them in InstructionSelect
• Verifying G_INTRINSIC operand types and numbers. There's missing API to do this.
• Verifying some target IR constraints, in an IR verifier
• Verifying some special block level properties for exec masking, which is the only category I would probably want to put in this pass

Yep, the first two sound like a job for verifyInstruction() and the third sounds like a one-shot pass before IRTranslator/SelectionDAG. The fourth does sound like it makes sense for this pass.

What's missing for the G_INTRINSIC one?

lib/Target/AArch64/AArch64MachineVerifier.cpp

80–93

I didn't know about that function. That will work nicely for the trivial checks on my list. Thanks.

[dsanders]

ASSERT_FALSE -> EXPECT_FALSE
fixed an compile error in the unit test caused by an API change

[arsenm]

In D64650#1583533, @dsanders wrote:

In D64650#1583208, @arsenm wrote:

The current target verification problems I have are:

• Verifying legal register bank mappings, so I don't have to defend against them in InstructionSelect
• Verifying G_INTRINSIC operand types and numbers. There's missing API to do this.

The only Intrinsic:: functions only deal with IR-level information. I didn't see a way to figure out the argument list without getting a concrete Function declaration. That would require converting from the argument LLTs to Type*, but you also don't know the context of which operands are needed for mangling without hardcoding it for specific intrinsics.

[arsenm]

Having it as an independent pass may be kind of annoying. Having verifier runs in -print-before-all/-print-after-all is annoying enough already, and this would double them

[jfb]

This comment was removed by jfb.

[dsanders]

In D64650#1583590, @arsenm wrote:

Having it as an independent pass may be kind of annoying. Having verifier runs in -print-before-all/-print-after-all is annoying enough already, and this would double them

The MachineVerifier itself is supposed to prevent it by calling PM->add() directly to add itself instead of going through addPass() but PM is private so I wasn't able to use the same method. Along the same lines the false, false in:

addPass(createAArch64MachineVerifier(Banner), false, false);

is supposed to prevent that. One of them prevents verifiers being added after the pass while the other prevents printers being added after the pass. Interestingly, TargetMachineVerifier successfully elides the printer on an older branch but not on trunk, while MachineVerifier fails on both. I'll take a look.

In D64650#1583566, @arsenm wrote:

In D64650#1583533, @dsanders wrote:

In D64650#1583208, @arsenm wrote:

The current target verification problems I have are:

• Verifying legal register bank mappings, so I don't have to defend against them in InstructionSelect
• Verifying G_INTRINSIC operand types and numbers. There's missing API to do this.

The only Intrinsic:: functions only deal with IR-level information. I didn't see a way to figure out the argument list without getting a concrete Function declaration. That would require converting from the argument LLTs to Type*, but you also don't know the context of which operands are needed for mangling without hardcoding it for specific intrinsics.

Ah ok. I haven't encountered a case where we need to reach back to the original argument list so I don't have any ideas for this to hand.

[dsanders]

Interestingly, TargetMachineVerifier successfully elides the printer on an older branch but not on trunk, while MachineVerifier fails on both.

Sorry, the older branch actually fails to elide it too. I was checking the wrong working copy.

[dsanders]

In D64650#1583590, @arsenm wrote:

Having it as an independent pass may be kind of annoying. Having verifier runs in -print-before-all/-print-after-all is annoying enough already, and this would double them

The MachineVerifier itself is supposed to prevent it by calling PM->add() directly to add itself instead of going through addPass() but PM is private so I wasn't able to use the same method. Along the same lines the false, false
in:

addPass(createAArch64MachineVerifier(Banner), false, false);

is supposed to prevent that. One of them prevents verifiers being added after the pass while the other prevents printers being added after the pass. Interestingly, TargetMachineVerifier successfully elides the printer on an older > branch but not on trunk, while MachineVerifier fails on both. I'll take a look.

It turns out the printing passes are added in two places and only one of them honours the printAfter argument to addPass(). https://reviews.llvm.org/D64681 is a quick hack that prevents the verifier triggering IR dumps

[luismarques]

@dsanders I agree that this patch addresses a real problem, and that there is currently a generalized lack of target-specific checks. AMDGPU seems to be the target with the most comprehensive set of checks (although I think it's mostly in the S.I. subtarget). Inspired by this patch we also submitted a patch to verify the immediates of MachineInstrs for the RISC-V target (D67397). The fact that invalid immediate operands just get silently processed (emitted in asm form, truncated in obj form) in nearly all targets is a good indicator that there is space for improvements, and that unrelated changes at any point might reveal latent bugs. Hopefully having more of this infrastructure in place will further encourage targets to add more checks. Still, I'm not sure adding a pass is the best way to accomplish this.

Why have the distinction between MachineVerifier and TargetMachineVerifier? Presumably you are going to update this patch to remove the TargetMachineVerifier::verifyMI method, since that functionality is already covered the by TargetInstrInfo::verifyInstruction hook. Is there any fundamental reason why the MF/MBB checks should be part of this new pass while the MI check is a hook? I think not. I think all three verifications should be hooks, appropriately called by the MachineVerifier. Having the MachineVerifier in charge of the high-level logic and delegating to the target to verify target-specific information is consistent with LLVM practice. Now, I'm assuming we can do the appropriate target-independent verifications in MachineVerifier before calling the hooks, which I agree nicely simplifies the target-specific checks, but I may be wrong about that. Assuming we can, there's IMO not really a good motivation to split the verifications in two passes. I imagine a single pass will be more cache friendly too. One possible issue is that it might not be clear where to put the MF/MBB hooks, since it seems none of the existing target interfaces currently are perfect matches. It might be worth considering adding a new target interface (or refactoring existing ones?). BTW, if you do that, consider if the TargetMachineVerifier::verifyMI hook should be moved there, or if it makes sense to remain part of the TargetMachineVerifier. Answering this question of where to put the hooks might also clarify what is the best way to verify machine operands. For my D67397 patch I did the MachineInstr verification based on the MCOperandPredicates, and I had to duplicate some of the logic for that, since the MC layer and the MachineInstrs have a different notion of immediates. Having the somewhat higher-level MachineInstr be checked based on the lower-level MC info is also not great, although not unprecedented. It might make sense to have a more neutral place as a ground truth that could be used to check properties common to MachineInstrs and MCInstrs, and that place might be where you would put those two additional hooks!

[arsenm]

D81821 has a use case where a target specific block/function level verifier makes sense

include/llvm/CodeGen/TargetMachineVerifier.h

+//===-- TargetMachineVerifier.h - Machine Code Verifier -------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Pass to verify target specific details of machine code.
+//
+// Based on MachineVerifier. Like that pass, it runs when -verify-machineinstrs
+// is given and when MachineFunction::verify() is called.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CODEGEN_TARGETMACHINEVERIFIER_H
+#define LLVM_CODEGEN_TARGETMACHINEVERIFIER_H
+
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/Passes.h"
+
+namespace llvm {
+class TargetMachineVerifier {
+public:
+  TargetMachineVerifier(const char *Banner) : Banner(Banner) {}
+  virtual ~TargetMachineVerifier() {}
+
+  /// Verify a given MF as well as the MBB's and MI's inside it.
+  unsigned verify(const MachineFunction &MF, const char *CallBanner = nullptr);
+
+  /// Override this to add MF verification
+  virtual void verifyMF(const MachineFunction &MF,
+                        const char *CallBanner = nullptr) {}
+  /// Override this to add MBB verification
+  virtual void verifyMBB(const MachineFunction &MF,
+                         const MachineBasicBlock &MBB,
+                         const char *CallBanner = nullptr) {}
+  /// Override this to add MI verification
+  virtual void verifyMI(const MachineFunction &MF, const MachineBasicBlock &MBB,
+                        const MachineInstr &MI,
+                        const char *CallBanner = nullptr) {}
+
+protected:
+  /// A banner to emit on error messages.
+  const char *Banner;
+  /// The number of errors discovered so far.
+  unsigned FoundErrors;
+
+  /// These functions report a verification issue with the given object and
+  /// prints the error message along with the scope and parent scopes.
+  void report(const char *CallBanner, const char *Msg,
+              const MachineFunction &MF);
+  void report(const char *CallBanner, const char *Msg,
+              const MachineBasicBlock &MBB);
+  void report(const char *CallBanner, const char *Msg, const MachineInstr &MI);
+  void report(const char *CallBanner, const char *Msg,
+              const MachineOperand &MO);
+};
+
+template<class VerifierClass>
+struct TargetMachineVerifierPass : public MachineFunctionPass {
+  const std::string Banner;
+
+  TargetMachineVerifierPass(char &ID, std::string banner = std::string())
+      : MachineFunctionPass(ID), Banner(std::move(banner)) {}
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.setPreservesAll();
+    MachineFunctionPass::getAnalysisUsage(AU);
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override {
+    unsigned FoundErrors = VerifierClass(Banner.c_str()).verify(MF);
+    if (FoundErrors)
+      report_fatal_error("Found " + Twine(FoundErrors) +
+                         " machine code errors.");
+    return false;
+  }
+};
+} // end namespace llvm
+
+#endif // ifndef LLVM_CODEGEN_TARGETMACHINEVERIFIER_H

include/llvm/CodeGen/TargetPassConfig.h

public:
 
   /// Add a pass to print the machine function if printing is enabled.
   void addPrintPass(const std::string &Banner);
 
   /// Add a pass to perform basic verification of the machine function if
   /// verification is enabled.
   void addVerifyPass(const std::string &Banner);
 
+  /// Add a pass to perform target specific verification of the machine function
+  /// if verification is enabled.
+  virtual void addTargetVerifyPass(const std::string &Banner) {}
+
   /// Check whether or not GlobalISel should abort on error.
   /// When this is disabled, GlobalISel will fall back on SDISel instead of
   /// erroring out.
   bool isGlobalISelAbortEnabled() const;
 
   /// Check whether or not a diagnostic should be emitted when GlobalISel
   /// uses the fallback path. In other words, it will emit a diagnostic
   /// when GlobalISel failed and isGlobalISelAbortEnabled is false.

include/llvm/Target/TargetMachine.h

 class SMRange;
 class Target;
 class TargetIntrinsicInfo;
 class TargetIRAnalysis;
 class TargetTransformInfo;
 class TargetLoweringObjectFile;
 class TargetPassConfig;
 class TargetSubtargetInfo;
+class TargetMachineVerifier;
 
 // The old pass manager infrastructure is hidden in a legacy namespace now.
 namespace legacy {
 class PassManagerBase;
 }
 using legacy::PassManagerBase;
 
 namespace yaml {
bool addPassesToEmitMC(PassManagerBase &PM, MCContext *&Ctx,
                          bool DisableVerify = true) override;
 
   /// Returns true if the target is expected to pass all machine verifier
   /// checks. This is a stopgap measure to fix targets one by one. We will
   /// remove this at some point and always enable the verifier when
   /// EXPENSIVE_CHECKS is enabled.
   virtual bool isMachineVerifierClean() const { return true; }
 
+  // Return a TargetMachineVerifier* that MachineFunction::verify() can use.
+  virtual TargetMachineVerifier *getTargetMachineVerifier() const {
+    return nullptr;
+  }
+
   /// Adds an AsmPrinter pass to the pipeline that prints assembly or
   /// machine code from the MI representation.
   bool addAsmPrinter(PassManagerBase &PM, raw_pwrite_stream &Out,
                      raw_pwrite_stream *DwoOut, CodeGenFileType FileTYpe,
                      MCContext &Context);
 
   /// True if the target uses physical regs at Prolog/Epilog insertion
   /// time. If true (most machines), all vregs must be allocated before

lib/CodeGen/CMakeLists.txt

add_llvm_library(LLVMCodeGen
   SwiftErrorValueTracking.cpp
   SwitchLoweringUtils.cpp
   TailDuplication.cpp
   TailDuplicator.cpp
   TargetFrameLoweringImpl.cpp
   TargetInstrInfo.cpp
   TargetLoweringBase.cpp
   TargetLoweringObjectFileImpl.cpp
+  TargetMachineVerifier.cpp
   TargetOptionsImpl.cpp
   TargetPassConfig.cpp
   TargetRegisterInfo.cpp
   TargetSchedule.cpp
   TargetSubtargetInfo.cpp
   TwoAddressInstructionPass.cpp
   UnreachableBlockElim.cpp
   ValueTypes.cpp

lib/CodeGen/MachineVerifier.cpp

 #include "llvm/CodeGen/MachineInstrBundle.h"
 #include "llvm/CodeGen/MachineMemOperand.h"
 #include "llvm/CodeGen/MachineOperand.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/PseudoSourceValue.h"
 #include "llvm/CodeGen/SlotIndexes.h"
 #include "llvm/CodeGen/StackMaps.h"
 #include "llvm/CodeGen/TargetInstrInfo.h"
+#include "llvm/CodeGen/TargetMachineVerifier.h"
 #include "llvm/CodeGen/TargetOpcodes.h"
 #include "llvm/CodeGen/TargetRegisterInfo.h"
 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/MC/LaneBitmask.h"

 
 INITIALIZE_PASS(MachineVerifierPass, "machineverifier",
                 "Verify generated machine code", false, false)
 
 FunctionPass *llvm::createMachineVerifierPass(const std::string &Banner) {
   return new MachineVerifierPass(Banner);
 }
 
-bool MachineFunction::verify(Pass *p, const char *Banner, bool AbortOnErrors)
-    const {
+bool MachineFunction::verify(Pass *p, const char *Banner,
+                             bool AbortOnErrors) const {
   MachineFunction &MF = const_cast<MachineFunction&>(*this);
   unsigned FoundErrors = MachineVerifier(p, Banner).verify(MF);
   if (AbortOnErrors && FoundErrors)
-    report_fatal_error("Found "+Twine(FoundErrors)+" machine code errors.");
+    report_fatal_error("Found " + Twine(FoundErrors) + " machine code errors.");
+
+  const LLVMTargetMachine &Target = MF.getTarget();
+  TargetMachineVerifier *Verifier = Target.getTargetMachineVerifier();
+  if (Verifier) {
+    FoundErrors = Verifier->verify(MF, Banner);
+    if (AbortOnErrors && FoundErrors)
+      report_fatal_error("Found " + Twine(FoundErrors) +
+                         " target machine code errors.");
+  }
+
   return FoundErrors == 0;
 }
 
 void MachineVerifier::verifySlotIndexes() const {
   if (Indexes == nullptr)
     return;
 
   // Ensure the IdxMBB list is sorted by slot indexes.

lib/CodeGen/TargetMachineVerifier.cpp

+//===-- TargetMachineVerifier.cpp - Machine Code Verifier -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Interface to verify Target specific details of machine code.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/TargetMachineVerifier.h"
+
+using namespace llvm;
+
+unsigned TargetMachineVerifier::verify(const MachineFunction &MF,
+                                       const char *CallBanner) {
+  FoundErrors = 0;
+
+  for (const auto &MBB : MF) {
+    verifyMBB(MF, MBB, CallBanner);
+    for (const auto &MI : MBB)
+      verifyMI(MF, MBB, MI, CallBanner);
+  }
+
+  return FoundErrors;
+}
+
+void TargetMachineVerifier::report(const char *CallBanner, const char *Msg,
+                                   const MachineFunction &MF) {
+  errs() << '\n';
+  if (!FoundErrors++) {
+    if (Banner)
+      errs() << "# " << Banner << '\n';
+    if (CallBanner)
+      errs() << "# " << CallBanner << '\n';
+  }
+  errs() << "*** Bad machine code: " << Msg << " ***\n"
+      << "- function:    " << MF.getName() << "\n";
+}
+
+void TargetMachineVerifier::report(const char *CallBanner, const char *Msg,
+                                   const MachineBasicBlock &MBB) {
+  errs() << '\n';
+  if (!FoundErrors++) {
+    if (Banner)
+      errs() << "# " << Banner << '\n';
+    if (CallBanner)
+      errs() << "# " << CallBanner << '\n';
+  }
+  const MachineFunction &MF = *MBB.getParent();
+  errs() << "*** Bad machine code: " << Msg << " ***\n"
+      << "- function:    " << MF.getName() << "\n"
+      << "- basic block: " << MBB.getFullName() << "\n";
+}
+
+void TargetMachineVerifier::report(const char *CallBanner, const char *Msg,
+                                   const MachineInstr &MI) {
+  errs() << '\n';
+  if (!FoundErrors++) {
+    if (Banner)
+      errs() << "# " << Banner << '\n';
+    if (CallBanner)
+      errs() << "# " << CallBanner << '\n';
+  }
+  const MachineBasicBlock &MBB = *MI.getParent();
+  const MachineFunction &MF = *MBB.getParent();
+  errs() << "*** Bad machine code: " << Msg << " ***\n"
+      << "- function:    " << MF.getName() << "\n"
+      << "- basic block: " << MBB.getFullName() << "\n"
+      << "- instruction: " << MI;
+}
+
+void TargetMachineVerifier::report(const char *CallBanner, const char *Msg,
+                                   const MachineOperand &MO) {
+  errs() << '\n';
+  if (!FoundErrors++) {
+    if (Banner)
+      errs() << "# " << Banner << '\n';
+    if (CallBanner)
+      errs() << "# " << CallBanner << '\n';
+  }
+  const MachineInstr &MI = *MO.getParent();
+  const MachineBasicBlock &MBB = *MI.getParent();
+  const MachineFunction &MF = *MBB.getParent();
+  errs() << "*** Bad machine code: " << Msg << " ***\n"
+      << "- function:    " << MF.getName() << "\n"
+      << "- basic block: " << MBB.getFullName() << "\n"
+      << "- instruction: " << MI
+      << "- operand:     " << MO << "\n";
+}

lib/CodeGen/TargetPassConfig.cpp

 }
 
 void TargetPassConfig::addVerifyPass(const std::string &Banner) {
   bool Verify = VerifyMachineCode == cl::BOU_TRUE;
 #ifdef EXPENSIVE_CHECKS
   if (VerifyMachineCode == cl::BOU_UNSET)
     Verify = TM->isMachineVerifierClean();
 #endif
-  if (Verify)
+  if (Verify) {
     PM->add(createMachineVerifierPass(Banner));
+    addTargetVerifyPass(Banner);
+  }
 }
 
 /// Add common target configurable passes that perform LLVM IR to IR transforms
 /// following machine independent optimization.
 void TargetPassConfig::addIRPasses() {
   switch (UseCFLAA) {
   case CFLAAType::Steensgaard:
     addPass(createCFLSteensAAWrapperPass());

lib/Target/AArch64/AArch64.h

 
 FunctionPass *createAArch64CleanupLocalDynamicTLSPass();
 
 FunctionPass *createAArch64CollectLOHPass();
 InstructionSelector *
 createAArch64InstructionSelector(const AArch64TargetMachine &,
                                  AArch64Subtarget &, AArch64RegisterBankInfo &);
 FunctionPass *createAArch64PreLegalizeCombiner();
+FunctionPass *createAArch64MachineVerifier(const std::string &Banner);
 
 void initializeAArch64A53Fix835769Pass(PassRegistry&);
 void initializeAArch64A57FPLoadBalancingPass(PassRegistry&);
 void initializeAArch64AdvSIMDScalarPass(PassRegistry&);
 void initializeAArch64BranchTargetsPass(PassRegistry&);
 void initializeAArch64CollectLOHPass(PassRegistry&);
 void initializeAArch64CondBrTuningPass(PassRegistry &);
 void initializeAArch64CompressJumpTablesPass(PassRegistry&);
 void initializeAArch64ConditionalComparesPass(PassRegistry&);
 void initializeAArch64ConditionOptimizerPass(PassRegistry&);
 void initializeAArch64DeadRegisterDefinitionsPass(PassRegistry&);
 void initializeAArch64ExpandPseudoPass(PassRegistry&);
 void initializeAArch64SpeculationHardeningPass(PassRegistry&);
 void initializeAArch64LoadStoreOptPass(PassRegistry&);
 void initializeAArch64SIMDInstrOptPass(PassRegistry&);
 void initializeAArch64PreLegalizerCombinerPass(PassRegistry&);
 void initializeAArch64PromoteConstantPass(PassRegistry&);
 void initializeAArch64RedundantCopyEliminationPass(PassRegistry&);
 void initializeAArch64StorePairSuppressPass(PassRegistry&);
 void initializeFalkorHWPFFixPass(PassRegistry&);
 void initializeFalkorMarkStridedAccessesLegacyPass(PassRegistry&);
 void initializeLDTLSCleanupPass(PassRegistry&);
+void initializeAArch64MachineVerifierPassPass(PassRegistry&);
 } // end namespace llvm
 
 #endif

lib/Target/AArch64/AArch64MachineVerifier.h

+//===-- AArch64MachineVerifier.h - Machine Code Verifier -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Pass to verify AArch64 specific details of machine code.
+//
+// Based on MachineVerifier. Like that pass, it runs when -verify-machineinstrs
+// is given and when MachineFunction::verify() is called.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_AARCH64_AARCH64MACHINEVERIFIER_H
+#define LLVM_LIB_TARGET_AARCH64_AARCH64MACHINEVERIFIER_H
+
+#include "llvm/CodeGen/TargetMachineVerifier.h"
+
+namespace llvm {
+struct AArch64MachineVerifier : public TargetMachineVerifier {
+  AArch64MachineVerifier(const char *Banner) : TargetMachineVerifier(Banner) {}
+
+  void verifyMI(const MachineFunction &MF, const MachineBasicBlock &MBB,
+                const MachineInstr &MI, const char *CallBanner) override;
+  void verifyBitFieldMove(const MachineInstr &MI, unsigned SizeInBits,
+                          const char *CallBanner);
+};
+} // end namespace llvm
+
+#endif // ifndef LLVM_LIB_TARGET_AARCH64_AARCH64MACHINEVERIFIER_H

lib/Target/AArch64/AArch64MachineVerifier.cpp

+//===-- AArch64MachineVerifier.cpp - Machine Code Verifier -------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Pass to verify AArch64 specific details of machine code.
+//
+// Based on MachineVerifier. Like that pass, it runs when -verify-machineinstrs
+// is given and when MachineFunction::verify() is called.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/CodeGen/TargetMachineVerifier.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Support/ScopedPrinter.h"
+
+#include "AArch64.h"
+#include "AArch64TargetMachine.h"
+
+using namespace llvm;
+
+namespace {
+struct AArch64MachineVerifierPass : public TargetMachineVerifierPass<AArch64MachineVerifier> {
+  static char ID; // Pass ID, replacement for typeid
+
+  AArch64MachineVerifierPass(std::string Banner = std::string())
+      : TargetMachineVerifierPass(ID, std::move(Banner)) {
+    initializeAArch64MachineVerifierPassPass(*PassRegistry::getPassRegistry());
+  }
+};
+char AArch64MachineVerifierPass::ID = 0;
+
+} // end anonymous namespace
+
+INITIALIZE_PASS(AArch64MachineVerifierPass, "aarch64-machineverifier",
+                "Verify generated AArch64 machine code", false, false)
+
+FunctionPass *llvm::createAArch64MachineVerifier(const std::string &Banner) {
+  return new AArch64MachineVerifierPass(Banner);
+}
+
+TargetMachineVerifier *
+AArch64TargetMachine::getTargetMachineVerifier() const {
+  if (TgtMachineVerifier)
+    return TgtMachineVerifier.get();
+  TgtMachineVerifier = make_unique<AArch64MachineVerifier>(nullptr);
+  return TgtMachineVerifier.get();
+}
+void AArch64MachineVerifier::verifyBitFieldMove(const MachineInstr &MI,
+                                                unsigned SizeInBits,
+                                                const char *CallBanner) {
+  if (MI.getOperand(2).isImm()) {
+    unsigned ImmR = MI.getOperand(2).getImm();
+    if (ImmR < 0 || ImmR >= SizeInBits)
+      report(CallBanner,
+             ("Expected immediate in range 0 - " + to_string(SizeInBits - 1))
+                 .c_str(),
+             MI.getOperand(2));
+  } else
+    report(CallBanner, "Expected immediate", MI.getOperand(2));
+
+  if (MI.getOperand(3).isImm()) {
+    unsigned ImmS = MI.getOperand(3).getImm();
+
+    if (ImmS < 0 || ImmS >= SizeInBits)
+      report(CallBanner,
+             ("Expected immediate in range 0 - " + to_string(SizeInBits - 1))
+                 .c_str(),
+             MI.getOperand(3));
+  } else
+    report(CallBanner, "Expected immediate", MI.getOperand(3));
+}
+
+void AArch64MachineVerifier::verifyMI(const MachineFunction &MF,
+                                      const MachineBasicBlock &MBB,
+                                      const MachineInstr &MI,
+                                      const char *CallBanner) {
+  switch (MI.getOpcode()) {
+  case AArch64::SBFMWri:
+  case AArch64::UBFMWri:
+    verifyBitFieldMove(MI, 32, CallBanner);
+    break;
+  case AArch64::SBFMXri:
+  case AArch64::UBFMXri:
+    verifyBitFieldMove(MI, 64, CallBanner);
+    break;
+  }
+}
+

lib/Target/AArch64/AArch64TargetMachine.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AARCH64_AARCH64TARGETMACHINE_H
 #define LLVM_LIB_TARGET_AARCH64_AARCH64TARGETMACHINE_H
 
 #include "AArch64InstrInfo.h"
 #include "AArch64Subtarget.h"
+#include "AArch64MachineVerifier.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/Target/TargetMachine.h"
 
 namespace llvm {
 
 class AArch64RegisterBankInfo;
 
 class AArch64TargetMachine : public LLVMTargetMachine {
 protected:
   std::unique_ptr<TargetLoweringObjectFile> TLOF;
   mutable StringMap<std::unique_ptr<AArch64Subtarget>> SubtargetMap;
+  mutable std::unique_ptr<TargetMachineVerifier> TgtMachineVerifier = nullptr;
 
 public:
   AArch64TargetMachine(const Target &T, const Triple &TT, StringRef CPU,
                        StringRef FS, const TargetOptions &Options,
                        Optional<Reloc::Model> RM, Optional<CodeModel::Model> CM,
                        CodeGenOpt::Level OL, bool JIT, bool IsLittleEndian);
 
   ~AArch64TargetMachine() override;
   const AArch64Subtarget *getSubtargetImpl(const Function &F) const override;
   // DO NOT IMPLEMENT: There is no such thing as a valid default subtarget,
   // subtargets are per-function entities based on the target-specific
   // attributes of each function.
   const AArch64Subtarget *getSubtargetImpl() const = delete;
 
   // Pass Pipeline Configuration
   TargetPassConfig *createPassConfig(PassManagerBase &PM) override;
 
   TargetTransformInfo getTargetTransformInfo(const Function &F) override;
 
   TargetLoweringObjectFile* getObjFileLowering() const override {
     return TLOF.get();
   }
 
+  TargetMachineVerifier *
+  getTargetMachineVerifier() const override;
+
 private:
   bool isLittle;
 };
 
 // AArch64 little endian target machine.
 //
 class AArch64leTargetMachine : public AArch64TargetMachine {
   virtual void anchor();

lib/Target/AArch64/AArch64TargetMachine.cpp

public:
   bool addRegBankSelect() override;
   void addPreGlobalInstructionSelect() override;
   bool addGlobalInstructionSelect() override;
   bool addILPOpts() override;
   void addPreRegAlloc() override;
   void addPostRegAlloc() override;
   void addPreSched2() override;
   void addPreEmitPass() override;
+  void addTargetVerifyPass(const std::string &Banner) override;
 
   std::unique_ptr<CSEConfigBase> getCSEConfig() const override;
 };
 
 } // end anonymous namespace
 
 TargetTransformInfo
 AArch64TargetMachine::getTargetTransformInfo(const Function &F) {
void AArch64PassConfig::addPreEmitPass() {
 
   if (TM->getOptLevel() != CodeGenOpt::None && EnableCompressJumpTables)
     addPass(createAArch64CompressJumpTablesPass());
 
   if (TM->getOptLevel() != CodeGenOpt::None && EnableCollectLOH &&
       TM->getTargetTriple().isOSBinFormatMachO())
     addPass(createAArch64CollectLOHPass());
 }
+
+void AArch64PassConfig::addTargetVerifyPass(const std::string &Banner) {
+  addPass(createAArch64MachineVerifier(Banner), false, false);
+}

lib/Target/AArch64/CMakeLists.txt

add_llvm_target(AArch64CodeGen
   AArch64SelectionDAGInfo.cpp
   AArch64SpeculationHardening.cpp
   AArch64StorePairSuppress.cpp
   AArch64Subtarget.cpp
   AArch64TargetMachine.cpp
   AArch64TargetObjectFile.cpp
   AArch64TargetTransformInfo.cpp
   AArch64SIMDInstrOpt.cpp
+  AArch64MachineVerifier.cpp
 
   DEPENDS
   intrinsics_gen
 )
 
 add_subdirectory(AsmParser)
 add_subdirectory(Disassembler)
 add_subdirectory(MCTargetDesc)
 add_subdirectory(TargetInfo)
 add_subdirectory(Utils)

test/Verifier/AArch64/lit.local.cfg

+if not 'AArch64' in config.root.targets:
+    config.unsupported = True

test/Verifier/AArch64/sbfm_ubfm.mir

+# RUN: not llc -mtriple=aarch64-- -run-pass=none -verify-machineinstrs %s -o - 2>%t
+# RUN: FileCheck %s < %t
+
+---
+name:            fn
+body:             |
+  bb.0:
+    liveins: $w0, $w1, $x0, $x1
+
+    %0:gpr32 = COPY $w0
+    %1:gpr32 = COPY $w1
+    %2:gpr32 = SBFMWri %0, 2, 1
+    $w0 = COPY %2:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, %vreg1, 1
+    ; CHECK-NEXT: operand: %vreg1
+    %3:gpr32 = SBFMWri %0, %1, 1
+    $w1 = COPY %3:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, 2, %vreg1
+    ; CHECK-NEXT: operand: %vreg1
+    %3:gpr32 = SBFMWri %0, 2, %1
+    $w1 = COPY %3:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, 32, 1
+    ; CHECK-NEXT: operand: 32
+    %3:gpr32 = SBFMWri %0, 32, 1
+    $w1 = COPY %3:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, 0, 32
+    ; CHECK-NEXT: operand: 32
+    %3:gpr32 = SBFMWri %0, 0, 32
+    $w1 = COPY %3:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, 64, 1
+    ; CHECK-NEXT: operand: 64
+    %3:gpr32 = SBFMWri %0, 64, 1
+    $w1 = COPY %3:gpr32
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg3<def> = SBFMWri %vreg0, 0, 64
+    ; CHECK-NEXT: operand: 64
+    %3:gpr32 = SBFMWri %0, 0, 64
+    $w1 = COPY %3:gpr32
+
+    %4:gpr64 = COPY $x0
+    %5:gpr64 = COPY $x1
+    %6:gpr64 = UBFMXri %4, 2, 1
+    $x0 = COPY %6:gpr64
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg7<def> = UBFMXri %vreg4, %vreg5, 1
+    ; CHECK-NEXT: operand: %vreg5
+    %7:gpr64 = UBFMXri %4, %5, 1
+    $x1 = COPY %7:gpr64
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg7<def> = UBFMXri %vreg4, 2, %vreg5
+    ; CHECK-NEXT: operand: %vreg5
+    %7:gpr64 = UBFMXri %4, 2, %5
+    $x1 = COPY %7:gpr64
+
+    %7:gpr64 = UBFMXri %4, 32, 1
+    $x1 = COPY %7:gpr64
+
+    %7:gpr64 = UBFMXri %4, 0, 32
+    $x1 = COPY %7:gpr64
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg7<def> = UBFMXri %vreg4, 64, 1
+    ; CHECK-NEXT: operand: 64
+    %7:gpr64 = UBFMXri %4, 64, 1
+    $x1 = COPY %7:gpr64
+
+    ; CHECK: Bad machine code: Expected immediate
+    ; CHECK-NEXT: function: fn
+    ; CHECK-NEXT: basic block: fn:BB0
+    ; CHECK-NEXT: instruction: %vreg7<def> = UBFMXri %vreg4, 0, 64
+    ; CHECK-NEXT: operand: 64
+    %7:gpr64 = UBFMXri %4, 0, 64
+    $x1 = COPY %7:gpr64
+
+    ; CHECK: LLVM ERROR: Found 10 target machine code errors.
+...

unittests/CodeGen/GlobalISel/PatternMatchTest.cpp

TEST(PatternMatchInstr, MatchMiscellaneous) {
   // Make multiple uses of this add.
   B.buildCast(LLT::pointer(0, 32), MIBAdd);
   B.buildCast(LLT::pointer(1, 32), MIBAdd);
   bool match = mi_match(MIBAdd.getReg(0), MRI, m_GAdd(m_Reg(), m_Reg()));
   EXPECT_TRUE(match);
   match = mi_match(MIBAdd.getReg(0), MRI, m_OneUse(m_GAdd(m_Reg(), m_Reg())));
   EXPECT_FALSE(match);
 }
+
+// This test isn't a pattern matcher but the boiler plate would be an exact
+// duplicate of the code so lets just put it here.
+TEST(TargetMachineVerifier, MachineFunctionVerify) {
+  LLVMContext Context;
+  std::unique_ptr<LLVMTargetMachine> TM = createTargetMachine();
+  if (!TM)
+    return;
+
+  auto ModuleMMIPair = createDummyModule(Context, *TM, R"MIR(
+  %4:gpr64 = COPY %0
+  %5:gpr64 = SBFMXri %4, 63, 1
+)MIR");
+  MachineFunction *MF =
+      getMFFromMMI(ModuleMMIPair.first.get(), ModuleMMIPair.second.get());
+  MachineBasicBlock &MBB = *MF->begin();
+  MachineInstr &MI = *(std::prev(MBB.end()));
+  // Break the instruction by changing the immediate to be out of range
+  MI.getOperand(2).setImm(64);
+  EXPECT_FALSE(MF->verify(nullptr, "Banner", false));
+}
+
 } // namespace
 
 int main(int argc, char **argv) {
   ::testing::InitGoogleTest(&argc, argv);
   initLLVM();
   return RUN_ALL_TESTS();
 }