Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator

[mehdi_amini]

Summary

In particular for Graph Regions, the terminator needs is just a
historical artifact of the generalization of MLIR from CFG region.
Operations like Module don't need a terminator, and before Module
migrated to be an operation with region there wasn't any needed.

To validate the feature, the ModuleOp is migrated to use this trait and
the ModuleTerminator operation is deleted.

This patch is likely to break clients, if you're in this case:

• you may iterate on a ModuleOp with getBody()->without_terminator(), the solution is simple: just remove the ->without_terminator!
• you created a builder with Builder::atBlockTerminator(module_body), just use Builder::atBlockEnd(module_body) instead.
• you were handling ModuleTerminator: it isn't needed anymore.
• for generic code, a Block::mayNotHaveTerminator() may be used.

[mehdi_amini]

Minor cleanup

[mehdi_amini]

Fix the examples

[mehdi_amini]

Update python bindings

[mehdi_amini]

Rebase

[mehdi_amini]

Rebase and add support for unregistered operation

[rriddle]

mlir/include/mlir/IR/Block.h

229 ↗

(On Diff #330782)

When would this be used outside of the verifier?

mlir/include/mlir/IR/OpDefinition.h

679

Can you verify this?

803

llvm::hasSingleElement

807–811

Isn't this already verified by the verifier?

864–865

This looks weird, is this necessary?

mlir/include/mlir/IR/OpImplementation.h

95

Can you document the rest of the parameters as well while you are here? (I wonder if we should also consider a flag builder here at some point, we now have 3 bool flags)

mlir/include/mlir/IR/RegionKindInterface.h

32

mlir/include/mlir/Parser.h

40–42

Can you check this clang-tidy message?

mlir/lib/IR/AsmPrinter.cpp

2545

Can you define a local variable for this?

mlir/lib/IR/SymbolTable.cpp

167

nit: Add braces to this if/else now, the body is quite large.

299

Can you cache the region to avoid computing it multiple times?

mlir/tools/mlir-tblgen/OpFormatGen.cpp

444

Drop trivial braces here.

[mehdi_amini]

mlir/include/mlir/IR/Block.h

229 ↗

(On Diff #330782)

I am not sure, but any pass that needs to be conservative in the same way.
I suspect it is similar to mightHaveTrait<...>()

mlir/include/mlir/IR/OpDefinition.h

679

I would need to include the RegionKind header in here, should I do that?

Also Chris mentioned on Discourse some non-graph region which aren't intended to return values and could also skip the terminator, what about removing the comment here?
(my original implementation was relying on graph region in some places but that was before I made it a "NoTerminator" trait).

803

(note this is just code motion, but still I fixed it)

807–811

That's a good point! (note this is code motion only again).

Unfortunately this is needed because the verifier won't kick in for nested block before verifying the enclosing operation itself: https://github.com/llvm/llvm-project/blob/main/mlir/test/IR/traits.mlir#L301-L306

864–865

indeed... (removed)

mlir/include/mlir/Parser.h

40–42

Yes, I'll need your magic to restore the version on the left, but without using ImplicitTerminatorOpT. The existing assert is actually failing to do what it is suppose to because if an op does not implement the SingleBlockImplicitTerminator it does not have the ImplicitTerminatorOpT in the first place!

[mehdi_amini]

Addressing comments (most)

[mehdi_amini]

Fix the trait detection for SingleBlockImplicitTerminatorOp

[rriddle]

mlir/include/mlir/IR/OpDefinition.h

863

nit: Block *body or getBody()->getOperations()...?

mlir/lib/IR/SymbolTable.cpp

299

Why not just region?

mlir/test/lib/Dialect/Test/TestDialect.cpp

38 ↗

(On Diff #331752)

Why is this in TestDialect.cpp?

[rriddle]

Did you make sure that the docs are up-to-date with this?

[mehdi_amini]

mlir/lib/IR/SymbolTable.cpp

299

Sure, renamed.

Do we verify that SymbolTable operations are single region by the way?
It's annoying I was planning to have an op with two regions and each would be a symbol table. I guess that's a problem for nested symbol refs...

[rriddle]

mlir/lib/IR/SymbolTable.cpp

299

Yes, it is verified. For multi-region symbol tables, it is theoretically possible but we would need to:
a) invent some special new syntax
b) A lot of the code in this file would need to be rewritten to not anchor on the symbol table operation itself.

[mehdi_amini]

In D98468#2638692, @rriddle wrote:

Did you make sure that the docs are up-to-date with this?

Excellent point, I haven't yet@

mlir/include/mlir/IR/OpDefinition.h

863

getBody()->getOperations() is better!

mlir/test/lib/Dialect/Test/TestDialect.cpp

38 ↗

(On Diff #331752)

I needed an op which defined the traits to be able to test that.
Otherwise we don't instantiate this anywhere in the codebase.

[mehdi_amini]

Address comments (doc not done yet)

[mehdi_amini]

Update LangRef

[mehdi_amini]

Minor TableGen rename for the trait list

[mehdi_amini]

Update Traits doc

[rriddle]

LGTM

[kiranchandramohan]

mlir/lib/IR/Block.cpp

294

This function is not used currently and there is an unused warning. Is there a plan to use it in future? There is an identical function in the verifier in this patch.

[stella.stamenova]

mlir/lib/IR/Block.cpp

294

+1, let's keep to one function instead of duplicating code in multiple places

[mehdi_amini]

mlir/lib/IR/Block.cpp

294

That was an oversight, it was here initially and I meant to move it to the Verifier and instead I duplicated it.

Removed in fcdf142ed59c !

mlir/docs/Tutorials/Toy/Ch-6.md

 ### Conversion Target
 
 For this conversion, aside from the top-level module, we will be lowering
 everything to the LLVM dialect.
 
 ```c++
   mlir::ConversionTarget target(getContext());
   target.addLegalDialect<mlir::LLVMDialect>();
-  target.addLegalOp<mlir::ModuleOp, mlir::ModuleTerminatorOp>();
+  target.addLegalOp<mlir::ModuleOp>();
 ```
 
 ### Type Converter
 
 This lowering will also transform the MemRef types which are currently being
 operated on into a representation in LLVM. To perform this conversion, we use a
 TypeConverter as part of the lowering. This converter specifies how one type
 maps to another. This is necessary now that we are performing more complicated

mlir/docs/Tutorials/UnderstandingTheIRStructure.md

"dialect.op2"() ( {
     "dialect.innerop3"(%0#0, %0#2, %0#3)[^bb1, ^bb2] : (i1, i32, i64) -> ()
   ^bb1(%1: i32):  // pred: ^bb0
     "dialect.innerop4"() : () -> ()
     "dialect.innerop5"() : () -> ()
   ^bb2(%2: i64):  // pred: ^bb0
     "dialect.innerop6"() : () -> ()
     "dialect.innerop7"() : () -> ()
   }) {"other attribute" = 42 : i64} : () -> ()
-  "module_terminator"() : () -> ()
 }) : () -> ()
 ```
 
 And will yield the following output:
 
 ```
 visiting op: 'module' with 0 operands and 0 results
  1 nested regions:
Block with 0 arguments, 0 successors, and 3 operations
              0 nested regions:
             visiting op: 'dialect.innerop5' with 0 operands and 0 results
              0 nested regions:
           Block with 1 arguments, 0 successors, and 2 operations
             visiting op: 'dialect.innerop6' with 0 operands and 0 results
              0 nested regions:
             visiting op: 'dialect.innerop7' with 0 operands and 0 results
              0 nested regions:
-      visiting op: 'module_terminator' with 0 operands and 0 results
        0 nested regions:
 ```
 
 ## Other IR Traversal Methods.
 
 In many cases, unwrapping the recursive structure of the IR is cumbersome and
 you may be interested in using other helpers.
 

mlir/examples/toy/Ch6/mlir/LowerToLLVM.cpp

 };
 } // end anonymous namespace
 
 void ToyToLLVMLoweringPass::runOnOperation() {
   // The first thing to define is the conversion target. This will define the
   // final target for this lowering. For this lowering, we are only targeting
   // the LLVM dialect.
   LLVMConversionTarget target(getContext());
-  target.addLegalOp<ModuleOp, ModuleTerminatorOp>();
+  target.addLegalOp<ModuleOp>();
 
   // During this lowering, we will also be lowering the MemRef types, that are
   // currently being operated on, to a representation in LLVM. To perform this
   // conversion we use a TypeConverter as part of the lowering. This converter
   // details how one type maps to another. This is necessary now that we will be
   // doing more complicated lowerings, involving loop region arguments.
   LLVMTypeConverter typeConverter(&getContext());
 

mlir/examples/toy/Ch7/mlir/LowerToLLVM.cpp

 };
 } // end anonymous namespace
 
 void ToyToLLVMLoweringPass::runOnOperation() {
   // The first thing to define is the conversion target. This will define the
   // final target for this lowering. For this lowering, we are only targeting
   // the LLVM dialect.
   LLVMConversionTarget target(getContext());
-  target.addLegalOp<ModuleOp, ModuleTerminatorOp>();
+  target.addLegalOp<ModuleOp>();
 
   // During this lowering, we will also be lowering the MemRef types, that are
   // currently being operated on, to a representation in LLVM. To perform this
   // conversion we use a TypeConverter as part of the lowering. This converter
   // details how one type maps to another. This is necessary now that we will be
   // doing more complicated lowerings, involving loop region arguments.
   LLVMTypeConverter typeConverter(&getContext());
 

mlir/include/mlir/IR/BuiltinOps.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef MLIR_IR_BUILTINOPS_H_
 #define MLIR_IR_BUILTINOPS_H_
 
 #include "mlir/IR/FunctionSupport.h"
 #include "mlir/IR/OwningOpRef.h"
+#include "mlir/IR/RegionKindInterface.h"
 #include "mlir/IR/SymbolTable.h"
 #include "mlir/Interfaces/CallInterfaces.h"
 #include "mlir/Interfaces/CastInterfaces.h"
 #include "mlir/Interfaces/SideEffectInterfaces.h"
 #include "llvm/Support/PointerLikeTypeTraits.h"
 
 //===----------------------------------------------------------------------===//
 // Dialect Operations

mlir/include/mlir/IR/BuiltinOps.td

 // necessary for the validity of and defining the IR.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef BUILTIN_OPS
 #define BUILTIN_OPS
 
 include "mlir/IR/BuiltinDialect.td"
+include "mlir/IR/RegionKindInterface.td"
 include "mlir/IR/SymbolInterfaces.td"
 include "mlir/Interfaces/CallInterfaces.td"
 include "mlir/Interfaces/CastInterfaces.td"
 include "mlir/Interfaces/SideEffectInterfaces.td"
 
 // Base class for Builtin dialect ops.
 class Builtin_Op<string mnemonic, list<OpTrait> traits = []> :
     Op<Builtin_Dialect, mnemonic, traits>;
]> {
   let verifier = [{ return ::verify(*this); }];
 }
 
 //===----------------------------------------------------------------------===//
 // ModuleOp
 //===----------------------------------------------------------------------===//
 
 def ModuleOp : Builtin_Op<"module", [
-  AffineScope, IsolatedFromAbove, NoRegionArguments, SymbolTable, Symbol,
-  SingleBlockImplicitTerminator<"ModuleTerminatorOp">
-]> {
+  AffineScope, IsolatedFromAbove, NoRegionArguments, SymbolTable, Symbol] 
+  # NoTerminator.traits> {
   let summary = "A top level container operation";
   let description = [{
     A `module` represents a top-level container operation. It contains a single
     SSACFG region containing a single block which can contain any
     operations. Operations within this region cannot implicitly capture values
     defined outside the module, i.e. Modules are `IsolatedFromAbove`. Modules
     have an optional symbol name which can be used to refer to them in
     operations.
def ModuleOp : Builtin_Op<"module", [
   let verifier = [{ return ::verify(*this); }];
 
   // We need to ensure the block inside the region is properly terminated;
   // the auto-generated builders do not guarantee that.
   let skipDefaultBuilders = 1;
 }
 
 //===----------------------------------------------------------------------===//
-// ModuleTerminatorOp
-//===----------------------------------------------------------------------===//
-
-def ModuleTerminatorOp : Builtin_Op<"module_terminator", [
-  Terminator, HasParent<"ModuleOp">
-]> {
-  let summary = "A pseudo op that marks the end of a module";
-  let description = [{
-    `module_terminator` is a special terminator operation for the body of a
-    `module`, it has no semantic meaning beyond keeping the body of a `module`
-    well-formed.
-  }];
-  let assemblyFormat = "attr-dict";
-}
-
-//===----------------------------------------------------------------------===//
 // UnrealizedConversionCastOp
 //===----------------------------------------------------------------------===//
 
 def UnrealizedConversionCastOp : Builtin_Op<"unrealized_conversion_cast", [
     DeclareOpInterfaceMethods<CastOpInterface>, NoSideEffect
   ]> {
   let summary = "An unrealized conversion from one set of types to another";
   let description = [{

mlir/include/mlir/IR/OpBase.td

def ElementwiseMappable {
   list<OpTrait> traits = [
     Elementwise,
     Scalarizable,
     Vectorizable,
     Tensorizable,
   ];
 }
 
+// Op's regions have a single block.
+def SingleBlock : NativeOpTrait<"SingleBlock">;
+
 // Op's regions have a single block with the specified terminator.
 class SingleBlockImplicitTerminator<string op>
     : ParamNativeOpTrait<"SingleBlockImplicitTerminator", op>;
 
 // Op's parent operation is the provided one.
 class HasParent<string op>
     : ParamNativeOpTrait<"HasParent", op>;
 

mlir/include/mlir/IR/OpDefinition.h

 /// results.
 template <typename ConcreteType>
 class VariadicResults
     : public detail::MultiResultTraitBase<ConcreteType, VariadicResults> {};
 
 //===----------------------------------------------------------------------===//
 // Terminator Traits
 
+/// This class indicates that the regions associated with this op don't have
+/// terminators. This is valid only for Graph regions (see RegionKind).
+template <typename ConcreteType>
+class NoTerminator : public TraitBase<ConcreteType, NoTerminator> {};
+
 /// This class provides the API for ops that are known to be terminators.
 template <typename ConcreteType>
 class IsTerminator : public TraitBase<ConcreteType, IsTerminator> {
 public:
   static LogicalResult verifyTrait(Operation *op) {
     return impl::verifyIsTerminator(op);
   }
 };

 /// This class provides the API for ops which have an unknown number of
 /// successors.
 template <typename ConcreteType>
 class VariadicSuccessors
     : public detail::MultiSuccessorTraitBase<ConcreteType, VariadicSuccessors> {
 };
 
 //===----------------------------------------------------------------------===//
+// SingleBlock
+
+/// This class provides APIs and verifiers for ops with regions having a single
+/// block.
+template <typename ConcreteType>
+struct SingleBlock : public TraitBase<ConcreteType, SingleBlock> {
+public:
+  static LogicalResult verifyTrait(Operation *op) {
+    for (unsigned i = 0, e = op->getNumRegions(); i < e; ++i) {
+      Region &region = op->getRegion(i);
+
+      // Empty regions are fine.
+      if (region.empty())
+        continue;
+
+      // Non-empty regions must contain a single basic block.
+      if (std::next(region.begin()) != region.end())
+        return op->emitOpError("expects region #")
+               << i << " to have 0 or 1 blocks";
+
+      if (!ConcreteType::template hasTrait<NoTerminator>()) {
+        Block &block = region.front();
+        if (block.empty())
+          return op->emitOpError() << "expects a non-empty block";
+      }
+    }
+    return success();
+  }
+
+  Block *getBody(unsigned idx = 0) {
+    Region &region = this->getOperation()->getRegion(idx);
+    assert(!region.empty() && "unexpected empty region");
+    return &region.front();
+  }
+  Region &getBodyRegion(unsigned idx = 0) {
+    return this->getOperation()->getRegion(idx);
+  }
+
+  //===------------------------------------------------------------------===//
+  // Single Region Utilities
+  //===------------------------------------------------------------------===//
+
+  /// The following are a set of methods only enabled when the parent
+  /// operation has a single region. Each of these methods take an additional
+  /// template parameter that represents the concrete operation so that we
+  /// can use SFINAE to disable the methods for non-single region operations.
+  template <typename OpT, typename T = void>
+  using enable_if_single_region =
+      typename std::enable_if_t<OpT::template hasTrait<OneRegion>(), T>;
+
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT, Block::iterator> begin() {
+    return getBody()->begin();
+  }
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT, Block::iterator> end() {
+    return getBody()->end();
+  }
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT, Operation &> front() {
+    return *begin();
+  }
+
+  /// Insert the operation into the back of the body.
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT> push_back(Operation *op) {
+    insert(Block::iterator(getBody()->end()), op);
+  }
+
+  /// Insert the operation at the given insertion point.
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT> insert(Operation *insertPt, Operation *op) {
+    insert(Block::iterator(insertPt), op);
+  }
+  template <typename OpT = ConcreteType>
+  enable_if_single_region<OpT> insert(Block::iterator insertPt, Operation *op) {
+    auto *body = getBody();
+    if (insertPt == body->end())
+      insertPt = Block::iterator(body->end());
+    body->getOperations().insert(insertPt, op);
+  }
+};
+
+//===----------------------------------------------------------------------===//
 // SingleBlockImplicitTerminator
 
 /// This class provides APIs and verifiers for ops with regions having a single
 /// block that must terminate with `TerminatorOpType`.
 template <typename TerminatorOpType>
 struct SingleBlockImplicitTerminator {
   template <typename ConcreteType>
-  class Impl : public TraitBase<ConcreteType, Impl> {
+  class Impl : public SingleBlock<ConcreteType> {
   private:
+    using Base = SingleBlock<ConcreteType>;
     /// Builds a terminator operation without relying on OpBuilder APIs to avoid
     /// cyclic header inclusion.
     static Operation *buildTerminator(OpBuilder &builder, Location loc) {
       OperationState state(loc, TerminatorOpType::getOperationName());
       TerminatorOpType::build(builder, state);
       return Operation::create(state);
     }
 
   public:
     /// The type of the operation used as the implicit terminator type.
     using ImplicitTerminatorOpT = TerminatorOpType;
 
     static LogicalResult verifyTrait(Operation *op) {
+      if (failed(Base::verifyTrait(op)))
+        return failure();
       for (unsigned i = 0, e = op->getNumRegions(); i < e; ++i) {
         Region &region = op->getRegion(i);
-
         // Empty regions are fine.
         if (region.empty())
           continue;
-
-        // Non-empty regions must contain a single basic block.
-        if (std::next(region.begin()) != region.end())
-          return op->emitOpError("expects region #")
-                 << i << " to have 0 or 1 blocks";
-
-        Block &block = region.front();
-        if (block.empty())
-          return op->emitOpError() << "expects a non-empty block";
-        Operation &terminator = block.back();
+        Operation &terminator = region.front().back();
         if (isa<TerminatorOpType>(terminator))
           continue;
 
         return op->emitOpError("expects regions to end with '" +
                                TerminatorOpType::getOperationName() +
                                "', found '" +
                                terminator.getName().getStringRef() + "'")
                    .attachNote()
static void ensureTerminator(Region &region, Builder &builder,
                                            buildTerminator);
     }
     static void ensureTerminator(Region &region, OpBuilder &builder,
                                  Location loc) {
       ::mlir::impl::ensureRegionTerminator(region, builder, loc,
                                            buildTerminator);
     }
 
-    Block *getBody(unsigned idx = 0) {
-      Region &region = this->getOperation()->getRegion(idx);
-      assert(!region.empty() && "unexpected empty region");
-      return &region.front();
-    }
-    Region &getBodyRegion(unsigned idx = 0) {
-      return this->getOperation()->getRegion(idx);
-    }
-
     //===------------------------------------------------------------------===//
     // Single Region Utilities
     //===------------------------------------------------------------------===//
+    using Base::getBody;
 
-    /// The following are a set of methods only enabled when the parent
-    /// operation has a single region. Each of these methods take an additional
-    /// template parameter that represents the concrete operation so that we
-    /// can use SFINAE to disable the methods for non-single region operations.
     template <typename OpT, typename T = void>
     using enable_if_single_region =
         typename std::enable_if_t<OpT::template hasTrait<OneRegion>(), T>;
 
-    template <typename OpT = ConcreteType>
-    enable_if_single_region<OpT, Block::iterator> begin() {
-      return getBody()->begin();
-    }
-    template <typename OpT = ConcreteType>
-    enable_if_single_region<OpT, Block::iterator> end() {
-      return getBody()->end();
-    }
-    template <typename OpT = ConcreteType>
-    enable_if_single_region<OpT, Operation &> front() {
-      return *begin();
-    }
-
     /// Insert the operation into the back of the body, before the terminator.
     template <typename OpT = ConcreteType>
     enable_if_single_region<OpT> push_back(Operation *op) {
       insert(Block::iterator(getBody()->getTerminator()), op);
     }
 
     /// Insert the operation at the given insertion point. Note: The operation
     /// is never inserted after the terminator, even if the insertion point is

mlir/include/mlir/IR/OpImplementation.h

public:
   virtual void
   printOptionalAttrDictWithKeyword(ArrayRef<NamedAttribute> attrs,
                                    ArrayRef<StringRef> elidedAttrs = {}) = 0;
 
   /// Print the entire operation with the default generic assembly form.
   virtual void printGenericOp(Operation *op) = 0;
 
   /// Prints a region.
+  /// If printEmptyBlock is true, then the block header is printed even if the
+  /// block is empty.
   virtual void printRegion(Region &blocks, bool printEntryBlockArgs = true,
-                           bool printBlockTerminators = true) = 0;
+                           bool printBlockTerminators = true,
+                           bool printEmptyBlock = false) = 0;
 
   /// Renumber the arguments for the specified region to the same names as the
   /// SSA values in namesToUse.  This may only be used for IsolatedFromAbove
   /// operations.  If any entry in namesToUse is null, the corresponding
   /// argument name is left alone.
   virtual void shadowRegionArgs(Region &region, ValueRange namesToUse) = 0;
 
   /// Prints an affine map of SSA ids, where SSA id names are used in place

mlir/include/mlir/IR/Region.h

public:
   Location getLoc();
 
   //===--------------------------------------------------------------------===//
   // Block list management
   //===--------------------------------------------------------------------===//
 
   using BlockListType = llvm::iplist<Block>;
   BlockListType &getBlocks() { return blocks; }
+  Block &emplaceBlock() {
+    push_back(new Block);
+    return back();
+  }
 
   // Iteration over the blocks in the region.
   using iterator = BlockListType::iterator;
   using reverse_iterator = BlockListType::reverse_iterator;
 
   iterator begin() { return blocks.begin(); }
   iterator end() { return blocks.end(); }
   reverse_iterator rbegin() { return blocks.rbegin(); }

mlir/include/mlir/IR/RegionKindInterface.h

 /// require SSA-Dominance. If a registered operation does not implement
 /// RegionKindInterface, then any regions it contains are assumed to be
 /// SSACFG regions.
 enum class RegionKind {
   SSACFG,
   Graph,
 };
 
+namespace OpTrait {
+/// A trait that specifies that an operation is only defining graph regions.
+template <typename ConcreteType>
+class HasOnlyGraphRegion : public TraitBase<ConcreteType, HasOnlyGraphRegion> {
+public:
+  static RegionKind getRegionKind(unsigned index) { return RegionKind::Graph; }
+  static bool hasSSADominance(unsigned index) { return false; }
+};
+} // namespace OpTrait
+
 } // namespace mlir
 
 #include "mlir/IR/RegionKindInterface.h.inc"
 
 #endif // MLIR_IR_REGIONKINDINTERFACE_H_

mlir/include/mlir/IR/RegionKindInterface.td

StaticInterfaceMethod<
       /*args=*/(ins "unsigned":$index),
       /*methodBody=*/[{
         return getRegionKind(index) == RegionKind::SSACFG;
       }]
     >,
   ];
 }
 
+def HasOnlyGraphRegion : NativeOpTrait<"HasOnlyGraphRegion">;
+
+// Op's regions that don't need a terminator: requires some other traits
+// so it defines a list that must be concatenated.
+def NoTerminator {
+  list<OpTrait> traits = [
+    NativeOpTrait<"NoTerminator">,
+    SingleBlock,
+    RegionKindInterface,
+    HasOnlyGraphRegion
+  ];
+}
+
 #endif // MLIR_IR_REGIONKINDINTERFACE

mlir/include/mlir/Parser.h

 /// operations within `parsedBlock` into the first block of the first region.
 /// `ContainerOpT` is required to have a single region containing a single
 /// block, and must implement the `SingleBlockImplicitTerminator` trait.
 template <typename ContainerOpT>
 inline OwningOpRef<ContainerOpT> constructContainerOpForParserIfNecessary(
     Block *parsedBlock, MLIRContext *context, Location sourceFileLoc) {
   static_assert(
       ContainerOpT::template hasTrait<OpTrait::OneRegion>() &&
-          std::is_base_of<typename OpTrait::SingleBlockImplicitTerminator<
-                              typename ContainerOpT::ImplicitTerminatorOpT>::
-                              template Impl<ContainerOpT>,
-                          ContainerOpT>::value,
+          (ContainerOpT::template hasTrait<OpTrait::NoTerminator>() ||
+           ContainerOpT::template hasTrait<
+               OpTrait::SingleBlockImplicitTerminator>()),
       "Expected `ContainerOpT` to have a single region with a single "
-      "block that has an implicit terminator");
+      "block that has an implicit terminator or does not require one");
 
   // Check to see if we parsed a single instance of this operation.
   if (llvm::hasSingleElement(*parsedBlock)) {
     if (ContainerOpT op = dyn_cast<ContainerOpT>(parsedBlock->front())) {
       op->remove();
       return op;
     }
   }

mlir/lib/Bindings/Python/mlir/dialects/_builtin_ops_ext.py

 #  Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 #  See https://llvm.org/LICENSE.txt for license information.
 #  SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 from ..ir import *
 
 
 class ModuleOp:
   """Specialization for the module op class."""
 
   def __init__(self, *, loc=None, ip=None):
     super().__init__(self.build_generic(results=[], operands=[], loc=loc,
                                         ip=ip))
     body = self.regions[0].blocks.append()
-    with InsertionPoint(body):
-      Operation.create("module_terminator")
 
   @property
   def body(self):
     return self.regions[0].blocks[0]
 
 
 class FuncOp:
   """Specialization for the func op class."""

mlir/lib/Conversion/AsyncToLLVM/AsyncToLLVM.cpp

static Type resumeFunctionType(MLIRContext *ctx) {
     auto i8Ptr = opaquePointerType(ctx);
     return LLVM::LLVMFunctionType::get(voidTy, {i8Ptr}, false);
   }
 };
 } // namespace
 
 /// Adds Async Runtime C API declarations to the module.
 static void addAsyncRuntimeApiDeclarations(ModuleOp module) {
-  auto builder = ImplicitLocOpBuilder::atBlockTerminator(module.getLoc(),
-                                                         module.getBody());
+  auto builder =
+      ImplicitLocOpBuilder::atBlockEnd(module.getLoc(), module.getBody());
 
   auto addFuncDecl = [&](StringRef name, FunctionType type) {
     if (module.lookupSymbol(name))
       return;
     builder.create<FuncOp>(name, type).setPrivate();
   };
 
   MLIRContext *ctx = module.getContext();
static void addLLVMFuncDecl(ModuleOp module, ImplicitLocOpBuilder &builder,
   builder.create<LLVM::LLVMFuncOp>(name, type);
 }
 
 /// Adds malloc/free declarations to the module.
 static void addCRuntimeDeclarations(ModuleOp module) {
   using namespace mlir::LLVM;
 
   MLIRContext *ctx = module.getContext();
-  ImplicitLocOpBuilder builder(module.getLoc(),
-                               module.getBody()->getTerminator());
+  auto builder =
+      ImplicitLocOpBuilder::atBlockEnd(module.getLoc(), module.getBody());
 
   auto voidTy = LLVMVoidType::get(ctx);
   auto i64 = IntegerType::get(ctx, 64);
   auto i8Ptr = LLVMPointerType::get(IntegerType::get(ctx, 8));
 
   addLLVMFuncDecl(module, builder, kMalloc, i8Ptr, {i64});
   addLLVMFuncDecl(module, builder, kFree, voidTy, {i8Ptr});
 }
 
 //===----------------------------------------------------------------------===//
 // Coroutine resume function wrapper.
 //===----------------------------------------------------------------------===//
 
 static constexpr const char *kResume = "__resume";
 
 /// A function that takes a coroutine handle and calls a `llvm.coro.resume`
 /// intrinsics. We need this function to be able to pass it to the async
 /// runtime execute API.
 static void addResumeFunction(ModuleOp module) {
   if (module.lookupSymbol(kResume))
     return;
 
   MLIRContext *ctx = module.getContext();
-
-  OpBuilder moduleBuilder(module.getBody()->getTerminator());
-  Location loc = module.getLoc();
+  auto loc = module.getLoc();
+  auto moduleBuilder = ImplicitLocOpBuilder::atBlockEnd(loc, module.getBody());
 
   auto voidTy = LLVM::LLVMVoidType::get(ctx);
   auto i8Ptr = LLVM::LLVMPointerType::get(IntegerType::get(ctx, 8));
 
   auto resumeOp = moduleBuilder.create<LLVM::LLVMFuncOp>(
-      loc, kResume, LLVM::LLVMFunctionType::get(voidTy, {i8Ptr}));
+      kResume, LLVM::LLVMFunctionType::get(voidTy, {i8Ptr}));
   resumeOp.setPrivate();
 
   auto *block = resumeOp.addEntryBlock();
   auto blockBuilder = ImplicitLocOpBuilder::atBlockEnd(loc, block);
 
   blockBuilder.create<LLVM::CoroResumeOp>(resumeOp.getArgument(0));
   blockBuilder.create<LLVM::ReturnOp>(ValueRange());
 }

mlir/lib/Conversion/GPUCommon/ConvertLaunchFuncToRuntimeCalls.cpp

 }
 
 LLVM::CallOp FunctionCallBuilder::create(Location loc, OpBuilder &builder,
                                          ArrayRef<Value> arguments) const {
   auto module = builder.getBlock()->getParent()->getParentOfType<ModuleOp>();
   auto function = [&] {
     if (auto function = module.lookupSymbol<LLVM::LLVMFuncOp>(functionName))
       return function;
-    return OpBuilder(module.getBody()->getTerminator())
+    return OpBuilder::atBlockEnd(module.getBody())
         .create<LLVM::LLVMFuncOp>(loc, functionName, functionType);
   }();
   return builder.create<LLVM::CallOp>(
       loc, const_cast<LLVM::LLVMFunctionType &>(functionType).getReturnType(),
       builder.getSymbolRefAttr(function), arguments);
 }
 
 // Returns whether all operands are of LLVM type.

mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp

void ConvertGpuLaunchFuncToVulkanLaunchFunc::runOnOperation() {
 
   for (auto spirvModule :
        llvm::make_early_inc_range(getOperation().getOps<spirv::ModuleOp>()))
     spirvModule.erase();
 }
 
 LogicalResult ConvertGpuLaunchFuncToVulkanLaunchFunc::declareVulkanLaunchFunc(
     Location loc, gpu::LaunchFuncOp launchOp) {
-  OpBuilder builder(getOperation().getBody()->getTerminator());
+  auto builder = OpBuilder::atBlockEnd(getOperation().getBody());
 
   // Workgroup size is written into the kernel. So to properly modelling
   // vulkan launch, we have to skip local workgroup size configuration here.
   SmallVector<Type, 8> gpuLaunchTypes(launchOp.getOperandTypes());
   // The first kVulkanLaunchNumConfigOperands of the gpu.launch_func op are the
   // same as the config operands for the vulkan launch call op.
   SmallVector<Type, 8> vulkanLaunchTypes(gpuLaunchTypes.begin(),
                                          gpuLaunchTypes.begin() +

mlir/lib/Conversion/GPUToVulkan/ConvertLaunchFuncToVulkanCalls.cpp

LogicalResult VulkanLaunchFuncToVulkanCallsPass::deduceMemRefRankAndType(
   rank = llvmDescriptorTy.getBody()[3]
              .cast<LLVM::LLVMArrayType>()
              .getNumElements();
   return success();
 }
 
 void VulkanLaunchFuncToVulkanCallsPass::declareVulkanFunctions(Location loc) {
   ModuleOp module = getOperation();
-  OpBuilder builder(module.getBody()->getTerminator());
+  auto builder = OpBuilder::atBlockEnd(module.getBody());
 
   if (!module.lookupSymbol(kSetEntryPoint)) {
     builder.create<LLVM::LLVMFuncOp>(
         loc, kSetEntryPoint,
         LLVM::LLVMFunctionType::get(getVoidType(),
                                     {getPointerType(), getPointerType()}));
   }
 

mlir/lib/Conversion/LinalgToLLVM/LinalgToLLVM.cpp

void ConvertLinalgToLLVMPass::runOnOperation() {
 
   // Convert to the LLVM IR dialect using the converter defined above.
   OwningRewritePatternList patterns;
   LLVMTypeConverter converter(&getContext());
   populateLinalgToLLVMConversionPatterns(converter, patterns);
 
   LLVMConversionTarget target(getContext());
   target.addIllegalOp<RangeOp>();
-  target.addLegalOp<ModuleOp, ModuleTerminatorOp, LLVM::DialectCastOp>();
+  target.addLegalOp<ModuleOp, LLVM::DialectCastOp>();
   if (failed(applyPartialConversion(module, target, std::move(patterns))))
     signalPassFailure();
 }
 
 std::unique_ptr<OperationPass<ModuleOp>> mlir::createConvertLinalgToLLVMPass() {
   return std::make_unique<ConvertLinalgToLLVMPass>();
 }

mlir/lib/Conversion/LinalgToSPIRV/LinalgToSPIRVPass.cpp

std::unique_ptr<ConversionTarget> target =
       spirv::SPIRVConversionTarget::get(targetAttr);
 
   SPIRVTypeConverter typeConverter(targetAttr);
   OwningRewritePatternList patterns;
   populateLinalgToSPIRVPatterns(context, typeConverter, patterns);
   populateBuiltinFuncToSPIRVPatterns(context, typeConverter, patterns);
 
   // Allow builtin ops.
-  target->addLegalOp<ModuleOp, ModuleTerminatorOp>();
+  target->addLegalOp<ModuleOp>();
   target->addDynamicallyLegalOp<FuncOp>([&](FuncOp op) {
     return typeConverter.isSignatureLegal(op.getType()) &&
            typeConverter.isLegal(&op.getBody());
   });
 
   if (failed(applyFullConversion(module, *target, std::move(patterns))))
     return signalPassFailure();
 }
 
 std::unique_ptr<OperationPass<ModuleOp>> mlir::createLinalgToSPIRVPass() {
   return std::make_unique<LinalgToSPIRVPass>();
 }

mlir/lib/Conversion/LinalgToStandard/LinalgToStandard.cpp

struct ConvertLinalgToStandardPass
   void runOnOperation() override;
 };
 } // namespace
 
 void ConvertLinalgToStandardPass::runOnOperation() {
   auto module = getOperation();
   ConversionTarget target(getContext());
   target.addLegalDialect<AffineDialect, scf::SCFDialect, StandardOpsDialect>();
-  target.addLegalOp<ModuleOp, FuncOp, ModuleTerminatorOp, ReturnOp>();
+  target.addLegalOp<ModuleOp, FuncOp, ReturnOp>();
   target.addLegalOp<linalg::ReshapeOp, linalg::RangeOp>();
   OwningRewritePatternList patterns;
   populateLinalgToStandardConversionPatterns(patterns, &getContext());
   if (failed(applyFullConversion(module, target, std::move(patterns))))
     signalPassFailure();
 }
 
 std::unique_ptr<OperationPass<ModuleOp>>
 mlir::createConvertLinalgToStandardPass() {
   return std::make_unique<ConvertLinalgToStandardPass>();
 }

mlir/lib/Conversion/SPIRVToLLVM/SPIRVToLLVM.cpp

class ModuleEndConversionPattern
     : public SPIRVToLLVMConversion<spirv::ModuleEndOp> {
 public:
   using SPIRVToLLVMConversion<spirv::ModuleEndOp>::SPIRVToLLVMConversion;
 
   LogicalResult
   matchAndRewrite(spirv::ModuleEndOp moduleEndOp, ArrayRef<Value> operands,
                   ConversionPatternRewriter &rewriter) const override {
 
-    rewriter.replaceOpWithNewOp<ModuleTerminatorOp>(moduleEndOp);
+    rewriter.eraseOp(moduleEndOp);
     return success();
   }
 };
 
 } // namespace
 
 //===----------------------------------------------------------------------===//
 // Pattern population

mlir/lib/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.cpp

void ConvertSPIRVToLLVMPass::runOnOperation() {
   populateSPIRVToLLVMModuleConversionPatterns(context, converter, patterns);
   populateSPIRVToLLVMConversionPatterns(context, converter, patterns);
   populateSPIRVToLLVMFunctionConversionPatterns(context, converter, patterns);
 
   ConversionTarget target(getContext());
   target.addIllegalDialect<spirv::SPIRVDialect>();
   target.addLegalDialect<LLVM::LLVMDialect>();
 
-  // Set `ModuleOp` and `ModuleTerminatorOp` as legal for `spv.module`
-  // conversion.
+  // Set `ModuleOp` as legal for `spv.module` conversion.
   target.addLegalOp<ModuleOp>();
-  target.addLegalOp<ModuleTerminatorOp>();
   if (failed(applyPartialConversion(module, target, std::move(patterns))))
     signalPassFailure();
 }
 
 std::unique_ptr<OperationPass<ModuleOp>> mlir::createConvertSPIRVToLLVMPass() {
   return std::make_unique<ConvertSPIRVToLLVMPass>();
 }

mlir/lib/Conversion/ShapeToStandard/ShapeToStandard.cpp

 } // namespace
 
 void ConvertShapeToStandardPass::runOnOperation() {
   // Setup target legality.
   MLIRContext &ctx = getContext();
   ConversionTarget target(ctx);
   target
       .addLegalDialect<StandardOpsDialect, SCFDialect, tensor::TensorDialect>();
-  target.addLegalOp<CstrRequireOp, FuncOp, ModuleOp, ModuleTerminatorOp>();
+  target.addLegalOp<CstrRequireOp, FuncOp, ModuleOp>();
 
   // Setup conversion patterns.
   OwningRewritePatternList patterns;
   populateShapeToStandardConversionPatterns(patterns, &ctx);
 
   // Apply conversion.
   auto module = getOperation();
   if (failed(applyPartialConversion(module, target, std::move(patterns))))

mlir/lib/Conversion/VectorToSPIRV/VectorToSPIRVPass.cpp

void LowerVectorToSPIRVPass::runOnOperation() {
   auto targetAttr = spirv::lookupTargetEnvOrDefault(module);
   std::unique_ptr<ConversionTarget> target =
       spirv::SPIRVConversionTarget::get(targetAttr);
 
   SPIRVTypeConverter typeConverter(targetAttr);
   OwningRewritePatternList patterns;
   populateVectorToSPIRVPatterns(context, typeConverter, patterns);
 
-  target->addLegalOp<ModuleOp, ModuleTerminatorOp>();
+  target->addLegalOp<ModuleOp>();
   target->addLegalOp<FuncOp>();
 
   if (failed(applyFullConversion(module, *target, std::move(patterns))))
     return signalPassFailure();
 }
 
 std::unique_ptr<OperationPass<ModuleOp>>
 mlir::createConvertVectorToSPIRVPass() {
   return std::make_unique<LowerVectorToSPIRVPass>();
 }

mlir/lib/Dialect/Async/Transforms/AsyncToAsyncRuntime.cpp

outlineExecuteOp(SymbolTable &symbolTable, ExecuteOp execute) {
   auto outputTypes = execute.getResultTypes();
 
   auto funcType = FunctionType::get(ctx, inputTypes, outputTypes);
   auto funcAttrs = ArrayRef<NamedAttribute>();
 
   // TODO: Derive outlined function name from the parent FuncOp (support
   // multiple nested async.execute operations).
   FuncOp func = FuncOp::create(loc, kAsyncFnPrefix, funcType, funcAttrs);
-  symbolTable.insert(func, Block::iterator(module.getBody()->getTerminator()));
+  symbolTable.insert(func);
 
   SymbolTable::setSymbolVisibility(func, SymbolTable::Visibility::Private);
 
   // Prepare a function for coroutine lowering by adding entry/cleanup/suspend
   // blocks, adding async.coro operations and setting up control flow.
   CoroMachinery coro = setupCoroMachinery(func);
 
   // Suspend async function at the end of an entry block, and resume it using

mlir/lib/Dialect/StandardOps/Transforms/FuncBufferize.cpp

void runOnOperation() override {
     });
     populateCallOpTypeConversionPattern(patterns, context, typeConverter);
     target.addDynamicallyLegalOp<CallOp>(
         [&](CallOp op) { return typeConverter.isLegal(op); });
 
     populateBranchOpInterfaceTypeConversionPattern(patterns, context,
                                                    typeConverter);
     populateReturnOpTypeConversionPattern(patterns, context, typeConverter);
-    target.addLegalOp<ModuleOp, ModuleTerminatorOp, TensorLoadOp,
-                      TensorToMemrefOp>();
+    target.addLegalOp<ModuleOp, TensorLoadOp, TensorToMemrefOp>();
 
     target.markUnknownOpDynamicallyLegal([&](Operation *op) {
       return isNotBranchOpInterfaceOrReturnLikeOp(op) ||
              isLegalForBranchOpInterfaceTypeConversionPattern(op,
                                                               typeConverter) ||
              isLegalForReturnOpTypeConversionPattern(op, typeConverter);
     });
 

mlir/lib/IR/AsmPrinter.cpp

void print(Block *block, bool printBlockArgs = true,
     auto range = llvm::make_range(
         block->begin(), std::prev(block->end(), printBlockTerminator ? 0 : 1));
     for (Operation &op : range)
       print(&op);
   }
 
   /// Print the given region.
   void printRegion(Region &region, bool printEntryBlockArgs,
-                   bool printBlockTerminators) override {
+                   bool printBlockTerminators,
+                   bool printEmptyBlock = false) override {
     if (region.empty())
       return;
 
     auto *entryBlock = &region.front();
     print(entryBlock, printEntryBlockArgs, printBlockTerminators);
     for (Block &b : llvm::drop_begin(region, 1))
       print(&b);
   }
public:
 
   /// Print an operation successor with the operands used for the block
   /// arguments.
   void printSuccessorAndUseList(Block *successor,
                                 ValueRange succOperands) override;
 
   /// Print the given region.
   void printRegion(Region &region, bool printEntryBlockArgs,
-                   bool printBlockTerminators) override;
+                   bool printBlockTerminators, bool printEmptyBlock) override;
 
   /// Renumber the arguments for the specified region to the same names as the
   /// SSA values in namesToUse. This may only be used for IsolatedFromAbove
   /// operations. If any entry in namesToUse is null, the corresponding
   /// argument name is left alone.
   void shadowRegionArgs(Region &region, ValueRange namesToUse) override {
     state->getSSANameState().shadowRegionArgs(region, namesToUse);
   }
if (op->getNumSuccessors() != 0) {
     os << ']';
   }
 
   // Print regions.
   if (op->getNumRegions() != 0) {
     os << " (";
     interleaveComma(op->getRegions(), [&](Region &region) {
       printRegion(region, /*printEntryBlockArgs=*/true,
-                  /*printBlockTerminators=*/true);
+                  /*printBlockTerminators=*/true, /*printEmptyBlock=*/true);
     });
     os << ')';
   }
 
   auto attrs = op->getAttrs();
   printOptionalAttrDict(attrs);
 
   // Print the type signature of the operation.
if (!block->getParent()) {
       os << "  // " << predIDs.size() << " preds: ";
 
       interleaveComma(predIDs, [&](std::pair<unsigned, Block *> pred) {
         printBlockName(pred.second);
       });
     }
     os << newLine;
   }
-
   currentIndent += indentWidth;
   auto range = llvm::make_range(
       block->begin(), std::prev(block->end(), printBlockTerminator ? 0 : 1));
   for (auto &op : range) {
     print(&op);
     os << newLine;
   }
   currentIndent -= indentWidth;
interleaveComma(succOperands,
                   [this](Value operand) { printValueID(operand); });
   os << " : ";
   interleaveComma(succOperands,
                   [this](Value operand) { printType(operand.getType()); });
   os << ')';
 }
 
 void OperationPrinter::printRegion(Region &region, bool printEntryBlockArgs,
-                                   bool printBlockTerminators) {
+                                   bool printBlockTerminators,
+                                   bool printEmptyBlock) {
   os << " {" << newLine;
   if (!region.empty()) {
     auto *entryBlock = &region.front();
-    print(entryBlock, printEntryBlockArgs && entryBlock->getNumArguments() != 0,
+    print(entryBlock,
+          (printEmptyBlock && entryBlock->empty()) ||
+              (printEntryBlockArgs && entryBlock->getNumArguments() != 0),
           printBlockTerminators);
     for (auto &b : llvm::drop_begin(region.getBlocks(), 1))
       print(&b);
   }
   os.indent(currentIndent) << "}";
 }
 
 void OperationPrinter::printAffineMapOfSSAIds(AffineMapAttr mapAttr,

mlir/lib/IR/Block.cpp

 }
 
 //===----------------------------------------------------------------------===//
 // Predecessors
 //===----------------------------------------------------------------------===//
 
 Block *PredecessorIterator::unwrap(BlockOperand &value) {
   return value.getOwner()->getBlock();
 }
 
 /// Get the successor number in the predecessor terminator.
 unsigned PredecessorIterator::getSuccessorIndex() const {
   return I->getOperandNumber();
 }
 
 //===----------------------------------------------------------------------===//
 // SuccessorRange
 //===----------------------------------------------------------------------===//
 
 SuccessorRange::SuccessorRange() : SuccessorRange(nullptr, 0) {}
 
 SuccessorRange::SuccessorRange(Block *block) : SuccessorRange() {
-  if (Operation *term = block->getTerminator())
+  if (!llvm::hasSingleElement(*block->getParent())) {
+    Operation *term = block->getTerminator();
     if ((count = term->getNumSuccessors()))
       base = term->getBlockOperands().data();
   }
+}
 
 SuccessorRange::SuccessorRange(Operation *term) : SuccessorRange() {
   if ((count = term->getNumSuccessors()))
     base = term->getBlockOperands().data();
 }
 
 //===----------------------------------------------------------------------===//
 // BlockRange

mlir/lib/IR/BuiltinDialect.cpp

 }
 
 //===----------------------------------------------------------------------===//
 // ModuleOp
 //===----------------------------------------------------------------------===//
 
 void ModuleOp::build(OpBuilder &builder, OperationState &state,
                      Optional<StringRef> name) {
-  ensureTerminator(*state.addRegion(), builder, state.location);
+  state.addRegion()->emplaceBlock();
   if (name) {
     state.attributes.push_back(builder.getNamedAttr(
         mlir::SymbolTable::getSymbolAttrName(), builder.getStringAttr(*name)));
   }
 }
 
 /// Construct a module from the given context.
 ModuleOp ModuleOp::create(Location loc, Optional<StringRef> name) {

mlir/lib/IR/SymbolTable.cpp

 /// there and rename it as necessary to avoid collisions.
 void SymbolTable::insert(Operation *symbol, Block::iterator insertPt) {
   // The symbol cannot be the child of another op and must be the child of the
   // symbolTableOp after this.
   //
   // TODO: consider if SymbolTable's constructor should behave the same.
   if (!symbol->getParentOp()) {
     auto &body = symbolTableOp->getRegion(0).front();
-    if (insertPt == Block::iterator() || insertPt == body.end())
-      insertPt = Block::iterator(body.getTerminator());
-
-    assert(insertPt->getParentOp() == symbolTableOp &&
+    if (insertPt == Block::iterator())
+      insertPt = Block::iterator(body.end());
+    else
+      assert((insertPt == body.end() ||
+              insertPt->getParentOp() == symbolTableOp) &&
              "expected insertPt to be in the associated module operation");
+    // Insert before the terminator, if any.
+    if (insertPt == Block::iterator(body.end()) && !body.empty() &&
+        std::prev(body.end())->hasTrait<OpTrait::IsTerminator>())
+      insertPt = std::prev(body.end());
 
     body.getOperations().insert(insertPt, symbol);
   }
   assert(symbol->getParentOp() == symbolTableOp &&
          "symbol is already inserted in another op");
 
   // Add this symbol to the symbol table, uniquing the name if a conflict is
   // detected.

 
 /// Returns the operation registered with the given symbol name with the
 /// regions of 'symbolTableOp'. 'symbolTableOp' is required to be an operation
 /// with the 'OpTrait::SymbolTable' trait. Returns nullptr if no valid symbol
 /// was found.
 Operation *SymbolTable::lookupSymbolIn(Operation *symbolTableOp,
                                        StringRef symbol) {
   assert(symbolTableOp->hasTrait<OpTrait::SymbolTable>());
+  if (symbolTableOp->getRegion(0).empty())
+    return nullptr;
 
   // Look for a symbol with the given name.
   Identifier symbolNameId = Identifier::get(SymbolTable::getSymbolAttrName(),
                                             symbolTableOp->getContext());
-  for (auto &op : symbolTableOp->getRegion(0).front().without_terminator())
+  for (auto &op : symbolTableOp->getRegion(0).front())
     if (getNameIfSymbol(&op, symbolNameId) == symbol)
       return &op;
   return nullptr;
 }
 Operation *SymbolTable::lookupSymbolIn(Operation *symbolTableOp,
                                        SymbolRefAttr symbol) {
   SmallVector<Operation *, 4> resolvedSymbols;
   if (failed(lookupSymbolIn(symbolTableOp, symbol, resolvedSymbols)))

mlir/lib/IR/Verifier.cpp

 public:
   explicit OperationVerifier(MLIRContext *ctx) : ctx(ctx) {}
 
   /// Verify the given operation.
   LogicalResult verify(Operation &op);
 
 private:
   /// Verify the given potentially nested region or block.
-  LogicalResult verifyRegion(Region &region);
-  LogicalResult verifyBlock(Block &block);
+  LogicalResult verifyRegion(Region &region,
+                             RegionKind kind = RegionKind::SSACFG);
+  LogicalResult verifyBlock(Block &block, RegionKind kind);
   LogicalResult verifyOperation(Operation &op);
 
   /// Verify the dominance property of operations within the given Region.
   LogicalResult verifyDominance(Region &region);
 
   /// Verify the dominance property of regions contained within the given
   /// Operation.
   LogicalResult verifyDominanceOfContainedRegions(Operation &op);
LogicalResult OperationVerifier::verify(Operation &op) {
   domInfo = &theDomInfo;
   if (failed(verifyDominanceOfContainedRegions(op)))
     return failure();
 
   domInfo = nullptr;
   return success();
 }
 
-LogicalResult OperationVerifier::verifyRegion(Region &region) {
+LogicalResult OperationVerifier::verifyRegion(Region &region, RegionKind kind) {
   if (region.empty())
     return success();
 
   // Verify the first block has no predecessors.
   auto *firstBB = &region.front();
   if (!firstBB->hasNoPredecessors())
     return mlir::emitError(region.getLoc(),
                            "entry block of region may not have predecessors");
 
   // Verify each of the blocks within the region.
   for (Block &block : region)
-    if (failed(verifyBlock(block)))
+    if (failed(verifyBlock(block, kind)))
       return failure();
   return success();
 }
 
-LogicalResult OperationVerifier::verifyBlock(Block &block) {
+LogicalResult OperationVerifier::verifyBlock(Block &block, RegionKind kind) {
   for (auto arg : block.getArguments())
     if (arg.getOwner() != &block)
       return emitError(block, "block argument not owned by block");
 
   // Verify that this block has a terminator.
-  if (block.empty())
-    return emitError(block, "block with no terminator");
+
+  if (block.empty() && kind != RegionKind::Graph)
+    return emitError(block, "empty block: expect at least a terminator");
 
   // Verify the non-terminator operations separately so that we can verify
   // they has no successors.
   for (auto &op : llvm::make_range(block.begin(), std::prev(block.end()))) {
     if (op.getNumSuccessors() != 0)
       return op.emitError(
           "operation with block successors must terminate its parent block");
 
     if (failed(verifyOperation(op)))
       return failure();
   }
 
   // Verify the terminator.
+
+  if (kind == RegionKind::Graph) {
+    // Graph region don't have terminators.
+    if (!block.empty())
+      return verifyOperation(block.back());
+    return success();
+  }
+
   Operation &terminator = block.back();
   if (failed(verifyOperation(terminator)))
     return failure();
   if (!terminator.mightHaveTrait<OpTrait::IsTerminator>())
-    return block.back().emitError("block with no terminator");
+    return block.back().emitError("block with no terminator, has ")
+           << terminator;
 
   // Verify that this block is not branching to a block of a different
   // region.
   for (Block *successor : block.getSuccessors())
     if (successor->getParent() != block.getParent())
       return block.back().emitOpError(
           "branching to block of a different region");
 
if (opInfo && failed(opInfo->verifyInvariants(&op)))
     return failure();
 
   auto kindInterface = dyn_cast<mlir::RegionKindInterface>(op);
 
   // Verify that all child regions are ok.
   unsigned numRegions = op.getNumRegions();
   for (unsigned i = 0; i < numRegions; i++) {
     Region &region = op.getRegion(i);
+    RegionKind kind =
+        kindInterface ? kindInterface.getRegionKind(i) : RegionKind::SSACFG;
     // Check that Graph Regions only have a single basic block. This is
     // similar to the code in SingleBlockImplicitTerminator, but doesn't
     // require the trait to be specified. This arbitrary limitation is
     // designed to limit the number of cases that have to be handled by
     // transforms and conversions until the concept stabilizes.
-    if (op.isRegistered() && kindInterface &&
-        kindInterface.getRegionKind(i) == RegionKind::Graph) {
+    if (op.isRegistered() && kind == RegionKind::Graph) {
       // Empty regions are fine.
       if (region.empty())
         continue;
 
       // Non-empty regions must contain a single basic block.
       if (std::next(region.begin()) != region.end())
         return op.emitOpError("expects graph region #")
                << i << " to have 0 or 1 blocks";
     }
-    if (failed(verifyRegion(region)))
+    if (failed(verifyRegion(region, kind)))
       return failure();
   }
 
   // If this is a registered operation, there is nothing left to do.
   if (opInfo)
     return success();
 
   // Otherwise, verify that the parent dialect allows un-registered operations.

mlir/lib/Parser/Parser.cpp

case Token::eof: {
       if (failed(verify(topLevelOp.get())))
         return failure();
 
       // Splice the blocks of the parsed operation over to the provided
       // top-level block.
       auto &parsedOps = (*topLevelOp)->getRegion(0).front().getOperations();
       auto &destOps = topLevelBlock->getOperations();
       destOps.splice(destOps.empty() ? destOps.end() : std::prev(destOps.end()),
-                     parsedOps, parsedOps.begin(), std::prev(parsedOps.end()));
+                     parsedOps, parsedOps.begin(), parsedOps.end());
       return success();
     }
 
     // If we got an error token, then the lexer already emitted an error, just
     // stop.  Someday we could introduce error recovery if there was demand
     // for it.
     case Token::error:
       return failure();

mlir/lib/Target/LLVMIR/ConvertFromLLVMIR.cpp

private:
   MLIRContext *context;
   /// The current module being created.
   ModuleOp module;
   /// The entry block of the current function being processed.
   Block *currentEntryBlock;
 
   /// Globals are inserted before the first function, if any.
   Block::iterator getGlobalInsertPt() {
-    auto i = module.getBody()->begin();
-    while (!isa<LLVMFuncOp, ModuleTerminatorOp>(i))
-      ++i;
-    return i;
+    auto it = module.getBody()->begin();
+    auto endIt = module.getBody()->end();
+    while (it != endIt && !isa<LLVMFuncOp>(it))
+      ++it;
+    return it;
   }
 
   /// Functions are always inserted before the module terminator.
   Block::iterator getFuncInsertPt() {
     return std::prev(module.getBody()->end());
   }
 
   /// Remapped blocks, for the current function.

mlir/lib/Transforms/SymbolDCE.cpp

if (failed(computeLiveness(symbolTableOp, symbolTable, symbolTableIsHidden,
     return signalPassFailure();
 
   // After computing the liveness, delete all of the symbols that were found to
   // be dead.
   symbolTableOp->walk([&](Operation *nestedSymbolTable) {
     if (!nestedSymbolTable->hasTrait<OpTrait::SymbolTable>())
       return;
     for (auto &block : nestedSymbolTable->getRegion(0)) {
-      for (Operation &op :
-           llvm::make_early_inc_range(block.without_terminator())) {
+      for (Operation &op : llvm::make_early_inc_range(block)) {
         if (isa<SymbolOpInterface>(&op) && !liveSymbols.count(&op))
           op.erase();
       }
     }
   });
 }
 
 /// Compute the liveness of the symbols within the given symbol table.
 /// `symbolTableIsHidden` is true if this symbol table is known to be
 /// unaccessible from operations in its parent regions.
 LogicalResult SymbolDCE::computeLiveness(Operation *symbolTableOp,
                                          SymbolTableCollection &symbolTable,
                                          bool symbolTableIsHidden,
                                          DenseSet<Operation *> &liveSymbols) {
   // A worklist of live operations to propagate uses from.
   SmallVector<Operation *, 16> worklist;
 
   // Walk the symbols within the current symbol table, marking the symbols that
   // are known to be live.
   for (auto &block : symbolTableOp->getRegion(0)) {
     // Add all non-symbols or symbols that can't be discarded.
-    for (Operation &op : block.without_terminator()) {
+    for (Operation &op : block) {
       SymbolOpInterface symbol = dyn_cast<SymbolOpInterface>(&op);
       if (!symbol) {
         worklist.push_back(&op);
         continue;
       }
       bool isDiscardable = (symbolTableIsHidden || symbol.isPrivate()) &&
                            symbol.canDiscardOnUseEmpty();
       if (!isDiscardable && liveSymbols.insert(&op).second)

mlir/lib/Transforms/Utils/RegionUtils.cpp

 }
 
 static LogicalResult deleteDeadness(MutableArrayRef<Region> regions,
                                     LiveMap &liveMap) {
   bool erasedAnything = false;
   for (Region &region : regions) {
     if (region.empty())
       continue;
+    bool hasSingleBlock = llvm::hasSingleElement(region);
 
     // We do the deletion in an order that deletes all uses before deleting
     // defs.
     // MLIR's SSA structural invariants guarantee that except for block
     // arguments, the use-def graph is acyclic, so this is possible with a
     // single walk of ops and then a final pass to clean up block arguments.
     //
     // To do this, we visit ops in an order that visits domtree children
     // before domtree parents. A CFG post-order (with reverse iteration with a
     // block) satisfies that without needing an explicit domtree calculation.
     for (Block *block : llvm::post_order(&region.front())) {
+      if (!hasSingleBlock)
         eraseTerminatorSuccessorOperands(block->getTerminator(), liveMap);
       for (Operation &childOp :
            llvm::make_early_inc_range(llvm::reverse(block->getOperations()))) {
         erasedAnything |=
             succeeded(deleteDeadness(childOp.getRegions(), liveMap));
         if (!liveMap.wasProvenLive(&childOp)) {
           erasedAnything = true;
           childOp.erase();
         }

mlir/test/Bindings/Python/context_managers.py

 
 run(testLocationEnterExit)
 
 
 # CHECK-LABEL: TEST: testInsertionPointEnterExit
 def testInsertionPointEnterExit():
   ctx1 = Context()
   m = Module.create(Location.unknown(ctx1))
-  ip = InsertionPoint.at_block_terminator(m.body)
+  ip = InsertionPoint(m.body)
 
   with ip:
     assert InsertionPoint.current is ip
     # Asserting a location from the same context should preserve.
     with Location.unknown(ctx1) as loc1:
       assert InsertionPoint.current is ip
       assert Location.current is loc1
     # Location should clear.

mlir/test/Bindings/Python/dialects.py

def createInput():
     op = Operation.create("pytest_dummy.intinput", results=[f32])
     # TODO: Auto result cast from operation
     return op.results[0]
 
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
     m = Module.create()
 
-    with InsertionPoint.at_block_terminator(m.body):
+    with InsertionPoint(m.body):
       f32 = F32Type.get()
       # Create via dialects context collection.
       input1 = createInput()
       input2 = createInput()
       op1 = ctx.dialects.std.AddFOp(input1.type, input1, input2)
 
       # Create via an import
       from mlir.dialects.std import AddFOp

mlir/test/Bindings/Python/dialects/linalg/ops.py

 
 
 # CHECK-LABEL: TEST: testStructuredOpOnTensors
 def testStructuredOpOnTensors():
   with Context() as ctx, Location.unknown():
     module = Module.create()
     f32 = F32Type.get()
     tensor_type = RankedTensorType.get((2, 3, 4), f32)
-    with InsertionPoint.at_block_terminator(module.body):
+    with InsertionPoint(module.body):
       func = builtin.FuncOp(name="matmul_test",
                             type=FunctionType.get(
                                 inputs=[tensor_type, tensor_type],
                                 results=[tensor_type]))
       with InsertionPoint(func.add_entry_block()):
         lhs, rhs = func.entry_block.arguments
         result = linalg.MatmulOp([lhs, rhs], results=[tensor_type]).result
         std.ReturnOp([result])
 
   # CHECK: %[[R:.*]] = linalg.matmul ins(%arg0, %arg1 : tensor<2x3x4xf32>, tensor<2x3x4xf32>) -> tensor<2x3x4xf32>
   print(module)
 
 
 run(testStructuredOpOnTensors)
 
 
 # CHECK-LABEL: TEST: testStructuredOpOnBuffers
 def testStructuredOpOnBuffers():
   with Context() as ctx, Location.unknown():
     module = Module.create()
     f32 = F32Type.get()
     memref_type = MemRefType.get((2, 3, 4), f32)
-    with InsertionPoint.at_block_terminator(module.body):
+    with InsertionPoint(module.body):
       func = builtin.FuncOp(name="matmul_test",
                             type=FunctionType.get(
                                 inputs=[memref_type, memref_type, memref_type],
                                 results=[]))
       with InsertionPoint(func.add_entry_block()):
         lhs, rhs, result = func.entry_block.arguments
         linalg.MatmulOp([lhs, rhs], outputs=[result])
         std.ReturnOp([])
 
   # CHECK: linalg.matmul ins(%arg0, %arg1 : memref<2x3x4xf32>, memref<2x3x4xf32>) outs(%arg2 : memref<2x3x4xf32>)
   print(module)
 
 
 run(testStructuredOpOnBuffers)

mlir/test/Bindings/Python/insertion_point.py

 
 run(test_insert_at_block_terminator_missing)
 
 
 # CHECK-LABEL: TEST: test_insert_at_end_with_terminator_errors
 def test_insert_at_end_with_terminator_errors():
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
-    m = Module.create()  # Module is created with a terminator.
-    with InsertionPoint(m.body):
+    module = Module.parse(r"""
+      func @foo() -> () {
+        return
+      }
+    """)
+    entry_block = module.body.operations[0].regions[0].blocks[0]
+    with InsertionPoint(entry_block):
       try:
         Operation.create("custom.op1", results=[], operands=[])
       except IndexError as e:
         # CHECK: ERROR: Cannot insert operation at the end of a block that already has a terminator.
         print(f"ERROR: {e}")
 
 run(test_insert_at_end_with_terminator_errors)
 

mlir/test/Bindings/Python/ir_operation.py

def testTraverseOpRegionBlockIterators():
 
   # CHECK: REGION 0:
   # CHECK:   BLOCK 0:
   # CHECK:     OP 0: func
   # CHECK:       REGION 0:
   # CHECK:         BLOCK 0:
   # CHECK:           OP 0: %0 = "custom.addi"
   # CHECK:           OP 1: return
-  # CHECK:    OP 1: module_terminator
   walk_operations("", op)
 
 run(testTraverseOpRegionBlockIterators)
 
 
 # Verify index based traversal of the op/region/block hierarchy.
 # CHECK-LABEL: TEST: testTraverseOpRegionBlockIndices
 def testTraverseOpRegionBlockIndices():
def testTraverseOpRegionBlockIndices():
 
   # CHECK: REGION 0:
   # CHECK:   BLOCK 0:
   # CHECK:     OP 0: func
   # CHECK:       REGION 0:
   # CHECK:         BLOCK 0:
   # CHECK:           OP 0: %0 = "custom.addi"
   # CHECK:           OP 1: return
-  # CHECK:    OP 1: module_terminator
   walk_operations("", module.operation)
 
 run(testTraverseOpRegionBlockIndices)
 
 
 # CHECK-LABEL: TEST: testBlockArgumentList
 def testBlockArgumentList():
   with Context() as ctx:
def testSingleResultProperty():
   print(module.body.operations[2])
 
 run(testSingleResultProperty)
 
 # CHECK-LABEL: TEST: testPrintInvalidOperation
 def testPrintInvalidOperation():
   ctx = Context()
   with Location.unknown(ctx):
-    module = Operation.create("module", regions=1)
-    # This block does not have a terminator, it may crash the custom printer.
-    # Verify that we fallback to the generic printer for safety.
+    module = Operation.create("module", regions=2)
+    # This module has two region and is invalid verify that we fallback
+    # to the generic printer for safety.
     block = module.regions[0].blocks.append()
     # CHECK: // Verification failed, printing generic form
     # CHECK: "module"() ( {
     # CHECK: }) : () -> ()
     print(module)
     # CHECK: .verify = False
     print(f".verify = {module.operation.verify()}")
 run(testPrintInvalidOperation)

mlir/test/Bindings/Python/ods_helpers.py

def testOdsBuildDefaultImplicitRegions():
 
   class TestVariadicRegionsOp(OpView):
     OPERATION_NAME = "custom.test_any_regions_op"
     _ODS_REGIONS = (2, False)
 
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
     m = Module.create()
-    with InsertionPoint.at_block_terminator(m.body):
+    with InsertionPoint(m.body):
       op = TestFixedRegionsOp.build_generic(results=[], operands=[])
       # CHECK: NUM_REGIONS: 2
       print(f"NUM_REGIONS: {len(op.regions)}")
       # Including a regions= that matches should be fine.
       op = TestFixedRegionsOp.build_generic(results=[], operands=[], regions=2)
       print(f"NUM_REGIONS: {len(op.regions)}")
       # Reject greater than.
       try:

 def testOdsBuildDefaultNonVariadic():
 
   class TestOp(OpView):
     OPERATION_NAME = "custom.test_op"
 
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
     m = Module.create()
-    with InsertionPoint.at_block_terminator(m.body):
+    with InsertionPoint(m.body):
       v0 = add_dummy_value()
       v1 = add_dummy_value()
       t0 = IntegerType.get_signless(8)
       t1 = IntegerType.get_signless(16)
       op = TestOp.build_generic(results=[t0, t1], operands=[v0, v1])
       # CHECK: %[[V0:.+]] = "custom.value"
       # CHECK: %[[V1:.+]] = "custom.value"
       # CHECK: "custom.test_op"(%[[V0]], %[[V1]])
def testOdsBuildDefaultSizedVariadic():
   class TestOp(OpView):
     OPERATION_NAME = "custom.test_op"
     _ODS_OPERAND_SEGMENTS = [1, -1, 0]
     _ODS_RESULT_SEGMENTS = [-1, 0, 1]
 
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
     m = Module.create()
-    with InsertionPoint.at_block_terminator(m.body):
+    with InsertionPoint(m.body):
       v0 = add_dummy_value()
       v1 = add_dummy_value()
       v2 = add_dummy_value()
       v3 = add_dummy_value()
       t0 = IntegerType.get_signless(8)
       t1 = IntegerType.get_signless(16)
       t2 = IntegerType.get_signless(32)
       t3 = IntegerType.get_signless(64)

 def testOdsBuildDefaultCastError():
 
   class TestOp(OpView):
     OPERATION_NAME = "custom.test_op"
 
   with Context() as ctx, Location.unknown():
     ctx.allow_unregistered_dialects = True
     m = Module.create()
-    with InsertionPoint.at_block_terminator(m.body):
+    with InsertionPoint(m.body):
       v0 = add_dummy_value()
       v1 = add_dummy_value()
       t0 = IntegerType.get_signless(8)
       t1 = IntegerType.get_signless(16)
       try:
         op = TestOp.build_generic(
             results=[t0, t1],
             operands=[None, v1])

mlir/test/Bindings/Python/pass_manager.py

 def testRunPipeline():
   with Context():
     pm = PassManager.parse("print-op-stats")
     module = Module.parse(r"""func @successfulParse() { return }""")
     pm.run(module)
 # CHECK: Operations encountered:
 # CHECK: func              , 1
 # CHECK: module            , 1
-# CHECK: module_terminator , 1
 # CHECK: std.return        , 1
 run(testRunPipeline)

mlir/test/CAPI/ir.c

int collectStats(MlirOperation operation) {
   fprintf(stderr, "Number of attributes: %u\n", stats.numAttributes);
   fprintf(stderr, "Number of blocks: %u\n", stats.numBlocks);
   fprintf(stderr, "Number of regions: %u\n", stats.numRegions);
   fprintf(stderr, "Number of values: %u\n", stats.numValues);
   fprintf(stderr, "Number of block arguments: %u\n", stats.numBlockArguments);
   fprintf(stderr, "Number of op results: %u\n", stats.numOpResults);
   // clang-format off
   // CHECK-LABEL: @stats
-  // CHECK: Number of operations: 13
+  // CHECK: Number of operations: 12
   // CHECK: Number of attributes: 4
   // CHECK: Number of blocks: 3
   // CHECK: Number of regions: 3
   // CHECK: Number of values: 9
   // CHECK: Number of block arguments: 3
   // CHECK: Number of op results: 6
   // clang-format on
   return 0;

mlir/test/CAPI/pass.c

"}"));
   if (mlirModuleIsNull(module)) {
     fprintf(stderr, "Unexpected failure parsing module.\n");
     exit(EXIT_FAILURE);
   }
 
   // Run the print-op-stats pass on the top-level module:
   // CHECK-LABEL: Operations encountered:
   // CHECK: func              , 1
-  // CHECK: module_terminator , 1
   // CHECK: std.addi          , 1
   // CHECK: std.return        , 1
   {
     MlirPassManager pm = mlirPassManagerCreate(ctx);
     MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats();
     mlirPassManagerAddOwnedPass(pm, printOpStatPass);
     MlirLogicalResult success = mlirPassManagerRun(pm, module);
     if (mlirLogicalResultIsFailure(success)) {

 "  }                                                                        \n"
 "}"));
   // clang-format on
   if (mlirModuleIsNull(module))
     exit(1);
 
   // Run the print-op-stats pass on functions under the top-level module:
   // CHECK-LABEL: Operations encountered:
-  // CHECK-NOT: module_terminator
   // CHECK: func              , 1
   // CHECK: std.addi          , 1
   // CHECK: std.return        , 1
   {
     MlirPassManager pm = mlirPassManagerCreate(ctx);
     MlirOpPassManager nestedFuncPm = mlirPassManagerGetNestedUnder(
         pm, mlirStringRefCreateFromCString("func"));
     MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats();
     mlirOpPassManagerAddOwnedPass(nestedFuncPm, printOpStatPass);
     MlirLogicalResult success = mlirPassManagerRun(pm, module);
     if (mlirLogicalResultIsFailure(success))
       exit(2);
     mlirPassManagerDestroy(pm);
   }
   // Run the print-op-stats pass on functions under the nested module:
   // CHECK-LABEL: Operations encountered:
-  // CHECK-NOT: module_terminator
   // CHECK: func              , 1
   // CHECK: std.addf          , 1
   // CHECK: std.return        , 1
   {
     MlirPassManager pm = mlirPassManagerCreate(ctx);
     MlirOpPassManager nestedModulePm = mlirPassManagerGetNestedUnder(
         pm, mlirStringRefCreateFromCString("module"));
     MlirOpPassManager nestedFuncPm = mlirOpPassManagerGetNestedUnder(

mlir/test/Dialect/GPU/outlining.mlir

 // CHECK-NEXT: %[[BDIM:.*]] = "gpu.block_dim"() {dimension = "x"} : () -> index
 // CHECK-NEXT: = "gpu.block_dim"() {dimension = "y"} : () -> index
 // CHECK-NEXT: = "gpu.block_dim"() {dimension = "z"} : () -> index
 // CHECK-NEXT: br ^[[BLOCK:.*]]
 // CHECK-NEXT: ^[[BLOCK]]:
 // CHECK-NEXT: "use"(%[[KERNEL_ARG0]]) : (f32) -> ()
 // CHECK-NEXT: "some_op"(%[[BID]], %[[BDIM]]) : (index, index) -> ()
 // CHECK-NEXT: = load %[[KERNEL_ARG1]][%[[TID]]] : memref<?xf32, 1>
-
-// -----
-
-// CHECK: module attributes {gpu.container_module}
-// CHECK-LABEL: @multiple_launches
-func @multiple_launches() {
-  // CHECK: %[[CST:.*]] = constant 8 : index
-  %cst = constant 8 : index
-  // CHECK: gpu.launch_func @multiple_launches_kernel::@multiple_launches_kernel blocks in (%[[CST]], %[[CST]], %[[CST]]) threads in (%[[CST]], %[[CST]], %[[CST]])
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %cst, %grid_y = %cst,
-                                       %grid_z = %cst)
-             threads(%tx, %ty, %tz) in (%block_x = %cst, %block_y = %cst,
-                                        %block_z = %cst) {
-    gpu.terminator
-  }
-  // CHECK: gpu.launch_func @multiple_launches_kernel_0::@multiple_launches_kernel blocks in (%[[CST]], %[[CST]], %[[CST]]) threads in (%[[CST]], %[[CST]], %[[CST]])
-  gpu.launch blocks(%bx2, %by2, %bz2) in (%grid_x2 = %cst, %grid_y2 = %cst,
-                                          %grid_z2 = %cst)
-             threads(%tx2, %ty2, %tz2) in (%block_x2 = %cst, %block_y2 = %cst,
-                                           %block_z2 = %cst) {
-    gpu.terminator
-  }
-  return
-}
-
-// CHECK: module @multiple_launches_kernel
-// CHECK: func @multiple_launches_kernel
-// CHECK: module @multiple_launches_kernel_0
-// CHECK: func @multiple_launches_kernel
-
-// -----
-
-// CHECK-LABEL: @extra_constants_not_inlined
-func @extra_constants_not_inlined(%arg0: memref<?xf32>) {
-  // CHECK: %[[CST:.*]] = constant 8 : index
-  %cst = constant 8 : index
-  %cst2 = constant 2 : index
-  %c0 = constant 0 : index
-  %cst3 = "secret_constant"() : () -> index
-  // CHECK: gpu.launch_func @extra_constants_not_inlined_kernel::@extra_constants_not_inlined_kernel blocks in (%[[CST]], %[[CST]], %[[CST]]) threads in (%[[CST]], %[[CST]], %[[CST]]) args({{.*}} : memref<?xf32>, {{.*}} : index)
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %cst, %grid_y = %cst,
-                                       %grid_z = %cst)
-             threads(%tx, %ty, %tz) in (%block_x = %cst, %block_y = %cst,
-                                        %block_z = %cst) {
-    "use"(%cst2, %arg0, %cst3) : (index, memref<?xf32>, index) -> ()
-    gpu.terminator
-  }
-  return
-}
-
-// CHECK-LABEL: func @extra_constants_not_inlined_kernel(%{{.*}}: memref<?xf32>, %{{.*}}: index)
-// CHECK: constant 2
-
-// -----
-
-// CHECK-LABEL: @extra_constants
-// CHECK-SAME: %[[ARG0:.*]]: memref<?xf32>
-func @extra_constants(%arg0: memref<?xf32>) {
-  // CHECK: %[[CST:.*]] = constant 8 : index
-  %cst = constant 8 : index
-  %cst2 = constant 2 : index
-  %c0 = constant 0 : index
-  %cst3 = dim %arg0, %c0 : memref<?xf32>
-  // CHECK: gpu.launch_func @extra_constants_kernel::@extra_constants_kernel blocks in (%[[CST]], %[[CST]], %[[CST]]) threads in (%[[CST]], %[[CST]], %[[CST]]) args(%[[ARG0]] : memref<?xf32>)
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %cst, %grid_y = %cst,
-                                       %grid_z = %cst)
-             threads(%tx, %ty, %tz) in (%block_x = %cst, %block_y = %cst,
-                                        %block_z = %cst) {
-    "use"(%cst2, %arg0, %cst3) : (index, memref<?xf32>, index) -> ()
-    gpu.terminator
-  }
-  return
-}
-
-// CHECK-LABEL: func @extra_constants_kernel(
-// CHECK-SAME: %[[KARG0:.*]]: memref<?xf32>
-// CHECK: constant 2
-// CHECK: constant 0
-// CHECK: dim %[[KARG0]]
-
-// -----
-
-// CHECK-LABEL: @extra_constants_noarg
-// CHECK-SAME: %[[ARG0:.*]]: memref<?xf32>, %[[ARG1:.*]]: memref<?xf32>
-func @extra_constants_noarg(%arg0: memref<?xf32>, %arg1: memref<?xf32>) {
-  // CHECK: %[[CST:.*]] = constant 8 : index
-  %cst = constant 8 : index
-  %cst2 = constant 2 : index
-  %c0 = constant 0 : index
-  // CHECK: dim %[[ARG1]]
-  %cst3 = dim %arg1, %c0 : memref<?xf32>
-  // CHECK: gpu.launch_func @extra_constants_noarg_kernel::@extra_constants_noarg_kernel blocks in (%[[CST]], %[[CST]], %[[CST]]) threads in (%[[CST]], %[[CST]], %[[CST]]) args(%[[ARG0]] : memref<?xf32>, {{.*}} : index)
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %cst, %grid_y = %cst,
-                                       %grid_z = %cst)
-             threads(%tx, %ty, %tz) in (%block_x = %cst, %block_y = %cst,
-                                        %block_z = %cst) {
-    "use"(%cst2, %arg0, %cst3) : (index, memref<?xf32>, index) -> ()
-    gpu.terminator
-  }
-  return
-}
-
-// CHECK-LABEL: func @extra_constants_noarg_kernel(
-// CHECK-SAME: %[[KARG0:.*]]: memref<?xf32>, %[[KARG1:.*]]: index
-// CHECK: %[[KCST:.*]] = constant 2
-// CHECK: "use"(%[[KCST]], %[[KARG0]], %[[KARG1]])
-
-// -----
-
-// CHECK-LABEL: @multiple_uses
-func @multiple_uses(%arg0 : memref<?xf32>) {
-  %c1 = constant 1 : index
-  %c2 = constant 2 : index
-  // CHECK: gpu.func {{.*}} {
-  // CHECK:   %[[C2:.*]] = constant 2 : index
-  // CHECK:   "use1"(%[[C2]], %[[C2]])
-  // CHECK:   "use2"(%[[C2]])
-  // CHECK:   gpu.return
-  // CHECK: }
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %c1, %grid_y = %c1,
-                                       %grid_z = %c1)
-             threads(%tx, %ty, %tz) in (%block_x = %c1, %block_y = %c1,
-                                        %block_z = %c1) {
-    "use1"(%c2, %c2) : (index, index) -> ()
-    "use2"(%c2) : (index) -> ()
-    gpu.terminator
-  }
-  return
-}
-
-// -----
-
-// CHECK-LABEL: @multiple_uses2
-func @multiple_uses2(%arg0 : memref<*xf32>) {
-  %c1 = constant 1 : index
-  %c2 = constant 2 : index
-  %d = dim %arg0, %c2 : memref<*xf32>
-  // CHECK: gpu.func {{.*}} {
-  // CHECK:   %[[C2:.*]] = constant 2 : index
-  // CHECK:   %[[D:.*]] = dim %[[ARG:.*]], %[[C2]]
-  // CHECK:   "use1"(%[[D]])
-  // CHECK:   "use2"(%[[C2]], %[[C2]])
-  // CHECK:   "use3"(%[[ARG]])
-  // CHECK:   gpu.return
-  // CHECK: }
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %c1, %grid_y = %c1,
-                                       %grid_z = %c1)
-             threads(%tx, %ty, %tz) in (%block_x = %c1, %block_y = %c1,
-                                        %block_z = %c1) {
-    "use1"(%d) : (index) -> ()
-    "use2"(%c2, %c2) : (index, index) -> ()
-    "use3"(%arg0) : (memref<*xf32>) -> ()
-    gpu.terminator
-  }
-  return
-}
-
-// -----
-
-llvm.mlir.global internal @global(42 : i64) : i64
-
-//CHECK-LABEL: @function_call
-func @function_call(%arg0 : memref<?xf32>) {
-  %cst = constant 8 : index
-  gpu.launch blocks(%bx, %by, %bz) in (%grid_x = %cst, %grid_y = %cst,
-                                       %grid_z = %cst)
-             threads(%tx, %ty, %tz) in (%block_x = %cst, %block_y = %cst,
-                                        %block_z = %cst) {
-    call @device_function() : () -> ()
-    call @device_function() : () -> ()
-    %0 = llvm.mlir.addressof @global : !llvm.ptr<i64>
-    gpu.terminator
-  }
-  return
-}
-
-func @device_function() {
-  call @recursive_device_function() : () -> ()
-  return
-}
-
-func @recursive_device_function() {
-  call @recursive_device_function() : () -> ()
-  return
-}
-
-// CHECK: gpu.module @function_call_kernel {
-// CHECK:   gpu.func @function_call_kernel()
-// CHECK:     call @device_function() : () -> ()
-// CHECK:     call @device_function() : () -> ()
-// CHECK:     llvm.mlir.addressof @global : !llvm.ptr<i64>
-// CHECK:     gpu.return
-//
-// CHECK:   llvm.mlir.global internal @global(42 : i64) : i64
-//
-// CHECK:   func @device_function()
-// CHECK:   func @recursive_device_function()
-// CHECK-NOT:   func @device_function

mlir/test/IR/invalid-module-op.mlir

 }
 
 // -----
 
 func @module_op() {
   // expected-error@+1 {{region should have no arguments}}
   module {
   ^bb1(%arg: i32):
-    "module_terminator"() : () -> ()
-  }
-  return
-}
-
-// -----
-
-func @module_op() {
-  // expected-error@below {{expects regions to end with 'module_terminator'}}
-  // expected-note@below {{the absence of terminator implies 'module_terminator'}}
-  module {
-    return
   }
   return
 }
 
 // -----
 
-func @module_op() {
-  // expected-error@+1 {{expects parent op 'module'}}
-  "module_terminator"() : () -> ()
-}
-
-// -----
-
 // expected-error@+1 {{can only contain attributes with dialect-prefixed names}}
 module attributes {attr} {
 }
 

mlir/test/IR/invalid.mlir

 ^bb42:
   return
 ^bb42:        // expected-error {{redefinition of block '^bb42'}}
   return
 }
 
 // -----
 
-func @no_terminator() {   // expected-error {{block with no terminator}}
+func @no_terminator() {   // expected-error {{empty block: expect at least a terminator}}
 ^bb40:
   return
 ^bb41:
 ^bb42:
   return
 }
 
 // -----

mlir/test/IR/module-op.mlir

 // RUN: mlir-opt -allow-unregistered-dialect %s -split-input-file -mlir-print-debuginfo -mlir-print-local-scope  | FileCheck %s
 
 // CHECK: module {
 module {
 }
 
-// CHECK: module {
-// CHECK-NEXT: }
-module {
-  "module_terminator"() : () -> ()
-}
+// -----
 
 // CHECK: module attributes {foo.attr = true} {
 module attributes {foo.attr = true} {
 }
 
+// -----
+
 // CHECK: module {
 module {
   // CHECK-NEXT: "foo.result_op"() : () -> i32
   %result = "foo.result_op"() : () -> i32
 }
 
 // -----
 

mlir/test/IR/print-ir-defuse.mlir

 // CHECK:   - Operand produced by operation 'dialect.op1'
 // CHECK: Has 0 results:
 // CHECK: Visiting op 'dialect.innerop1' with 2 operands:
 // CHECK:   - Operand produced by Block argument, number 0
 // CHECK:   - Operand produced by operation 'dialect.op1'
 // CHECK: Has 0 results:
 // CHECK: Visiting op 'dialect.op3' with 0 operands:
 // CHECK: Has 0 results:
-// CHECK: Visiting op 'module_terminator' with 0 operands:
-// CHECK: Has 0 results:
 // CHECK: Visiting op 'module' with 0 operands:
 // CHECK: Has 0 results:
 
 %results:4 = "dialect.op1"() : () -> (i1, i16, i32, i64)
 "dialect.op2"(%results#0, %results#2) : (i1, i32) -> ()
 "dialect.op3"() ({
   ^bb0(%arg0 : i1):
     "dialect.innerop1"(%arg0, %results#2) : (i1, i32) -> ()
 }) : () -> ()

mlir/test/IR/print-ir-nesting.mlir

 // RUN: mlir-opt -test-print-nesting  -allow-unregistered-dialect %s | FileCheck %s
 
 // CHECK: visiting op: 'module' with 0 operands and 0 results
 // CHECK:  1 nested regions:
 // CHECK:   Region with 1 blocks:
-// CHECK:     Block with 0 arguments, 0 successors, and 3 operations
+// CHECK:     Block with 0 arguments, 0 successors, and 2 operations
 module {
 
 
 // CHECK:       visiting op: 'dialect.op1' with 0 operands and 4 results
 // CHECK:       1 attributes:
 // CHECK:        - 'attribute name' : '42 : i32'
 // CHECK:        0 nested regions:
   %results:4 = "dialect.op1"() { "attribute name" = 42 : i32 } : () -> (i1, i16, i32, i64)

 // CHECK:             visiting op: 'dialect.innerop6' with 0 operands and 0 results
 // CHECK:              0 nested regions:
     "dialect.innerop6"() : () -> ()
 // CHECK:             visiting op: 'dialect.innerop7' with 0 operands and 0 results
 // CHECK:              0 nested regions:
     "dialect.innerop7"() : () -> ()
   }) : () -> ()
 
-// CHECK:       visiting op: 'module_terminator' with 0 operands and 0 results
-
 } // module

mlir/test/Transforms/test-legalizer-analysis.mlir

 // RUN: mlir-opt -allow-unregistered-dialect -test-legalize-patterns -verify-diagnostics -test-legalize-mode=analysis %s | FileCheck %s
 // expected-remark@-2 {{op 'module' is legalizable}}
-// expected-remark@-3 {{op 'module_terminator' is legalizable}}
 
 // expected-remark@+1 {{op 'func' is legalizable}}
 func @test(%arg0: f32) {
   // expected-remark@+1 {{op 'test.illegal_op_a' is legalizable}}
   %result = "test.illegal_op_a"() : () -> (i32)
   "foo.region"() ({
       // expected-remark@+1 {{op 'test.invalid' is legalizable}}
       "test.invalid"() : () -> ()
   }) : () -> ()
   return
 }
 
 // Check that none of the legalizable operations were modified.
 // CHECK-LABEL: func @test
 // CHECK-NEXT: "test.illegal_op_a"
 // CHECK: "test.invalid"

mlir/test/lib/Dialect/Test/TestPatterns.cpp

void runOnOperation() override {
     patterns.insert<TestDropOpSignatureConversion>(&getContext(), converter);
     mlir::populateFuncOpTypeConversionPattern(patterns, &getContext(),
                                               converter);
     mlir::populateCallOpTypeConversionPattern(patterns, &getContext(),
                                               converter);
 
     // Define the conversion target used for the test.
     ConversionTarget target(getContext());
-    target.addLegalOp<ModuleOp, ModuleTerminatorOp>();
+    target.addLegalOp<ModuleOp>();
     target.addLegalOp<LegalOpA, LegalOpB, TestCastOp, TestValidOp,
                       TerminatorOp>();
     target
         .addIllegalOp<ILLegalOpF, TestRegionBuilderOp, TestOpWithRegionFold>();
     target.addDynamicallyLegalOp<TestReturnOp>([](TestReturnOp op) {
       // Don't allow F32 operands.
       return llvm::none_of(op.getOperandTypes(),
                            [](Type type) { return type.isF32(); });

 
 struct TestRemappedValue
     : public mlir::PassWrapper<TestRemappedValue, FunctionPass> {
   void runOnFunction() override {
     mlir::OwningRewritePatternList patterns;
     patterns.insert<OneVResOneVOperandOp1Converter>(&getContext());
 
     mlir::ConversionTarget target(getContext());
-    target.addLegalOp<ModuleOp, ModuleTerminatorOp, FuncOp, TestReturnOp>();
+    target.addLegalOp<ModuleOp, FuncOp, TestReturnOp>();
     // We make OneVResOneVOperandOp1 legal only when it has more that one
     // operand. This will trigger the conversion that will replace one-operand
     // OneVResOneVOperandOp1 with two-operand OneVResOneVOperandOp1.
     target.addDynamicallyLegalOp<OneVResOneVOperandOp1>(
         [](Operation *op) -> bool {
           return std::distance(op->operand_begin(), op->operand_end()) > 1;
         });
 
: public PassWrapper<TestMergeBlocksPatternDriver,
                          OperationPass<ModuleOp>> {
   void runOnOperation() override {
     mlir::OwningRewritePatternList patterns;
     MLIRContext *context = &getContext();
     patterns
         .insert<TestMergeBlock, TestUndoBlocksMerge, TestMergeSingleBlockOps>(
             context);
     ConversionTarget target(*context);
-    target.addLegalOp<FuncOp, ModuleOp, ModuleTerminatorOp, TerminatorOp,
-                      TestBranchOp, TestTypeConsumerOp, TestTypeProducerOp,
-                      TestReturnOp>();
+    target.addLegalOp<FuncOp, ModuleOp, TerminatorOp, TestBranchOp,
+                      TestTypeConsumerOp, TestTypeProducerOp, TestReturnOp>();
     target.addIllegalOp<ILLegalOpF>();
 
     /// Expect the op to have a single block after legalization.
     target.addDynamicallyLegalOp<TestMergeBlocksOp>(
         [&](TestMergeBlocksOp op) -> bool {
           return llvm::hasSingleElement(op.body());
         });
 

mlir/test/lib/Transforms/TestConvVectorization.cpp

 
 void TestConvVectorization::runOnOperation() {
   MLIRContext *context = &getContext();
   ModuleOp module = getOperation();
 
   ConversionTarget target(*context);
   target.addLegalDialect<AffineDialect, scf::SCFDialect, StandardOpsDialect,
                          VectorDialect>();
-  target.addLegalOp<ModuleOp, FuncOp, ModuleTerminatorOp, ReturnOp>();
+  target.addLegalOp<ModuleOp, FuncOp, ReturnOp>();
   target.addLegalOp<linalg::FillOp, linalg::YieldOp>();
 
   SmallVector<OwningRewritePatternList, 4> stage1Patterns;
   linalg::populateConvVectorizationPatterns(context, stage1Patterns, tileSizes);
   SmallVector<FrozenRewritePatternList, 4> frozenStage1Patterns;
   llvm::move(stage1Patterns, std::back_inserter(frozenStage1Patterns));
 
   OwningRewritePatternList stage2Patterns =

mlir/tools/mlir-tblgen/OpFormatGen.cpp

OperationFormat(const Operator &op)
       : allOperands(false), allOperandTypes(false), allResultTypes(false) {
     operandTypes.resize(op.getNumOperands(), TypeResolution());
     resultTypes.resize(op.getNumResults(), TypeResolution());
 
     hasImplicitTermTrait =
         llvm::any_of(op.getTraits(), [](const OpTrait &trait) {
           return trait.getDef().isSubClassOf("SingleBlockImplicitTerminator");
         });
+
+    hasSingleBlockTrait =
+        hasImplicitTermTrait ||
+        llvm::any_of(op.getTraits(), [](const OpTrait &trait) {
+          if (auto *native = dyn_cast<NativeOpTrait>(&trait)) {
+            return native->getTrait() == "::mlir::OpTrait::SingleBlock";
+          }
+          return false;
+        });
   }
 
   /// Generate the operation parser from this format.
   void genParser(Operator &op, OpClass &opClass);
   /// Generate the parser code for a specific format element.
   void genElementParser(Element *element, OpMethodBody &body,
                         FmtContext &attrTypeCtx);
   /// Generate the c++ to resolve the types of operands and results during
struct OperationFormat {
   /// A flag indicating if all operand/result types were seen. If the format
   /// contains these, it can not contain individual type resolvers.
   bool allOperands, allOperandTypes, allResultTypes;
 
   /// A flag indicating if this operation has the SingleBlockImplicitTerminator
   /// trait.
   bool hasImplicitTermTrait;
 
+  /// A flag indicating if this operation has the SingleBlock trait.
+  bool hasSingleBlockTrait;
+
   /// A map of buildable types to indices.
   llvm::MapVector<StringRef, int, llvm::StringMap<int>> buildableTypes;
 
   /// The index of the buildable type, if valid, for every operand and result.
   std::vector<TypeResolution> operandTypes, resultTypes;
 
   /// The set of attributes explicitly used within the format.
   SmallVector<const NamedAttribute *, 8> usedAttributes;

 /// The code snippet used to ensure a list of regions have terminators.
 ///
 /// {0}: The name of the region list.
 const char *regionListEnsureTerminatorParserCode = R"(
   for (auto &region : {0}Regions)
     ensureTerminator(*region, parser.getBuilder(), result.location);
 )";
 
+/// The code snippet used to ensure a list of regions have a block.
+///
+/// {0}: The name of the region list.
+const char *regionListEnsureSingleBlockParserCode = R"(
+  for (auto &region : {0}Regions)
+    if (region.empty()) *{0}Region.emplaceBlock();
+)";
+
 /// The code snippet used to generate a parser call for an optional region.
 ///
 /// {0}: The name of the region.
 const char *optionalRegionParserCode = R"(
   {
      auto parseResult = parser.parseOptionalRegion(*{0}Region);
      if (parseResult.hasValue() && failed(*parseResult))
        return ::mlir::failure();

 
 /// The code snippet used to ensure a region has a terminator.
 ///
 /// {0}: The name of the region.
 const char *regionEnsureTerminatorParserCode = R"(
   ensureTerminator(*{0}Region, parser.getBuilder(), result.location);
 )";
 
+/// The code snippet used to ensure a region has a block.
+///
+/// {0}: The name of the region.
+const char *regionEnsureSingleBlockParserCode = R"(
+  if ({0}Region->empty()) {0}Region->emplaceBlock();
+)";
+
 /// The code snippet used to generate a parser call for a successor list.
 ///
 /// {0}: The name for the successor list.
 const char *successorListParserCode = R"(
   {
     ::mlir::Block *succ;
     auto firstSucc = parser.parseOptionalSuccessor(succ);
     if (firstSucc.hasValue()) {
if (auto *attrVar = dyn_cast<AttributeVariable>(firstElement)) {
       if (region->isVariadic()) {
         genElementParser(regionVar, body, attrTypeCtx);
         body << "  if (!" << region->name << "Regions.empty()) {\n";
       } else {
         body << llvm::formatv(optionalRegionParserCode, region->name);
         body << "  if (!" << region->name << "Region->empty()) {\n  ";
         if (hasImplicitTermTrait)
           body << llvm::formatv(regionEnsureTerminatorParserCode, region->name);
+        else if (hasSingleBlockTrait)
+          body << llvm::formatv(regionEnsureSingleBlockParserCode,
+                                region->name);
       }
     }
 
     // If the anchor is a unit attribute, we don't need to print it. When
     // parsing, we will add this attribute if this group is present.
     Element *elidedAnchorElement = nullptr;
     auto *anchorAttr = dyn_cast<AttributeVariable>(optional->getAnchor());
     if (anchorAttr && anchorAttr != firstElement && anchorAttr->isUnitAttr()) {
else if (lengthKind == ArgumentLengthKind::Optional)
       body << llvm::formatv(optionalOperandParserCode, name);
     else
       body << formatv(operandParserCode, name);
 
   } else if (auto *region = dyn_cast<RegionVariable>(element)) {
     bool isVariadic = region->getVar()->isVariadic();
     body << llvm::formatv(isVariadic ? regionListParserCode : regionParserCode,
                           region->getVar()->name);
-    if (hasImplicitTermTrait) {
+    if (hasImplicitTermTrait)
       body << llvm::formatv(isVariadic ? regionListEnsureTerminatorParserCode
                                        : regionEnsureTerminatorParserCode,
                             region->getVar()->name);
-    }
+    else if (hasSingleBlockTrait)
+      body << llvm::formatv(isVariadic ? regionListEnsureSingleBlockParserCode
+                                       : regionEnsureSingleBlockParserCode,
+                            region->getVar()->name);
 
   } else if (auto *successor = dyn_cast<SuccessorVariable>(element)) {
     bool isVariadic = successor->getVar()->isVariadic();
     body << formatv(isVariadic ? successorListParserCode : successorParserCode,
                     successor->getVar()->name);
 
     /// Directives.
   } else if (auto *attrDict = dyn_cast<AttrDictDirective>(element)) {
     body << "  if (parser.parseOptionalAttrDict"
          << (attrDict->isWithKeyword() ? "WithKeyword" : "")
          << "(result.attributes))\n"
          << "    return ::mlir::failure();\n";
   } else if (auto *customDir = dyn_cast<CustomDirective>(element)) {
     genCustomDirectiveParser(customDir, body);
 
   } else if (isa<OperandsDirective>(element)) {
     body << "  ::llvm::SMLoc allOperandLoc = parser.getCurrentLocation();\n"
          << "  if (parser.parseOperandList(allOperands))\n"
          << "    return ::mlir::failure();\n";
 
   } else if (isa<RegionsDirective>(element)) {
     body << llvm::formatv(regionListParserCode, "full");
     if (hasImplicitTermTrait)
       body << llvm::formatv(regionListEnsureTerminatorParserCode, "full");
+    else if (hasSingleBlockTrait)
+      body << llvm::formatv(regionListEnsureSingleBlockParserCode, "full");
 
   } else if (isa<SuccessorsDirective>(element)) {
     body << llvm::formatv(successorListParserCode, "full");
 
   } else if (auto *dir = dyn_cast<TypeDirective>(element)) {
     ArgumentLengthKind lengthKind;
     StringRef listName = getTypeListName(dir->getOperand(), lengthKind);
     if (lengthKind == ArgumentLengthKind::Variadic)