[mlir][python] Normalize asm-printing IR behavior.

[stellaraccident]

Summary

While working on an integration, I found a lot of inconsistencies on IR printing and verification. It turns out that we were:

• Only doing "soft fail" verification on IR printing of Operation, not of a Module.
• Failed verification was interacting badly with binary=True IR printing (causing a TypeError trying to pass an str to a bytes based handle).
• For systematic integrations, it is often desirable to control verification yourself so that you can explicitly handle errors.

This patch:

• Trues up the "soft fail" semantics by having Module.__str__ delegate to Operation.__str__ vs having a shortcut implementation.
• Fixes soft fail in the presence of binary=True (and adds an additional happy path test case to make sure the binary functionality works).
• Adds an assume_verified boolean flag to the print/get_asm methods which disables internal verification, presupposing that the caller has taken care of it.

It turns out that we had a number of tests which were generating illegal IR but it wasn't being caught because they were doing a print on the Module vs operation. All except two were trivially fixed:

• linalg/ops.py : Had two tests for direct constructing a Matmul incorrectly. Fixing them made them just like the next two tests so just deleted (no need to test the verifier only at this level).
• linalg/opdsl/emit_structured_generic.py : Hand coded conv and pooling tests appear to be using illegal shaped inputs/outputs, causing a verification failure. I just used the assume_verified= flag to restore the original behavior and left a TODO. Will get someone who owns that to fix it properly in a followup (would also be nice to break this file up into multiple test modules as it is hard to tell exactly what is failing).

Notes to downstreams:

• If, like some of our tests, you get verification failures after this patch, it is likely that your IR was always invalid and you will need to fix the root cause. To temporarily revert to prior (broken) behavior, replace calls like print(module) with print(module.operation.get_asm(assume_verified=True)).

[mehdi_amini]

Seems like you went down a rabbit hole to get there! Thanks for fixing it all up :)

mlir/lib/Bindings/Python/IRCore.cpp

enable_debug_info: Whether to print debug/location information. Defaults
     to False.
   pretty_debug_info: Whether to format debug information for easier reading
     by a human (warning: the result is unparseable).
   print_generic_op_form: Whether to print the generic assembly forms of all
     ops. Defaults to False.
   use_local_Scope: Whether to print in a way that is more optimized for
     multi-threaded access but may not be consistent with how the overall
     module prints.
+  assume_verified: By default, if not printing generic form, the verifier
+    will be run and if it fails, generic form will be printed with a comment
+    about failed verification. While a reasonable default for interactive use,
+    for systematic use, it is often better for the caller to verify explicitly
+    and report failures in a more robust fashion. Set this to True if doing this
+    in order to avoid running a redundant verification. If the IR is actually
+    invalid, behavior is undefined.
 )";
 
 static const char kOperationGetAsmDocstring[] =
     R"(Gets the assembly form of the operation with all options available.
 
 Args:
   binary: Whether to return a bytes (True) or str (False) object. Defaults to
     False.
void PyOperation::checkValid() const {
   if (!valid) {
     throw SetPyError(PyExc_RuntimeError, "the operation has been invalidated");
   }
 }
 
 void PyOperationBase::print(py::object fileObject, bool binary,
                             llvm::Optional<int64_t> largeElementsLimit,
                             bool enableDebugInfo, bool prettyDebugInfo,
-                            bool printGenericOpForm, bool useLocalScope) {
+                            bool printGenericOpForm, bool useLocalScope,
+                            bool assumeVerified) {
   PyOperation &operation = getOperation();
   operation.checkValid();
   if (fileObject.is_none())
     fileObject = py::module::import("sys").attr("stdout");
 
-  if (!printGenericOpForm && !mlirOperationVerify(operation)) {
-    fileObject.attr("write")("// Verification failed, printing generic form\n");
+  if (!assumeVerified && !printGenericOpForm &&
+      !mlirOperationVerify(operation)) {
+    std::string message("// Verification failed, printing generic form\n");
+    if (binary) {
+      fileObject.attr("write")(py::bytes(message));
+    } else {
+      fileObject.attr("write")(py::str(message));
+    }
     printGenericOpForm = true;
   }
 
   MlirOpPrintingFlags flags = mlirOpPrintingFlagsCreate();
   if (largeElementsLimit)
     mlirOpPrintingFlagsElideLargeElementsAttrs(flags, *largeElementsLimit);
   if (enableDebugInfo)
     mlirOpPrintingFlagsEnableDebugInfo(flags, /*prettyForm=*/prettyDebugInfo);
   if (printGenericOpForm)
     mlirOpPrintingFlagsPrintGenericOpForm(flags);
 
   PyFileAccumulator accum(fileObject, binary);
   py::gil_scoped_release();
   mlirOperationPrintWithFlags(operation, flags, accum.getCallback(),
                               accum.getUserData());
   mlirOpPrintingFlagsDestroy(flags);
 }
 
 py::object PyOperationBase::getAsm(bool binary,
                                    llvm::Optional<int64_t> largeElementsLimit,
                                    bool enableDebugInfo, bool prettyDebugInfo,
-                                   bool printGenericOpForm,
-                                   bool useLocalScope) {
+                                   bool printGenericOpForm, bool useLocalScope,
+                                   bool assumeVerified) {
   py::object fileObject;
   if (binary) {
     fileObject = py::module::import("io").attr("BytesIO")();
   } else {
     fileObject = py::module::import("io").attr("StringIO")();
   }
   print(fileObject, /*binary=*/binary,
         /*largeElementsLimit=*/largeElementsLimit,
         /*enableDebugInfo=*/enableDebugInfo,
         /*prettyDebugInfo=*/prettyDebugInfo,
         /*printGenericOpForm=*/printGenericOpForm,
-        /*useLocalScope=*/useLocalScope);
+        /*useLocalScope=*/useLocalScope,
+        /*assumeVerified=*/assumeVerified);
 
   return fileObject.attr("getvalue")();
 }
 
 void PyOperationBase::moveAfter(PyOperationBase &other) {
   PyOperation &operation = getOperation();
   PyOperation &otherOp = other.getOperation();
   operation.checkValid();
py::class_<PyModule>(m, "Module", py::module_local())
       .def(
           "dump",
           [](PyModule &self) {
             mlirOperationDump(mlirModuleGetOperation(self.get()));
           },
           kDumpDocstring)
       .def(
           "__str__",
-          [](PyModule &self) {
-            MlirOperation operation = mlirModuleGetOperation(self.get());
-            PyPrintAccumulator printAccum;
-            mlirOperationPrint(operation, printAccum.getCallback(),
-                               printAccum.getUserData());
-            return printAccum.join();
+          [](py::object self) {
+            // Defer to the operation's __str__.
+            return self.attr("operation").attr("__str__")();
           },
           kOperationStrDunderDocstring);
 
   //----------------------------------------------------------------------------
   // Mapping of Operation.
   //----------------------------------------------------------------------------
   py::class_<PyOperationBase>(m, "_OperationBase", py::module_local())
       .def_property_readonly(MLIR_PYTHON_CAPI_PTR_ATTR,
py::class_<PyOperationBase>(m, "_OperationBase", py::module_local())
       .def(
           "__str__",
           [](PyOperationBase &self) {
             return self.getAsm(/*binary=*/false,
                                /*largeElementsLimit=*/llvm::None,
                                /*enableDebugInfo=*/false,
                                /*prettyDebugInfo=*/false,
                                /*printGenericOpForm=*/false,
-                               /*useLocalScope=*/false);
+                               /*useLocalScope=*/false,
+                               /*assumeVerified=*/false);
           },
           "Returns the assembly form of the operation.")
       .def("print", &PyOperationBase::print,
            // Careful: Lots of arguments must match up with print method.
            py::arg("file") = py::none(), py::arg("binary") = false,
            py::arg("large_elements_limit") = py::none(),
            py::arg("enable_debug_info") = false,
            py::arg("pretty_debug_info") = false,
            py::arg("print_generic_op_form") = false,
-           py::arg("use_local_scope") = false, kOperationPrintDocstring)
+           py::arg("use_local_scope") = false,
+           py::arg("assume_verified") = false, kOperationPrintDocstring)
       .def("get_asm", &PyOperationBase::getAsm,
            // Careful: Lots of arguments must match up with get_asm method.
            py::arg("binary") = false,
            py::arg("large_elements_limit") = py::none(),
            py::arg("enable_debug_info") = false,
            py::arg("pretty_debug_info") = false,
            py::arg("print_generic_op_form") = false,
-           py::arg("use_local_scope") = false, kOperationGetAsmDocstring)
+           py::arg("use_local_scope") = false,
+           py::arg("assume_verified") = false, kOperationGetAsmDocstring)
       .def(
           "verify",
           [](PyOperationBase &self) {
             return mlirOperationVerify(self.getOperation());
           },
           "Verify the operation and return true if it passes, false if it "
           "fails.")
       .def("move_after", &PyOperationBase::moveAfter, py::arg("other"),

mlir/lib/Bindings/Python/IRModule.h

 /// Base class for PyOperation and PyOpView which exposes the primary, user
 /// visible methods for manipulating it.
 class PyOperationBase {
 public:
   virtual ~PyOperationBase() = default;
   /// Implements the bound 'print' method and helps with others.
   void print(pybind11::object fileObject, bool binary,
              llvm::Optional<int64_t> largeElementsLimit, bool enableDebugInfo,
-             bool prettyDebugInfo, bool printGenericOpForm, bool useLocalScope);
+             bool prettyDebugInfo, bool printGenericOpForm, bool useLocalScope,
+             bool assumeVerified);
   pybind11::object getAsm(bool binary,
                           llvm::Optional<int64_t> largeElementsLimit,
                           bool enableDebugInfo, bool prettyDebugInfo,
-                          bool printGenericOpForm, bool useLocalScope);
+                          bool printGenericOpForm, bool useLocalScope,
+                          bool assumeVerified);
 
   /// Moves the operation before or after the other operation.
   void moveAfter(PyOperationBase &other);
   void moveBefore(PyOperationBase &other);
 
   /// Each must provide access to the raw Operation.
   virtual PyOperation &getOperation() = 0;
 };

mlir/test/python/dialects/builtin.py

def testBuildFuncOp():
   # CHECK:   return %arg0 : tensor<2x3x4xf32>
   # CHECK:  }
   print(m)
 
 
 # CHECK-LABEL: TEST: testFuncArgumentAccess
 @run
 def testFuncArgumentAccess():
-  with Context(), Location.unknown():
+  with Context() as ctx, Location.unknown():
+    ctx.allow_unregistered_dialects = True
     module = Module.create()
     f32 = F32Type.get()
     f64 = F64Type.get()
     with InsertionPoint(module.body):
       func = builtin.FuncOp("some_func", ([f32, f32], [f32, f32]))
       with InsertionPoint(func.add_entry_block()):
         std.ReturnOp(func.arguments)
       func.arg_attrs = ArrayAttr.get([
           DictAttr.get({
-              "foo": StringAttr.get("bar"),
-              "baz": UnitAttr.get()
+              "custom_dialect.foo": StringAttr.get("bar"),
+              "custom_dialect.baz": UnitAttr.get()
           }),
-          DictAttr.get({"qux": ArrayAttr.get([])})
+          DictAttr.get({"custom_dialect.qux": ArrayAttr.get([])})
       ])
       func.result_attrs = ArrayAttr.get([
-          DictAttr.get({"res1": FloatAttr.get(f32, 42.0)}),
-          DictAttr.get({"res2": FloatAttr.get(f64, 256.0)})
+          DictAttr.get({"custom_dialect.res1": FloatAttr.get(f32, 42.0)}),
+          DictAttr.get({"custom_dialect.res2": FloatAttr.get(f64, 256.0)})
       ])
 
       other = builtin.FuncOp("other_func", ([f32, f32], []))
       with InsertionPoint(other.add_entry_block()):
         std.ReturnOp([])
       other.arg_attrs = [
-          DictAttr.get({"foo": StringAttr.get("qux")}),
+          DictAttr.get({"custom_dialect.foo": StringAttr.get("qux")}),
           DictAttr.get()
       ]
 
-  # CHECK: [{baz, foo = "bar"}, {qux = []}]
+  # CHECK: [{custom_dialect.baz, custom_dialect.foo = "bar"}, {custom_dialect.qux = []}]
   print(func.arg_attrs)
 
-  # CHECK: [{res1 = 4.200000e+01 : f32}, {res2 = 2.560000e+02 : f64}]
+  # CHECK: [{custom_dialect.res1 = 4.200000e+01 : f32}, {custom_dialect.res2 = 2.560000e+02 : f64}]
   print(func.result_attrs)
 
   # CHECK: func @some_func(
-  # CHECK: %[[ARG0:.*]]: f32 {baz, foo = "bar"},
-  # CHECK: %[[ARG1:.*]]: f32 {qux = []}) ->
-  # CHECK: f32 {res1 = 4.200000e+01 : f32},
-  # CHECK: f32 {res2 = 2.560000e+02 : f64})
+  # CHECK: %[[ARG0:.*]]: f32 {custom_dialect.baz, custom_dialect.foo = "bar"},
+  # CHECK: %[[ARG1:.*]]: f32 {custom_dialect.qux = []}) ->
+  # CHECK: f32 {custom_dialect.res1 = 4.200000e+01 : f32},
+  # CHECK: f32 {custom_dialect.res2 = 2.560000e+02 : f64})
   # CHECK: return %[[ARG0]], %[[ARG1]] : f32, f32
   #
   # CHECK: func @other_func(
-  # CHECK: %{{.*}}: f32 {foo = "qux"},
+  # CHECK: %{{.*}}: f32 {custom_dialect.foo = "qux"},
   # CHECK: %{{.*}}: f32)
   print(module)

mlir/test/python/dialects/linalg/opdsl/emit_structured_generic.py

with InsertionPoint(module.body):
     # Just check that we don't assert out on name mismatch.
     # CHECK-LABEL: @test_non_default_op_name
     @builtin.FuncOp.from_py_func(
         RankedTensorType.get((42,), f32), RankedTensorType.get((42,), f32))
     def test_non_default_op_name(input, init_result):
       return non_default_op_name(input, outs=[init_result])
 
 
-print(module)
+# TODO: Fix me! Conv and pooling ops above do not verify, which was uncovered
+# when switching to more robust module verification. For now, reverting to the
+# old behavior which does not verify on module print.
+print(module.operation.get_asm(assume_verified=True))

mlir/test/python/dialects/linalg/ops.py

with InsertionPoint(module.body):
       @builtin.FuncOp.from_py_func(MemRefType.get((12, -1), f32))
       def fill_buffer(out):
         zero = arith.ConstantOp(value=FloatAttr.get(f32, 0.), result=f32).result
         linalg.FillOp(output=out, value=zero)
 
   print(module)
 
 
-# CHECK-LABEL: TEST: testStructuredOpOnTensors
-@run
-def testStructuredOpOnTensors():
-  with Context() as ctx, Location.unknown():
-    module = Module.create()
-    f32 = F32Type.get()
-    tensor_type = RankedTensorType.get((2, 3, 4), f32)
-    with InsertionPoint(module.body):
-      func = builtin.FuncOp(
-          name="matmul_test",
-          type=FunctionType.get(
-              inputs=[tensor_type, tensor_type], results=[tensor_type]))
-      with InsertionPoint(func.add_entry_block()):
-        lhs, rhs = func.entry_block.arguments
-        result = linalg.MatmulOp([lhs, rhs], results=[tensor_type]).result
-        std.ReturnOp([result])
-
-  # CHECK: %[[R:.*]] = linalg.matmul ins(%arg0, %arg1 : tensor<2x3x4xf32>, tensor<2x3x4xf32>) -> tensor<2x3x4xf32>
-  print(module)
-
-
-# CHECK-LABEL: TEST: testStructuredOpOnBuffers
-@run
-def testStructuredOpOnBuffers():
-  with Context() as ctx, Location.unknown():
-    module = Module.create()
-    f32 = F32Type.get()
-    memref_type = MemRefType.get((2, 3, 4), f32)
-    with InsertionPoint(module.body):
-      func = builtin.FuncOp(
-          name="matmul_test",
-          type=FunctionType.get(
-              inputs=[memref_type, memref_type, memref_type], results=[]))
-      with InsertionPoint(func.add_entry_block()):
-        lhs, rhs, result = func.entry_block.arguments
-        # TODO: prperly hook up the region.
-        linalg.MatmulOp([lhs, rhs], outputs=[result])
-        std.ReturnOp([])
-
-  # CHECK: linalg.matmul ins(%arg0, %arg1 : memref<2x3x4xf32>, memref<2x3x4xf32>) outs(%arg2 : memref<2x3x4xf32>)
-  print(module)
-
-
 # CHECK-LABEL: TEST: testNamedStructuredOpCustomForm
 @run
 def testNamedStructuredOpCustomForm():
   with Context() as ctx, Location.unknown():
     module = Module.create()
     f32 = F32Type.get()
     with InsertionPoint(module.body):
 

mlir/test/python/dialects/shape.py

 def testConstShape():
   with Context() as ctx, Location.unknown():
     module = Module.create()
     f32 = F32Type.get()
     with InsertionPoint(module.body):
       @builtin.FuncOp.from_py_func(
           RankedTensorType.get((12, -1), f32))
       def const_shape_tensor(arg):
-        return shape.ConstShapeOp(DenseElementsAttr.get(np.array([10, 20])))
+        return shape.ConstShapeOp(
+          DenseElementsAttr.get(np.array([10, 20]), type=IndexType.get()))
 
     # CHECK-LABEL: func @const_shape_tensor(%arg0: tensor<12x?xf32>)
     # CHECK: shape.const_shape [10, 20] : tensor<2xindex>
     print(module)

mlir/test/python/dialects/std.py

 
 # CHECK: = arith.constant 10 : index
 
 
 # CHECK-LABEL: TEST: testFunctionCalls
 @constructAndPrintInModule
 def testFunctionCalls():
   foo = builtin.FuncOp("foo", ([], []))
+  foo.sym_visibility = StringAttr.get("private")
   bar = builtin.FuncOp("bar", ([], [IndexType.get()]))
+  bar.sym_visibility = StringAttr.get("private")
   qux = builtin.FuncOp("qux", ([], [F32Type.get()]))
+  qux.sym_visibility = StringAttr.get("private")
 
   with InsertionPoint(builtin.FuncOp("caller", ([], [])).add_entry_block()):
     std.CallOp(foo, [])
     std.CallOp([IndexType.get()], "bar", [])
     std.CallOp([F32Type.get()], FlatSymbolRefAttr.get("qux"), [])
     std.ReturnOp([])
 
 
-# CHECK: func @foo()
-# CHECK: func @bar() -> index
-# CHECK: func @qux() -> f32
+# CHECK: func private @foo()
+# CHECK: func private @bar() -> index
+# CHECK: func private @qux() -> f32
 # CHECK: func @caller() {
 # CHECK:   call @foo() : () -> ()
 # CHECK:   %0 = call @bar() : () -> index
 # CHECK:   %1 = call @qux() : () -> f32
 # CHECK:   return
 # CHECK: }

mlir/test/python/ir/module.py

 # RUN: %PYTHON %s | FileCheck %s
 
 import gc
 from mlir.ir import *
 
 def run(f):
   print("\nTEST:", f.__name__)
   f()
   gc.collect()
   assert Context._get_live_count() == 0
+  return f
 
 
 # Verify successful parse.
 # CHECK-LABEL: TEST: testParseSuccess
 # CHECK: module @successfulParse
+@run
 def testParseSuccess():
   ctx = Context()
   module = Module.parse(r"""module @successfulParse {}""", ctx)
   assert module.context is ctx
   print("CLEAR CONTEXT")
   ctx = None  # Ensure that module captures the context.
   gc.collect()
   module.dump()  # Just outputs to stderr. Verifies that it functions.
   print(str(module))
 
-run(testParseSuccess)
-
 
 # Verify parse error.
 # CHECK-LABEL: TEST: testParseError
 # CHECK: testParseError: Unable to parse module assembly (see diagnostics)
+@run
 def testParseError():
   ctx = Context()
   try:
     module = Module.parse(r"""}SYNTAX ERROR{""", ctx)
   except ValueError as e:
     print("testParseError:", e)
   else:
     print("Exception not produced")
 
-run(testParseError)
-
 
 # Verify successful parse.
 # CHECK-LABEL: TEST: testCreateEmpty
 # CHECK: module {
+@run
 def testCreateEmpty():
   ctx = Context()
   loc = Location.unknown(ctx)
   module = Module.create(loc)
   print("CLEAR CONTEXT")
   ctx = None  # Ensure that module captures the context.
   gc.collect()
   print(str(module))
 
-run(testCreateEmpty)
-
 
 # Verify round-trip of ASM that contains unicode.
 # Note that this does not test that the print path converts unicode properly
 # because MLIR asm always normalizes it to the hex encoding.
 # CHECK-LABEL: TEST: testRoundtripUnicode
 # CHECK: func private @roundtripUnicode()
 # CHECK: foo = "\F0\9F\98\8A"
+@run
 def testRoundtripUnicode():
   ctx = Context()
   module = Module.parse(r"""
     func private @roundtripUnicode() attributes { foo = "😊" }
   """, ctx)
   print(str(module))
 
-run(testRoundtripUnicode)
+
+# Verify round-trip of ASM that contains unicode.
+# Note that this does not test that the print path converts unicode properly
+# because MLIR asm always normalizes it to the hex encoding.
+# CHECK-LABEL: TEST: testRoundtripBinary
+# CHECK: func private @roundtripUnicode()
+# CHECK: foo = "\F0\9F\98\8A"
+@run
+def testRoundtripBinary():
+  with Context():
+    module = Module.parse(r"""
+      func private @roundtripUnicode() attributes { foo = "😊" }
+    """)
+    binary_asm = module.operation.get_asm(binary=True)
+    assert isinstance(binary_asm, bytes)
+    module = Module.parse(binary_asm)
+    print(module)
 
 
 # Tests that module.operation works and correctly interns instances.
 # CHECK-LABEL: TEST: testModuleOperation
+@run
 def testModuleOperation():
   ctx = Context()
   module = Module.parse(r"""module @successfulParse {}""", ctx)
   assert ctx._get_live_module_count() == 1
   op1 = module.operation
   assert ctx._get_live_operation_count() == 1
   # CHECK: module @successfulParse
   print(op1)
def testModuleOperation():
   # Collect and verify lifetime.
   op1 = None
   op2 = None
   gc.collect()
   print("LIVE OPERATIONS:", ctx._get_live_operation_count())
   assert ctx._get_live_operation_count() == 0
   assert ctx._get_live_module_count() == 0
 
-run(testModuleOperation)
-
 
 # CHECK-LABEL: TEST: testModuleCapsule
+@run
 def testModuleCapsule():
   ctx = Context()
   module = Module.parse(r"""module @successfulParse {}""", ctx)
   assert ctx._get_live_module_count() == 1
   # CHECK: "mlir.ir.Module._CAPIPtr"
   module_capsule = module._CAPIPtr
   print(module_capsule)
   module_dup = Module._CAPICreate(module_capsule)
   assert module is module_dup
   assert module_dup.context is ctx
   # Gc and verify destructed.
   module = None
   module_capsule = None
   module_dup = None
   gc.collect()
   assert ctx._get_live_module_count() == 0
 
-
-run(testModuleCapsule)

mlir/test/python/ir/operation.py

except ValueError as e:
     print(e)
   else:
     assert False, "Expected exception"
 
   # CHECK: %1 = "custom.one_result"() : () -> f32
   print(module.body.operations[2])
 
 
-# CHECK-LABEL: TEST: testPrintInvalidOperation
+def create_invalid_operation():
+  # This module has two region and is invalid verify that we fallback
+  # to the generic printer for safety.
+  op = Operation.create("builtin.module", regions=2)
+  op.regions[0].blocks.append()
+  return op
+
+# CHECK-LABEL: TEST: testInvalidOperationStrSoftFails
 @run
-def testPrintInvalidOperation():
+def testInvalidOperationStrSoftFails():
   ctx = Context()
   with Location.unknown(ctx):
-    module = Operation.create("builtin.module", regions=2)
-    # This module has two region and is invalid verify that we fallback
-    # to the generic printer for safety.
-    block = module.regions[0].blocks.append()
+    invalid_op = create_invalid_operation()
+    # Verify that we fallback to the generic printer for safety.
     # CHECK: // Verification failed, printing generic form
     # CHECK: "builtin.module"() ( {
     # CHECK: }) : () -> ()
-    print(module)
+    print(invalid_op)
     # CHECK: .verify = False
-    print(f".verify = {module.operation.verify()}")
+    print(f".verify = {invalid_op.operation.verify()}")
+
+
+# CHECK-LABEL: TEST: testInvalidModuleStrSoftFails
+@run
+def testInvalidModuleStrSoftFails():
+  ctx = Context()
+  with Location.unknown(ctx):
+    module = Module.create()
+    with InsertionPoint(module.body):
+      invalid_op = create_invalid_operation()
+    # Verify that we fallback to the generic printer for safety.
+    # CHECK: // Verification failed, printing generic form
+    print(module)
+
+
+# CHECK-LABEL: TEST: testInvalidOperationGetAsmBinarySoftFails
+@run
+def testInvalidOperationGetAsmBinarySoftFails():
+  ctx = Context()
+  with Location.unknown(ctx):
+    invalid_op = create_invalid_operation()
+    # Verify that we fallback to the generic printer for safety.
+    # CHECK: b'// Verification failed, printing generic form\n
+    print(invalid_op.get_asm(binary=True))
 
 
 # CHECK-LABEL: TEST: testCreateWithInvalidAttributes
 @run
 def testCreateWithInvalidAttributes():
   ctx = Context()
   with Location.unknown(ctx):
     try: