[mlir][linalg] Move some bufferization patterns to std

[silvas]

Summary

ConstantOp and TensorCastOp are part of std, so it makes sense for their
bufferizations to live there.

Also, add a couple TODO's for cases that don't work yet: rank-0 tensor
constants and tensor constants with an extent that is 0.

[silvas]

fix BUILD_SHARED_LIBS

[herhut]

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

42

Wouldn't the most straight-forward way be to just allocate a size-0 memref?

56

Use getNumElements() instead.

62

So this entire vector dance is only required because store cannot write tensor values? That seems like a short cut to me rather than the best way to implement this. Why not use a tensor constant followed by a tensor_store? That seems more canonical to me and also should handle 0-element tensors correctly.

80

Can we at least avoid the dependency on linalg here and instead use a memref_reshape here?

[silvas]

Note that this is just moving code with some light touch-up. We don't need to block on architectural discussion for the pattern itself.

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

62

Why not use a tensor constant followed by a tensor_store?

This is the pattern that lowers tensor constants :)

I agree in general though, that this pattern is likely in need some some rethinking (possibly based on Rahul's work on std.global_memref)

[silvas]

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

42

Good idea. This patch is just a mechanical move with a light touch-op, so I added that in a comment.

80

That op doesn't seem to exist upstream. Maybe you folks could upstream the one from mhlo?

[silvas]

address comments

[herhut]

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

80

That is in the work and I believe the op is landing as I write this. I am ok with moving this over to std even though it is using a vector but I'd rather avoid the dependency on linalg. Linalg depending on std seems natural but std depending on linalg seems wrong.

The reshape and its lowering should be available soon (FYI @pifon2a).

[silvas]

Abandoning. This transition is already moving forward in a different way.

mlir/include/mlir/Dialect/Linalg/Transforms/Transforms.h

public:
   LinalgOpConverter(MLIRContext *context, BufferizeTypeConverter &converter)
       : BufferizeConversionPattern(context, converter) {}
 
   LogicalResult
   matchAndRewrite(Operation *op, ArrayRef<Value> operands,
                   ConversionPatternRewriter &rewriter) const final;
 };
 
-/// TensorConstantOp conversion inserts a linearized 1-D vector constant that is
-/// stored in memory. A linalg.reshape is introduced to convert to the desired
-/// n-D buffer form.
-class TensorConstantOpConverter
-    : public BufferizeOpConversionPattern<ConstantOp> {
-public:
-  using BufferizeOpConversionPattern<ConstantOp>::BufferizeOpConversionPattern;
-
-  LogicalResult
-  matchAndRewrite(ConstantOp op, ArrayRef<Value> operands,
-                  ConversionPatternRewriter &rewriter) const final;
-};
-
-/// TensorCastOp converts 1-1 to MemRefCastOp.
-class TensorCastOpConverter
-    : public BufferizeOpConversionPattern<TensorCastOp> {
-public:
-  using BufferizeOpConversionPattern<
-      TensorCastOp>::BufferizeOpConversionPattern;
-
-  LogicalResult
-  matchAndRewrite(TensorCastOp op, ArrayRef<Value> operands,
-                  ConversionPatternRewriter &rewriter) const final;
-};
-
 //===----------------------------------------------------------------------===//
 // Support for staged pattern application.
 //===----------------------------------------------------------------------===//
 /// Helper function to allow applying rewrite patterns, interleaved with more
 /// global transformations, in a staged fashion:
 ///   1. the first stage consists of a list of OwningRewritePatternList. Each
 ///   OwningRewritePatternList in this list is applied once, in order.
 ///   2. the second stage consists of a single OwningRewritePattern that is

mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.td

 def ExpandAtomic : FunctionPass<"expand-atomic"> {
   let summary = "Expands AtomicRMWOp into GenericAtomicRMWOp.";
   let constructor = "mlir::createExpandAtomicPass()";
 }
 
 def StdBufferize : FunctionPass<"std-bufferize"> {
   let summary = "Bufferize the std dialect";
   let constructor = "mlir::createStdBufferizePass()";
-  let dependentDialects = ["scf::SCFDialect"];
+  let dependentDialects = [
+    "linalg::LinalgDialect",
+    "scf::SCFDialect",
+    "vector::VectorDialect"
+  ];
 }
 
 #endif // MLIR_DIALECT_STANDARD_TRANSFORMS_PASSES

mlir/lib/Dialect/Linalg/Transforms/Bufferize.cpp

 //===- Bufferize.cpp - Bufferization of linalg ops ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "mlir/Transforms/Bufferize.h"
 #include "PassDetail.h"
 #include "mlir/Dialect/Linalg/IR/LinalgOps.h"
 #include "mlir/Dialect/Linalg/Passes.h"
 #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
 #include "mlir/Dialect/Linalg/Utils/Utils.h"
+#include "mlir/Dialect/StandardOps/Transforms/Passes.h"
 #include "mlir/Dialect/Vector/VectorOps.h"
 #include "mlir/IR/Function.h"
 #include "mlir/IR/Operation.h"
 #include "mlir/Pass/Pass.h"
 
 using namespace ::mlir;
 using namespace ::mlir::linalg;
 
if (auto genericOp = dyn_cast<linalg::GenericOp>(op)) {
     return success();
   }
 
   finalizeBufferAllocation(rewriter, linalgOp, adaptor.inputs(),
                            newOutputBuffers);
   return success();
 }
 
-LogicalResult mlir::linalg::TensorConstantOpConverter::matchAndRewrite(
-    ConstantOp op, ArrayRef<Value> operands,
-    ConversionPatternRewriter &rewriter) const {
-  RankedTensorType rankedTensorType = op.getType().dyn_cast<RankedTensorType>();
-  if (!rankedTensorType)
-    return failure();
-  if (llvm::any_of(rankedTensorType.getShape(), [](int64_t s) {
-        return s == 0 || ShapedType::isDynamic(s);
-      }))
-    return failure();
-
-  int64_t nElements = 1;
-  for (int64_t s : rankedTensorType.getShape())
-    nElements *= s;
-  Type elementType = rankedTensorType.getElementType();
-  MemRefType memrefType =
-      converter.convertType(op.getType()).cast<MemRefType>();
-  VectorType flatVectorType = VectorType::get({nElements}, elementType);
-  MemRefType memrefOfFlatVectorType = MemRefType::get({}, flatVectorType);
-  MemRefType flatMemrefType = MemRefType::get({nElements}, elementType);
-
-  Location loc = op.getLoc();
-  auto attr = op.getValue().cast<DenseElementsAttr>();
-  Value alloc =
-      rewriter.create<AllocOp>(loc, memrefOfFlatVectorType, ValueRange{});
-  Value cstVec = rewriter.create<ConstantOp>(loc, flatVectorType,
-                                             attr.reshape(flatVectorType));
-  rewriter.create<StoreOp>(loc, cstVec, alloc);
-
-  Value memref =
-      rewriter.create<vector::TypeCastOp>(loc, flatMemrefType, alloc);
-  if (rankedTensorType.getRank() > 1) {
-    // Introduce a linalg.reshape to flatten the memref.
-    AffineMap collapseAllDims = AffineMap::getMultiDimIdentityMap(
-        /*numDims=*/rankedTensorType.getRank(), op.getContext());
-    memref = rewriter.create<linalg::ReshapeOp>(
-        loc, memrefType, memref,
-        rewriter.getAffineMapArrayAttr(collapseAllDims));
-  }
-  rewriter.replaceOp(op, memref);
-
-  return success();
-}
-
-LogicalResult mlir::linalg::TensorCastOpConverter::matchAndRewrite(
-    TensorCastOp op, ArrayRef<Value> operands,
-    ConversionPatternRewriter &rewriter) const {
-  if (op.getType().hasRank())
-    return failure();
-  Type t = UnrankedMemRefType::get(op.getType().getElementType(),
-                                   /*memorySpace=*/0);
-  rewriter.replaceOpWithNewOp<MemRefCastOp>(op, t, operands.front());
-  return success();
-}
-
 namespace {
 
 /// Converts Linalg operations that work on tensor-type operands or results to
 /// work on buffers.
 struct LinalgBufferizePass : public LinalgBufferizeBase<LinalgBufferizePass> {
   void runOnOperation() override {
     MLIRContext &context = getContext();
     ConversionTarget target(context);
target.addDynamicallyLegalOp<FuncOp>([&](FuncOp funcOp) {
              converter.isLegal(&funcOp.getBody());
     });
 
     converter.setResultConversionKind<RankedTensorType, MemRefType>(
         BufferizeTypeConverter::AppendToArgumentsList);
 
     OwningRewritePatternList patterns;
     populateLinalgBufferizePatterns(&context, converter, patterns);
+    populateStdBufferizePatterns(&context, converter, patterns);
     populateWithBufferizeOpConversionPatterns<mlir::ReturnOp, mlir::ReturnOp,
                                               linalg::CopyOp>(
         &context, converter, patterns);
     if (failed(applyFullConversion(this->getOperation(), target, patterns)))
       this->signalPassFailure();
   }
 };
 } // end anonymous namespace
 
 std::unique_ptr<OperationPass<ModuleOp>> mlir::createLinalgBufferizePass() {
   return std::make_unique<LinalgBufferizePass>();
 }
 void mlir::linalg::populateLinalgBufferizePatterns(
     MLIRContext *context, BufferizeTypeConverter &converter,
     OwningRewritePatternList &patterns) {
-  patterns.insert<
-      // clang-format off
-      LinalgOpConverter,
-      TensorCastOpConverter,
-      TensorConstantOpConverter
-      // clang-format on
-      >(context, converter);
+  patterns.insert<LinalgOpConverter>(context, converter);
 }

mlir/lib/Dialect/Linalg/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgTransforms
   MLIRLinalgAnalysis
   MLIRLinalgEDSC
   MLIRLinalg
   MLIRLinalgUtils
   MLIRSCF
   MLIRSCFTransforms
   MLIRPass
   MLIRStandard
+  MLIRStandardOpsTransforms
   MLIRStandardToLLVM
   MLIRTransforms
   MLIRTransformUtils
   MLIRVector
   MLIRVectorToSCF
 )

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

 //===- Bufferize.cpp - Bufferization for std ops --------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements bufferization of std ops.
 //
 //===----------------------------------------------------------------------===//
 
 #include "mlir/Transforms/Bufferize.h"
 #include "PassDetail.h"
+#include "mlir/Dialect/Linalg/IR/LinalgOps.h"
 #include "mlir/Dialect/SCF/SCF.h"
 #include "mlir/Dialect/StandardOps/IR/Ops.h"
 #include "mlir/Dialect/StandardOps/Transforms/Passes.h"
+#include "mlir/Dialect/Vector/VectorOps.h"
 #include "mlir/IR/BlockAndValueMapping.h"
 #include "mlir/Transforms/DialectConversion.h"
 
 using namespace mlir;
 
+// BufferizeConstantOp conversion inserts a linearized 1-D vector constant that
+// is stored in memory. A linalg.reshape is introduced to convert to the desired
+// n-D buffer form.
+//
+// TODO: It's not known how well this strategy works for very large constants,
+// such as large weight buffers for large machine learning models.
+class BufferizeConstantOp : public OpConversionPattern<ConstantOp> {
+public:
+  using OpConversionPattern::OpConversionPattern;
+
+  LogicalResult
+  matchAndRewrite(ConstantOp op, ArrayRef<Value> operands,
+                  ConversionPatternRewriter &rewriter) const final {
+    auto rankedTensorType = op.getType().dyn_cast<RankedTensorType>();
+    if (!rankedTensorType)
+      return rewriter.notifyMatchFailure(op, "requires RankedTensorType");
+
+    // TODO: Make this case work.
+    // Because we lower through a vector, and vectors cannot have extent 0, our
+    // strategy here doesn't work for tensors of extent 0.
+    // We could do something slightly more complex here and convert as though
+    // the tensor was size 1, and then take a slice of size 0 from that.
+    if (llvm::any_of(rankedTensorType.getShape(),
+                     [](int64_t extent) { return extent == 0; }))
+      return failure();
+
+    // TODO: Make this case work.
+    // It seems to fail when verifying the linalg.reshape op.
+    if (rankedTensorType.getRank() == 0)
+      return failure();
+
+    int64_t totalElements = 1;
+    for (int64_t s : rankedTensorType.getShape())
+      totalElements *= s;
+    Type elementType = rankedTensorType.getElementType();
+    MemRefType memrefType =
+        getTypeConverter()->convertType(op.getType()).cast<MemRefType>();
+    VectorType flatVectorType = VectorType::get({totalElements}, elementType);
+    MemRefType memrefOfFlatVectorType = MemRefType::get({}, flatVectorType);
+    MemRefType flatMemrefType = MemRefType::get({totalElements}, elementType);
+
+    Location loc = op.getLoc();
+    auto attr = op.getValue().cast<DenseElementsAttr>();
+    Value alloc =
+        rewriter.create<AllocOp>(loc, memrefOfFlatVectorType, ValueRange{});
+    Value constantVector = rewriter.create<ConstantOp>(
+        loc, flatVectorType, attr.reshape(flatVectorType));
+    rewriter.create<StoreOp>(loc, constantVector, alloc);
+
+    Value memref =
+        rewriter.create<vector::TypeCastOp>(loc, flatMemrefType, alloc);
+    if (rankedTensorType.getRank() != 1) {
+      // Introduce a linalg.reshape to get a memref of the final desired shape.
+      AffineMap collapseAllDims = AffineMap::getMultiDimIdentityMap(
+          /*numDims=*/rankedTensorType.getRank(), op.getContext());
+      memref = rewriter.create<linalg::ReshapeOp>(
+          loc, memrefType, memref,
+          rewriter.getAffineMapArrayAttr(collapseAllDims));
+    }
+    rewriter.replaceOp(op, memref);
+
+    return success();
+  }
+};
+
 namespace {
 class BufferizeDynamicTensorFromElementsOp
     : public OpConversionPattern<DynamicTensorFromElementsOp> {
 public:
   using OpConversionPattern::OpConversionPattern;
 
   LogicalResult
   matchAndRewrite(DynamicTensorFromElementsOp op, ArrayRef<Value> operands,
matchAndRewrite(TensorFromElementsOp op, ArrayRef<Value> operands,
     return success();
   }
 };
 } // namespace
 
 void mlir::populateStdBufferizePatterns(MLIRContext *context,
                                         BufferizeTypeConverter &typeConverter,
                                         OwningRewritePatternList &patterns) {
-  patterns
-      .insert<BufferizeDynamicTensorFromElementsOp, BufferizeExtractElementOp,
-              BufferizeTensorCastOp, BufferizeTensorFromElementsOp>(
-          typeConverter, context);
+  patterns.insert<BufferizeConstantOp, BufferizeDynamicTensorFromElementsOp,
+                  BufferizeExtractElementOp, BufferizeTensorCastOp,
+                  BufferizeTensorFromElementsOp>(typeConverter, context);
 }
 
 namespace {
 struct StdBufferizePass : public StdBufferizeBase<StdBufferizePass> {
   void runOnFunction() override {
     auto *context = &getContext();
     BufferizeTypeConverter typeConverter;
     OwningRewritePatternList patterns;
     ConversionTarget target(*context);
 
+    target.addLegalDialect<linalg::LinalgDialect>();
     target.addLegalDialect<StandardOpsDialect>();
     target.addLegalDialect<scf::SCFDialect>();
+    target.addLegalDialect<vector::VectorDialect>();
 
     populateStdBufferizePatterns(context, typeConverter, patterns);
     target.addIllegalOp<DynamicTensorFromElementsOp, ExtractElementOp,
                         TensorCastOp, TensorFromElementsOp>();
+    target.addDynamicallyLegalOp<ConstantOp>(
+        [&](ConstantOp op) { return typeConverter.isLegal(op.getType()); });
 
     if (failed(applyPartialConversion(getFunction(), target, patterns)))
       signalPassFailure();
   }
 };
 } // namespace
 
 std::unique_ptr<Pass> mlir::createStdBufferizePass() {
   return std::make_unique<StdBufferizePass>();
 }

mlir/lib/Dialect/StandardOps/Transforms/CMakeLists.txt

 add_mlir_dialect_library(MLIRStandardOpsTransforms
   Bufferize.cpp
   ExpandAtomic.cpp
   ExpandTanh.cpp
   FuncConversions.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/StandardOps/Transforms
 
   DEPENDS
   MLIRStandardTransformsIncGen
 
   LINK_LIBS PUBLIC
   MLIRIR
+  MLIRLinalg
   MLIRPass
   MLIRSCF
   MLIRStandard
   MLIRTransforms
+  MLIRVector
   )

mlir/lib/Dialect/StandardOps/Transforms/PassDetail.h

 //===- PassDetail.h - GPU Pass class details --------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_
 #define DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_
 
+#include "mlir/Dialect/Linalg/IR/LinalgOps.h"
 #include "mlir/Dialect/SCF/SCF.h"
+#include "mlir/Dialect/Vector/VectorOps.h"
 #include "mlir/Pass/Pass.h"
 
 namespace mlir {
 
 class AtomicRMWOp;
 
 #define GEN_PASS_CLASSES
 #include "mlir/Dialect/StandardOps/Transforms/Passes.h.inc"
 
 } // end namespace mlir
 
 #endif // DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_

mlir/test/Dialect/Linalg/bufferize.mlir

 // CHECK: linalg.copy(%[[OUT_BUF_1]], %[[OUT_1]]) : [[TYPE]], [[TYPE]]
 // CHECK: dealloc %[[OUT_BUF_1]] : [[TYPE]]
 // CHECK: linalg.copy(%[[OUT_BUF_2]], %[[OUT_2]]) : [[TYPE]], [[TYPE]]
 // CHECK: dealloc %[[OUT_BUF_2]] : [[TYPE]]
 // CHECK: return
 
 // -----
 
-func @foo() -> tensor<2x3xf32> {
-// CHECK-LABEL: func @foo(
-//  CHECK-SAME:   %[[A:[0-9a-z]*]]: memref<2x3xf32>) {
-
-  %0 = constant dense<[[0.0, 1.0, 2.0], [3.0, 4.0, 5.0]]> : tensor<2x3xf32>
-//  CHECK-NEXT:   %[[ALLOC:.*]] = alloc() : memref<vector<6xf32>>
-//  CHECK-NEXT:   %[[CST:.*]] = constant dense<[0.000000e+00, 1.000000e+00, 2.000000e+00, 3.000000e+00, 4.000000e+00, 5.000000e+00]> : vector<6xf32>
-//  CHECK-NEXT:   store %[[CST]], %[[ALLOC]][] : memref<vector<6xf32>>
-//  CHECK-NEXT:   %[[FLAT:.*]] = vector.type_cast %[[ALLOC]] : memref<vector<6xf32>> to memref<6xf32>
-//  CHECK-NEXT:   %[[RES:.*]] = linalg.reshape %[[FLAT]] {{.*}} : memref<6xf32> into memref<2x3xf32>
-
-  return %0 : tensor<2x3xf32>
-//  CHECK-NEXT:   linalg.copy(%[[RES]], %[[A]]) : memref<2x3xf32>, memref<2x3xf32>
-//  CHECK-NEXT:   dealloc %[[ALLOC]] : memref<vector<6xf32>>
-//  CHECK-NEXT:   return
-}
-
-func @bar() {
-// CHECK-LABEL: func @bar() {
-
-  %0 = call @foo() : () -> tensor<2x3xf32>
-//  CHECK-NEXT:   %[[ALLOC:.*]] = alloc() : memref<2x3xf32>
-//  CHECK-NEXT:   call @foo(%[[ALLOC]]) : (memref<2x3xf32>) -> ()
-
-  // Instead of relying on tensor_store which introduces aliasing, we rely on
-  // the conversion of print_memref_f32(tensor<*xf32>) to
-  // print_memref_f32(memref<*xf32>).
-  // Note that this is skipping a step and we would need at least some function
-  // attribute to declare that this conversion is valid (e.g. when we statically
-  // know that things will play nicely at the C ABI boundary).
-  %unranked = tensor_cast %0 : tensor<2x3xf32> to tensor<*xf32>
-//  CHECK-NEXT:   %[[UNRANKED:.*]] = memref_cast %[[ALLOC]] :
-//  CHECK-SAME:     memref<2x3xf32> to memref<*xf32>
+// CHECK-LABEL:   func @test_print_memref_f32(
+// CHECK-SAME:                                %[[ARG:.*]]: memref<2x3xf32>) {
 
+func @test_print_memref_f32(%arg0: tensor<2x3xf32>) {
+  // CHECK:           %[[RANK_ERASED:.*]] = memref_cast %[[ARG]] : memref<2x3xf32> to memref<*xf32>
+  %unranked = tensor_cast %arg0 : tensor<2x3xf32> to tensor<*xf32>
+  // CHECK:           call @print_memref_f32(%[[RANK_ERASED]]) : (memref<*xf32>) -> ()
   call @print_memref_f32(%unranked) : (tensor<*xf32>) -> ()
-//  CHECK-NEXT:   call @print_memref_f32(%[[UNRANKED]]) : (memref<*xf32>) -> ()
-
+  // CHECK:           return
   return
-//  CHECK-NEXT:   dealloc %[[ALLOC]] : memref<2x3xf32>
-//  CHECK-NEXT:   return
 }
 
 // This gets converted to a function operating on memref<*xf32>.
 // Note that this is skipping a step and we would need at least some function
 // attribute to declare that this conversion is valid (e.g. when we statically
 // know that things will play nicely at the C ABI boundary).
 func @print_memref_f32(%ptr : tensor<*xf32>)
 // CHECK-LABEL: func @print_memref_f32(memref<*xf32>)

mlir/test/Dialect/Standard/bufferize.mlir

 // RUN: mlir-opt %s -std-bufferize | FileCheck %s
 
+// CHECK-LABEL:   func @tensor_constant() -> tensor<2x3xf32> {
+// CHECK:           %[[MEMREF_OF_VECTOR:.*]] = alloc() : memref<vector<6xf32>>
+// CHECK:           %[[VECTOR_CONSTANT:.*]] = constant dense<[0.000000e+00, 1.000000e+00, 2.000000e+00, 3.000000e+00, 4.000000e+00, 5.000000e+00]> : vector<6xf32>
+// CHECK:           store %[[VECTOR_CONSTANT]], %[[MEMREF_OF_VECTOR]][] : memref<vector<6xf32>>
+// CHECK:           %[[MEMREF_OF_SCALARS:.*]] = vector.type_cast %[[MEMREF_OF_VECTOR]] : memref<vector<6xf32>> to memref<6xf32>
+// CHECK:           %[[RESHAPED:.*]] = linalg.reshape %[[MEMREF_OF_SCALARS]] [#map0] : memref<6xf32> into memref<2x3xf32>
+// CHECK:           %[[RET:.*]] = tensor_load %[[RESHAPED]] : memref<2x3xf32>
+// CHECK:           return %[[RET]] : tensor<2x3xf32>
+// CHECK:         }
+func @tensor_constant() -> tensor<2x3xf32> {
+  %0 = constant dense<[[0.0, 1.0, 2.0], [3.0, 4.0, 5.0]]> : tensor<2x3xf32>
+  return %0 : tensor<2x3xf32>
+}
+
+// CHECK-LABEL: func @no_convert_yet_tensor_constant_rank_0() -> tensor<f32>
+// TODO: Make this work.
+func @no_convert_yet_tensor_constant_rank_0() -> tensor<f32> {
+  // CHECK: constant dense<0.000000e+00> : tensor<f32
+  %0 = constant dense<0.0> : tensor<f32>
+  return %0 : tensor<f32>
+}
+
+// CHECK-LABEL: func @no_convert_yet_tensor_constant_extent_0() -> tensor<0xf32>
+// TODO: Make this work.
+func @no_convert_yet_tensor_constant_extent_0() -> tensor<0xf32> {
+  // CHECK: constant dense<> : tensor<0xf32>
+  %0 = constant dense<> : tensor<0xf32>
+  return %0 : tensor<0xf32>
+}
+
+// CHECK-LABEL: func @no_convert_scalar_constant() -> i32
+func @no_convert_scalar_constant() -> i32 {
+  // CHECK: constant 0 : i32
+  %0 = constant 0 : i32
+  return %0 : i32
+}
+
 // CHECK-LABEL:   func @dynamic_tensor_from_elements(
 // CHECK-SAME:                                       %[[ARG:.*]]: tensor<*xf32>,
 // CHECK-SAME:                                       %[[DYNAMIC_EXTENT:.*]]: index) -> tensor<?xindex> {
 // CHECK:           %[[MEMREF:.*]] = alloc(%[[DYNAMIC_EXTENT]]) : memref<?xindex>
 // CHECK:           %[[C0:.*]] = constant 0 : index
 // CHECK:           %[[C1:.*]] = constant 1 : index
 // CHECK:           scf.parallel (%[[I:.*]]) = (%[[C0]]) to (%[[DYNAMIC_EXTENT]]) step (%[[C1]]) {
 // CHECK:             %[[ELEM:.*]] = dim %[[ARG]], %[[I]] : tensor<*xf32>