[mlir][tosa] Constant optimizations for reduce operations

[amirBish]

Summary

Replace the different reduce operations which is getting
a constant tensor as an input argument with a constant
tensor.

As the arguement of the reduce operation is constant tensor
and has only a single user we could calculate the resulted
constant tensor in compilation time and replace it
with reduced memory tensor

This optimization has been implemented for:
tosa.reduce_sum
tosa.reduce_prod
tosa.reduce_any
tosa.reduce_all
tosa.reduce_max
tosa.reduce_min

[rsuderman]

mlir/include/mlir/Dialect/Tosa/IR/TosaOps.td

1412

Mind breaking this into a couple lines? clang-format does not understand tablegen sadly.

1444

Ditto.

mlir/lib/Dialect/Tosa/Transforms/TosaReduceConstantOptimization.cpp

37 ↗

(On Diff #538581)

You are in an empty namespace so you don't need static. Also, declare this outside of the struct.

63 ↗

(On Diff #538581)

Ditto.

64 ↗

(On Diff #538581)

Can you use a more simple type for this? This feels likely simpler as a regular attribute.

67 ↗

(On Diff #538581)

You should be using llvm::SmallVector and llvm::ArrayRef instead of std::vector and std::vector&

68 ↗

(On Diff #538581)

You only return a single thing so its better to compute and return than take a std::vector&.

75 ↗

(On Diff #538581)

You do not need to recompute the position every time. Just determine what the striding is and do position = initial + index * stride.

77 ↗

(On Diff #538581)

Initialize the value before the loop and start iterating at 0. This avoids doing a index == 0 check every iteration.

90 ↗

(On Diff #538581)

You will also want to check the resultTy and verify the return type is known and statically shaped. Otherwise you could cause some type changes for under-defined programs.

111 ↗

(On Diff #538581)

This should be a llvm::SmallVector

113 ↗

(On Diff #538581)

you can request the new shape from the result type. Just invoke getType() and case to a shape type.

117 ↗

(On Diff #538581)

Same about llvm::SmallVector<APInt>

119 ↗

(On Diff #538581)

Name is extremely verbose. See if you can reduce it to something simpler. E.g. reductionIndex.

mlir/test/Dialect/Tosa/reduce-constant-optimizations.mlir

1 ↗

(On Diff #538581)

Delete extra startine line.

1 ↗

(On Diff #538581)

Please merge this file with the other tosa constant folding tests. We should not have 3 different files for constant folding.

481 ↗

(On Diff #538581)

Missing newline at end.

[amirBish]

updating the commit based on rsuderman reply

[amirBish]

mlir/lib/Dialect/Tosa/Transforms/TosaReduceConstantOptimization.cpp

64 ↗

(On Diff #538581)

sure, passed the mlir::elementsAttr instead of the complex type

67 ↗

(On Diff #538581)

I've modified all the code to use SmallVector/ArrayRef. also used the MutableArrayRef for non const reference, thanks

68 ↗

(On Diff #538581)

modified the function to return the llvm::APInt as a reducedValue, and there is no need to take the std::vector&

90 ↗

(On Diff #538581)

thanks, I've missed it

119 ↗

(On Diff #538581)

modified to reductionIndex

[amirBish]

@rsuderman would appreciate your feedback regarding the recent changes

[rsuderman]

Sorry for the delay

mlir/lib/Dialect/Tosa/Transforms/TosaReduceConstantOptimization.cpp

64 ↗

(On Diff #538958)

llvm::MutableArrayRef only makes sense when you are intending to inplace mutate the ArrayRef for a return. See the change below, but you should only need position to be a single integer.

72 ↗

(On Diff #538958)

Make the uint64_t match the element type of oldShape instead of casting the value of oldShape[reductionAxis]

79 ↗

(On Diff #538958)

Prefer a new value to mutation. It is better in this case to do

index = indexAtOldTensor += stride * reductionAxisVal;

and use index instead.

130 ↗

(On Diff #538958)

You should be this work into the calculateReduceValue as it is only used in there. You also do not need the newShape if you have the oldShape and reductionAxis, so only need to pass the reductionIndex in instead of the position.

137 ↗

(On Diff #538958)

Why not just use op.getType().cast<RankedTensorType>() instead of constructing a new one?

mlir/test/Dialect/Tosa/constant-op-fold.mlir

577

I do not see a single test for floating-point numbers. Have you tested what its behavior is with floating point?

[amirBish]

Additional changes according to rsuderman's review

[amirBish]

thanks for the reply @rsuderman , uploaded a new revision

mlir/lib/Dialect/Tosa/Transforms/TosaReduceConstantOptimization.cpp

64 ↗

(On Diff #538958)

you're right, It's not intended to be a return

72 ↗

(On Diff #538958)

done :)

79 ↗

(On Diff #538958)

modified

mlir/test/Dialect/Tosa/constant-op-fold.mlir

577

Hey, maybe It wasn't clear enough in the summary of the patch. But, right now I'm allowing this optimization just within integer element type. (there is a check at mlir/lib/Dialect/Tosa/Transforms/TosaReduceConstantOptimization.cpp::117).

would upload a new patch for the APfloat handling (would require some changes also reduceAny/reduceAll won't support the float tensors)

[amirBish]

@rsuderman would be glad for your feedback regarding the recent changes

[amirBish]

Ping

[rsuderman]

Overall I would move these next to the other constant folders. Looking standing back they are all the same as the other layerwise folders and should be kept together. There is not much value is separating the reduction folders into a separate file. Once thats done I can approve and land.

[amirBish]

Moving the constant reduce optimization to the tosa folders

[amirBish]

ping

[amirBish]

It's my first commit, could you please commit it for me?
thanks in advance
(https://llvm.org/docs/Phabricator.html#committing-someone-s-change-from-phabricator)
@rsuderman

[eric-k256]

I'll try to help land this, but the printing for tosa ops changed recently to a new format. Can you rebase this onto the latest to make it easier to land?

[amirBish]

Rebasing into the latest

mlir/include/mlir/Dialect/Tosa/IR/TosaOps.td

def Tosa_ReduceAllOp : Tosa_InferTensorTypeOp<"reduce_all"> {
   );
 
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the AND result between two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      return leftOperand & rightOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // Operator: reduce_any
 //===----------------------------------------------------------------------===//
 def Tosa_ReduceAnyOp : Tosa_InferTensorTypeOp<"reduce_any"> {
   let summary = "Reduce Any operator";
def Tosa_ReduceAnyOp : Tosa_InferTensorTypeOp<"reduce_any"> {
   );
 
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the OR result between two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      return leftOperand | rightOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // Operator: reduce_max
 //===----------------------------------------------------------------------===//
 def Tosa_ReduceMaxOp : Tosa_InferTensorTypeOp<"reduce_max"> {
   let summary = "Reduce Max operator";
def Tosa_ReduceMaxOp : Tosa_InferTensorTypeOp<"reduce_max"> {
   );
 
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the max of the two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      const llvm::APInt subtractRes = leftOperand - rightOperand;
+      return (!subtractRes.isNegative()) ? leftOperand : rightOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // Operator: reduce_min
 //===----------------------------------------------------------------------===//
 def Tosa_ReduceMinOp : Tosa_InferTensorTypeOp<"reduce_min"> {
   let summary = "Reduce Min operator";
def Tosa_ReduceMinOp : Tosa_InferTensorTypeOp<"reduce_min"> {
   );
 
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the min of the two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      const llvm::APInt subtractRes = leftOperand - rightOperand;
+      return (!subtractRes.isNegative()) ? rightOperand : leftOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // Operator: reduce_prod
 //===----------------------------------------------------------------------===//
 def Tosa_ReduceProdOp : Tosa_InferTensorTypeOp<"reduce_prod"> {
   let summary = "Reduce Prod operator";
def Tosa_ReduceProdOp : Tosa_InferTensorTypeOp<"reduce_prod"> {
   );
 
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the prod of the two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      return leftOperand * rightOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // Operator: reduce_sum
 //===----------------------------------------------------------------------===//
 def Tosa_ReduceSumOp : Tosa_InferTensorTypeOp<"reduce_sum"> {
   let summary = "Reduce Sum operator";
 
   let description = [{
     Reduce a tensor along the given axis by computing the sum of the axis.
   }];
 
   let arguments = (ins
     Tosa_Tensor:$input,
     I32Attr:$axis
   );
 
   let results = (outs
     Tosa_Tensor:$output
   );
-
   let hasFolder = 1;
 
   let extraClassDeclaration = [{
     /// Returns true when two result types are compatible for this op;
     /// Method used by InferTypeOpInterface.
     static bool isCompatibleReturnTypes(TypeRange l, TypeRange r);
+
+    /// Return the sum of the two integer operands
+    static inline APInt calcOneElement(APInt leftOperand, APInt rightOperand) {
+      return leftOperand + rightOperand;
+    }
   }];
 }
 
 //===----------------------------------------------------------------------===//
 // TOSA Spec Section 2.9
 // Operator Class: Data Layout / Memory Reinterpretation.
 //===----------------------------------------------------------------------===//
 

mlir/include/mlir/Dialect/Tosa/Transforms/Passes.h

 void populateTosaDecomposeTransposeConv(MLIRContext *ctx,
                                         RewritePatternSet &patterns);
 void populateTosaDecomposeDepthwise(MLIRContext *ctx,
                                     RewritePatternSet &patterns);
 void populateTosaFoldConstantReciprocalPatterns(MLIRContext *ctx,
                                                 RewritePatternSet &patterns);
 void populateTosaFoldConstantTransposePatterns(MLIRContext *ctx,
                                                RewritePatternSet &patterns);
+void populateTosaConstantReduction(MLIRContext *ctx,
+                                   RewritePatternSet &patterns);
 
 std::unique_ptr<Pass> createTosaLayerwiseConstantFoldPass();
 std::unique_ptr<Pass> createTosaInferShapesPass();
 std::unique_ptr<Pass> createTosaMakeBroadcastablePass();
 std::unique_ptr<Pass> createTosaTestQuantUtilAPIPass();
 std::unique_ptr<Pass> createTosaOptionalDecompositions();
 
 struct ValidationOptions {

mlir/lib/Dialect/Tosa/Transforms/TosaFolders.cpp

 //===- TosaFolders.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Fold TOSA operations
 //
 //===----------------------------------------------------------------------===//
 
 #include <functional>
+#include <numeric>
 
 #include "mlir/Dialect/Tosa/IR/TosaOps.h"
 #include "mlir/Dialect/Tosa/Transforms/Passes.h"
 #include "mlir/Dialect/Utils/IndexingUtils.h"
 #include "mlir/IR/BuiltinAttributes.h"
 #include "mlir/IR/BuiltinTypes.h"
 #include "mlir/IR/Matchers.h"
 #include "mlir/Pass/Pass.h"
auto newTensor = applyElementWise<APFloat, APFloat, FloatType>(
         cast<FloatType>(inputValues.getElementType()));
 
     // Replace the use of the reciprocal with the transformed tensor
     rewriter.replaceOpWithNewOp<ConstOp>(recip, newTensor.getType(), newTensor);
     return success();
   }
 };
 
+/// Getting the axes position of the element which is located
+/// in the tensor at the counter index
+
+llvm::SmallVector<int64_t>
+getPositionFromIndex(int64_t index, llvm::ArrayRef<int64_t> tensorShape) {
+  int64_t remaining = index;
+  llvm::SmallVector<int64_t> position(tensorShape.size(), 0);
+  for (int64_t i = tensorShape.size() - 1; i >= 0; --i) {
+    position[i] = remaining % tensorShape[i];
+    remaining /= tensorShape[i];
+  }
+  return position;
+}
+
+/// Getting the index of the element which is located at the
+/// axes position in the tensor
+
+int64_t getIndexFromPosition(llvm::ArrayRef<int64_t> position,
+                             llvm::ArrayRef<int64_t> tensorShape) {
+  int64_t index = 0;
+  int64_t multiplierTmp = 1;
+  for (int64_t i = position.size() - 1; i >= 0; --i) {
+    index += position[i] * multiplierTmp;
+    multiplierTmp *= tensorShape[i];
+  }
+  return index;
+}
+
+template <typename OperationType>
+llvm::APInt calculateReducedValue(const mlir::ElementsAttr &oldTensorAttr,
+                                  llvm::ArrayRef<int64_t> oldShape,
+                                  int64_t reductionAxis,
+                                  int64_t reductionIndex) {
+
+  llvm::SmallVector<int64_t> newShape(oldShape);
+  newShape[reductionAxis] = 1;
+  /// Let's calculate the position of the index
+  llvm::SmallVector<int64_t> position =
+      getPositionFromIndex(reductionIndex, newShape);
+  auto oldTensor = oldTensorAttr.getValues<llvm::APInt>();
+  /// Starting from the first positon along the reduction axis
+  position[reductionAxis] = 0;
+  int64_t indexAtOldTensor = getIndexFromPosition(position, oldShape);
+  llvm::APInt reducedValue = oldTensor[indexAtOldTensor];
+
+  for (int64_t reductionAxisVal = 1; reductionAxisVal < oldShape[reductionAxis];
+       ++reductionAxisVal) {
+
+    int64_t stride = std::accumulate(oldShape.begin() + reductionAxis + 1,
+                                     oldShape.end(), 1, std::multiplies<int>());
+    int64_t index = indexAtOldTensor + stride * reductionAxisVal;
+    reducedValue =
+        OperationType::calcOneElement(reducedValue, oldTensor[index]);
+  }
+  return reducedValue;
+}
+
+template <typename OperationType>
+struct ReduceConstantOptimization : public OpRewritePattern<OperationType> {
+
+  using OpRewritePattern<OperationType>::OpRewritePattern;
+
+  LogicalResult matchAndRewrite(OperationType op,
+                                PatternRewriter &rewriter) const override {
+    Value inputOp = op.getInput();
+    auto constOp = inputOp.getDefiningOp<tosa::ConstOp>();
+
+    if (!constOp)
+      return rewriter.notifyMatchFailure(
+          op, "reduce input must be const operation");
+
+    if (!inputOp.hasOneUse())
+      return rewriter.notifyMatchFailure(
+          op, "input operation has more than one user");
+
+    auto resultType = cast<ShapedType>(op.getOutput().getType());
+
+    if (!resultType.hasStaticShape())
+      return rewriter.notifyMatchFailure(op, "result type shape is not static");
+
+    auto reductionAxis = op.getAxis();
+    const auto denseElementsAttr = constOp.getValue();
+    const auto shapedOldElementsValues =
+        denseElementsAttr.getType().cast<ShapedType>();
+
+    if (!llvm::isa<IntegerType>(shapedOldElementsValues.getElementType()))
+      return rewriter.notifyMatchFailure(
+          op, "reduce input currently supported with integer type");
+
+    auto oldShape = shapedOldElementsValues.getShape();
+    auto newShape = resultType.getShape();
+
+    auto newNumOfElements = std::accumulate(newShape.begin(), newShape.end(), 1,
+                                            std::multiplies<int>());
+    llvm::SmallVector<APInt> newReducedTensor(newNumOfElements);
+
+    for (int64_t reductionIndex = 0; reductionIndex < newNumOfElements;
+         ++reductionIndex) {
+
+      /// Let's reduce all the elements along this reduction axis
+      newReducedTensor[reductionIndex] = calculateReducedValue<OperationType>(
+          denseElementsAttr, oldShape, reductionAxis, reductionIndex);
+    }
+
+    auto rankedTensorType = cast<RankedTensorType>(resultType);
+    auto denseAttr =
+        mlir::DenseElementsAttr::get(rankedTensorType, newReducedTensor);
+    rewriter.replaceOpWithNewOp<tosa::ConstOp>(op, rankedTensorType, denseAttr);
+    return success();
+  }
+};
+
 } // namespace
 
+void mlir::tosa::populateTosaConstantReduction(MLIRContext *ctx,
+                                               RewritePatternSet &patterns) {
+  patterns.add<ReduceConstantOptimization<ReduceAllOp>>(ctx);
+  patterns.add<ReduceConstantOptimization<ReduceAnyOp>>(ctx);
+  patterns.add<ReduceConstantOptimization<ReduceMaxOp>>(ctx);
+  patterns.add<ReduceConstantOptimization<ReduceMinOp>>(ctx);
+  patterns.add<ReduceConstantOptimization<ReduceProdOp>>(ctx);
+  patterns.add<ReduceConstantOptimization<ReduceSumOp>>(ctx);
+}
+
 void mlir::tosa::populateTosaFoldConstantTransposePatterns(
     MLIRContext *ctx, RewritePatternSet &patterns) {
   patterns.add<TosaFoldConstantTranspose>(ctx);
 }
 
 void mlir::tosa::populateTosaFoldConstantReciprocalPatterns(
     MLIRContext *ctx, RewritePatternSet &patterns) {
   patterns.add<TosaFoldConstantReciprocal>(ctx);
 }

mlir/lib/Dialect/Tosa/Transforms/TosaLayerwiseConstantFoldPass.cpp

: public tosa::impl::TosaLayerwiseConstantFoldPassBase<
           TosaLayerwiseConstantFoldPass> {
   void runOnOperation() override {
     auto *ctx = &getContext();
     RewritePatternSet patterns(ctx);
     auto func = getOperation();
 
     mlir::tosa::populateTosaFoldConstantReciprocalPatterns(ctx, patterns);
     mlir::tosa::populateTosaFoldConstantTransposePatterns(ctx, patterns);
+    mlir::tosa::populateTosaConstantReduction(ctx, patterns);
     populateTosaOpsCanonicalizationPatterns(ctx, patterns);
 
     if (applyPatternsAndFoldGreedily(func, std::move(patterns)).failed())
       signalPassFailure();
   }
 };
 
 } // namespace
 
 std::unique_ptr<Pass> mlir::tosa::createTosaLayerwiseConstantFoldPass() {
   return std::make_unique<TosaLayerwiseConstantFoldPass>();
 }

mlir/test/Dialect/Tosa/constant-op-fold.mlir

 // CHECK-LABEL: @reverse_length_one
 func.func @reverse_length_one(%arg0 : tensor<10x1xi32>) -> (tensor<10x1xi32>, tensor<10x1xi32>) {
   %nofold = tosa.reverse %arg0 { axis = 0 : i32 } : (tensor<10x1xi32>) -> tensor<10x1xi32>
   %fold = tosa.reverse %arg0 { axis = 1 : i32 } : (tensor<10x1xi32>) -> tensor<10x1xi32>
   // CHECK: %[[NOFOLD:.+]] = tosa.reverse %arg0 {axis = 0 : i32}
   // CHECK: return %[[NOFOLD]], %arg0
   return %nofold, %fold : tensor<10x1xi32>, tensor<10x1xi32>
 }
+
+// -----
+
+  func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+    // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+    // CHECK:    %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}5, 7, 9]]> : tensor<1x3xi32>}> : () -> tensor<1x3xi32>
+    // CHECK:         return %[[VAL_0]] : tensor<1x3xi32>
+
+    %const = "tosa.const"() {value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>} : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_sum %const {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+    return %0 : tensor<1x3xi32>
+  }
+
+// -----
+
+  func.func @reduce_sum_constant() -> tensor<2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}6], [15]]> : tensor<2x1xi32>}> : () -> tensor<2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1xi32>
+  // CHECK:         }
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_sum %const {axis = 1 : i32} : (tensor<2x3xi32>) -> tensor<2x1xi32>
+    return %0 : tensor<2x1xi32>
+  }
+
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<3x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<3x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}6], [15], [24]]> : tensor<3x1xi32>}> : () -> tensor<3x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[1, 2, 3], [4, 5, 6], [7, 8, 9]]> : tensor<3x3xi32>}> : () -> tensor<3x3xi32>
+  %0 = tosa.reduce_sum %const {axis = 1 : i32} : (tensor<3x3xi32>) -> tensor<3x1xi32>
+  return %0 : tensor<3x1xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<2x1x4xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<2x1x4xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[15, 18, 21, 24]], {{\[\[}}51, 54, 57, 60]]]> : tensor<2x1x4xi32>}> : () -> tensor<2x1x4xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24]]]> : tensor<2x3x4xi32>}> : () -> tensor<2x3x4xi32>
+  %0 = tosa.reduce_sum %const {axis = 1 : i32} : (tensor<2x3x4xi32>) -> tensor<2x1x4xi32>
+  return %0 : tensor<2x1x4xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<1x3x3xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<1x3x3xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[30, 33, 36], [39, 42, 45], [48, 51, 54]]]> : tensor<1x3x3xi32>}> : () -> tensor<1x3x3xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x3x3xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]], [[19, 20, 21], [22, 23, 24], [25, 26, 27]]]> : tensor<3x3x3xi32>}> : () -> tensor<3x3x3xi32>
+  %0 = tosa.reduce_sum %const {axis = 0 : i32} : (tensor<3x3x3xi32>) -> tensor<1x3x3xi32>
+  return %0 : tensor<1x3x3xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}{{\[\[}}3], [7]], {{\[\[}}11], [15]]], {{\[\[}}[19], [23]], {{\[\[}}27], [31]]]]> : tensor<2x2x2x1xi32>}> : () -> tensor<2x2x2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x2x2x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]> : tensor<2x2x2x2xi32>}> : () -> tensor<2x2x2x2xi32>
+  %0 = tosa.reduce_sum %const {axis = 3 : i32} : (tensor<2x2x2x2xi32>) -> tensor<2x2x2x1xi32>
+  return %0 : tensor<2x2x2x1xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<1x1x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<1x1x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<42> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x1x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[42]]]> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  %0 = tosa.reduce_sum %const {axis = 0 : i32} : (tensor<1x1x1xi32>) -> tensor<1x1x1xi32>
+  return %0 : tensor<1x1x1xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<2x3x1x5xi32> {
+  // CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<2x3x1x5xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}{{\[\[}}34, 38, 42, 46, 50]], {{\[\[}}114, 118, 122, 126, 130]], {{\[\[}}194, 198, 202, 206, 210]]], {{\[\[}}[274, 278, 282, 286, 290]], {{\[\[}}354, 358, 362, 366, 370]], {{\[\[}}434, 438, 442, 446, 450]]]]> : tensor<2x3x1x5xi32>}> : () -> tensor<2x3x1x5xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x3x1x5xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20]], [[21, 22, 23, 24, 25], [26, 27, 28, 29, 30], [31, 32, 33, 34, 35], [36, 37, 38, 39, 40]], [[41, 42, 43, 44, 45], [46, 47, 48, 49, 50], [51, 52, 53, 54, 55], [56, 57, 58, 59, 60]]], [[[61, 62, 63, 64, 65], [66, 67, 68, 69, 70], [71, 72, 73, 74, 75], [76, 77, 78, 79, 80]], [[81, 82, 83, 84, 85], [86, 87, 88, 89, 90], [91, 92, 93, 94, 95], [96, 97, 98, 99, 100]], [[101, 102, 103, 104, 105], [106, 107, 108, 109, 110], [111, 112, 113, 114, 115], [116, 117, 118, 119, 120]]]]> : tensor<2x3x4x5xi32>}> : () -> tensor<2x3x4x5xi32>
+  %0 = tosa.reduce_sum %const {axis = 2 : i32} : (tensor<2x3x4x5xi32>) -> tensor<2x3x1x5xi32>
+  return %0 : tensor<2x3x1x5xi32>
+}
+
+// -----
+
+  func.func @reduce_prod_constant() -> tensor<1x3xi32> {
+    // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<1x3xi32> {
+    // CHECK:    %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}4, 10, 18]]> : tensor<1x3xi32>}> : () -> tensor<1x3xi32>
+    // CHECK:         return %[[VAL_0]] : tensor<1x3xi32>
+
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_prod %const {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+    return %0 : tensor<1x3xi32>
+  }
+
+// -----
+
+  func.func @reduce_prod_constant() -> tensor<2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}6], [120]]> : tensor<2x1xi32>}> : () -> tensor<2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1xi32>
+  // CHECK:         }
+
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_prod %const {axis = 1 : i32} : (tensor<2x3xi32>) -> tensor<2x1xi32>
+    return %0 : tensor<2x1xi32>
+  }
+
+// -----
+
+func.func @reduce_prod_constant() -> tensor<3x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<3x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}6], [120], [504]]> : tensor<3x1xi32>}> : () -> tensor<3x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[1, 2, 3], [4, 5, 6], [7, 8, 9]]> : tensor<3x3xi32>}> : () -> tensor<3x3xi32>
+  %0 = tosa.reduce_prod %const {axis = 1 : i32} : (tensor<3x3xi32>) -> tensor<3x1xi32>
+  return %0 : tensor<3x1xi32>
+}
+
+// -----
+
+func.func @reduce_prod_constant() -> tensor<2x1x4xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<2x1x4xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[45, 120, 231, 384]], {{\[\[}}4641, 5544, 6555, 7680]]]> : tensor<2x1x4xi32>}> : () -> tensor<2x1x4xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24]]]> : tensor<2x3x4xi32>}> : () -> tensor<2x3x4xi32>
+  %0 = tosa.reduce_prod %const {axis = 1 : i32} : (tensor<2x3x4xi32>) -> tensor<2x1x4xi32>
+  return %0 : tensor<2x1x4xi32>
+}
+
+// -----
+
+func.func @reduce_prod_constant() -> tensor<1x3x3xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<1x3x3xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[190, 440, 756], [1144, 1610, 2160], [2800, 3536, 4374]]]> : tensor<1x3x3xi32>}> : () -> tensor<1x3x3xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x3x3xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]], [[19, 20, 21], [22, 23, 24], [25, 26, 27]]]> : tensor<3x3x3xi32>}> : () -> tensor<3x3x3xi32>
+  %0 = tosa.reduce_prod %const {axis = 0 : i32} : (tensor<3x3x3xi32>) -> tensor<1x3x3xi32>
+  return %0 : tensor<1x3x3xi32>
+}
+
+// -----
+
+func.func @reduce_prod_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}{{\[\[}}2], [12]], {{\[\[}}30], [56]]], {{\[\[}}[90], [132]], {{\[\[}}182], [240]]]]> : tensor<2x2x2x1xi32>}> : () -> tensor<2x2x2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x2x2x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]> : tensor<2x2x2x2xi32>}> : () -> tensor<2x2x2x2xi32>
+  %0 = tosa.reduce_prod %const {axis = 3 : i32} : (tensor<2x2x2x2xi32>) -> tensor<2x2x2x1xi32>
+  return %0 : tensor<2x2x2x1xi32>
+}
+
+// -----
+
+func.func @reduce_prod_constant() -> tensor<1x1x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_prod_constant() -> tensor<1x1x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<42> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x1x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[42]]]> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  %0 = tosa.reduce_prod %const {axis = 0 : i32} : (tensor<1x1x1xi32>) -> tensor<1x1x1xi32>
+  return %0 : tensor<1x1x1xi32>
+}
+
+// -----
+
+  func.func @reduce_max_constant() -> tensor<1x3xi32> {
+    // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<1x3xi32> {
+    // CHECK:    %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}4, 5, 6]]> : tensor<1x3xi32>}> : () -> tensor<1x3xi32>
+    // CHECK:         return %[[VAL_0]] : tensor<1x3xi32>
+
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_max %const {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+    return %0 : tensor<1x3xi32>
+  }
+
+// -----
+
+  func.func @reduce_max_constant() -> tensor<2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}3], [6]]> : tensor<2x1xi32>}> : () -> tensor<2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1xi32>
+  // CHECK:         }
+
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_max %const {axis = 1 : i32} : (tensor<2x3xi32>) -> tensor<2x1xi32>
+    return %0 : tensor<2x1xi32>
+  }
+
+// -----
+
+func.func @reduce_max_constant() -> tensor<3x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<3x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}3], [6], [9]]> : tensor<3x1xi32>}> : () -> tensor<3x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[1, 2, 3], [4, 5, 6], [7, 8, 9]]> : tensor<3x3xi32>}> : () -> tensor<3x3xi32>
+  %0 = tosa.reduce_max %const {axis = 1 : i32} : (tensor<3x3xi32>) -> tensor<3x1xi32>
+  return %0 : tensor<3x1xi32>
+}
+
+// -----
+
+func.func @reduce_max_constant() -> tensor<2x1x4xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<2x1x4xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[9, 10, 11, 12]], {{\[\[}}21, 22, 23, 24]]]> : tensor<2x1x4xi32>}> : () -> tensor<2x1x4xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24]]]> : tensor<2x3x4xi32>}> : () -> tensor<2x3x4xi32>
+  %0 = tosa.reduce_max %const {axis = 1 : i32} : (tensor<2x3x4xi32>) -> tensor<2x1x4xi32>
+  return %0 : tensor<2x1x4xi32>
+}
+
+// -----
+
+func.func @reduce_max_constant() -> tensor<1x3x3xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<1x3x3xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[19, 20, 21], [22, 23, 24], [25, 26, 27]]]> : tensor<1x3x3xi32>}> : () -> tensor<1x3x3xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x3x3xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]], [[19, 20, 21], [22, 23, 24], [25, 26, 27]]]> : tensor<3x3x3xi32>}> : () -> tensor<3x3x3xi32>
+  %0 = tosa.reduce_max %const {axis = 0 : i32} : (tensor<3x3x3xi32>) -> tensor<1x3x3xi32>
+  return %0 : tensor<1x3x3xi32>
+}
+
+// -----
+
+func.func @reduce_max_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}{{\[\[}}2], [4]], {{\[\[}}6], [8]]], {{\[\[}}[10], [12]], {{\[\[}}14], [16]]]]> : tensor<2x2x2x1xi32>}> : () -> tensor<2x2x2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x2x2x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]> : tensor<2x2x2x2xi32>}> : () -> tensor<2x2x2x2xi32>
+  %0 = tosa.reduce_max %const {axis = 3 : i32} : (tensor<2x2x2x2xi32>) -> tensor<2x2x2x1xi32>
+  return %0 : tensor<2x2x2x1xi32>
+}
+
+// -----
+
+func.func @reduce_max_constant() -> tensor<1x1x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_max_constant() -> tensor<1x1x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<42> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x1x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[42]]]> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  %0 = tosa.reduce_max %const {axis = 0 : i32} : (tensor<1x1x1xi32>) -> tensor<1x1x1xi32>
+  return %0 : tensor<1x1x1xi32>
+}
+
+// -----
+
+  func.func @reduce_min_constant() -> tensor<1x3xi32> {
+    // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<1x3xi32> {
+    // CHECK:    %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}1, 2, 3]]> : tensor<1x3xi32>}> : () -> tensor<1x3xi32>
+    // CHECK:         return %[[VAL_0]] : tensor<1x3xi32>
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_min %const {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+    return %0 : tensor<1x3xi32>
+  }
+
+
+// -----
+
+  func.func @reduce_min_constant() -> tensor<2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}1], [4]]> : tensor<2x1xi32>}> : () -> tensor<2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1xi32>
+  // CHECK:         }
+
+    %const = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+    %0 = tosa.reduce_min %const {axis = 1 : i32} : (tensor<2x3xi32>) -> tensor<2x1xi32>
+    return %0 : tensor<2x1xi32>
+  }
+
+// -----
+
+func.func @reduce_min_constant() -> tensor<3x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<3x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}1], [4], [7]]> : tensor<3x1xi32>}> : () -> tensor<3x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[1, 2, 3], [4, 5, 6], [7, 8, 9]]> : tensor<3x3xi32>}> : () -> tensor<3x3xi32>
+  %0 = tosa.reduce_min %const {axis = 1 : i32} : (tensor<3x3xi32>) -> tensor<3x1xi32>
+  return %0 : tensor<3x1xi32>
+}
+
+// -----
+
+func.func @reduce_min_constant() -> tensor<2x1x4xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<2x1x4xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[1, 2, 3, 4]], {{\[\[}}13, 14, 15, 16]]]> : tensor<2x1x4xi32>}> : () -> tensor<2x1x4xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], [[13, 14, 15, 16], [17, 18, 19, 20], [21, 22, 23, 24]]]> : tensor<2x3x4xi32>}> : () -> tensor<2x3x4xi32>
+  %0 = tosa.reduce_min %const {axis = 1 : i32} : (tensor<2x3x4xi32>) -> tensor<2x1x4xi32>
+  return %0 : tensor<2x1x4xi32>
+}
+
+// -----
+
+func.func @reduce_min_constant() -> tensor<1x3x3xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<1x3x3xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[1, 2, 3], [4, 5, 6], [7, 8, 9]]]> : tensor<1x3x3xi32>}> : () -> tensor<1x3x3xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x3x3xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[1, 2, 3], [4, 5, 6], [7, 8, 9]], [[10, 11, 12], [13, 14, 15], [16, 17, 18]], [[19, 20, 21], [22, 23, 24], [25, 26, 27]]]> : tensor<3x3x3xi32>}> : () -> tensor<3x3x3xi32>
+  %0 = tosa.reduce_min %const {axis = 0 : i32} : (tensor<3x3x3xi32>) -> tensor<1x3x3xi32>
+  return %0 : tensor<1x3x3xi32>
+}
+
+// -----
+
+func.func @reduce_min_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<2x2x2x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}{{\[\[}}1], [3]], {{\[\[}}5], [7]]], {{\[\[}}[9], [11]], {{\[\[}}13], [15]]]]> : tensor<2x2x2x1xi32>}> : () -> tensor<2x2x2x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<2x2x2x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]> : tensor<2x2x2x2xi32>}> : () -> tensor<2x2x2x2xi32>
+  %0 = tosa.reduce_min %const {axis = 3 : i32} : (tensor<2x2x2x2xi32>) -> tensor<2x2x2x1xi32>
+  return %0 : tensor<2x2x2x1xi32>
+}
+
+// -----
+
+func.func @reduce_min_constant() -> tensor<1x1x1xi32> {
+  // CHECK-LABEL:   func.func @reduce_min_constant() -> tensor<1x1x1xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<42> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  // CHECK:           return %[[VAL_0]] : tensor<1x1x1xi32>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[42]]]> : tensor<1x1x1xi32>}> : () -> tensor<1x1x1xi32>
+  %0 = tosa.reduce_min %const {axis = 0 : i32} : (tensor<1x1x1xi32>) -> tensor<1x1x1xi32>
+  return %0 : tensor<1x1x1xi32>
+}
+
+// -----
+
+func.func @reduce_any_constant() -> tensor<1x3xi1> {
+  // CHECK-LABEL:   func.func @reduce_any_constant() -> tensor<1x3xi1> {
+  // CHECK:    %[[VAL_0:.*]] = "tosa.const"() <{value = dense<true> : tensor<1x3xi1>}> : () -> tensor<1x3xi1>
+  // CHECK:         return %[[VAL_0]] : tensor<1x3xi1>
+
+  %const = "tosa.const"() <{value = dense<[[true,true,true], [true,false,true]]> : tensor<2x3xi1>}> : () -> tensor<2x3xi1>
+  %0 = tosa.reduce_any %const {axis = 0 : i32} : (tensor<2x3xi1>) -> tensor<1x3xi1>
+  return %0 : tensor<1x3xi1>
+}
+
+
+// -----
+
+func.func @reduce_any_constant() -> tensor<2x1xi1> {
+// CHECK-LABEL:   func.func @reduce_any_constant() -> tensor<2x1xi1> {
+// CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<true> : tensor<2x1xi1>}> : () -> tensor<2x1xi1>
+// CHECK:           return %[[VAL_0]] : tensor<2x1xi1>
+// CHECK:         }
+
+  %const = "tosa.const"() <{value = dense<[[true,true,true], [true,false,true]]> : tensor<2x3xi1>}> : () -> tensor<2x3xi1>
+  %0 = tosa.reduce_any %const {axis = 1 : i32} : (tensor<2x3xi1>) -> tensor<2x1xi1>
+  return %0 : tensor<2x1xi1>
+}
+
+// -----
+
+func.func @reduce_any_constant() -> tensor<3x1xi1> {
+  // CHECK-LABEL:   func.func @reduce_any_constant() -> tensor<3x1xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}true], [false], [true]]> : tensor<3x1xi1>}> : () -> tensor<3x1xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi1>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[true, false, false], [false, false, false], [false, false, true]]> : tensor<3x3xi1>}> : () -> tensor<3x3xi1>
+  %0 = tosa.reduce_any %const {axis = 1 : i32} : (tensor<3x3xi1>) -> tensor<3x1xi1>
+  return %0 : tensor<3x1xi1>
+}
+
+// -----
+
+func.func @reduce_any_constant() -> tensor<2x1x4xi1> {
+  // CHECK-LABEL:   func.func @reduce_any_constant() -> tensor<2x1x4xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}[true, false, true, true]], {{\[\[}}true, false, true, false]]]> : tensor<2x1x4xi1>}> : () -> tensor<2x1x4xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi1>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[true, false, false, true], [false, false, true, false], [true, false, true, true]], [[false, false, false, false], [false, false, true, false], [true, false, true, false]]]> : tensor<2x3x4xi1>}> : () -> tensor<2x3x4xi1>
+  %0 = tosa.reduce_any %const {axis = 1 : i32} : (tensor<2x3x4xi1>) -> tensor<2x1x4xi1>
+  return %0 : tensor<2x1x4xi1>
+}
+
+// -----
+
+  func.func @reduce_all_constant() -> tensor<1x3xi1> {
+  // CHECK-LABEL:   func.func @reduce_all_constant() -> tensor<1x3xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}true, false, true]]> : tensor<1x3xi1>}> : () -> tensor<1x3xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<1x3xi1>
+  // CHECK:         }
+    %const = "tosa.const"() <{value = dense<[[true,true,true], [true,false,true]]> : tensor<2x3xi1>}> : () -> tensor<2x3xi1>
+    %0 = tosa.reduce_all %const {axis = 0 : i32} : (tensor<2x3xi1>) -> tensor<1x3xi1>
+    return %0 : tensor<1x3xi1>
+  }
+
+// -----
+
+  func.func @reduce_all_constant() -> tensor<2x1xi1> {
+  // CHECK-LABEL:   func.func @reduce_all_constant() -> tensor<2x1xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}true], [false]]> : tensor<2x1xi1>}> : () -> tensor<2x1xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1xi1>
+  // CHECK:         }
+    %const = "tosa.const"() <{value = dense<[[true,true,true], [true,false,true]]> : tensor<2x3xi1>}> : () -> tensor<2x3xi1>
+    %0 = tosa.reduce_all %const {axis = 1 : i32} : (tensor<2x3xi1>) -> tensor<2x1xi1>
+    return %0 : tensor<2x1xi1>
+  }
+
+// -----
+
+func.func @reduce_all_constant() -> tensor<3x1xi1> {
+  // CHECK-LABEL:   func.func @reduce_all_constant() -> tensor<3x1xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<false> : tensor<3x1xi1>}> : () -> tensor<3x1xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<3x1xi1>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[true, false, false], [false, false, false], [false, false, true]]> : tensor<3x3xi1>}> : () -> tensor<3x3xi1>
+  %0 = tosa.reduce_all %const {axis = 1 : i32} : (tensor<3x3xi1>) -> tensor<3x1xi1>
+  return %0 : tensor<3x1xi1>
+}
+
+// -----
+
+func.func @reduce_all_constant() -> tensor<2x1x4xi1> {
+  // CHECK-LABEL:   func.func @reduce_all_constant() -> tensor<2x1x4xi1> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<false> : tensor<2x1x4xi1>}> : () -> tensor<2x1x4xi1>
+  // CHECK:           return %[[VAL_0]] : tensor<2x1x4xi1>
+  // CHECK:         }
+  %const = "tosa.const"() <{value = dense<[[[true, false, false, true], [false, false, true, false], [true, false, true, true]], [[false, false, false, false], [false, false, true, false], [true, false, true, false]]]> : tensor<2x3x4xi1>}> : () -> tensor<2x3x4xi1>
+  %0 = tosa.reduce_all %const {axis = 1 : i32} : (tensor<2x3x4xi1>) -> tensor<2x1x4xi1>
+  return %0 : tensor<2x1x4xi1>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+// CHECK-LABEL:   func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+// CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<2> : tensor<1x3xi32>}> : () -> tensor<1x3xi32>
+// CHECK:           return %[[VAL_0]] : tensor<1x3xi32>
+// CHECK:         }
+  %const = "tosa.const"() <{value = dense<1> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+  %0 = tosa.reduce_sum %const {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+  return %0 : tensor<1x3xi32>
+}
+
+// -----
+
+func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+  // CHECK-LABEL:     func.func @reduce_sum_constant() -> tensor<1x3xi32> {
+  // CHECK:           %[[VAL_0:.*]] = "tosa.const"() <{value = dense<{{\[\[}}1, 2, 3], [4, 5, 6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+  // CHECK:           %[[VAL_1:.*]] = "tosa.const"() <{value = dense<{{\[\[}}1, 2, 3], [4, 5, 7]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+  // CHECK:           %[[VAL_2:.*]] = tosa.add %[[VAL_0]], %[[VAL_1]] : (tensor<2x3xi32>, tensor<2x3xi32>) -> tensor<2x3xi32>
+  // CHECK:           %[[VAL_3:.*]] = tosa.reduce_sum %[[VAL_2]] {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+  // CHECK:           return %[[VAL_3]] : tensor<1x3xi32>
+  %arg0 = "tosa.const"() <{value = dense<[[1,2,3], [4,5,6]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+  %arg1 = "tosa.const"() <{value = dense<[[1,2,3], [4,5,7]]> : tensor<2x3xi32>}> : () -> tensor<2x3xi32>
+  %arg2 = tosa.add %arg0, %arg1 : (tensor<2x3xi32>, tensor<2x3xi32>) -> tensor<2x3xi32>
+  %0 = tosa.reduce_sum %arg2 {axis = 0 : i32} : (tensor<2x3xi32>) -> tensor<1x3xi32>
+  return %0 : tensor<1x3xi32>
+}