[MLIR] Add explicit initial values for loop.parallel op.

[pifon2a]

Summary

Add initial value operands to loop.parallel. The operands correspond
to the result types and loop.reduce ops in the body of the op.

[herhut]

Thanks. This looks great with some minor changes addressed.

mlir/include/mlir/Dialect/LoopOps/LoopOps.td

238

This confused me when I read it. The initial values are not optional. You need the same number of initial values as there are return values (which is mentioned below).

mlir/lib/Dialect/LoopOps/LoopOps.cpp

322

Drop the const and & and avoid auto if the actual type is concise. for (Value init : initVals).

346

I think it is preferred to not mix the two forms and just use op.emitOpError() << ...

428

This should reflect what you actually parsed. So lower.size(), upper.size(), steps.size() and initVals.size().

[pifon2a]

Addressed the comments.

[rriddle]

mlir/lib/Dialect/LoopOps/LoopOps.cpp

416

nit: Can you please add /*variableName=*/ comments when using constant arguments?

[mehdi_amini]

mlir/lib/Dialect/LoopOps/LoopOps.cpp

413

Nit: the comment needs an update here.

433

Have you considered having the type closer to the init?

%res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) init (%zero : f32) {

or

%res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) init (%zero) -> f32 {

Having to skip visually the body is not the most readable to me.

[pifon2a]

mlir/lib/Dialect/LoopOps/LoopOps.cpp

433

https://reviews.llvm.org/D75832 it is more readable indeed.

mlir/include/mlir/Dialect/LoopOps/LoopOps.td

let description = [{
     argument of this region. This SSA value always has type index, which is the
     size of the machine word. The step is a value of type index, required to be
     positive.
     The lower and upper bounds specify a half-open range: the range includes
     the lower bound but does not include the upper bound.
 
     The body region must contain exactly one block that terminates with
     "loop.yield".  Calling ForOp::build will create such a region and insert
-    the terminator implicitly if none is defined, so will the parsing even 
+    the terminator implicitly if none is defined, so will the parsing even
     in cases when it is absent from the custom format. For example:
 
     ```mlir
        loop.for %iv = %lb to %ub step %step {
          ... // body
        }
     ```
 
     "loop.for" can also operate on loop-carried variables and returns the final values
-    after loop termination. The initial values of the variables are passed as additional SSA 
+    after loop termination. The initial values of the variables are passed as additional SSA
     operands to the "loop.for" following the 3 loop control SSA values mentioned above
-    (lower bound, upper bound and step). The operation region has equivalent arguments 
+    (lower bound, upper bound and step). The operation region has equivalent arguments
     for each variable representing the value of the variable at the current iteration.
-    
-    The region must terminate with a "loop.yield" that passes all the current iteration 
+    The region must terminate with a "loop.yield" that passes all the current iteration
     variables to the next iteration, or to the "loop.for" result, if at the last iteration.
-    "loop.for" results hold the final values after the last iteration. 
-    
+    "loop.for" results hold the final values after the last iteration.
     For example, to sum-reduce a memref:
-    
     ```mlir
     func @reduce(%buffer: memref<1024xf32>, %lb: index, %ub: index, %step: index) -> (f32) {
       // Initial sum set to 0.
       %sum_0 = constant 0.0 : f32
       // iter_args binds initial values to the loop's region arguments.
       %sum = loop.for %iv = %lb to %ub step %step iter_args(%sum_iter = %sum_0) -> (f32) {
         %t = load %buffer[%iv] : memref<1024xf32>
         %sum_next = addf %sum_iter, %t : f32
         // Yield current iteration sum to next iteration %sum_iter or to %sum if final iteration.
         loop.yield %sum_next : f32
       }
       return %sum : f32
-    }   
+    }
     ```
 
-    If the "loop.for" defines any values, a yield must be explicitly present. 
-    The number and types of the "loop.for" results must match the initial values 
+    If the "loop.for" defines any values, a yield must be explicitly present.
+    The number and types of the "loop.for" results must match the initial values
     in the "iter_args" binding and the yield operands.
-    
-    Another example with a nested "loop.if" (see "loop.if" for details) 
+    Another example with a nested "loop.if" (see "loop.if" for details)
     to perform conditional reduction:
 
     ```mlir
     func @conditional_reduce(%buffer: memref<1024xf32>, %lb: index, %ub: index, %step: index) -> (f32) {
       %sum_0 = constant 0.0 : f32
       %c0 = constant 0.0 : f32
       %sum = loop.for %iv = %lb to %ub step %step iter_args(%sum_iter = %sum_0) -> (f32) {
         %t = load %buffer[%iv] : memref<1024xf32>
func @conditional_reduce(%buffer: memref<1024xf32>, %lb: index, %ub: index, %step: index) -> (f32) {
       return %sum : f32
     }
   ```
   }];
   let arguments = (ins Index:$lowerBound,
                        Index:$upperBound,
                        Index:$step,
                        Variadic<AnyType>:$initArgs);
-  let results = (outs Variadic<AnyType>:$results);                       
+  let results = (outs Variadic<AnyType>:$results);
   let regions = (region SizedRegion<1>:$region);
 
   let skipDefaultBuilders = 1;
   let builders = [
     OpBuilder<"Builder *builder, OperationState &result, "
               "Value lowerBound, Value upperBound, Value step">
   ];
 
   let extraClassDeclaration = [{
     Block *getBody() { return &region().front(); }
     Value getInductionVar() { return getBody()->getArgument(0); }
     OpBuilder getBodyBuilder() {
       return OpBuilder(getBody(), std::prev(getBody()->end()));
     }
     iterator_range<Block::args_iterator> getRegionIterArgs() {
       return getBody()->getArguments().drop_front();
     }
     iterator_range<Operation::operand_iterator> getIterOperands() {
       return getOperands().drop_front(getNumControlOperands());
     }
 
     void setLowerBound(Value bound) { getOperation()->setOperand(0, bound); }
     void setUpperBound(Value bound) { getOperation()->setOperand(1, bound); }
     void setStep(Value step) { getOperation()->setOperand(2, step); }
-    
     /// Number of region arguments for loop-carried values
     unsigned getNumRegionIterArgs() {
-      return getBody()->getNumArguments() - 1; 
+      return getBody()->getNumArguments() - 1;
     }
     /// Number of operands controlling the loop: lb, ub, step
     unsigned getNumControlOperands() { return 3; }
     /// Does the operation hold operands for loop-carried values
-    bool hasIterOperands() { 
+    bool hasIterOperands() {
       return getOperation()->getNumOperands() > getNumControlOperands();
     }
     /// Get Number of loop-carried values
     unsigned getNumIterOperands() {
       return getOperation()->getNumOperands() - getNumControlOperands();
     }
   }];
 }
```mlir
        loop.if %b  {
          ...
        } else {
          ...
        }
     ```
 
     "loop.if" may also return results that are defined in its regions. The values
-    defined are determined by which execution path is taken. 
+    defined are determined by which execution path is taken.
     For example:
     ```mlir
        %x, %y = loop.if %b -> (f32, f32) {
          %x_true = ...
          %y_true = ...
          loop.yield %x_true, %y_true : f32, f32
        } else {
-         %x_false = ... 
-         %y_false = ... 
+         %x_false = ...
+         %y_false = ...
          loop.yield %x_false, %y_false : f32, f32
        }
     ```
 
     "loop.if" regions are always terminated with "loop.yield". If "loop.if"
     defines no values, the "loop.yield" can be left out, and will be
     inserted implicitly. Otherwise, it must be explicit.
     Also, if "loop.if" defines one or more values, the 'else' block cannot
-    be omitted. 
+    be omitted.
 
     For example:
     ```mlir
        loop.if %b  {
          ...
        }
     ```
   }];
OpBuilder getElseBodyBuilder() {
       assert(!elseRegion().empty() && "Unexpected empty 'else' region.");
       Block &body = elseRegion().front();
       return OpBuilder(&body, std::prev(body.end()));
     }
   }];
 }
 
 def ParallelOp : Loop_Op<"parallel",
-    [SameVariadicOperandSize, SingleBlockImplicitTerminator<"YieldOp">]> {
+    [AttrSizedOperandSegments, SingleBlockImplicitTerminator<"YieldOp">]> {
   let summary = "parallel for operation";
   let description = [{
-    The "loop.parallel" operation represents a loop nest taking 3 groups of SSA
-    values as operands that represent the lower bounds, upper bounds and steps,
-    respectively. The operation defines a variadic number of SSA values for its
-    induction variables. It has one region capturing the loop body. The
-    induction variables are represented as an argument of this region. These SSA
-    values always have type index, which is the size of the machine word. The
-    steps are values of type index, required to be positive.
-    The lower and upper bounds specify a half-open range: the range includes the
-    lower bound but does not include the upper bound.
+    The "loop.parallel" operation represents a loop nest taking 4 groups of SSA
+    values as operands that represent the lower bounds, upper bounds, steps and
+    optional initial values, respectively. The operation defines a variadic
+    number of SSA values for its induction variables. It has one region
+    capturing the loop body. The induction variables are represented as an
+    argument of this region. These SSA values always have type index, which is
+    the size of the machine word. The steps are values of type index, required
+    to be positive. The lower and upper bounds specify a half-open range: the
+    range includes the lower bound but does not include the upper bound. The
+    initial values have the same types as results of "loop.parallel".
 
     Semantically we require that the iteration space can be iterated in any
     order, and the loop body can be executed in parallel. If there are data
     races, the behavior is undefined.
 
     The parallel loop operation supports reduction of values produced by
     individual iterations into a single result. This is modeled using the
     loop.reduce operation (see loop.reduce for details). Each result of a
-    loop.parallel operation is associated with a reduce operation that is an
-    immediate child. Reduces are matched to result values in order of their
-    appearance in the body. Consequently, we require that the body region has
-    the same number of results as it has reduce operations.
+    loop.parallel operation is associated with an initial value operand and
+    reduce operation that is an immediate child. Reductions are matched to result
+    and initial values in order of their appearance in the body. Consequently,
+    we require that the body region has the same number of results and initial
+    values as it has reduce operations.
 
     The body region must contain exactly one block that terminates with
     "loop.yield" without operands. Parsing ParallelOp will create such a region
     and insert the terminator when it is absent from the custom format. For example:
 
     ```mlir
        loop.parallel (%iv) = (%lb) to (%ub) step (%step) {
          %zero = constant 0.0 : f32
          loop.reduce(%zero) {
            ^bb0(%lhs : f32, %rhs: f32):
              %res = addf %lhs, %rhs : f32
              loop.reduce.return %res : f32
          } : f32
        }
     ```
    }];
 
   let arguments = (ins Variadic<Index>:$lowerBound,
                        Variadic<Index>:$upperBound,
-                       Variadic<Index>:$step);
+                       Variadic<Index>:$step,
+                       Variadic<AnyType>:$initVals);
   let results = (outs Variadic<AnyType>:$results);
   let regions = (region SizedRegion<1>:$region);
 
   let skipDefaultBuilders = 1;
   let builders = [
     OpBuilder<"Builder *builder, OperationState &result, "
               "ValueRange lowerBounds, ValueRange upperBounds, "
-              "ValueRange steps">,
-    OpBuilder<"Builder *builder, OperationState &result, "
-              "ValueRange lowerBounds, ValueRange upperBounds, "
-              "ValueRange steps, ArrayRef<Type> resultTypes">
+              "ValueRange steps, ValueRange initVals = {}">,
   ];
 
   let extraClassDeclaration = [{
     Block *getBody() { return &region().front(); }
     iterator_range<Block::args_iterator> getInductionVars() {
-      return {getBody()->args_begin(), getBody()->args_end()};
+      return getBody()->getArguments();
     }
     unsigned getNumLoops() { return step().size(); }
+    unsigned getNumReductions() { return initVals().size(); }
   }];
 }
 
 def ReduceOp : Loop_Op<"reduce", [HasParent<"ParallelOp">]> {
   let summary = "reduce operation for parallel for";
   let description = [{
     "loop.reduce" is an operation occurring inside "loop.parallel" operations.
     It consists of one block with two arguments which have the same type as the
def ReduceReturnOp :
   let assemblyFormat = "$result attr-dict `:` type($result)";
 }
 
 def YieldOp : Loop_Op<"yield", [Terminator]> {
   let summary = "loop yield and termination operation";
   let description = [{
     "loop.yield" yields an SSA value from a loop dialect op region and
     terminates the regions. The semantics of how the values are yielded
-    is defined by the parent operation. 
+    is defined by the parent operation.
     If "loop.yield" has any operands, the operands must match the parent
-    operation's results. 
-    If the parent operation defines no values, then the "loop.yield" may be 
+    operation's results.
+    If the parent operation defines no values, then the "loop.yield" may be
     left out in the custom syntax and the builders will insert one implicitly.
     Otherwise, it has to be present in the syntax to indicate which values
-    are yielded. 
+    are yielded.
   }];
 
   let arguments = (ins Variadic<AnyType>:$results);
   let builders = [
     OpBuilder<"Builder *builder, OperationState &result", [{ /* nothing to do */ }]>
   ];
 }
 #endif // LOOP_OPS

mlir/lib/Dialect/LoopOps/LoopOps.cpp

static void print(OpAsmPrinter &p, IfOp op) {
   p.printOptionalAttrDict(op.getAttrs());
 }
 
 //===----------------------------------------------------------------------===//
 // ParallelOp
 //===----------------------------------------------------------------------===//
 
 void ParallelOp::build(Builder *builder, OperationState &result, ValueRange lbs,
-                       ValueRange ubs, ValueRange steps) {
+                       ValueRange ubs, ValueRange steps, ValueRange initVals) {
   result.addOperands(lbs);
   result.addOperands(ubs);
   result.addOperands(steps);
+  result.addOperands(initVals);
+  result.addAttribute(
+      ParallelOp::getOperandSegmentSizeAttr(),
+      builder->getI32VectorAttr({static_cast<int32_t>(lbs.size()),
+                                 static_cast<int32_t>(ubs.size()),
+                                 static_cast<int32_t>(steps.size()),
+                                 static_cast<int32_t>(initVals.size())}));
   Region *bodyRegion = result.addRegion();
   ParallelOp::ensureTerminator(*bodyRegion, *builder, result.location);
   for (size_t i = 0, e = steps.size(); i < e; ++i)
     bodyRegion->front().addArgument(builder->getIndexType());
+  for (const auto &init : initVals) {
+    result.addTypes(init.getType());
   }
-
-void ParallelOp::build(Builder *builder, OperationState &result, ValueRange lbs,
-                       ValueRange ubs, ValueRange steps,
-                       ArrayRef<Type> resultTypes) {
-  result.addTypes(resultTypes);
-  build(builder, result, lbs, ubs, steps);
 }
 
 static LogicalResult verify(ParallelOp op) {
   // Check that there is at least one value in lowerBound, upperBound and step.
   // It is sufficient to test only step, because it is ensured already that the
   // number of elements in lowerBound, upperBound and step are the same.
   Operation::operand_range stepValues = op.step();
   if (stepValues.empty())
     return op.emitOpError(
         "needs at least one tuple element for lowerBound, upperBound and step");
 
   // Check whether all constant step values are positive.
   for (Value stepValue : stepValues)
     if (auto cst = dyn_cast_or_null<ConstantIndexOp>(stepValue.getDefiningOp()))
       if (cst.getValue() <= 0)
         return op.emitOpError("constant step operand must be positive");
 
   // Check that the body defines the same number of block arguments as the
   // number of tuple elements in step.
   Block *body = op.getBody();
   if (body->getNumArguments() != stepValues.size())
-    return op.emitOpError(
-        "expects the same number of induction variables as bound and step "
-        "values");
+    return op.emitOpError("expects the same number of induction variables: ")
+           << body->getNumArguments()
+           << " as bound and step values: " << stepValues.size();
   for (auto arg : body->getArguments())
     if (!arg.getType().isIndex())
       return op.emitOpError(
           "expects arguments for the induction variable to be of index type");
 
   // Check that the number of results is the same as the number of ReduceOps.
   SmallVector<ReduceOp, 4> reductions(body->getOps<ReduceOp>());
-  if (op.results().size() != reductions.size())
-    return op.emitOpError(
-        "expects number of results to be the same as number of reductions");
+  auto resultsSize = op.results().size();
+  auto reductionsSize = reductions.size();
+  auto initValsSize = op.initVals().size();
+  if (resultsSize != reductionsSize)
+    return op.emitOpError("expects number of results: ")
+           << resultsSize
+           << " to be the same as number of reductions: " << reductionsSize;
+  if (resultsSize != initValsSize)
+    return op.emitOpError("expects number of results: ")
+           << resultsSize
+           << " to be the same as number of initial values: " << initValsSize;
 
   // Check that the types of the results and reductions are the same.
   for (auto resultAndReduce : llvm::zip(op.results(), reductions)) {
     auto resultType = std::get<0>(resultAndReduce).getType();
     auto reduceOp = std::get<1>(resultAndReduce);
     auto reduceType = reduceOp.operand().getType();
     if (resultType != reduceType)
       return reduceOp.emitOpError()
-             << "expects type of reduce to be the same as result type: "
-             << resultType;
+             << "expects type of reduce: " << reduceType
+             << " to be the same as result type: " << resultType;
   }
   return success();
 }
 
 static ParseResult parseParallelOp(OpAsmParser &parser,
                                    OperationState &result) {
   auto &builder = parser.getBuilder();
   // Parse an opening `(` followed by induction variables followed by `)`
static ParseResult parseParallelOp(OpAsmParser &parser,
   // Parse step value.
   SmallVector<OpAsmParser::OperandType, 4> steps;
   if (parser.parseKeyword("step") ||
       parser.parseOperandList(steps, ivs.size(),
                               OpAsmParser::Delimiter::Paren) ||
       parser.resolveOperands(steps, builder.getIndexType(), result.operands))
     return failure();
 
+  // Parse step value.
+  SmallVector<OpAsmParser::OperandType, 4> initVals;
+  if (succeeded(parser.parseOptionalKeyword("init"))) {
+    if (parser.parseOperandList(initVals, -1, OpAsmParser::Delimiter::Paren))
+      return failure();
+  }
+
   // Now parse the body.
   Region *body = result.addRegion();
   SmallVector<Type, 4> types(ivs.size(), builder.getIndexType());
   if (parser.parseRegion(*body, ivs, types))
     return failure();
 
+  // Set `operand_segment_sizes` attribute.
+  int32_t numLoops = ivs.size();
+  result.addAttribute(
+      ParallelOp::getOperandSegmentSizeAttr(),
+      builder.getI32VectorAttr({numLoops, numLoops, numLoops,
+                                static_cast<int32_t>(initVals.size())}));
+
   // Parse attributes and optional results (in case there is a reduce).
   if (parser.parseOptionalAttrDict(result.attributes) ||
       parser.parseOptionalColonTypeList(result.types))
     return failure();
 
+  if (!initVals.empty())
+    parser.resolveOperands(initVals, result.types, parser.getNameLoc(),
+                           result.operands);
   // Add a terminator if none was parsed.
   ForOp::ensureTerminator(*body, builder, result.location);
 
   return success();
 }
 
 static void print(OpAsmPrinter &p, ParallelOp op) {
   p << op.getOperationName() << " (" << op.getBody()->getArguments() << ") = ("
     << op.lowerBound() << ") to (" << op.upperBound() << ") step (" << op.step()
     << ")";
+  if (!op.initVals().empty())
+    p << " init (" << op.initVals() << ")";
   p.printRegion(op.region(), /*printEntryBlockArgs=*/false);
-  p.printOptionalAttrDict(op.getAttrs());
+  p.printOptionalAttrDict(
+      op.getAttrs(), /*elidedAttrs=*/ParallelOp::getOperandSegmentSizeAttr());
   if (!op.results().empty())
     p << " : " << op.getResultTypes();
 }
 
 ParallelOp mlir::loop::getParallelForInductionVarOwner(Value val) {
   auto ivArg = val.dyn_cast<BlockArgument>();
   if (!ivArg)
     return ParallelOp();

mlir/test/Dialect/Loops/invalid.mlir

 // -----
 
 func @parallel_body_arguments_wrong_type(
     %arg0: index, %arg1: index, %arg2: index) {
   // expected-error@+1 {{'loop.parallel' op expects arguments for the induction variable to be of index type}}
   "loop.parallel"(%arg0, %arg1, %arg2) ({
     ^bb0(%i0: f32):
       loop.yield
-  }): (index, index, index) -> ()
+  }) {operand_segment_sizes = dense<[1, 1, 1, 0]>: vector<4xi32>}: (index, index, index) -> ()
   return
 }
 
 // -----
 
 func @parallel_body_wrong_number_of_arguments(
     %arg0: index, %arg1: index, %arg2: index) {
-  // expected-error@+1 {{'loop.parallel' op expects the same number of induction variables as bound and step values}}
+  // expected-error@+1 {{'loop.parallel' op expects the same number of induction variables: 2 as bound and step values: 1}}
   "loop.parallel"(%arg0, %arg1, %arg2) ({
     ^bb0(%i0: index, %i1: index):
       loop.yield
-  }): (index, index, index) -> ()
+  }) {operand_segment_sizes = dense<[1, 1, 1, 0]>: vector<4xi32>}: (index, index, index) -> ()
   return
 }
 
 // -----
 
 func @parallel_no_tuple_elements() {
   // expected-error@+1 {{'loop.parallel' op needs at least one tuple element for lowerBound, upperBound and step}}
   loop.parallel () = () to () step () {
func @parallel_step_not_positive(
   }
   return
 }
 
 // -----
 
 func @parallel_fewer_results_than_reduces(
     %arg0 : index, %arg1: index, %arg2: index) {
-  // expected-error@+1 {{expects number of results to be the same as number of reductions}}
+  // expected-error@+1 {{expects number of results: 0 to be the same as number of reductions: 1}}
   loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) {
     %c0 = constant 1.0 : f32
     loop.reduce(%c0) {
       ^bb0(%lhs: f32, %rhs: f32):
         loop.reduce.return %lhs : f32
     } : f32
   }
   return
 }
 
 // -----
 
 func @parallel_more_results_than_reduces(
     %arg0 : index, %arg1 : index, %arg2 : index) {
-  // expected-error@+1 {{expects number of results to be the same as number of reductions}}
-  %res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) {
+  // expected-error@+2 {{expects number of results: 1 to be the same as number of reductions: 0}}
+  %zero = constant 1.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) init (%zero) {
   } : f32
 
   return
 }
 
 // -----
 
-func @parallel_different_types_of_results_and_reduces(
+func @parallel_more_results_than_initial_values(
     %arg0 : index, %arg1: index, %arg2: index) {
+  // expected-error@+1 {{expects number of results: 1 to be the same as number of initial values: 0}}
   %res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) {
-    // expected-error@+1 {{expects type of reduce to be the same as result type: 'f32'}}
+    loop.reduce(%arg0) {
+      ^bb0(%lhs: index, %rhs: index):
+        loop.reduce.return %lhs : index
+    } : index
+  } : f32
+  return
+}
+
+// -----
+
+func @parallel_different_types_of_results_and_reduces(
+    %arg0 : index, %arg1: index, %arg2: index) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) init (%zero) {
+    // expected-error@+1 {{expects type of reduce: 'index' to be the same as result type: 'f32'}}
     loop.reduce(%arg0) {
       ^bb0(%lhs: index, %rhs: index):
         loop.reduce.return %lhs : index
     } : index
   } : f32
   return
 }
 
 // -----
 
 func @top_level_reduce(%arg0 : f32) {
   // expected-error@+1 {{expects parent op 'loop.parallel'}}
   loop.reduce(%arg0) {
     ^bb0(%lhs : f32, %rhs : f32):
       loop.reduce.return %lhs : f32
   } : f32
   return
 }
 
 // -----
 
 func @reduce_empty_block(%arg0 : index, %arg1 : f32) {
-  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) init (%zero) {
     // expected-error@+1 {{the block inside reduce should not be empty}}
     loop.reduce(%arg1) {
       ^bb0(%lhs : f32, %rhs : f32):
     } : f32
   } : f32
   return
 }
 
 // -----
 
 func @reduce_too_many_args(%arg0 : index, %arg1 : f32) {
-  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) init (%zero) {
     // expected-error@+1 {{expects two arguments to reduce block of type 'f32'}}
     loop.reduce(%arg1) {
       ^bb0(%lhs : f32, %rhs : f32, %other : f32):
         loop.reduce.return %lhs : f32
     } : f32
   } : f32
   return
 }
 
 // -----
 
 func @reduce_wrong_args(%arg0 : index, %arg1 : f32) {
-  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) init (%zero) {
     // expected-error@+1 {{expects two arguments to reduce block of type 'f32'}}
     loop.reduce(%arg1) {
       ^bb0(%lhs : f32, %rhs : i32):
         loop.reduce.return %lhs : f32
     } : f32
   } : f32
   return
 }
 
 
 // -----
 
 func @reduce_wrong_terminator(%arg0 : index, %arg1 : f32) {
-  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) init (%zero) {
     // expected-error@+1 {{the block inside reduce should be terminated with a 'loop.reduce.return' op}}
     loop.reduce(%arg1) {
       ^bb0(%lhs : f32, %rhs : f32):
         loop.yield
     } : f32
   } : f32
   return
 }
 
 // -----
 
 func @reduceReturn_wrong_type(%arg0 : index, %arg1: f32) {
-  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) {
+  %zero = constant 0.0 : f32
+  %res = loop.parallel (%i0) = (%arg0) to (%arg0) step (%arg0) init (%zero) {
     loop.reduce(%arg1) {
       ^bb0(%lhs : f32, %rhs : f32):
         %c0 = constant 1 : index
         // expected-error@+1 {{needs to have type 'f32' (the type of the enclosing ReduceOp)}}
         loop.reduce.return %c0 : index
     } : f32
   } : f32
   return

 func @parallel_invalid_yield(
     %arg0: index, %arg1: index, %arg2: index) {
   loop.parallel (%i0) = (%arg0) to (%arg1) step (%arg2) {
     %c0 = constant 1.0 : f32
     // expected-error@+1 {{yield inside loop.parallel is not allowed to have operands}}
     loop.yield %c0 : f32
   }
   return
-}
+}
 

mlir/test/Dialect/Loops/ops.mlir

-// RUN: mlir-opt %s | FileCheck %s
+// RUN: mlir-opt %s | FileCheck %s --dump-input-on-failure
 // Verify the printed output can be parsed.
-// RUN: mlir-opt %s | mlir-opt | FileCheck %s
+// RUN: mlir-opt %s | mlir-opt | FileCheck %s --dump-input-on-failure
 // Verify the generic form can be parsed.
-// RUN: mlir-opt -mlir-print-op-generic %s | mlir-opt | FileCheck %s
+// RUN: mlir-opt -mlir-print-op-generic %s | mlir-opt | FileCheck %s --dump-input-on-failure
 
 func @std_for(%arg0 : index, %arg1 : index, %arg2 : index) {
   loop.for %i0 = %arg0 to %arg1 step %arg2 {
     loop.for %i1 = %arg0 to %arg1 step %arg2 {
       %min_cmp = cmpi "slt", %i0, %i1 : index
       %min = select %min_cmp, %i0, %i1 : index
       %max_cmp = cmpi "sge", %i0, %i1 : index
       %max = select %max_cmp, %i0, %i1 : index
func @std_parallel_loop(%arg0 : index, %arg1 : index, %arg2 : index,
                         %arg3 : index, %arg4 : index) {
   %step = constant 1 : index
   loop.parallel (%i0, %i1) = (%arg0, %arg1) to (%arg2, %arg3)
                                           step (%arg4, %step) {
     %min_cmp = cmpi "slt", %i0, %i1 : index
     %min = select %min_cmp, %i0, %i1 : index
     %max_cmp = cmpi "sge", %i0, %i1 : index
     %max = select %max_cmp, %i0, %i1 : index
-    %red = loop.parallel (%i2) = (%min) to (%max) step (%i1) {
-      %zero = constant 0.0 : f32
-      loop.reduce(%zero) {
+    %red = loop.parallel (%i2) = (%min) to (%max) step (%i1) init (%zero) {
+      %one = constant 1.0 : f32
+      loop.reduce(%one) {
         ^bb0(%lhs : f32, %rhs: f32):
           %res = addf %lhs, %rhs : f32
           loop.reduce.return %res : f32
       } : f32
     } : f32
   }
   return
 }
 // CHECK-LABEL: func @std_parallel_loop(
 //  CHECK-SAME: %[[ARG0:[A-Za-z0-9]+]]:
 //  CHECK-SAME: %[[ARG1:[A-Za-z0-9]+]]:
 //  CHECK-SAME: %[[ARG2:[A-Za-z0-9]+]]:
 //  CHECK-SAME: %[[ARG3:[A-Za-z0-9]+]]:
 //  CHECK-SAME: %[[ARG4:[A-Za-z0-9]+]]:
 //       CHECK:   %[[STEP:.*]] = constant 1 : index
 //  CHECK-NEXT:   loop.parallel (%[[I0:.*]], %[[I1:.*]]) = (%[[ARG0]], %[[ARG1]]) to
 //       CHECK:   (%[[ARG2]], %[[ARG3]]) step (%[[ARG4]], %[[STEP]]) {
 //  CHECK-NEXT:     %[[MIN_CMP:.*]] = cmpi "slt", %[[I0]], %[[I1]] : index
 //  CHECK-NEXT:     %[[MIN:.*]] = select %[[MIN_CMP]], %[[I0]], %[[I1]] : index
 //  CHECK-NEXT:     %[[MAX_CMP:.*]] = cmpi "sge", %[[I0]], %[[I1]] : index
 //  CHECK-NEXT:     %[[MAX:.*]] = select %[[MAX_CMP]], %[[I0]], %[[I1]] : index
-//  CHECK-NEXT:     loop.parallel (%{{.*}}) = (%[[MIN]]) to (%[[MAX]]) step (%[[I1]]) {
-//  CHECK-NEXT:       %[[ZERO:.*]] = constant 0.000000e+00 : f32
-//  CHECK-NEXT:       loop.reduce(%[[ZERO]]) {
+//  CHECK-NEXT:     %[[ZERO:.*]] = constant 0.000000e+00 : f32
+//  CHECK-NEXT:     loop.parallel (%{{.*}}) = (%[[MIN]]) to (%[[MAX]])
+//  CHECK-SAME:           step (%[[I1]]) init (%[[ZERO]]) {
+//  CHECK-NEXT:       %[[ONE:.*]] = constant 1.000000e+00 : f32
+//  CHECK-NEXT:       loop.reduce(%[[ONE]]) {
 //  CHECK-NEXT:       ^bb0(%[[LHS:.*]]: f32, %[[RHS:.*]]: f32):
 //  CHECK-NEXT:         %[[RES:.*]] = addf %[[LHS]], %[[RHS]] : f32
 //  CHECK-NEXT:         loop.reduce.return %[[RES]] : f32
 //  CHECK-NEXT:       } : f32
 //  CHECK-NEXT:       loop.yield
 //  CHECK-NEXT:     } : f32
 //  CHECK-NEXT:     loop.yield