[mlir][vector] Lowering of transfers with out-of-bounds non-transfer dims

[springerm]

Summary

The starting point ("indices") of every transfer dimension may now be out-of-bounds, including non-transfer dimensions. (Negative starting points are still not allowed.)

An additional VectorToSCF pattern is added, which turns an out-of-bounds non-transfer dimension into an in-bounds transfer dimension by adding an scf.if check. (Such out-of-bounds ops are ignored by other patterns.)

Out-of-bounds starting points of transfer dimensions were already supported (by VectorToSCF in the (n>1)-D case and by the mask materialization pattern in the 1-D case). This revision merely adds new test cases for such ops.

Depends On: D155205

[springerm]

update

[dcaballe]

Hey Matthias, could you please help me understand what transfer dimensions and non-transfer dimensions are? I can't parse this sentence combining both (maybe it's because it's a bit late over here :)): The starting point ("indices") of every transfer dimension may now be out-of-bounds, including non-transfer dimensions.

mlir/lib/Conversion/VectorToSCF/VectorToSCF.cpp

178

typo: a -> an

[springerm]

Putting this on hold for the moment. We may not need it.

mlir/include/mlir/Dialect/Vector/IR/VectorOps.td

let description = [{
     into a [vector](../LangRef.md#vector-type) of the same base elemental type.
 
     A memref/tensor operand with vector element type, must have its vector
     element type match a suffix (shape and element type) of the vector (e.g.
     memref<3x2x6x4x3xf32>, vector<1x1x4x3xf32>).
 
     The slice is further defined by a full-rank index within the MemRef/Tensor,
     supplied as the operands `[1 .. 1 + rank(memref/tensor))` that defines the
-    starting point of the transfer (e.g. `%A[%i0, %i1, %i2]`).
+    starting point of the transfer (e.g. `%A[%i0, %i1, %i2]`). All indices must
+    be positive.
 
     The permutation_map [attribute](../LangRef.md#attributes) is an
     [affine-map](Affine.md#affine-maps) which specifies the transposition on the
     slice to match the vector shape. The permutation map may be implicit and
     omitted from parsing and printing if it is the canonical minor identity map
     (i.e. if it does not permute or broadcast any dimension).
 
     The size of the slice is specified by the size of the vector, given as the
     return type.
 
     An SSA value `padding` of the same elemental type as the MemRef/Tensor is
     provided to specify a fallback value in the case of out-of-bounds accesses
     and/or masking.
 
     An optional SSA value `mask` may be specified to mask out elements read from
     the MemRef/Tensor. The `mask` type is an `i1` vector with a shape that
     matches how elements are read from the MemRef/Tensor, *before* any
     permutation or broadcasting. Elements whose corresponding mask element is
     `0` are masked out and replaced with `padding`.
 
     An optional boolean array attribute `in_bounds` specifies for every tensor/
     memref dimension if the transfer is guaranteed to be within the source
-    bounds. While the starting point of the transfer has to be in-bounds,
-    accesses may run out-of-bounds as indices increase. If specified, the
-    `in_bounds` array length has to be equal to the rank of the source. In
-    absence of the attribute, accesses along all dimensions may run
-    out-of-bounds. A `vector.transfer_read` can be lowered to a simple load if
-    all dimensions are specified to be within bounds and no `mask` was
-    specified.
+    bounds. If specified, the `in_bounds` array length has to be equal to the
+    rank of the source. In absence of the attribute, accesses along all
+    dimensions may run out-of-bounds. A `vector.transfer_read` can be lowered to
+    a simple load if all dimensions are specified to be within bounds and no
+    `mask` was specified.
 
     This operation is called 'read' by opposition to 'load' because the
     super-vector granularity is generally not representable with a single
     hardware register. A `vector.transfer_read` is thus a mid-level abstraction
     that supports super-vectorization with non-effecting padding for full-tile
     only operations.
 
     More precisely, let's dive deeper into the permutation_map for the following
let description = [{
 
     A vector memref/tensor operand must have its vector element type match a
     suffix (shape and element type) of the vector (e.g. memref<3x2x6x4x3xf32>,
     vector<1x1x4x3xf32>). If the operand is a tensor, the operation returns a
     new tensor of the same type.
 
     The slice is further defined by a full-rank index within the MemRef/Tensor,
     supplied as the operands `[2 .. 2 + rank(memref/tensor))` that defines the
-    starting point of the transfer (e.g. `%A[%i0, %i1, %i2, %i3]`).
+    starting point of the transfer (e.g. `%A[%i0, %i1, %i2, %i3]`). All indices
+    must be positive.
 
     The permutation_map [attribute](../LangRef.md#attributes) is an
     [affine-map](Affine.md#affine-maps) which specifies the transposition on the
     slice to match the vector shape. The permutation map may be implicit and
     omitted from parsing and printing if it is the canonical minor identity map
     (i.e. if it does not permute any dimension). In contrast to `transfer_read`,
     write ops cannot have broadcast dimensions.
 
     The size of the slice is specified by the size of the vector.
 
     An optional SSA value `mask` may be specified to mask out elements written
     to the MemRef/Tensor. The `mask` type is an `i1` vector with a shape that
     matches how elements are written into the MemRef/Tensor, *after* applying
     any permutation. Elements whose corresponding mask element is `0` are
     masked out.
 
     An optional SSA value `mask` of the same shape as the vector type may be
     specified to mask out elements. Elements whose corresponding mask element
     is `0` are masked out.
 
     An optional boolean array attribute `in_bounds` specifies for every tensor/
     memref dimension if the transfer is guaranteed to be within the source
-    bounds. While the starting point of the transfer has to be in-bounds,
-    accesses may run out-of-bounds as indices increase. If specified, the
-    `in_bounds` array length has to be equal to the rank of the source. In
-    absence of the attribute, accesses along all dimensions may run
-    out-of-bounds. A `vector.transfer_write` can be lowered to a simple store
-    if all dimensions are specified to be within bounds and no `mask` was
-    specified.
+    bounds. If specified, the `in_bounds` array length has to be equal to the
+    rank of the source. In absence of the attribute, accesses along all
+    dimensions may run out-of-bounds. A `vector.transfer_write` can be lowered
+    to a simple store if all dimensions are specified to be within bounds and no
+    `mask` was specified.
 
     This operation is called 'write' by opposition to 'store' because the
     super-vector granularity is generally not representable with a single
     hardware register. A `vector.transfer_write` is thus a
     mid-level abstraction that supports super-vectorization with non-effecting
     padding for full-tile-only code. It is the responsibility of
     `vector.transfer_write`'s implementation to ensure the memory writes are
     valid. Different lowerings may be pertinent depending on the hardware

mlir/lib/Conversion/VectorToSCF/VectorToSCF.cpp

 /// Attribute name used for labeling transfer ops during progressive lowering.
 static const char kPassLabel[] = "__vector_to_scf_lowering__";
 
 /// Patterns that inherit from this struct have access to
 /// VectorTransferToSCFOptions.
 template <typename OpTy>
 struct VectorToSCFPattern : public OpRewritePattern<OpTy> {
   explicit VectorToSCFPattern(MLIRContext *context,
-                              VectorTransferToSCFOptions opt)
-      : OpRewritePattern<OpTy>(context), options(opt) {}
+                              VectorTransferToSCFOptions opt,
+                              PatternBenefit benefit = 1)
+      : OpRewritePattern<OpTy>(context, benefit), options(opt) {}
 
   VectorTransferToSCFOptions options;
 };
 
 /// Given a vector transfer op, calculate which dimension of the `source`
 /// memref should be unpacked in the next application of TransferOpConversion.
 /// A return value of std::nullopt indicates a broadcast.
 template <typename OpTy>

 /// ```
 ///
 /// If the transfer is 1D and has a mask, this function generates a more complex
 /// check also accounts for potentially masked out elements.
 ///
 /// This function variant returns the value returned by `inBoundsCase` or
 /// `outOfBoundsCase`. The MLIR type of the return value must be specified in
 /// `resultTypes`.
+///
+/// Note: `iv` is optional. If it is not provided, a "if (%a < %d)" check will
+/// be generated.
 template <typename OpTy>
 static Value generateInBoundsCheck(
     OpBuilder &b, OpTy xferOp, Value iv, std::optional<int64_t> dim,
     TypeRange resultTypes,
     function_ref<Value(OpBuilder &, Location)> inBoundsCase,
     function_ref<Value(OpBuilder &, Location)> outOfBoundsCase = nullptr) {
   bool hasRetVal = !resultTypes.empty();
   Value cond; // Condition to be built...
 
   // Condition check 1: Access in-bounds?
   bool isBroadcast = !dim; // No in-bounds check for broadcasts.
   Location loc = xferOp.getLoc();
   ImplicitLocOpBuilder lb(xferOp.getLoc(), b);
   if (!isBroadcast && !xferOp.isDimInBounds(*dim)) {
     Value memrefDim =
         vector::createOrFoldDimOp(b, loc, xferOp.getSource(), *dim);
+    Value memrefIdx;
+    if (iv) {
       AffineExpr d0, d1;
       bindDims(xferOp.getContext(), d0, d1);
       Value base = xferOp.getIndices()[*dim];
-    Value memrefIdx =
-        affine::makeComposedAffineApply(b, loc, d0 + d1, {base, iv});
+      memrefIdx = affine::makeComposedAffineApply(b, loc, d0 + d1, {base, iv});
+    } else {
+      memrefIdx = xferOp.getIndices()[*dim];
+    }
     cond = lb.create<arith::CmpIOp>(arith::CmpIPredicate::sgt, memrefDim,
                                     memrefIdx);
   }
 
-  // Condition check 2: Masked in?
+  // Condition check 2: Masked in? If no `iv` is provided, this is a check for
+  // a non-transfer dimensions (which does not have a mask).
+  if (iv) {
     if (auto maskCond = generateMaskCheck(b, xferOp, iv)) {
       if (cond)
         cond = lb.create<arith::AndIOp>(cond, maskCond);
       else
         cond = maskCond;
     }
+  } else {
+#ifndef NDEBUG
+    if (dim.has_value()) {
+      for (AffineExpr expr : xferOp.getPermutationMap().getResults()) {
+        if (auto dimExpr = expr.dyn_cast<AffineDimExpr>()) {
+          assert(dimExpr.getPosition() != *dim && "expected non-transfer dim");
+        }
+      }
+    }
+#endif // NDEBUG
+  }
 
   // If the condition is non-empty, generate an SCF::IfOp.
   if (cond) {
     auto check = lb.create<scf::IfOp>(
         cond,
         /*thenBuilder=*/
         [&](OpBuilder &b, Location loc) {
           maybeYieldValue(b, loc, hasRetVal, inBoundsCase(b, loc));
rewriter.replaceOpWithNewOp<scf::ForOp>(
           Strategy1d<OpTy>::generateForLoopBody(b, loc, xferOp, iv, loopState);
         });
 
     return success();
   }
 };
 
 } // namespace lowering_1_d
+
+/// This pattern turn an out-of-bounds non-transfer dimension into an in-bound
+/// dimension by inserting the corresponding in-bounds check as an scf.if op.
+///
+/// Example:
+/// ```
+/// %vec = vector.transfer_read %A[%a, %b] %cst {in_bounds = [false, false]}
+///     : vector<4xf32>, memref<?x?xf32>
+/// ```
+/// Generated IR:
+/// ```
+/// %d = memref.dim %A, %c0 : memref<?x?xf32>
+/// if (%a < %d) {
+///   %vec = vector.transfer_read %A[%a, %b] %cst {in_bounds = [true, false]}
+///       : vector<4xf32>, memref<?x?xf32>
+///   scf.yield %vec
+/// } else {
+///   %vec = vector.splat %cst
+///   scf.yield %vec
+/// }
+/// ```
+///
+/// This pattern ignores all transfer dims such as the %b in the example above.
+/// Such dims are handles by other patterns (e.g., as part of unrolling or
+/// materialization of masks).
+template <typename OpTy>
+struct TransferOpOutOfBoundsNonTransferDims : public VectorToSCFPattern<OpTy> {
+  using VectorToSCFPattern<OpTy>::VectorToSCFPattern;
+
+  /// Out-of-bounds read: return padding value.
+  static Value handleOutOfBoundsDim(OpBuilder &b, TransferReadOp xferOp) {
+    return b.create<vector::SplatOp>(xferOp.getLoc(), xferOp.getVectorType(),
+                                     xferOp.getPadding());
+  }
+
+  /// Out-of-bounds write: nothing to do.
+  static Value handleOutOfBoundsDim(OpBuilder &b, TransferWriteOp xferOp) {
+    if (xferOp.getNumResults() == 0)
+      return Value();
+    assert(xferOp.getNumResults() == 1 && "expected 0 or 1 result");
+    return xferOp.getSource();
+  }
+
+  LogicalResult matchAndRewrite(OpTy xferOp,
+                                PatternRewriter &rewriter) const override {
+    // Find out-of-bounds non-transfer dim.
+    int64_t dim = -1;
+    DenseSet<int64_t> xferDims;
+    for (AffineExpr expr : xferOp.getPermutationMap().getResults()) {
+      if (auto dimExpr = expr.template dyn_cast<AffineDimExpr>())
+        xferDims.insert(dimExpr.getPosition());
+    }
+    for (int64_t i = 0, e = xferOp.getShapedType().getRank(); i < e; ++i) {
+      if (!xferDims.contains(i) && !xferOp.isDimInBounds(i)) {
+        dim = i;
+        break;
+      }
+    }
+
+    // Nothing to do if all non-transfer dims are in-bounds.
+    if (dim == -1)
+      return failure();
+
+    Value result = generateInBoundsCheck(
+        rewriter, xferOp, /*iv=*/Value(), dim, xferOp->getResultTypes(),
+        /*inBoundsCase=*/
+        [&](OpBuilder &b, Location loc) -> Value {
+          // Clone xfer op and set in_bounds to "true".
+          OpTy cloned = cast<OpTy>(b.clone(*xferOp.getOperation()));
+          SmallVector<bool> inBounds(xferOp.getShapedType().getRank(), false);
+          if (xferOp.getInBounds().has_value())
+            inBounds =
+                extractFromIntegerArrayAttr<bool>(xferOp.getInBoundsAttr());
+          inBounds[dim] = true;
+          cloned.setInBoundsAttr(b.getBoolArrayAttr(inBounds));
+          if (cloned->getNumResults() == 0)
+            return Value();
+          assert(cloned->getNumResults() == 1 && "expected 0 or 1 result");
+          return cloned->getResult(0);
+        },
+        /*outOfBoundsCase=*/
+        [&](OpBuilder &b, Location loc) -> Value {
+          return handleOutOfBoundsDim(b, xferOp);
+        });
+
+    if (xferOp->getNumResults() == 0) {
+      assert(!result && "expected that check has no result");
+      rewriter.eraseOp(xferOp);
+    } else {
+      rewriter.replaceOp(xferOp, result);
+    }
+    return success();
+  }
+};
 } // namespace
 
 void mlir::populateVectorToSCFConversionPatterns(
     RewritePatternSet &patterns, const VectorTransferToSCFOptions &options) {
   if (options.unroll) {
     patterns.add<lowering_n_d_unrolled::UnrollTransferReadConversion,
                  lowering_n_d_unrolled::UnrollTransferWriteConversion>(
         patterns.getContext(), options);
   } else {
     patterns.add<lowering_n_d::PrepareTransferReadConversion,
                  lowering_n_d::PrepareTransferWriteConversion,
                  lowering_n_d::TransferOpConversion<TransferReadOp>,
                  lowering_n_d::TransferOpConversion<TransferWriteOp>>(
         patterns.getContext(), options);
   }
 
   if (options.targetRank == 1) {
     patterns.add<lowering_1_d::TransferOp1dConversion<TransferReadOp>,
                  lowering_1_d::TransferOp1dConversion<TransferWriteOp>>(
         patterns.getContext(), options);
   }
+
+  patterns.add<TransferOpOutOfBoundsNonTransferDims<TransferReadOp>,
+               TransferOpOutOfBoundsNonTransferDims<TransferWriteOp>>(
+      patterns.getContext(), options, /*benefit=*/2);
 }
 
 namespace {
 
 struct ConvertVectorToSCFPass
     : public impl::ConvertVectorToSCFBase<ConvertVectorToSCFPass> {
   ConvertVectorToSCFPass() = default;
   ConvertVectorToSCFPass(const VectorTransferToSCFOptions &options) {

mlir/lib/Dialect/Vector/IR/VectorOps.cpp

verifyTransferOp(VectorTransferOpInterface op, ShapedType shapedType,
   if (inBounds) {
     if (shapedType.getRank() != static_cast<int64_t>(inBounds.size()))
       return op->emitOpError("expects the optional in_bounds attr of same rank "
                              "as the source type: ")
              << shapedType.getRank()
              << " vs inBounds of size: " << inBounds.size();
   }
 
-  // Make sure that all non-transfer dimensions are in-bounds.
-  SmallVector<bool> inBoundsVals(op.getShapedType().getRank(), false);
-  if (inBounds)
-    inBoundsVals = extractFromIntegerArrayAttr<bool>(inBounds);
-  DenseSet<int64_t> xferDims;
-  for (AffineExpr expr : permutationMap.getResults()) {
-    if (auto dimExpr = expr.template dyn_cast<AffineDimExpr>())
-      xferDims.insert(dimExpr.getPosition());
-  }
-  for (int64_t i = 0, e = op.getShapedType().getRank(); i < e; ++i)
-    if (!xferDims.contains(i) && !op.isDimInBounds(i)) {
-      return op->emitOpError(
-          "expects that all non-transfer dims are in-bounds");
-    }
-
   return success();
 }
 
 static void printTransferAttrs(OpAsmPrinter &p, VectorTransferOpInterface op) {
   SmallVector<StringRef, 3> elidedAttrs;
   elidedAttrs.push_back(TransferReadOp::getOperandSegmentSizeAttr());
   if (op.getPermutationMap().isMinorIdentity())
     elidedAttrs.push_back(op.getPermutationMapAttrStrName());

mlir/test/Conversion/VectorToSCF/vector-to-scf.mlir

 // CHECK:           scf.for %[[IDX:.*]] = %[[C_0]] to %[[UB]] step %[[STEP]] {
 // CHECK:             %[[MASK_VAL:.*]] = vector.extractelement %[[MASK_VEC]][%[[IDX]] : index] : vector<[16]xi1>
 // CHECK:             scf.if %[[MASK_VAL]] {
 // CHECK:               %[[VAL_TO_STORE:.*]] = vector.extractelement %{{.*}}[%[[IDX]] : index] : vector<[16]xf32>
 // CHECK:               memref.store %[[VAL_TO_STORE]], %[[ARG_0]][%[[IDX]]] : memref<?xf32, strided<[?], offset: ?>>
 // CHECK:             } else {
 // CHECK:             }
 // CHECK:           }
+
+// -----
+
+// CHECK-LABEL: func @read_non_transfer_dim_out_of_bounds(
+//       CHECK:   %[[r5:.*]] = scf.if {{.*}} {
+//       CHECK:     %[[r2:.*]] = scf.if {{.*}} {
+//   CHECK-NOT:       scf.if
+//       CHECK:       %[[r:.*]] = vector.transfer_read {{.*}} {in_bounds = [true, true, true, false]} : tensor<?x?x?x?xf32>, vector<5xf32>
+//       CHECK:       scf.yield %[[r]]
+//       CHECK:     } else {
+///      CHECK:       %[[r3:.*]] = vector.splat {{.*}} : vector<5xf32>
+//       CHECK:       scf.yield %[[r3]]
+//       CHECK:     }
+//       CHECK:     scf.yield %[[r2]]
+//       CHECK:   } else {
+//       CHECK:     %[[r4:.*]] = vector.splat {{.*}} : vector<5xf32>
+//       CHECK:     scf.yield %[[r4]]
+//       CHECK:   }
+//       CHECK:   return %[[r5]]
+func.func @read_non_transfer_dim_out_of_bounds(%t: tensor<?x?x?x?xf32>, %idx: index, %f: f32) -> vector<5xf32> {
+  %0 = vector.transfer_read %t[%idx, %idx, %idx, %idx], %f {in_bounds = [false, true, false, false]} : tensor<?x?x?x?xf32>, vector<5xf32>
+  return %0 : vector<5xf32>
+}
+
+// -----
+
+// CHECK-LABEL: func @write_non_transfer_dim_out_of_bounds(
+//  CHECK-SAME:     %[[t:.*]]: tensor
+//       CHECK:   scf.if {{.*}} {
+//       CHECK:     scf.if {{.*}} {
+//   CHECK-NOT:       scf.if
+//       CHECK:       %[[r:.*]] = vector.transfer_write {{.*}} {in_bounds = [true, true, true, false]} : vector<5xf32>, tensor<?x?x?x?xf32>
+//       CHECK:       scf.yield %[[r]]
+//       CHECK:     } else {
+//       CHECK:       scf.yield %[[t]]
+//       CHECK:     }
+//       CHECK:   } else {
+//       CHECK:     scf.yield %[[t]]
+//       CHECK:   }
+func.func @write_non_transfer_dim_out_of_bounds(%t: tensor<?x?x?x?xf32>, %idx: index, %f: f32, %v: vector<5xf32>) -> tensor<?x?x?x?xf32> {
+  %0 = vector.transfer_write %v, %t[%idx, %idx, %idx, %idx] {in_bounds = [false, true, false, false]} : vector<5xf32>, tensor<?x?x?x?xf32>
+  return %0 : tensor<?x?x?x?xf32>
+}
+
+// -----
+
+// CHECK-LABEL: func @read_memref_non_transfer_dim_out_of_bounds(
+//       CHECK:   scf.if {{.*}} {
+//       CHECK:     scf.if {{.*}} {
+//   CHECK-NOT:       scf.if
+//       CHECK:       vector.transfer_read {{.*}} {in_bounds = [true, true, true, false]} : memref<?x?x?x?xf32>, vector<5xf32>
+//       CHECK:     } else {
+//       CHECK:       vector.splat {{.*}} : vector<5xf32>
+//       CHECK:     }
+//       CHECK:   } else {
+//       CHECK:     vector.splat {{.*}} : vector<5xf32>
+//       CHECK:   }
+func.func @read_memref_non_transfer_dim_out_of_bounds(%t: memref<?x?x?x?xf32>, %idx: index, %f: f32) -> vector<5xf32> {
+  %0 = vector.transfer_read %t[%idx, %idx, %idx, %idx], %f {in_bounds = [false, true, false, false]} : memref<?x?x?x?xf32>, vector<5xf32>
+  return %0 : vector<5xf32>
+}
+
+// -----
+
+// CHECK-LABEL: func @memref_write_non_transfer_dim_out_of_bounds(
+//  CHECK-SAME:     %[[t:.*]]: memref
+//       CHECK:   scf.if {{.*}} {
+//       CHECK:     scf.if {{.*}} {
+//   CHECK-NOT:       scf.if
+//       CHECK:       vector.transfer_write {{.*}} {in_bounds = [true, true, true, false]} : vector<5xf32>, memref<?x?x?x?xf32>
+//       CHECK:     } else {
+//       CHECK:     }
+//       CHECK:   } else {
+//       CHECK:   }
+func.func @memref_write_non_transfer_dim_out_of_bounds(%t: memref<?x?x?x?xf32>, %idx: index, %f: f32, %v: vector<5xf32>) {
+  vector.transfer_write %v, %t[%idx, %idx, %idx, %idx] {in_bounds = [false, true, false, false]} : vector<5xf32>, memref<?x?x?x?xf32>
+  return
+}

mlir/test/Dialect/Vector/invalid.mlir

 func.func @integer_vector_contract(%arg0: vector<16x32xsi8>, %arg1: vector<32x16xsi8>, %arg2: vector<16x16xsi32>) -> vector<16x16xsi32> {
   // expected-error@+1 {{op only supports signless integer types}}
   %0 = vector.contract {
     indexing_maps = [affine_map<(d0, d1, d2) -> (d0, d2)>, affine_map<(d0, d1, d2) -> (d2, d1)>, affine_map<(d0, d1, d2) -> (d0, d1)>],
     iterator_types = ["parallel", "parallel", "reduction"], kind = #vector.kind<add>
   } %arg0, %arg1, %arg2 : vector<16x32xsi8>, vector<32x16xsi8> into vector<16x16xsi32>
   return %0: vector<16x16xsi32>
 }
-
-// -----
-
-func.func @out_of_bounds_non_transfer_dim(%arg0: tensor<?x?xf32>, %pos: index, %f: f32) -> vector<5xf32> {
-  // expected-error @below{{expects that all non-transfer dims are in-bounds}}
-  %0 = vector.transfer_read %arg0[%pos, %pos], %f : tensor<?x?xf32>, vector<5xf32>
-  return %0 : vector<5xf32>
-}

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-1d.mlir

func.func @transfer_read_1d_in_bounds(
   %f = vector.transfer_read %A[%base1, %base2], %fm42
       {permutation_map = affine_map<(d0, d1) -> (d0)>, in_bounds = [true, true]}
       : memref<?x?xf32>, vector<3xf32>
   vector.print %f: vector<3xf32>
   return
 }
 
 // Non-contiguous, strided load.
+func.func @transfer_read_1d_out_of_bounds(
+    %A : memref<?x?xf32>, %base1 : index, %base2 : index) {
+  %fm42 = arith.constant -42.0: f32
+  %f = vector.transfer_read %A[%base1, %base2], %fm42
+      {permutation_map = affine_map<(d0, d1) -> (d0)>, in_bounds = [false, false]}
+      : memref<?x?xf32>, vector<3xf32>
+  vector.print %f: vector<3xf32>
+  return
+}
+
+// Non-contiguous, strided load.
 func.func @transfer_read_1d_mask(
     %A : memref<?x?xf32>, %base1 : index, %base2 : index) {
   %fm42 = arith.constant -42.0: f32
   %mask = arith.constant dense<[1, 0, 1, 0, 1, 1, 1, 0, 1]> : vector<9xi1>
   %f = vector.transfer_read %A[%base1, %base2], %fm42, %mask
       {permutation_map = affine_map<(d0, d1) -> (d0)>, in_bounds=[false, true]}
       : memref<?x?xf32>, vector<9xf32>
   vector.print %f: vector<9xf32>
func.func @transfer_write_1d_mask(%A : memref<?x?xf32>, %base1 : index, %base2 : index) {
   return
 }
 
 func.func @entry() {
   %c0 = arith.constant 0: index
   %c1 = arith.constant 1: index
   %c2 = arith.constant 2: index
   %c3 = arith.constant 3: index
+  %c10 = arith.constant 10 : index
   %0 = memref.get_global @gv : memref<5x6xf32>
   %A = memref.cast %0 : memref<5x6xf32> to memref<?x?xf32>
 
   // 1. Read from 2D memref on first dimension. Cannot be lowered to an LLVM
   //    vector load. Instead, generates scalar loads.
   call @transfer_read_1d(%A, %c1, %c2) : (memref<?x?xf32>, index, index) -> ()
   // CHECK: ( 12, 22, 32, 42, -42, -42, -42, -42, -42 )
 
   // 2.a. Read 1D vector from 2D memref with non-unit stride on first dim.
   call @transfer_read_1d_unit_stride(%A) : (memref<?x?xf32>) -> ()
   // CHECK: ( 10, 11 )
   // CHECK: ( 13, 14 )
   // CHECK: ( 30, 31 )
   // CHECK: ( 33, 34 )
 
   // 2.b. Read 1D vector from 2D memref with non-unit stride on first dim.
   //      Strides are non-static.
   call @transfer_read_1d_non_static_unit_stride(%A) : (memref<?x?xf32>) -> ()
   // CHECK: ( 31, 32, 33, 34 )
 
+  // 2.c. Read 1D vector from 2D memref with out-of-bounds non-transfer dim
+  //      starting point.
+  call @transfer_read_1d_out_of_bounds(%A, %c10, %c1)
+      : (memref<?x?xf32>, index, index) -> ()
+  // CHECK: ( -42, -42, -42 )
+
+  // 2.d. Read 1D vector from 2D memref with out-of-bounds transfer dim starting
+  //      point.
+  call @transfer_read_1d_out_of_bounds(%A, %c1, %c10)
+      : (memref<?x?xf32>, index, index) -> ()
+  // CHECK: ( -42, -42, -42 )
+
   // 3. Read 1D vector from 2D memref with non-unit stride on second dim.
   call @transfer_read_1d_non_unit_stride(%A) : (memref<?x?xf32>) -> ()
   // CHECK: ( 22, 24, -42 )
 
   // 4. Write to 2D memref on first dimension. Cannot be lowered to an LLVM
   //    vector store. Instead, generates scalar stores.
   call @transfer_write_1d(%A, %c3, %c2) : (memref<?x?xf32>, index, index) -> ()
 

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-2d.mlir

func.func @transfer_write_2d_mask(%A : memref<?x?xf32>, %base1: index, %base2: index) {
   return
 }
 
 func.func @entry() {
   %c0 = arith.constant 0: index
   %c1 = arith.constant 1: index
   %c2 = arith.constant 2: index
   %c3 = arith.constant 3: index
+  %c10 = arith.constant 10 : index
   %0 = memref.get_global @gv : memref<3x4xf32>
   %A = memref.cast %0 : memref<3x4xf32> to memref<?x?xf32>
 
-  // 1. Read 2D vector from 2D memref.
+  // 1.a. Read 2D vector from 2D memref.
   call @transfer_read_2d(%A, %c1, %c2) : (memref<?x?xf32>, index, index) -> ()
   // CHECK: ( ( 12, 13, -42, -42, -42, -42, -42, -42, -42 ), ( 22, 23, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ) )
 
+  // 1.b. Read 2D vector from 2D memref. Starting position of first dim is
+  //      out-of-bounds.
+  call @transfer_read_2d(%A, %c3, %c2) : (memref<?x?xf32>, index, index) -> ()
+  // CHECK: ( ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ) )
+
+  // 1.c. Read 2D vector from 2D memref. Starting position of second dim is
+  //      out-of-bounds.
+  call @transfer_read_2d(%A, %c1, %c10) : (memref<?x?xf32>, index, index) -> ()
+  // CHECK: ( ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ) )
+
   // 2. Read 2D vector from 2D memref at specified location and transpose the
   //    result.
   call @transfer_read_2d_transposed(%A, %c1, %c2)
       : (memref<?x?xf32>, index, index) -> ()
   // CHECK: ( ( 12, 22, -42, -42, -42, -42, -42, -42, -42 ), ( 13, 23, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ), ( -42, -42, -42, -42, -42, -42, -42, -42, -42 ) )
 
   // 3. Read 2D vector from 2D memref with a 2D mask. In addition, some
   //    accesses are out-of-bounds.