[mlir][spirv] Define `spirv.*Dot` integer dot product ops

[kuhar]

Summary

This covers SDot, SUDot, and UDot. The *AccSat version will be
added in a follow-up revision.

[kuhar]

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVArithmeticOps.td

580

FYI, this got reordered when I ran the script to generate new instructions. I can undo this if it makes this revision confusing.

[antiagainst]

Awesome! Thanks for working this through!

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVArithmeticOps.td

580

SG, thanks for the explanation!

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVIntegerDotProductOps.td

47

Missing optional packed_vector_format?

57

Missing optional packed_vector_format?

mlir/lib/Dialect/SPIRV/IR/SPIRVEnums.cpp

57 ↗

(On Diff #479793)

While we are here can we add everything that gotten promoted to v1.6 so we won't need to come to this place again? List here https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html#_changes_from_version_1_5

Some of the extensions might not be defined yet in SPIRVBase.td; feel free to add them. (Also might be good to extract changes to this place to a separate prior patch.)

mlir/lib/Dialect/SPIRV/IR/SPIRVOps.cpp

4807

.. for both operands. (Calling them as vector operands is a bit confusing here given operands can be of integer type too..)

4822

Missing validation rules for "When Vector 1 and Vector 2 are scalar integer types, Packed Vector Format must be specified to select how the integers are to be interpreted as vectors."?

mlir/test/Dialect/SPIRV/IR/target-env.mlir

128

These tests are not implied capabilities? They are suitable extensions and capabilities. Can we also add tests for

• Full list of necessary of capabilities, but missing extensions
• Full list of necessary of extensions, but missing some capabilities
• Using v1.6 to imply extensions

[kuhar]

Address comments and rebase

[kuhar]

mlir/lib/Dialect/SPIRV/IR/SPIRVEnums.cpp

57 ↗

(On Diff #479793)

Done

mlir/lib/Dialect/SPIRV/IR/SPIRVOps.cpp

49

FYI, I only sorted these lexicographically and added one more name

4807

There can be 2 or 3 operands, so 'both operands' is not precise.
This is how the spec calls them. Initially, I had them as 'factors', but decided to follow the spec language instead, which calls the first two 'vectors' and the last one 'accumulator'. WDYT about changing this to 'factors' or keeping as 'vectors'?

[kuhar]

Update examples

[kuhar]

Fix a typo

[antiagainst]

mlir/lib/Dialect/SPIRV/IR/SPIRVOps.cpp

4807

ah okay. fine to keep as-is to be consistent with the spec then.

mlir/test/Dialect/SPIRV/IR/integer-dot-product-ops.mlir

14

I think we can improve the assembly to print something like spirv.SDot <PackedVectorFormat4x8Bit> %a %b to make this nicer by let assemblyFormat = "$format $vector1 , $vector2 : type(operands) -> type(results) or something. Fine to me to land this revision; but please feel free to play with it a bit.

[kuhar]

mlir/test/Dialect/SPIRV/IR/integer-dot-product-ops.mlir

14

Ack. I'll commit this as-is so that we don't keep the size of this patch under control.

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVArithmeticOps.td

class SPIRV_ArithmeticExtendedBinaryOp<string mnemonic,
   let builders = [
     OpBuilder<(ins "Value":$operand1, "Value":$operand2), [{
       build($_builder, $_state,
             ::mlir::spirv::StructType::get({operand1.getType(), operand1.getType()}),
             operand1, operand2);
     }]>
   ];
 
-  // These op require a custom verifier.
+  // These ops require a custom verifier.
   let hasVerifier = 1;
 }
 
 // -----
 
 def SPIRV_FAddOp : SPIRV_ArithmeticBinaryOp<"FAdd", SPIRV_Float, [Commutative]> {
   let summary = "Floating-point addition of Operand 1 and Operand 2.";
 
let description = [{
     ```
   }];
 
   let hasFolder = 1;
 }
 
 // -----
 
-def SPIRV_SMulExtendedOp : SPIRV_ArithmeticExtendedBinaryOp<"SMulExtended",
-                                                            [Pure, Commutative]> {
-  let summary = [{
-    Result is the full value of the signed integer multiplication of Operand
-    1 and Operand 2.
-  }];
-
-  let description = [{
-    Result Type must be from OpTypeStruct.  The struct must have two
-    members, and the two members must be the same type.  The member type
-    must be a scalar or vector of integer type.
-
-    Operand 1 and Operand 2 must have the same type as the members of Result
-    Type. These are consumed as signed integers.
-
-    Results are computed per component.
-
-    Member 0 of the result gets the low-order bits of the multiplication.
-
-    Member 1 of the result gets the high-order bits of the multiplication.
-
-    <!-- End of AutoGen section -->
-
-    #### Example:
-
-    ```mlir
-    %2 = spirv.SMulExtended %0, %1 : !spirv.struct<(i32, i32)>
-    %2 = spirv.SMulExtended %0, %1 : !spirv.struct<(vector<2xi32>, vector<2xi32>)>
-    ```
-  }];
-}
-
-// -----
-
-def SPIRV_UMulExtendedOp : SPIRV_ArithmeticExtendedBinaryOp<"UMulExtended",
-                                                            [Pure, Commutative]> {
-  let summary = [{
-    Result is the full value of the unsigned integer multiplication of
-    Operand 1 and Operand 2.
-  }];
-
-  let description = [{
-    Result Type must be from OpTypeStruct.  The struct must have two
-    members, and the two members must be the same type.  The member type
-    must be a scalar or vector of integer type, whose Signedness operand is
-    0.
-
-    Operand 1 and Operand 2 must have the same type as the members of Result
-    Type. These are consumed as unsigned integers.
-
-    Results are computed per component.
-
-    Member 0 of the result gets the low-order bits of the multiplication.
-
-    Member 1 of the result gets the high-order bits of the multiplication.
-
-    <!-- End of AutoGen section -->
-
-    #### Example:
-
-    ```mlir
-    %2 = spirv.UMulExtended %0, %1 : !spirv.struct<(i32, i32)>
-    %2 = spirv.UMulExtended %0, %1 : !spirv.struct<(vector<2xi32>, vector<2xi32>)>
-    ```
-  }];
-}
-
-// -----
-
 def SPIRV_ISubOp : SPIRV_ArithmeticBinaryOp<"ISub",
                                         SPIRV_Integer,
                                         [UsableInSpecConstantOp]> {
   let summary = "Integer subtraction of Operand 2 from Operand 1.";
 
   let description = [{
     Result Type must be a scalar or vector of integer type.
 
let description = [{
     %5 = spirv.SMod %2, %3 : vector<4xi32>
 
     ```
   }];
 }
 
 // -----
 
+def SPIRV_SMulExtendedOp : SPIRV_ArithmeticExtendedBinaryOp<"SMulExtended",
+                                                            [Pure, Commutative]> {
+  let summary = [{
+    Result is the full value of the signed integer multiplication of Operand
+    1 and Operand 2.
+  }];
+
+  let description = [{
+    Result Type must be from OpTypeStruct.  The struct must have two
+    members, and the two members must be the same type.  The member type
+    must be a scalar or vector of integer type.
+
+    Operand 1 and Operand 2 must have the same type as the members of Result
+    Type. These are consumed as signed integers.
+
+    Results are computed per component.
+
+    Member 0 of the result gets the low-order bits of the multiplication.
+
+    Member 1 of the result gets the high-order bits of the multiplication.
+
+    <!-- End of AutoGen section -->
+
+    #### Example:
+
+    ```mlir
+    %2 = spirv.SMulExtended %0, %1 : !spirv.struct<(i32, i32)>
+    %2 = spirv.SMulExtended %0, %1 : !spirv.struct<(vector<2xi32>, vector<2xi32>)>
+    ```
+  }];
+}
+
+// -----
+
 def SPIRV_SNegateOp : SPIRV_ArithmeticUnaryOp<"SNegate",
                                           SPIRV_Integer,
                                           [UsableInSpecConstantOp]> {
   let summary = "Signed-integer subtract of Operand from zero.";
 
   let description = [{
     Result Type must be a scalar or vector of integer type.
 
-    Operand’s type  must be a scalar or vector of integer type.  It must
+    Operand's type  must be a scalar or vector of integer type.  It must
     have the same number of components as Result Type.  The component width
     must equal the component width in Result Type.
 
      Results are computed per component.
 
     <!-- End of AutoGen section -->
 
     #### Example:
let description = [{
     %5 = spirv.UDiv %2, %3 : vector<4xi32>
 
     ```
   }];
 }
 
 // -----
 
+def SPIRV_UMulExtendedOp : SPIRV_ArithmeticExtendedBinaryOp<"UMulExtended",
+                                                            [Pure, Commutative]> {
+  let summary = [{
+    Result is the full value of the unsigned integer multiplication of
+    Operand 1 and Operand 2.
+  }];
+
+  let description = [{
+    Result Type must be from OpTypeStruct.  The struct must have two
+    members, and the two members must be the same type.  The member type
+    must be a scalar or vector of integer type, whose Signedness operand is
+    0.
+
+    Operand 1 and Operand 2 must have the same type as the members of Result
+    Type. These are consumed as unsigned integers.
+
+    Results are computed per component.
+
+    Member 0 of the result gets the low-order bits of the multiplication.
+
+    Member 1 of the result gets the high-order bits of the multiplication.
+
+    <!-- End of AutoGen section -->
+
+    #### Example:
+
+    ```mlir
+    %2 = spirv.UMulExtended %0, %1 : !spirv.struct<(i32, i32)>
+    %2 = spirv.UMulExtended %0, %1 : !spirv.struct<(vector<2xi32>, vector<2xi32>)>
+    ```
+  }];
+}
+
+// -----
+
 def SPIRV_VectorTimesScalarOp : SPIRV_Op<"VectorTimesScalar", [Pure]> {
   let summary = "Scale a floating-point vector.";
 
   let description = [{
     Result Type must be a vector of floating-point type.
 
      The type of Vector must be the same as Result Type. Each component of
     Vector is multiplied by Scalar.

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVBase.td

SPIRV_I32EnumAttr<"StorageClass", "valid SPIR-V StorageClass", "storage_class", [
       SPIRV_SC_Workgroup, SPIRV_SC_CrossWorkgroup, SPIRV_SC_Private, SPIRV_SC_Function,
       SPIRV_SC_Generic, SPIRV_SC_PushConstant, SPIRV_SC_AtomicCounter, SPIRV_SC_Image,
       SPIRV_SC_StorageBuffer, SPIRV_SC_CallableDataKHR, SPIRV_SC_IncomingCallableDataKHR,
       SPIRV_SC_RayPayloadKHR, SPIRV_SC_HitAttributeKHR, SPIRV_SC_IncomingRayPayloadKHR,
       SPIRV_SC_ShaderRecordBufferKHR, SPIRV_SC_PhysicalStorageBuffer,
       SPIRV_SC_CodeSectionINTEL, SPIRV_SC_DeviceOnlyINTEL, SPIRV_SC_HostOnlyINTEL
     ]>;
 
+def SPIRV_PVF_PackedVectorFormat4x8Bit : I32EnumAttrCase<"PackedVectorFormat4x8Bit", 0> {
+  list<Availability> availability = [
+    MinVersion<SPIRV_V_1_6>,
+    Extension<[SPV_KHR_integer_dot_product]>
+  ];
+}
+
+def SPIRV_PackedVectorFormatAttr :
+    SPIRV_I32EnumAttr<"PackedVectorFormat", "valid SPIR-V PackedVectorFormat", "packed_vector_format", [
+      SPIRV_PVF_PackedVectorFormat4x8Bit
+    ]>;
+
 // End enum section. Generated from SPIR-V spec; DO NOT MODIFY!
 
 // Enums added manually that are not part of SPIR-V spec
 
 def SPIRV_IDI_NoDepth      : I32EnumAttrCase<"NoDepth", 0>;
 def SPIRV_IDI_IsDepth      : I32EnumAttrCase<"IsDepth", 1>;
 def SPIRV_IDI_DepthUnknown : I32EnumAttrCase<"DepthUnknown", 2>;
 

 def SPIRV_OC_OpGroupNonUniformFMul        : I32EnumAttrCase<"OpGroupNonUniformFMul", 352>;
 def SPIRV_OC_OpGroupNonUniformSMin        : I32EnumAttrCase<"OpGroupNonUniformSMin", 353>;
 def SPIRV_OC_OpGroupNonUniformUMin        : I32EnumAttrCase<"OpGroupNonUniformUMin", 354>;
 def SPIRV_OC_OpGroupNonUniformFMin        : I32EnumAttrCase<"OpGroupNonUniformFMin", 355>;
 def SPIRV_OC_OpGroupNonUniformSMax        : I32EnumAttrCase<"OpGroupNonUniformSMax", 356>;
 def SPIRV_OC_OpGroupNonUniformUMax        : I32EnumAttrCase<"OpGroupNonUniformUMax", 357>;
 def SPIRV_OC_OpGroupNonUniformFMax        : I32EnumAttrCase<"OpGroupNonUniformFMax", 358>;
 def SPIRV_OC_OpSubgroupBallotKHR          : I32EnumAttrCase<"OpSubgroupBallotKHR", 4421>;
+def SPIRV_OC_OpSDot                       : I32EnumAttrCase<"OpSDot", 4450>;
+def SPIRV_OC_OpUDot                       : I32EnumAttrCase<"OpUDot", 4451>;
+def SPIRV_OC_OpSUDot                      : I32EnumAttrCase<"OpSUDot", 4452>;
+def SPIRV_OC_OpSDotAccSat                 : I32EnumAttrCase<"OpSDotAccSat", 4453>;
+def SPIRV_OC_OpUDotAccSat                 : I32EnumAttrCase<"OpUDotAccSat", 4454>;
+def SPIRV_OC_OpSUDotAccSat                : I32EnumAttrCase<"OpSUDotAccSat", 4455>;
 def SPIRV_OC_OpTypeCooperativeMatrixNV    : I32EnumAttrCase<"OpTypeCooperativeMatrixNV", 5358>;
 def SPIRV_OC_OpCooperativeMatrixLoadNV    : I32EnumAttrCase<"OpCooperativeMatrixLoadNV", 5359>;
 def SPIRV_OC_OpCooperativeMatrixStoreNV   : I32EnumAttrCase<"OpCooperativeMatrixStoreNV", 5360>;
 def SPIRV_OC_OpCooperativeMatrixMulAddNV  : I32EnumAttrCase<"OpCooperativeMatrixMulAddNV", 5361>;
 def SPIRV_OC_OpCooperativeMatrixLengthNV  : I32EnumAttrCase<"OpCooperativeMatrixLengthNV", 5362>;
 def SPIRV_OC_OpSubgroupBlockReadINTEL     : I32EnumAttrCase<"OpSubgroupBlockReadINTEL", 5575>;
 def SPIRV_OC_OpSubgroupBlockWriteINTEL    : I32EnumAttrCase<"OpSubgroupBlockWriteINTEL", 5576>;
 def SPIRV_OC_OpAssumeTrueKHR              : I32EnumAttrCase<"OpAssumeTrueKHR", 5630>;
SPIRV_I32EnumAttr<"Opcode", "valid SPIR-V instructions", "opcode", [
       SPIRV_OC_OpGroupNonUniformBroadcast, SPIRV_OC_OpGroupNonUniformBallot,
       SPIRV_OC_OpGroupNonUniformShuffle, SPIRV_OC_OpGroupNonUniformShuffleXor,
       SPIRV_OC_OpGroupNonUniformShuffleUp, SPIRV_OC_OpGroupNonUniformShuffleDown,
       SPIRV_OC_OpGroupNonUniformIAdd, SPIRV_OC_OpGroupNonUniformFAdd,
       SPIRV_OC_OpGroupNonUniformIMul, SPIRV_OC_OpGroupNonUniformFMul,
       SPIRV_OC_OpGroupNonUniformSMin, SPIRV_OC_OpGroupNonUniformUMin,
       SPIRV_OC_OpGroupNonUniformFMin, SPIRV_OC_OpGroupNonUniformSMax,
       SPIRV_OC_OpGroupNonUniformUMax, SPIRV_OC_OpGroupNonUniformFMax,
-      SPIRV_OC_OpSubgroupBallotKHR, SPIRV_OC_OpTypeCooperativeMatrixNV,
+      SPIRV_OC_OpSubgroupBallotKHR, SPIRV_OC_OpSDot, SPIRV_OC_OpUDot,
+      SPIRV_OC_OpSUDot, SPIRV_OC_OpSDotAccSat, SPIRV_OC_OpUDotAccSat,
+      SPIRV_OC_OpSUDotAccSat, SPIRV_OC_OpTypeCooperativeMatrixNV,
       SPIRV_OC_OpCooperativeMatrixLoadNV, SPIRV_OC_OpCooperativeMatrixStoreNV,
       SPIRV_OC_OpCooperativeMatrixMulAddNV, SPIRV_OC_OpCooperativeMatrixLengthNV,
       SPIRV_OC_OpSubgroupBlockReadINTEL, SPIRV_OC_OpSubgroupBlockWriteINTEL,
       SPIRV_OC_OpAssumeTrueKHR, SPIRV_OC_OpAtomicFAddEXT, SPIRV_OC_OpGroupIMulKHR,
       SPIRV_OC_OpGroupFMulKHR,
 
       SPIRV_OC_OpTypeJointMatrixINTEL, SPIRV_OC_OpJointMatrixLoadINTEL,
       SPIRV_OC_OpJointMatrixStoreINTEL, SPIRV_OC_OpJointMatrixMadINTEL,
class SPIRV_Op<string mnemonic, list<Trait> traits = []> :
   // Availability specification for this op itself.
   list<Availability> availability = [
     MinVersion<SPIRV_V_1_0>,
     MaxVersion<SPIRV_V_1_6>,
     Extension<[]>,
     Capability<[]>
   ];
 
+  // Controls whether to auto-generate this op's availability specification.
+  // If set, generates the following methods:
+  //
+  // ```c++
+  // SmallVector<ArrayRef<Capability>, 1> OpTy::getCapabilities();
+  // SmallVector<ArrayRef<Extension>, 1>  OpTy::getExtensions();
+  // Optional<Version>                    OpTy::getMinVersion();
+  // Optional<Version>                    OpTy::getMaxVersion();
+  // ```
+  //
+  // When not set, manual implementation of these methods is required.
+  bit autogenAvailability = 1;
+
   // For each SPIR-V op, the following static functions need to be defined
   // in SPIRVOps.cpp:
   //
   // * ParseResult <op-c++-class-name>::parse(OpAsmParser &parser,
   //                                          OperationState &result)
   // * void <op-c++-class-name>::print(OpAsmPrinter &p)
   // * LogicalResult <op-c++-class-name>::verify()
   let hasCustomAssemblyFormat = 1;

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVIntegerDotProductOps.td

+//===-- SPIRVIntegerDotProductOps.td - MLIR SPIR-V IDP Ops -*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains arithmetic ops for the SPIR-V dialect. It corresponds
+// to instructions defined by the "SPV_KHR_integer_dot_product" SPIR-V
+// extension.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef MLIR_DIALECT_SPIRV_IR_INTEGER_DOT_PRODUCT_OPS
+#define MLIR_DIALECT_SPIRV_IR_INTEGER_DOT_PRODUCT_OPS
+
+include "mlir/Dialect/SPIRV/IR/SPIRVBase.td"
+include "mlir/Interfaces/InferTypeOpInterface.td"
+include "mlir/Interfaces/SideEffectInterfaces.td"
+
+class SPIRV_IntegerDotProductOp<string mnemonic,
+                                list<Trait> traits = []> :
+      SPIRV_Op<mnemonic, !listconcat(traits, [Pure])> {
+  let results = (outs
+    SPIRV_Integer:$result
+  );
+
+  let assemblyFormat = [{
+    operands attr-dict `:` `(` type(operands) `)` `->` type($result)
+  }];
+
+  // These ops require dynamic availability specification based on operand and
+  // result types.
+  bit autogenAvailability = 0;
+
+  // These ops require a custom verifier.
+  let hasVerifier = 1;
+}
+
+class SPIRV_IntegerDotProductBinaryOp<string mnemonic,
+                                      list<Trait> traits = []> :
+      SPIRV_IntegerDotProductOp<mnemonic, traits> {
+  let arguments = (ins
+    SPIRV_ScalarOrVectorOf<SPIRV_Integer>:$vector1,
+    SPIRV_ScalarOrVectorOf<SPIRV_Integer>:$vector2,
+    OptionalAttr<SPIRV_PackedVectorFormatAttr>:$format
+  );
+}
+
+class SPIRV_IntegerDotProductTernaryOp<string mnemonic,
+                                       list<Trait> traits = []> :
+      SPIRV_IntegerDotProductOp<mnemonic, traits> {
+  let arguments = (ins
+    SPIRV_ScalarOrVectorOf<SPIRV_Integer>:$vector1,
+    SPIRV_ScalarOrVectorOf<SPIRV_Integer>:$vector2,
+    SPIRV_Integer:$accumulator,
+    OptionalAttr<SPIRV_PackedVectorFormatAttr>:$format
+  );
+}
+
+// -----
+
+def SPIRV_SDotOp : SPIRV_IntegerDotProductBinaryOp<"SDot",
+                                                   [SignedOp, Commutative]> {
+  let summary = "Signed integer dot product of Vector 1 and Vector 2.";
+
+  let description = [{
+    Result Type must be an integer type whose Width must be greater than or
+    equal to that of the components of Vector 1 and Vector 2.
+
+    Vector 1 and Vector 2 must have the same type.
+
+    Vector 1 and Vector 2 must be either 32-bit integers (enabled by the
+    DotProductInput4x8BitPacked capability) or vectors of integer type
+    (enabled by the DotProductInput4x8Bit or DotProductInputAll capability).
+
+    When Vector 1 and Vector 2 are scalar integer types, Packed Vector
+    Format must be specified to select how the integers are to be
+    interpreted as vectors.
+
+    All components of the input vectors are sign-extended to the bit width
+    of the result's type. The sign-extended input vectors are then
+    multiplied component-wise and all components of the vector resulting
+    from the component-wise multiplication are added together. The resulting
+    value will equal the low-order N bits of the correct result R, where N
+    is the result width and R is computed with enough precision to avoid
+    overflow and underflow.
+
+    <!-- End of AutoGen section -->
+
+    #### Example:
+
+    ```mlir
+    %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+    %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+    %r = spirv.SDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+    ```
+  }];
+}
+
+// -----
+
+def SPIRV_SUDotOp : SPIRV_IntegerDotProductBinaryOp<"SUDot",
+                                                    [SignedOp, UnsignedOp]> {
+  let summary = [{
+    Mixed-signedness integer dot product of Vector 1 and Vector 2.
+    Components of Vector 1 are treated as signed, components of Vector 2 are
+    treated as unsigned.
+  }];
+
+  let description = [{
+    Result Type must be an integer type whose Width must be greater than or
+    equal to that of the components of Vector 1 and Vector 2.
+
+    Vector 1 and Vector 2 must be either 32-bit integers (enabled by the
+    DotProductInput4x8BitPacked capability) or vectors of integer type with
+    the same number of components and same component Width (enabled by the
+    DotProductInput4x8Bit or DotProductInputAll capability). When Vector 1
+    and Vector 2 are vectors, the components of Vector 2 must have a
+    Signedness of 0.
+
+    When Vector 1 and Vector 2 are scalar integer types, Packed Vector
+    Format must be specified to select how the integers are to be
+    interpreted as vectors.
+
+    All components of Vector 1 are sign-extended to the bit width of the
+    result's type. All components of Vector 2 are zero-extended to the bit
+    width of the result's type. The sign- or zero-extended input vectors are
+    then multiplied component-wise and all components of the vector
+    resulting from the component-wise multiplication are added together. The
+    resulting value will equal the low-order N bits of the correct result R,
+    where N is the result width and R is computed with enough precision to
+    avoid overflow and underflow.
+
+    <!-- End of AutoGen section -->
+
+    #### Example:
+
+    ```mlir
+    %r = spirv.SUDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+    %r = spirv.SUDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+    %r = spirv.SUDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+    ```
+  }];
+}
+
+// -----
+
+def SPIRV_UDotOp : SPIRV_IntegerDotProductBinaryOp<"UDot",
+                                                   [UnsignedOp, Commutative]> {
+  let summary = "Unsigned integer dot product of Vector 1 and Vector 2.";
+
+  let description = [{
+    Result Type must be an integer type with Signedness of 0 whose Width
+    must be greater than or equal to that of the components of Vector 1 and
+    Vector 2.
+
+    Vector 1 and Vector 2 must have the same type.
+
+    Vector 1 and Vector 2 must be either 32-bit integers (enabled by the
+    DotProductInput4x8BitPacked capability) or vectors of integer type with
+    Signedness of 0 (enabled by the DotProductInput4x8Bit or
+    DotProductInputAll capability).
+
+    When Vector 1 and Vector 2 are scalar integer types, Packed Vector
+    Format must be specified to select how the integers are to be
+    interpreted as vectors.
+
+    All components of the input vectors are zero-extended to the bit width
+    of the result's type. The zero-extended input vectors are then
+    multiplied component-wise and all components of the vector resulting
+    from the component-wise multiplication are added together. The resulting
+    value will equal the low-order N bits of the correct result R, where N
+    is the result width and R is computed with enough precision to avoid
+    overflow and underflow.
+
+    <!-- End of AutoGen section -->
+
+    #### Example:
+
+    ```mlir
+    %r = spirv.UDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+    %r = spirv.UDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+    %r = spirv.UDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+    ```
+  }];
+}
+
+#endif // MLIR_DIALECT_SPIRV_IR_INTEGER_DOT_PRODUCT_OPS

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVOps.td

 include "mlir/Dialect/SPIRV/IR/SPIRVCastOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVCompositeOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVControlFlowOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVCooperativeMatrixOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVJointMatrixOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVGLOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVGroupOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVImageOps.td"
+include "mlir/Dialect/SPIRV/IR/SPIRVIntegerDotProductOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVLogicalOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVMatrixOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVMemoryOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVMiscOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVNonUniformOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVCLOps.td"
 include "mlir/Dialect/SPIRV/IR/SPIRVStructureOps.td"
 include "mlir/Interfaces/SideEffectInterfaces.td"
 
 #endif // MLIR_DIALECT_SPIRV_IR_OPS

mlir/lib/Dialect/SPIRV/IR/SPIRVOps.cpp

 #include "mlir/IR/FunctionImplementation.h"
 #include "mlir/IR/OpDefinition.h"
 #include "mlir/IR/OpImplementation.h"
 #include "mlir/IR/Operation.h"
 #include "mlir/IR/TypeUtilities.h"
 #include "mlir/Interfaces/CallInterfaces.h"
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/APInt.h"
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/StringExtras.h"
-#include "llvm/ADT/bit.h"
+#include "llvm/Support/FormatVariadic.h"
 #include <cassert>
 #include <numeric>
 
 using namespace mlir;
 
 // TODO: generate these strings using ODS.
-constexpr char kMemoryAccessAttrName[] = "memory_access";
-constexpr char kSourceMemoryAccessAttrName[] = "source_memory_access";
 constexpr char kAlignmentAttrName[] = "alignment";
-constexpr char kSourceAlignmentAttrName[] = "source_alignment";
 constexpr char kBranchWeightAttrName[] = "branch_weights";
 constexpr char kCallee[] = "callee";
 constexpr char kClusterSize[] = "cluster_size";
 constexpr char kControl[] = "control";
 constexpr char kDefaultValueAttrName[] = "default_value";
-constexpr char kExecutionScopeAttrName[] = "execution_scope";
 constexpr char kEqualSemanticsAttrName[] = "equal_semantics";
+constexpr char kExecutionScopeAttrName[] = "execution_scope";
 constexpr char kFnNameAttrName[] = "fn";
 constexpr char kGroupOperationAttrName[] = "group_operation";
 constexpr char kIndicesAttrName[] = "indices";
 constexpr char kInitializerAttrName[] = "initializer";
 constexpr char kInterfaceAttrName[] = "interface";
+constexpr char kMemoryAccessAttrName[] = "memory_access";
 constexpr char kMemoryScopeAttrName[] = "memory_scope";
+constexpr char kPackedVectorFormatAttrName[] = "format";
 constexpr char kSemanticsAttrName[] = "semantics";
+constexpr char kSourceAlignmentAttrName[] = "source_alignment";
+constexpr char kSourceMemoryAccessAttrName[] = "source_memory_access";
 constexpr char kSpecIdAttrName[] = "spec_id";
 constexpr char kTypeAttrName[] = "type";
 constexpr char kUnequalSemanticsAttrName[] = "unequal_semantics";
 constexpr char kValueAttrName[] = "value";
 constexpr char kValuesAttrName[] = "values";
 constexpr char kCompositeSpecConstituentsName[] = "constituents";
 
 //===----------------------------------------------------------------------===//

 LogicalResult spirv::GroupUMaxOp::verify() { return verifyGroupOp(*this); }
 
 LogicalResult spirv::GroupSMaxOp::verify() { return verifyGroupOp(*this); }
 
 LogicalResult spirv::GroupIMulKHROp::verify() { return verifyGroupOp(*this); }
 
 LogicalResult spirv::GroupFMulKHROp::verify() { return verifyGroupOp(*this); }
 
+//===----------------------------------------------------------------------===//
+// Integer Dot Product ops
+//===----------------------------------------------------------------------===//
+
+static LogicalResult verifyIntegerDotProduct(Operation *op) {
+  assert(llvm::is_contained({2u, 3u}, op->getNumOperands()) &&
+         "Not an integer dot product op?");
+  assert(op->getNumResults() == 1 && "Expected a single result");
+
+  Type factorTy = op->getOperand(0).getType();
+  if (op->getOperand(1).getType() != factorTy)
+    return op->emitOpError("requires the same type for both vector operands");
+
+  if (auto intTy = factorTy.dyn_cast<IntegerType>()) {
+    auto packedVectorFormat =
+        op->getAttr(kPackedVectorFormatAttrName)
+            .dyn_cast_or_null<spirv::PackedVectorFormatAttr>();
+    if (!packedVectorFormat)
+      return op->emitOpError("requires Packed Vector Format attribute for "
+                             "integer vector operands");
+
+    assert(packedVectorFormat.getValue() ==
+               spirv::PackedVectorFormat::PackedVectorFormat4x8Bit &&
+           "unknown Packed Vector format");
+    if (intTy.getWidth() != 32)
+      return op->emitOpError(
+          llvm::formatv("with specified Packed Vector Format ({0}) requires "
+                        "integer vector operands to be 32-bits wide",
+                        packedVectorFormat.getValue()));
+  }
+
+  if (op->getAttrs().size() > 1)
+    return op->emitError(
+        "op only supports the 'format' #spirv.packed_vector_format attribute");
+
+  Type resultTy = op->getResultTypes().front();
+  unsigned factorBitWidth = getBitWidth(factorTy);
+  unsigned resultBitWidth = getBitWidth(resultTy);
+  if (factorBitWidth > resultBitWidth)
+    return op->emitOpError(
+        llvm::formatv("result type has insufficient bit-width ({0} bits) "
+                      "for the specified vector operand type ({1} bits)",
+                      resultBitWidth, factorBitWidth));
+
+  return success();
+}
+
+static Optional<spirv::Version> getIntegerDotProductMinVersion() {
+  return spirv::Version::V_1_0; // Available in SPIR-V >= 1.0.
+}
+
+static Optional<spirv::Version> getIntegerDotProductMaxVersion() {
+  return spirv::Version::V_1_6; // Available in SPIR-V <= 1.6.
+}
+
+static SmallVector<ArrayRef<spirv::Extension>, 1>
+getIntegerDotProductExtensions() {
+  // Requires the SPV_KHR_integer_dot_product extension, specified either
+  // explicitly or implied by target env's SPIR-V version >= 1.6.
+  static const auto extension = spirv::Extension::SPV_KHR_integer_dot_product;
+  return {extension};
+}
+
+static SmallVector<ArrayRef<spirv::Capability>, 1>
+getIntegerDotProductCapabilities(Operation *op) {
+  // Requires the the DotProduct capability and capabilities that depend on
+  // exact op types.
+  static const auto dotProductCap = spirv::Capability::DotProduct;
+  static const auto dotProductInput4x8BitPackedCap =
+      spirv::Capability::DotProductInput4x8BitPacked;
+  static const auto dotProductInput4x8BitCap =
+      spirv::Capability::DotProductInput4x8Bit;
+  static const auto dotProductInputAllCap =
+      spirv::Capability::DotProductInputAll;
+
+  SmallVector<ArrayRef<spirv::Capability>, 1> capabilities = {dotProductCap};
+
+  Type factorTy = op->getOperand(0).getType();
+  if (auto intTy = factorTy.dyn_cast<IntegerType>()) {
+    auto formatAttr = op->getAttr(kPackedVectorFormatAttrName)
+                          .cast<spirv::PackedVectorFormatAttr>();
+    if (formatAttr.getValue() ==
+        spirv::PackedVectorFormat::PackedVectorFormat4x8Bit)
+      capabilities.push_back(dotProductInput4x8BitPackedCap);
+
+    return capabilities;
+  }
+
+  auto vecTy = factorTy.cast<VectorType>();
+  if (vecTy.getElementTypeBitWidth() == 8) {
+    capabilities.push_back(dotProductInput4x8BitCap);
+    return capabilities;
+  }
+
+  capabilities.push_back(dotProductInputAllCap);
+  return capabilities;
+}
+
+#define SPIRV_IMPL_INTEGER_DOT_PRODUCT_OP(OpName)                              \
+  LogicalResult OpName::verify() { return verifyIntegerDotProduct(*this); }    \
+  SmallVector<ArrayRef<spirv::Extension>, 1> OpName::getExtensions() {         \
+    return getIntegerDotProductExtensions();                                   \
+  }                                                                            \
+  SmallVector<ArrayRef<spirv::Capability>, 1> OpName::getCapabilities() {      \
+    return getIntegerDotProductCapabilities(*this);                            \
+  }                                                                            \
+  Optional<spirv::Version> OpName::getMinVersion() {                           \
+    return getIntegerDotProductMinVersion();                                   \
+  }                                                                            \
+  Optional<spirv::Version> OpName::getMaxVersion() {                           \
+    return getIntegerDotProductMaxVersion();                                   \
+  }
+
+SPIRV_IMPL_INTEGER_DOT_PRODUCT_OP(spirv::SDotOp)
+SPIRV_IMPL_INTEGER_DOT_PRODUCT_OP(spirv::SUDotOp)
+SPIRV_IMPL_INTEGER_DOT_PRODUCT_OP(spirv::UDotOp)
+
+#undef SPIRV_IMPL_INTEGER_DOT_PRODUCT_OP
+
 // TableGen'erated operation interfaces for querying versions, extensions, and
 // capabilities.
 #include "mlir/Dialect/SPIRV/IR/SPIRVAvailability.cpp.inc"
 
 // TablenGen'erated operation definitions.
 #define GET_OP_CLASSES
 #include "mlir/Dialect/SPIRV/IR/SPIRVOps.cpp.inc"
 
 namespace mlir {
 namespace spirv {
 // TableGen'erated operation availability interface implementations.
 #include "mlir/Dialect/SPIRV/IR/SPIRVOpAvailabilityImpl.inc"
 } // namespace spirv
 } // namespace mlir

mlir/test/Dialect/SPIRV/IR/availability.mlir

 func.func @module_physical_storage_buffer64_vulkan() {
   // CHECK: spirv.module min version: v1.0
   // CHECK: spirv.module max version: v1.6
   // CHECK: spirv.module extensions: [ [SPV_EXT_physical_storage_buffer, SPV_KHR_physical_storage_buffer] [SPV_KHR_vulkan_memory_model] ]
   // CHECK: spirv.module capabilities: [ [PhysicalStorageBufferAddresses] [VulkanMemoryModel] ]
   spirv.module PhysicalStorageBuffer64 Vulkan { }
   return
 }
+
+//===----------------------------------------------------------------------===//
+// Integer Dot Product ops
+//===----------------------------------------------------------------------===//
+
+// CHECK-LABEL: sdot_scalar_i32_i32
+func.func @sdot_scalar_i32_i32(%a: i32) -> i32 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8BitPacked] ]
+  %r = spirv.SDot %a, %a {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r: i32
+}
+
+// CHECK-LABEL: sdot_vector_4xi8_i64
+func.func @sdot_vector_4xi8_i64(%a: vector<4xi8>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8Bit] ]
+  %r = spirv.SDot %a, %a: (vector<4xi8>, vector<4xi8>) -> i64
+  return %r: i64
+}
+
+// CHECK-LABEL: sdot_vector_4xi16_i64
+func.func @sdot_vector_4xi16_i64(%a: vector<4xi16>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInputAll] ]
+  %r = spirv.SDot %a, %a: (vector<4xi16>, vector<4xi16>) -> i64
+  return %r: i64
+}
+
+// CHECK-LABEL: sudot_scalar_i32_i32
+func.func @sudot_scalar_i32_i32(%a: i32) -> i32 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8BitPacked] ]
+  %r = spirv.SUDot %a, %a {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r: i32
+}
+
+// CHECK-LABEL: sudot_vector_4xi8_i64
+func.func @sudot_vector_4xi8_i64(%a: vector<4xi8>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8Bit] ]
+  %r = spirv.SUDot %a, %a: (vector<4xi8>, vector<4xi8>) -> i64
+  return %r: i64
+}
+
+// CHECK-LABEL: sudot_vector_4xi16_i64
+func.func @sudot_vector_4xi16_i64(%a: vector<4xi16>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInputAll] ]
+  %r = spirv.SUDot %a, %a: (vector<4xi16>, vector<4xi16>) -> i64
+  return %r: i64
+}
+
+// CHECK-LABEL: udot_scalar_i32_i32
+func.func @udot_scalar_i32_i32(%a: i32) -> i32 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8BitPacked] ]
+  %r = spirv.UDot %a, %a {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r: i32
+}
+
+// CHECK-LABEL: udot_vector_4xi8_i64
+func.func @udot_vector_4xi8_i64(%a: vector<4xi8>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInput4x8Bit] ]
+  %r = spirv.UDot %a, %a: (vector<4xi8>, vector<4xi8>) -> i64
+  return %r: i64
+}
+
+// CHECK-LABEL: udot_vector_4xi16_i64
+func.func @udot_vector_4xi16_i64(%a: vector<4xi16>) -> i64 {
+  // CHECK: min version: v1.0
+  // CHECK: max version: v1.6
+  // CHECK: extensions: [ [SPV_KHR_integer_dot_product] ]
+  // CHECK: capabilities: [ [DotProduct] [DotProductInputAll] ]
+  %r = spirv.UDot %a, %a: (vector<4xi16>, vector<4xi16>) -> i64
+  return %r: i64
+}

mlir/test/Dialect/SPIRV/IR/integer-dot-product-ops.mlir

+// RUN: mlir-opt -split-input-file -verify-diagnostics %s | FileCheck %s
+
+// This test covers the Integer Dot Product ops defined in the
+// SPV_KHR_integer_dot_product extension.
+
+//===----------------------------------------------------------------------===//
+// spirv.SDot
+//===----------------------------------------------------------------------===//
+
+// CHECK: @sdot_scalar_i32
+func.func @sdot_scalar_i32(%a: i32, %b: i32) -> i32 {
+  // CHECK-NEXT: spirv.SDot
+  %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r : i32
+}
+
+// CHECK: @sdot_scalar_i64
+func.func @sdot_scalar_i64(%a: i32, %b: i32) -> i64 {
+  // CHECK-NEXT: spirv.SDot
+  %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+  return %r : i64
+}
+
+// CHECK: @sdot_vector_4xi8
+func.func @sdot_vector_4xi8(%a: vector<4xi8>, %b: vector<4xi8>) -> i32 {
+  // CHECK-NEXT: spirv.SDot
+  %r = spirv.SDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+  return %r : i32
+}
+
+// CHECK: @sdot_vector_4xi16
+func.func @sdot_vector_4xi16(%a: vector<4xi16>, %b: vector<4xi16>) -> i64 {
+  // CHECK-NEXT: spirv.SDot
+  %r = spirv.SDot %a, %b : (vector<4xi16>, vector<4xi16>) -> i64
+  return %r : i64
+}
+
+// CHECK: @sdot_vector_8xi8
+func.func @sdot_vector_8xi8(%a: vector<8xi8>, %b: vector<8xi8>) -> i64 {
+  // CHECK-NEXT: spirv.SDot
+  %r = spirv.SDot %a, %b : (vector<8xi8>, vector<8xi8>) -> i64
+  return %r : i64
+}
+
+// -----
+
+func.func @sdot_scalar_bad_types(%a: i32, %b: i64) -> i32 {
+  // expected-error @+1 {{op requires the same type for both vector operands}}
+  %r = spirv.SDot %a, %b : (i32, i64) -> i32
+  return %r : i32
+}
+
+// -----
+
+func.func @sdot_scalar_i32_bad_attr(%a: i32, %b: i32) -> i32 {
+  // expected-error @+1 {{op only supports the 'format' #spirv.packed_vector_format attribute}}
+  %r = spirv.SDot %a, %b {volatile = #spirv.decoration<Volatile>,
+                          format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r : i32
+}
+
+// -----
+
+func.func @sdot_scalar_bad_types(%a: i32, %b: i32) -> i16 {
+  // expected-error @+1 {{op result type has insufficient bit-width (16 bits) for the specified vector operand type (32 bits)}}
+  %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i16
+  return %r : i16
+}
+
+// -----
+
+func.func @sdot_scalar_bad_types(%a: i64, %b: i64) -> i64 {
+  // expected-error @+1 {{op with specified Packed Vector Format (PackedVectorFormat4x8Bit) requires integer vector operands to be 32-bits wide}}
+  %r = spirv.SDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i64, i64) -> i64
+  return %r : i64
+}
+
+// -----
+
+//===----------------------------------------------------------------------===//
+// spirv.SUDot
+//===----------------------------------------------------------------------===//
+
+// CHECK: @sudot_scalar_i32
+func.func @sudot_scalar_i32(%a: i32, %b: i32) -> i32 {
+  // CHECK-NEXT: spirv.SUDot
+  %r = spirv.SUDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r : i32
+}
+
+// CHECK: @sudot_scalar_i64
+func.func @sudot_scalar_i64(%a: i32, %b: i32) -> i64 {
+  // CHECK-NEXT: spirv.SUDot
+  %r = spirv.SUDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+  return %r : i64
+}
+
+// CHECK: @sudot_vector_4xi8
+func.func @sudot_vector_4xi8(%a: vector<4xi8>, %b: vector<4xi8>) -> i32 {
+  // CHECK-NEXT: spirv.SUDot
+  %r = spirv.SUDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+  return %r : i32
+}
+
+// CHECK: @sudot_vector_4xi16
+func.func @sudot_vector_4xi16(%a: vector<4xi16>, %b: vector<4xi16>) -> i64 {
+  // CHECK-NEXT: spirv.SUDot
+  %r = spirv.SUDot %a, %b : (vector<4xi16>, vector<4xi16>) -> i64
+  return %r : i64
+}
+
+// CHECK: @sudot_vector_8xi8
+func.func @sudot_vector_8xi8(%a: vector<8xi8>, %b: vector<8xi8>) -> i64 {
+  // CHECK-NEXT: spirv.SUDot
+  %r = spirv.SUDot %a, %b : (vector<8xi8>, vector<8xi8>) -> i64
+  return %r : i64
+}
+
+// -----
+
+//===----------------------------------------------------------------------===//
+// spirv.UDot
+//===----------------------------------------------------------------------===//
+
+// CHECK: @udot_scalar_i32
+func.func @udot_scalar_i32(%a: i32, %b: i32) -> i32 {
+  // CHECK-NEXT: spirv.UDot
+  %r = spirv.UDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i32
+  return %r : i32
+}
+
+// CHECK: @udot_scalar_i64
+func.func @udot_scalar_i64(%a: i32, %b: i32) -> i64 {
+  // CHECK-NEXT: spirv.UDot
+  %r = spirv.UDot %a, %b {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> i64
+  return %r : i64
+}
+
+// CHECK: @udot_vector_4xi8
+func.func @udot_vector_4xi8(%a: vector<4xi8>, %b: vector<4xi8>) -> i32 {
+  // CHECK-NEXT: spirv.UDot
+  %r = spirv.UDot %a, %b : (vector<4xi8>, vector<4xi8>) -> i32
+  return %r : i32
+}

mlir/test/Dialect/SPIRV/IR/target-env.mlir

 // spirv.BitReverse is available in all SPIR-V versions under Shader capability.
 
 // spirv.GroupNonUniformBallot is available starting from SPIR-V 1.3 under
 // GroupNonUniform capability.
 
 // spirv.KHR.SubgroupBallot is available under in all SPIR-V versions under
 // SubgroupBallotKHR capability and SPV_KHR_shader_ballot extension.
 
+// Integer Dot Product ops (spirv.*Dot*) require the
+// SPV_KHR_integer_dot_product extension and a number of related capabilities.
+
 // The GeometryPointSize capability implies the Geometry capability, which
 // implies the Shader capability.
 
 // PhysicalStorageBuffer64 addressing model is available via extension
 // SPV_EXT_physical_storage_buffer or SPV_KHR_physical_storage_buffer;
 // both extensions are incorporated into SPIR-V 1.5.
 
 // Vulkan memory model is available via extension SPV_KHR_vulkan_memory_model,

 func.func @bit_reverse_recursively_implied_capability(%operand: i32) -> i32 attributes {
   spirv.target_env = #spirv.target_env<#spirv.vce<v1.0, [GeometryPointSize], []>, #spirv.resource_limits<>>
 } {
   // CHECK: spirv.BitReverse
   %0 = "test.convert_to_bit_reverse_op"(%operand): (i32) -> (i32)
   return %0: i32
 }
 
+// CHECK-LABEL: @sdot_scalar_i32_i32_capabilities
+func.func @sdot_scalar_i32_i32_capabilities(%operand: i32) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct, DotProductInput4x8BitPacked], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: spirv.SDot
+  %0 = "test.convert_to_sdot_op"(%operand, %operand) {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @sdot_scalar_i32_i32_missing_capability1
+func.func @sdot_scalar_i32_i32_missing_capability1(%operand: i32) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_sdot_op
+  %0 = "test.convert_to_sdot_op"(%operand, %operand) {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @sdot_scalar_i32_i32_missing_capability2
+func.func @sdot_scalar_i32_i32_missing_capability2(%operand: i32) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProductInput4x8BitPacked], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_sdot_op
+  %0 = "test.convert_to_sdot_op"(%operand, %operand) {format = #spirv.packed_vector_format<PackedVectorFormat4x8Bit>}: (i32, i32) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @sudot_vector_4xi8_i32_capabilities
+func.func @sudot_vector_4xi8_i32_capabilities(%operand: vector<4xi8>) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct, DotProductInput4x8Bit], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: spirv.SUDot
+  %0 = "test.convert_to_sudot_op"(%operand, %operand): (vector<4xi8>, vector<4xi8>) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @sudot_vector_4xi8_i32_missing_capability1
+func.func @sudot_vector_4xi8_i32_missing_capability1(%operand: vector<4xi8>) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_sudot_op
+  %0 = "test.convert_to_sudot_op"(%operand, %operand): (vector<4xi8>, vector<4xi8>) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @sudot_vector_4xi8_i32_missing_capability2
+func.func @sudot_vector_4xi8_i32_missing_capability2(%operand: vector<4xi8>) -> i32 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProductInput4x8Bit], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_sudot_op
+  %0 = "test.convert_to_sudot_op"(%operand, %operand): (vector<4xi8>, vector<4xi8>) -> (i32)
+  return %0: i32
+}
+
+// CHECK-LABEL: @udot_vector_4xi16_i64_capabilities
+func.func @udot_vector_4xi16_i64_capabilities(%operand: vector<4xi16>) -> i64 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct, DotProductInputAll, Int16, Int64], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: spirv.UDot
+  %0 = "test.convert_to_udot_op"(%operand, %operand): (vector<4xi16>, vector<4xi16>) -> (i64)
+  return %0: i64
+}
+
+// CHECK-LABEL: @udot_vector_4xi16_i64_missing_capability1
+func.func @udot_vector_4xi16_i64_missing_capability1(%operand: vector<4xi16>) -> i64 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProduct, Int16, Int64], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_udot_op
+  %0 = "test.convert_to_udot_op"(%operand, %operand): (vector<4xi16>, vector<4xi16>) -> (i64)
+  return %0: i64
+}
+
+// CHECK-LABEL: @udot_vector_4xi16_i64_missing_capability2
+func.func @udot_vector_4xi16_i64_missing_capability2(%operand: vector<4xi16>) -> i64 attributes {
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.0,
+    [DotProductInputAll, Int16, Int64], [SPV_KHR_integer_dot_product]>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_udot_op
+  %0 = "test.convert_to_udot_op"(%operand, %operand): (vector<4xi16>, vector<4xi16>) -> (i64)
+  return %0: i64
+}
+
 //===----------------------------------------------------------------------===//
 // Extension
 //===----------------------------------------------------------------------===//
 
 // CHECK-LABEL: @subgroup_ballot_suitable_extension
 func.func @subgroup_ballot_suitable_extension(%predicate: i1) -> vector<4xi32> attributes {
   spirv.target_env = #spirv.target_env<#spirv.vce<v1.4, [SubgroupBallotKHR], [SPV_KHR_shader_ballot]>, #spirv.resource_limits<>>
 } {

 func.func @module_implied_extension() attributes {
   // Version 1.5 implies SPV_KHR_vulkan_memory_model and SPV_KHR_physical_storage_buffer.
   spirv.target_env = #spirv.target_env<#spirv.vce<v1.5, [VulkanMemoryModel, PhysicalStorageBufferAddresses], []>, #spirv.resource_limits<>>
 } {
   // CHECK: spirv.module PhysicalStorageBuffer64 Vulkan
   "test.convert_to_module_op"() : () -> ()
   return
 }
+
+// CHECK-LABEL: @udot_vector_4xi16_i64_implied_extension
+func.func @udot_vector_4xi16_i64_implied_extension(%operand: vector<4xi16>) -> i64 attributes {
+  // Version 1.6 implies SPV_KHR_integer_to_product.
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.6,
+    [DotProduct, DotProductInputAll, Int16, Int64], []>, #spirv.resource_limits<>>
+} {
+  // CHECK: spirv.UDot
+  %0 = "test.convert_to_udot_op"(%operand, %operand): (vector<4xi16>, vector<4xi16>) -> (i64)
+  return %0: i64
+}
+
+// CHECK-LABEL: @udot_vector_4xi16_i64_missing_extension
+func.func @udot_vector_4xi16_i64_missing_extension(%operand: vector<4xi16>) -> i64 attributes {
+  // Version 1.5 does not imply SPV_KHR_integer_to_product.
+  spirv.target_env = #spirv.target_env<#spirv.vce<v1.5,
+    [DotProduct, DotProductInputAll, Int16, Int64], []>, #spirv.resource_limits<>>
+} {
+  // CHECK: test.convert_to_udot_op
+  %0 = "test.convert_to_udot_op"(%operand, %operand): (vector<4xi16>, vector<4xi16>) -> (i64)
+  return %0: i64
+}

mlir/test/lib/Dialect/SPIRV/TestAvailability.cpp

struct ConvertToSubgroupBallot : RewritePattern {
   LogicalResult matchAndRewrite(Operation *op,
                                 PatternRewriter &rewriter) const override {
     Value predicate = op->getOperand(0);
     rewriter.replaceOpWithNewOp<spirv::KHRSubgroupBallotOp>(
         op, op->getResult(0).getType(), predicate);
     return success();
   }
 };
+
+template <const char *TestOpName, typename SPIRVOp>
+struct ConvertToIntegerDotProd : RewritePattern {
+  ConvertToIntegerDotProd(MLIRContext *context)
+      : RewritePattern(TestOpName, 1, context, {SPIRVOp::getOperationName()}) {}
+
+  LogicalResult matchAndRewrite(Operation *op,
+                                PatternRewriter &rewriter) const override {
+    rewriter.replaceOpWithNewOp<SPIRVOp>(op, op->getResultTypes(),
+                                         op->getOperands(), op->getAttrs());
+    return success();
+  }
+};
 } // namespace
 
 void ConvertToTargetEnv::runOnOperation() {
   MLIRContext *context = &getContext();
   func::FuncOp fn = getOperation();
 
   auto targetEnv = fn.getOperation()
                        ->getAttr(spirv::getTargetEnvAttrName())
                        .cast<spirv::TargetEnvAttr>();
   if (!targetEnv) {
     fn.emitError("missing 'spirv.target_env' attribute");
     return signalPassFailure();
   }
 
   auto target = SPIRVConversionTarget::get(targetEnv);
 
+  static constexpr char sDotTestOpName[] = "test.convert_to_sdot_op";
+  static constexpr char suDotTestOpName[] = "test.convert_to_sudot_op";
+  static constexpr char uDotTestOpName[] = "test.convert_to_udot_op";
+
   RewritePatternSet patterns(context);
   patterns.add<ConvertToAtomCmpExchangeWeak, ConvertToBitReverse,
                ConvertToGroupNonUniformBallot, ConvertToModule,
-               ConvertToSubgroupBallot>(context);
+               ConvertToSubgroupBallot,
+               ConvertToIntegerDotProd<sDotTestOpName, spirv::SDotOp>,
+               ConvertToIntegerDotProd<suDotTestOpName, spirv::SUDotOp>,
+               ConvertToIntegerDotProd<uDotTestOpName, spirv::UDotOp>>(context);
 
   if (failed(applyPartialConversion(fn, *target, std::move(patterns))))
     return signalPassFailure();
 }
 
 namespace mlir {
 void registerConvertToTargetEnvPass() {
   PassRegistration<ConvertToTargetEnv>();
 }
 } // namespace mlir

mlir/tools/mlir-tblgen/SPIRVUtilsGen.cpp

 
 static bool emitAvailabilityImpl(const RecordKeeper &recordKeeper,
                                  raw_ostream &os) {
   llvm::emitSourceFileHeader("SPIR-V Op Availability Implementations", os);
 
   auto defs = recordKeeper.getAllDerivedDefinitions("SPIRV_Op");
   for (const auto *def : defs) {
     Operator op(def);
+    if (def->getValueAsBit("autogenAvailability"))
       emitAvailabilityImpl(op, os);
   }
   return false;
 }
 
 //===----------------------------------------------------------------------===//
 // Op Availability Implementation Hook Registration
 //===----------------------------------------------------------------------===//
 

mlir/utils/spirv/define_enum.sh

 #!/bin/bash
 
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 # Script for defining a new enum attr using SPIR-V spec from the Internet.
 #
 # Run as:
 # ./define_enum.sh <enum-class-name>
 #
 # The 'operand_kinds' dict of spirv.core.grammar.json contains all supported
 # SPIR-V enum classes.
 #
-# If <enum-name> is missing, this script updates existing ones.
+# If <enum-class-name> is missing, this script updates existing ones.
 
 set -e
 
 new_enum=$1
 
 current_file="$(readlink -f "$0")"
 current_dir="$(dirname "$current_file")"
 
 python3 ${current_dir}/gen_spirv_dialect.py \
   --base-td-path ${current_dir}/../../include/mlir/Dialect/SPIRV/IR/SPIRVBase.td \
   --new-enum "${new_enum}"