[MLIR] Add complex numbers to standard dialect

[frgossen]

Summary

Add CreateComplexOp, ReOp, and ImOp to the standard dialect.
This is the first step to support complex numbers.

[pifon2a]

awesome! thanks!

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1564

verifier is already included in Std_Op, you don't have to add it here.

mlir/lib/Conversion/StandardToLLVM/StandardToLLVM.cpp

197

const can be omitted in LLVM codebase.

1347

auto loc = op->getLoc(); and reuse it 3 times?

[bondhugula]

mlir/include/mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h

161

These comments should all be triple /// I believe.

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1548

Please document the operands.

1957

Please document the operands.

mlir/lib/Dialect/StandardOps/IR/Ops.cpp

1905 ↗

(On Diff #261148)

dyn_cast -> cast

[bondhugula]

mlir/test/mlir-cpu-runner/complex_numbers.mlir

6 ↗

(On Diff #261148)

CHECK-LABEL to prevent things from matching past this.

21 ↗

(On Diff #261148)

CHECK-LABEL here please.

[frgossen]

Document operands, small changes.

[frgossen]

mlir/test/mlir-cpu-runner/complex_numbers.mlir

6 ↗

(On Diff #261148)

I double-checked the output of mlir-opt %s -convert-std-to-llvm | mlir-cpu-runner -e create_and_extract_real -entry-point-result=f32 and it is no more than the result of the computation, i.e. 1.200000e+00.
Because the two functions are compiled and executed separately, I believe matching past things is not an issue here and I don't see how I can use CHECK-LABEL in a useful way.
Do let me know if there is a more elegant way to write this test.

21 ↗

(On Diff #261148)

See above.

[ftynse]

Looks good in general! I have some advice on the op syntax, feel free to argue against it.

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1008

Could we use a full word for the name, $complex. We generate functions with this name, it's more readable but reasonably short

1011

Do we want to support "mixed" types, e.g. f32, f64 -> complex<f64>? If not, I would just keep the type($cplx) here.

1553

This example looks wrong: the spec below says there is only one operand

1561

I think complex<X> -> X is quite redundant here, keeping just the complex type should suffice.

1963

Same as above

mlir/lib/Conversion/StandardToLLVM/StandardToLLVM.cpp

197

I think you can just call convertType(type.getElementType()) and let it be dispatched for you.

1332

We tend to use the kRealIdxInCpxNumStruct style for constants.

1345

Could you rather derive a new ComplexStructBuilder from `StructBuilder and use it here? Look at MemRefDescriptor for an example.

mlir/lib/Dialect/StandardOps/IR/Ops.cpp

1230 ↗

(On Diff #261183)

Could you please add a test for this? test/IR/invalid-ops.mlir sounds like the right file (although the file itself is misplaced)

mlir/test/Conversion/StandardToLLVM/convert-to-llvmir.mlir

78

You don't need an explicit label for the entry block in MLIR functions (and most other operations)

mlir/test/mlir-cpu-runner/complex_numbers.mlir

1 ↗

(On Diff #261183)

I think the conversion is straightforward enough to trust the individual components and drop this test. The -runner tests are intended to check that the runner itself works, your patch does not alter it in any way.

[frgossen]

Thanks for the many helpful comments.
There is one comment for which I'd like clarification.

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1011

I would love to have only type($cplx) here.
The reason for the current format is that tablegen complained when I dropped the other types.
I can fix this with a hand-written parser and printer but before I do that I want to ask if this is also possible with the assemblyFormat thingy.
What is the elegant way to do this?

[ftynse]

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1011

Indeed, auto-generated printer and parser needs a way to resolve _all_ types and cannot do so in this case. Sometimes, it can rely on the traits like SameOperandAndResultTypes, but not here. Ideally, we would have some way of specifying "type deduction rules"... Until that exists, you will need printer and parser functions. To make life easier, you can just take those generated by mlir-tblgen today and amend them a bit to deduce the types.

[rriddle]

mlir/include/mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h

18

Please use forward declarations instead of includes in header files when you can.

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

991

Same comment as below.

1011

You can do that with TypesMatchWith, it was intended for this purpose..

1541

Don't provide a syntax if you have the declarative assembly(assemblyFormat). It is included automatically in the docs.

1951

Same here.

mlir/lib/Dialect/StandardOps/IR/Ops.cpp

1228 ↗

(On Diff #261183)

You should be able to verify this in tablegen with TypesMatchWith.

1649 ↗

(On Diff #261183)

Same comment as the other verifier.

1905 ↗

(On Diff #261183)

Same here.

[frgossen]

Simplified assembly format, introduced ComplexStructBuilder.

[frgossen]

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

1008

Also changed $imag to $imaginary.

1011

Nice. Thanks!

[ftynse]

Thanks!

mlir/include/mlir/Dialect/StandardOps/EDSC/Intrinsics.h

35

Nit: let's use std_re to be consistent with the assembly spelling.

mlir/test/IR/invalid-ops.mlir

1236

Nit: let's not hardcode SSA value names in checks, I think it's fine to shorten the expected-error message to expects different type than prior uses: 'f32' vs 'f64'

[frgossen]

Nits

[frgossen]

Address nits.

[frgossen]

Apply clang tidy suggestions

mlir/include/mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h

 // converts Standard ops and types and provides hooks for dialect-specific
 // extensions to the conversion.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef MLIR_CONVERSION_STANDARDTOLLVM_CONVERTSTANDARDTOLLVM_H
 #define MLIR_CONVERSION_STANDARDTOLLVM_CONVERTSTANDARDTOLLVM_H
 
 #include "mlir/Transforms/DialectConversion.h"
 
 namespace llvm {
 class IntegerType;
 class LLVMContext;
 class Module;
 class Type;
 } // namespace llvm
 
 namespace mlir {
 
+class ComplexType;
 class LLVMTypeConverter;
 class UnrankedMemRefType;
 
 namespace LLVM {
 class LLVMDialect;
 class LLVMType;
 } // namespace LLVM
 
public:
   unsigned getIndexTypeBitwidth() { return customizations.indexBitwidth; }
 
 protected:
   /// LLVM IR module used to parse/create types.
   llvm::Module *module;
   LLVM::LLVMDialect *llvmDialect;
 
 private:
-  // Convert a function type.  The arguments and results are converted one by
-  // one.  Additionally, if the function returns more than one value, pack the
-  // results into an LLVM IR structure type so that the converted function type
-  // returns at most one result.
+  /// Convert a function type.  The arguments and results are converted one by
+  /// one.  Additionally, if the function returns more than one value, pack the
+  /// results into an LLVM IR structure type so that the converted function type
+  /// returns at most one result.
   Type convertFunctionType(FunctionType type);
 
-  // Convert the index type.  Uses llvmModule data layout to create an integer
-  // of the pointer bitwidth.
+  /// Convert the index type.  Uses llvmModule data layout to create an integer
+  /// of the pointer bitwidth.
   Type convertIndexType(IndexType type);
 
-  // Convert an integer type `i*` to `!llvm<"i*">`.
+  /// Convert an integer type `i*` to `!llvm<"i*">`.
   Type convertIntegerType(IntegerType type);
 
-  // Convert a floating point type: `f16` to `!llvm.half`, `f32` to
-  // `!llvm.float` and `f64` to `!llvm.double`.  `bf16` is not supported
-  // by LLVM.
+  /// Convert a floating point type: `f16` to `!llvm.half`, `f32` to
+  /// `!llvm.float` and `f64` to `!llvm.double`.  `bf16` is not supported
+  /// by LLVM.
   Type convertFloatType(FloatType type);
 
+  /// Convert complex number type: `complex<f16>` to `!llvm<"{ half, half }">`,
+  /// `complex<f32>` to `!llvm<"{ float, float }">`, and `complex<f64>` to
+  /// `!llvm<"{ double, double }">`. `complex<bf16>` is not supported.
+  Type convertComplexType(ComplexType type);
+
   /// Convert a memref type into an LLVM type that captures the relevant data.
   Type convertMemRefType(MemRefType type);
 
   /// Convert a memref type into a list of non-aggregate LLVM IR types that
   /// contain all the relevant data. In particular, the list will contain:
   /// - two pointers to the memref element type, followed by
   /// - an integer offset, followed by
   /// - one integer size per dimension of the memref, followed by

 
 protected:
   /// Builds IR to extract a value from the struct at position pos
   Value extractPtr(OpBuilder &builder, Location loc, unsigned pos);
   /// Builds IR to set a value in the struct at position pos
   void setPtr(OpBuilder &builder, Location loc, unsigned pos, Value ptr);
 };
 
+class ComplexStructBuilder : public StructBuilder {
+public:
+  /// Construct a helper for the given complex number value.
+  using StructBuilder::StructBuilder;
+  /// Build IR creating an `undef` value of the complex number type.
+  static ComplexStructBuilder undef(OpBuilder &builder, Location loc,
+                                    Type type);
+
+  // Build IR extracting the real value from the complex number struct.
+  Value real(OpBuilder &builder, Location loc);
+  // Build IR inserting the real value into the complex number struct.
+  void setReal(OpBuilder &builder, Location loc, Value real);
+
+  // Build IR extracting the imaginary value from the complex number struct.
+  Value imaginary(OpBuilder &builder, Location loc);
+  // Build IR inserting the imaginary value into the complex number struct.
+  void setImaginary(OpBuilder &builder, Location loc, Value imaginary);
+};
+
 /// Helper class to produce LLVM dialect operations extracting or inserting
 /// elements of a MemRef descriptor. Wraps a Value pointing to the descriptor.
 /// The Value may be null, in which case none of the operations are valid.
 class MemRefDescriptor : public StructBuilder {
 public:
   /// Construct a helper for the given descriptor value.
   explicit MemRefDescriptor(Value descriptor);
   /// Builds IR creating an `undef` value of the descriptor type.
public:
   matchAndRewrite(Operation *op, ArrayRef<Value> operands,
                   ConversionPatternRewriter &rewriter) const override {
     return LLVM::detail::oneToOneRewrite(op, TargetOp::getOperationName(),
                                          operands, this->typeConverter,
                                          rewriter);
   }
 };
 
-/// Basic lowering implementation for rewriting from Ops to LLVM Dialect Ops
-/// with one result. This supports higher-dimensional vector types.
+/// Basic lowering implementation to rewrite Ops with just one result to the
+/// LLVM Dialect. This supports higher-dimensional vector types.
 template <typename SourceOp, typename TargetOp>
 class VectorConvertToLLVMPattern : public ConvertOpToLLVMPattern<SourceOp> {
 public:
   using ConvertOpToLLVMPattern<SourceOp>::ConvertOpToLLVMPattern;
   using Super = VectorConvertToLLVMPattern<SourceOp, TargetOp>;
 
   LogicalResult
   matchAndRewrite(Operation *op, ArrayRef<Value> operands,

mlir/include/mlir/Dialect/StandardOps/EDSC/Intrinsics.h

 namespace edsc {
 namespace intrinsics {
 
 using std_addi = ValueBuilder<AddIOp>;
 using std_addf = ValueBuilder<AddFOp>;
 using std_alloc = ValueBuilder<AllocOp>;
 using std_alloca = ValueBuilder<AllocaOp>;
 using std_call = OperationBuilder<CallOp>;
+using std_create_complex = ValueBuilder<CreateComplexOp>;
 using std_constant = ValueBuilder<ConstantOp>;
 using std_constant_float = ValueBuilder<ConstantFloatOp>;
 using std_constant_index = ValueBuilder<ConstantIndexOp>;
 using std_constant_int = ValueBuilder<ConstantIntOp>;
 using std_dealloc = OperationBuilder<DeallocOp>;
 using std_dim = ValueBuilder<DimOp>;
 using std_extract_element = ValueBuilder<ExtractElementOp>;
+using std_im = ValueBuilder<ImOp>;
 using std_index_cast = ValueBuilder<IndexCastOp>;
 using std_muli = ValueBuilder<MulIOp>;
 using std_mulf = ValueBuilder<MulFOp>;
 using std_memref_cast = ValueBuilder<MemRefCastOp>;
+using std_re = ValueBuilder<ReOp>;
 using std_ret = OperationBuilder<ReturnOp>;
 using std_select = ValueBuilder<SelectOp>;
 using std_load = ValueBuilder<LoadOp>;
 using std_splat = ValueBuilder<SplatOp>;
 using std_store = OperationBuilder<StoreOp>;
 using std_subi = ValueBuilder<SubIOp>;
 using std_sub_view = ValueBuilder<SubViewOp>;
 using std_tanh = ValueBuilder<TanhOp>;

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

def CmpIOp : Std_Op<"cmpi",
   let verifier = [{ return success(); }];
 
   let hasFolder = 1;
 
   let assemblyFormat = "$predicate `,` $lhs `,` $rhs attr-dict `:` type($lhs)";
 }
 
 //===----------------------------------------------------------------------===//
+// CreateComplexOp
+//===----------------------------------------------------------------------===//
+
+def CreateComplexOp : Std_Op<"create_complex",
+    [NoSideEffect,
+     AllTypesMatch<["real", "imaginary"]>,
+     TypesMatchWith<"complex element type matches real operand type",
+                    "complex", "real",
+                    "$_self.cast<ComplexType>().getElementType()">,
+     TypesMatchWith<"complex element type matches imaginary operand type",
+                    "complex", "imaginary",
+                    "$_self.cast<ComplexType>().getElementType()">]> {
+  let summary = "creates a complex number";
+  let description = [{
+    The `create_complex` operation creates a complex number from two
+    floating-point operands, the real and the imaginary part.
+
+    Example:
+
+    ```mlir
+    %a = create_complex %b, %c : complex<f32>
+    ```
+  }];
+
+  let arguments = (ins AnyFloat:$real, AnyFloat:$imaginary);
+  let results = (outs Complex<AnyFloat>:$complex);
+
+  let assemblyFormat = "$real `,` $imaginary attr-dict `:` type($complex)";
+
+  // `CreateComplexOp` is fully verified by its traits.
+  let verifier = ?;
+}
+
+//===----------------------------------------------------------------------===//
 // CondBranchOp
 //===----------------------------------------------------------------------===//
 
 def CondBranchOp : Std_Op<"cond_br",
     [AttrSizedOperandSegments,
      DeclareOpInterfaceMethods<BranchOpInterface, ["getSuccessorForOperands"]>,
      NoSideEffect, Terminator]> {
   let summary = "conditional branch operation";
let extraClassDeclaration = [{
     /// the operation.
     static bool areCastCompatible(Type a, Type b);
   }];
 
   let hasFolder = 0;
 }
 
 //===----------------------------------------------------------------------===//
+// ImOp
+//===----------------------------------------------------------------------===//
+
+def ImOp : Std_Op<"im",
+    [NoSideEffect,
+     TypesMatchWith<"complex element type matches result type",
+                    "complex", "imaginary",
+                    "$_self.cast<ComplexType>().getElementType()">]> {
+  let summary = "extracts the imaginary part of a complex number";
+  let description = [{
+    The `im` operation takes a single complex number as its operand and extracts
+    the imaginary part as a floating-point value.
+
+    Example:
+
+    ```mlir
+    %a = im %b : complex<f32>
+    ```
+  }];
+
+  let arguments = (ins Complex<AnyFloat>:$complex);
+  let results = (outs AnyFloat:$imaginary);
+
+  let assemblyFormat = "$complex attr-dict `:` type($complex)";
+
+  // `ImOp` is fully verified by its traits.
+  let verifier = ?;
+}
+
+//===----------------------------------------------------------------------===//
 // IndexCastOp
 //===----------------------------------------------------------------------===//
 
 def IndexCastOp : CastOp<"index_cast">, Arguments<(ins AnyType:$in)> {
   let summary = "cast between index and integer types";
   let description = [{
     Casts between integer scalars and 'index' scalars. Index is an integer of
     platform-specific bit width. If casting to a wider integer, the value is
"OpBuilder &builder, OperationState &result, Value tensor", [{
       build(builder, result, indexType, tensor);
     }]>];
 
   let hasFolder = 1;
   let assemblyFormat = "operands attr-dict `:` type(operands)";
 }
 
 //===----------------------------------------------------------------------===//
+// ReOp
+//===----------------------------------------------------------------------===//
+
+def ReOp : Std_Op<"re",
+    [NoSideEffect,
+     TypesMatchWith<"complex element type matches result type",
+                    "complex", "real",
+                    "$_self.cast<ComplexType>().getElementType()">]> {
+  let summary = "extracts the real part of a complex number";
+  let description = [{
+    The `re` operation takes a single complex number as its operand and extracts
+    the real part as a floating-point value.
+
+    Example:
+
+    ```mlir
+    %a = re %b : complex<f32>
+    ```
+  }];
+
+  let arguments = (ins Complex<AnyFloat>:$complex);
+  let results = (outs AnyFloat:$real);
+
+  let assemblyFormat = "$complex attr-dict `:` type($complex)";
+
+  // `ReOp` is fully verified by its traits.
+  let verifier = ?;
+}
+
+//===----------------------------------------------------------------------===//
 // RemFOp
 //===----------------------------------------------------------------------===//
 
 def RemFOp : FloatArithmeticOp<"remf"> {
   let summary = "floating point division remainder operation";
 }
 
 //===----------------------------------------------------------------------===//

mlir/lib/Conversion/StandardToLLVM/StandardToLLVM.cpp

: llvmDialect(ctx->getRegisteredDialect<LLVM::LLVMDialect>()),
       customizations(customs) {
   assert(llvmDialect && "LLVM IR dialect is not registered");
   module = &llvmDialect->getLLVMModule();
   if (customizations.indexBitwidth == kDeriveIndexBitwidthFromDataLayout)
     customizations.indexBitwidth =
         module->getDataLayout().getPointerSizeInBits();
 
   // Register conversions for the standard types.
+  addConversion([&](ComplexType type) { return convertComplexType(type); });
   addConversion([&](FloatType type) { return convertFloatType(type); });
   addConversion([&](FunctionType type) { return convertFunctionType(type); });
   addConversion([&](IndexType type) { return convertIndexType(type); });
   addConversion([&](IntegerType type) { return convertIntegerType(type); });
   addConversion([&](MemRefType type) { return convertMemRefType(type); });
   addConversion(
       [&](UnrankedMemRefType type) { return convertUnrankedMemRefType(type); });
   addConversion([&](VectorType type) { return convertVectorType(type); });
case mlir::StandardTypes::BF16: {
     return emitError(UnknownLoc::get(mlirContext), "unsupported type: BF16"),
            Type();
   }
   default:
     llvm_unreachable("non-float type in convertFloatType");
   }
 }
 
+// Convert a `ComplexType` to an LLVM type. The result is a complex number
+// struct with entries for the
+//   1. real part and for the
+//   2. imaginary part.
+static constexpr unsigned kRealPosInComplexNumberStruct = 0;
+static constexpr unsigned kImaginaryPosInComplexNumberStruct = 1;
+Type LLVMTypeConverter::convertComplexType(ComplexType type) {
+  auto elementType = convertType(type.getElementType()).cast<LLVM::LLVMType>();
+  return LLVM::LLVMType::getStructTy(llvmDialect, {elementType, elementType});
+}
+
 // Except for signatures, MLIR function types are converted into LLVM
 // pointer-to-function types.
 Type LLVMTypeConverter::convertFunctionType(FunctionType type) {
   SignatureConversion conversion(type.getNumInputs());
   LLVM::LLVMType converted =
       convertFunctionSignature(type, /*isVariadic=*/false, conversion);
   return converted.getPointerTo();
 }
ConvertToLLVMPattern::ConvertToLLVMPattern(StringRef rootOpName,
                                            LLVMTypeConverter &typeConverter_,
                                            PatternBenefit benefit)
     : ConversionPattern(rootOpName, benefit, context),
       typeConverter(typeConverter_) {}
 
 /*============================================================================*/
 /* StructBuilder implementation                                               */
 /*============================================================================*/
+
 StructBuilder::StructBuilder(Value v) : value(v) {
   assert(value != nullptr && "value cannot be null");
   structType = value.getType().dyn_cast<LLVM::LLVMType>();
   assert(structType && "expected llvm type");
 }
 
 Value StructBuilder::extractPtr(OpBuilder &builder, Location loc,
                                 unsigned pos) {
   Type type = structType.cast<LLVM::LLVMType>().getStructElementType(pos);
   return builder.create<LLVM::ExtractValueOp>(loc, type, value,
                                               builder.getI64ArrayAttr(pos));
 }
 
 void StructBuilder::setPtr(OpBuilder &builder, Location loc, unsigned pos,
                            Value ptr) {
   value = builder.create<LLVM::InsertValueOp>(loc, structType, value, ptr,
                                               builder.getI64ArrayAttr(pos));
 }
+
+/*============================================================================*/
+/* ComplexStructBuilder implementation                                        */
+/*============================================================================*/
+
+ComplexStructBuilder ComplexStructBuilder::undef(OpBuilder &builder,
+                                                 Location loc, Type type) {
+  Value val = builder.create<LLVM::UndefOp>(loc, type.cast<LLVM::LLVMType>());
+  return ComplexStructBuilder(val);
+}
+
+void ComplexStructBuilder::setReal(OpBuilder &builder, Location loc,
+                                   Value real) {
+  setPtr(builder, loc, kRealPosInComplexNumberStruct, real);
+}
+
+Value ComplexStructBuilder::real(OpBuilder &builder, Location loc) {
+  return extractPtr(builder, loc, kRealPosInComplexNumberStruct);
+}
+
+void ComplexStructBuilder ::setImaginary(OpBuilder &builder, Location loc,
+                                         Value imaginary) {
+  setPtr(builder, loc, kImaginaryPosInComplexNumberStruct, imaginary);
+}
+
+Value ComplexStructBuilder ::imaginary(OpBuilder &builder, Location loc) {
+  return extractPtr(builder, loc, kImaginaryPosInComplexNumberStruct);
+}
+
 /*============================================================================*/
 /* MemRefDescriptor implementation                                            */
 /*============================================================================*/
 
 /// Construct a helper for the given descriptor value.
 MemRefDescriptor::MemRefDescriptor(Value descriptor)
     : StructBuilder(descriptor) {
   assert(value != nullptr && "value cannot be null");

 using UnsignedDivIOpLowering =
     VectorConvertToLLVMPattern<UnsignedDivIOp, LLVM::UDivOp>;
 using UnsignedRemIOpLowering =
     VectorConvertToLLVMPattern<UnsignedRemIOp, LLVM::URemOp>;
 using UnsignedShiftRightOpLowering =
     OneToOneConvertToLLVMPattern<UnsignedShiftRightOp, LLVM::LShrOp>;
 using XOrOpLowering = VectorConvertToLLVMPattern<XOrOp, LLVM::XOrOp>;
 
+// Lowerings for operations on complex numbers, `CreateComplexOp`, `ReOp`, and
+// `ImOp`.
+
+struct CreateComplexOpLowering
+    : public ConvertOpToLLVMPattern<CreateComplexOp> {
+  using ConvertOpToLLVMPattern<CreateComplexOp>::ConvertOpToLLVMPattern;
+
+  LogicalResult
+  matchAndRewrite(Operation *op, ArrayRef<Value> operands,
+                  ConversionPatternRewriter &rewriter) const override {
+    auto complexOp = cast<CreateComplexOp>(op);
+    OperandAdaptor<CreateComplexOp> transformed(operands);
+
+    // Pack real and imaginary part in a complex number struct.
+    auto loc = op->getLoc();
+    auto structType = typeConverter.convertType(complexOp.getType());
+    auto complexStruct = ComplexStructBuilder::undef(rewriter, loc, structType);
+    complexStruct.setReal(rewriter, loc, transformed.real());
+    complexStruct.setImaginary(rewriter, loc, transformed.imaginary());
+
+    rewriter.replaceOp(op, {complexStruct});
+    return success();
+  }
+};
+
+struct ReOpLowering : public ConvertOpToLLVMPattern<ReOp> {
+  using ConvertOpToLLVMPattern<ReOp>::ConvertOpToLLVMPattern;
+
+  LogicalResult
+  matchAndRewrite(Operation *op, ArrayRef<Value> operands,
+                  ConversionPatternRewriter &rewriter) const override {
+    OperandAdaptor<ReOp> transformed(operands);
+
+    // Extract real part from the complex number struct.
+    ComplexStructBuilder complexStruct(transformed.complex());
+    Value real = complexStruct.real(rewriter, op->getLoc());
+    rewriter.replaceOp(op, real);
+
+    return success();
+  }
+};
+
+struct ImOpLowering : public ConvertOpToLLVMPattern<ImOp> {
+  using ConvertOpToLLVMPattern<ImOp>::ConvertOpToLLVMPattern;
+
+  LogicalResult
+  matchAndRewrite(Operation *op, ArrayRef<Value> operands,
+                  ConversionPatternRewriter &rewriter) const override {
+    OperandAdaptor<ImOp> transformed(operands);
+
+    // Extract imaginary part from the complex number struct.
+    ComplexStructBuilder complexStruct(transformed.complex());
+    Value imaginary = complexStruct.imaginary(rewriter, op->getLoc());
+    rewriter.replaceOp(op, imaginary);
+
+    return success();
+  }
+};
+
 // Check if the MemRefType `type` is supported by the lowering. We currently
 // only support memrefs with identity maps.
 static bool isSupportedMemRefType(MemRefType type) {
   return type.getAffineMaps().empty() ||
          llvm::all_of(type.getAffineMaps(),
                       [](AffineMap map) { return map.isIdentity(); });
 }
 
patterns.insert<
       CallOpLowering,
       CeilFOpLowering,
       CmpFOpLowering,
       CmpIOpLowering,
       CondBranchOpLowering,
       CopySignOpLowering,
       CosOpLowering,
       ConstLLVMOpLowering,
+      CreateComplexOpLowering,
       DialectCastOpLowering,
       DivFOpLowering,
       ExpOpLowering,
       Exp2OpLowering,
       GenericAtomicRMWOpLowering,
       LogOpLowering,
       Log10OpLowering,
       Log2OpLowering,
       FPExtLowering,
       FPTruncLowering,
+      ImOpLowering,
       IndexCastOpLowering,
       MulFOpLowering,
       MulIOpLowering,
       NegFOpLowering,
       OrOpLowering,
       PrefetchOpLowering,
+      ReOpLowering,
       RemFOpLowering,
       ReturnOpLowering,
       RsqrtOpLowering,
       SIToFPLowering,
       SelectOpLowering,
       ShiftLeftOpLowering,
       SignExtendIOpLowering,
       SignedDivIOpLowering,

mlir/test/Conversion/StandardToLLVM/convert-to-llvmir.mlir

^bb3: // pred: ^bb2
   br ^bb2(%2 : index)
 
 // CHECK:      ^bb4:  // pred: ^bb2
 // CHECK-NEXT:  llvm.return
 ^bb4: // pred: ^bb2
   return
 }
 
+// CHECK-LABEL: llvm.func @complex_numbers()
+// CHECK-NEXT:    %[[REAL0:.*]] = llvm.mlir.constant(1.200000e+00 : f32) : !llvm.float
+// CHECK-NEXT:    %[[IMAG0:.*]] = llvm.mlir.constant(3.400000e+00 : f32) : !llvm.float
+// CHECK-NEXT:    %[[CPLX0:.*]] = llvm.mlir.undef : !llvm<"{ float, float }">
+// CHECK-NEXT:    %[[CPLX1:.*]] = llvm.insertvalue %[[REAL0]], %[[CPLX0]][0] : !llvm<"{ float, float }">
+// CHECK-NEXT:    %[[CPLX2:.*]] = llvm.insertvalue %[[IMAG0]], %[[CPLX1]][1] : !llvm<"{ float, float }">
+// CHECK-NEXT:    %[[REAL1:.*]] = llvm.extractvalue %[[CPLX2:.*]][0] : !llvm<"{ float, float }">
+// CHECK-NEXT:    %[[IMAG1:.*]] = llvm.extractvalue %[[CPLX2:.*]][1] : !llvm<"{ float, float }">
+// CHECK-NEXT:    llvm.return
+func @complex_numbers() {
+  %real0 = constant 1.2 : f32
+  %imag0 = constant 3.4 : f32
+  %cplx2 = create_complex %real0, %imag0 : complex<f32>
+  %real1 = re %cplx2 : complex<f32>
+  %imag1 = im %cplx2 : complex<f32>
+  return
+}
+
 // CHECK-LABEL: func @simple_caller() {
 // CHECK-NEXT:  llvm.call @simple_loop() : () -> ()
 // CHECK-NEXT:  llvm.return
 // CHECK-NEXT: }
 func @simple_caller() {
 ^bb0:
   call @simple_loop() : () -> ()
   return

 ^bb12:  // pred: ^bb2
   return
 }
 
 // CHECK-LABEL: func @get_i64() -> !llvm.i64
 func @get_i64() -> (i64)
 // CHECK-LABEL: func @get_f32() -> !llvm.float
 func @get_f32() -> (f32)
+// CHECK-LABEL: func @get_c16() -> !llvm<"{ half, half }">
+func @get_c16() -> (complex<f16>)
+// CHECK-LABEL: func @get_c32() -> !llvm<"{ float, float }">
+func @get_c32() -> (complex<f32>)
+// CHECK-LABEL: func @get_c64() -> !llvm<"{ double, double }">
+func @get_c64() -> (complex<f64>)
 // CHECK-LABEL: func @get_memref() -> !llvm<"{ float*, float*, i64, [4 x i64], [4 x i64] }">
 // CHECK32-LABEL: func @get_memref() -> !llvm<"{ float*, float*, i32, [4 x i32], [4 x i32] }">
 func @get_memref() -> (memref<42x?x10x?xf32>)
 
 // CHECK-LABEL: func @multireturn() -> !llvm<"{ i64, float, { float*, float*, i64, [4 x i64], [4 x i64] } }"> {
 // CHECK32-LABEL: func @multireturn() -> !llvm<"{ i64, float, { float*, float*, i32, [4 x i32], [4 x i32] } }"> {
 func @multireturn() -> (i64, f32, memref<42x?x10x?xf32>) {
 ^bb0:

mlir/test/IR/core-ops.mlir

^bb42(%t: tensor<4x4x?xf32>, %f: f32, %i: i32, %idx : index, %j: i64, %half: f16):
   %i5 = subi %i4, %i4 : i32
 
   // CHECK: %12 = mulf %2, %2 : f32
   %f6 = mulf %f2, %f2 : f32
 
   // CHECK: %13 = muli %4, %4 : i32
   %i6 = muli %i2, %i2 : i32
 
+  // CHECK: %[[C0:.*]] = create_complex %[[F2:.*]], %[[F2]] : complex<f32>
+  %c0 = "std.create_complex"(%f2, %f2) : (f32, f32) -> complex<f32>
+
+  // CHECK: %[[C1:.*]] = create_complex %[[F2]], %[[F2]] : complex<f32>
+  %c1 = create_complex %f2, %f2 : complex<f32>
+
+  // CHECK: %[[REAL0:.*]] = re %[[CPLX0:.*]] : complex<f32>
+  %real0 = "std.re"(%c0) : (complex<f32>) -> f32
+
+  // CHECK: %[[REAL1:.*]] = re %[[CPLX0]] : complex<f32>
+  %real1 = re %c0 : complex<f32>
+
+  // CHECK: %[[IMAG0:.*]] = im %[[CPLX0]] : complex<f32>
+  %imag0 = "std.im"(%c0) : (complex<f32>) -> f32
+
+  // CHECK: %[[IMAG1:.*]] = im %[[CPLX0]] : complex<f32>
+  %imag1 = im %c0 : complex<f32>
+
   // CHECK: %c42_i32 = constant 42 : i32
   %x = "std.constant"(){value = 42 : i32} : () -> i32
 
   // CHECK: %c42_i32_0 = constant 42 : i32
   %7 = constant 42 : i32
 
   // CHECK: %c43 = constant {crazy = "std.foo"} 43 : index
   %8 = constant {crazy = "std.foo"} 43: index

mlir/test/IR/invalid-ops.mlir

 
 // -----
 
 "alloca_without_scoped_alloc_parent"() ( {
   std.alloca() : memref<1xf32>
   // expected-error@-1 {{requires an ancestor op with AutomaticAllocationScope trait}}
   return
 }) : () -> ()
+
+// -----
+
+func @complex_number_from_non_float_operands(%real: i32, %imag: i32) {
+  // expected-error@+1 {{'complex' must be complex type with floating-point elements, but got 'complex<i32>'}}
+  std.create_complex %real, %imag : complex<i32>
+  return
+}
+
+// -----
+
+// expected-note@+1 {{prior use here}}
+func @complex_number_from_different_float_types(%real: f32, %imag: f64) {
+  // expected-error@+1 {{expects different type than prior uses: 'f32' vs 'f64'}}
+  std.create_complex %real, %imag : complex<f32>
+  return
+}
+
+// -----
+
+// expected-note@+1 {{prior use here}}
+func @complex_number_from_incompatible_float_type(%real: f32, %imag: f32) {
+  // expected-error@+1 {{expects different type than prior uses: 'f64' vs 'f32'}}
+  std.create_complex %real, %imag : complex<f64>
+  return
+}
+
+// -----
+
+// expected-note@+1 {{prior use here}}
+func @real_part_from_incompatible_complex_type(%cplx: complex<f32>) {
+  // expected-error@+1 {{expects different type than prior uses: 'complex<f64>' vs 'complex<f32>'}}
+  std.re %cplx : complex<f64>
+  return
+}
+
+// -----
+
+// expected-note@+1 {{prior use here}}
+func @imaginary_part_from_incompatible_complex_type(%cplx: complex<f64>) {
+  // expected-error@+1 {{expects different type than prior uses: 'complex<f32>' vs 'complex<f64>'}}
+  std.re %cplx : complex<f32>
+  return
+}