[mlir][llvm] Opaque pointer support for atomic and call ops.

[gysit]

Summary

This revision adapts the printers and parsers of the LLVM Dialect
AtomicRMWOp, AtomicCmpXchgOp, CallOp, and InvokeOp to support both
opaque and typed pointers by printing the pointer types explicitly.
Previously, the printers and parser of these operations silently assumed
typed pointers. This assumption is problematic if a lowering or the
LLVM IR import produce LLVM Dialect with opaque pointers and the IR is
then printed and parsed, for example, when running mlir-translate. In
LLVM IR itself all tests with typed pointers are already gone. It is
thus important to start switching to opaque pointers.

This revision can be seen as a preparation step for the switch of the
LLVM Dialect to opaque pointers. Once printing and parsing works
seamlessly, all lowerings to LLVM Dialect can be switched to produce
opaque pointers. After a transition period, LLVM Dialect itself can by
simplified to support opaque pointers only.

[zero9178]

LGTM, especially with all the red from the assembly format :-)
I'll leave final approval to someone else however given the impact on syntax.

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

1693–1694

tiny nit in wording

[Dinistro]

Overall, this is looking good to me.
Please wait for the opinions of others as they might have other concerns.

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

548

Personally, prefer to just use the functional-type assembly format here: (!llvm.ptr, f32) -> (). Separating the types but only applying the result types to the second one seems a bit odd.

While this would force us to manually construct the functional types, some other parts of the parsing and printing could be simplified.

Any other opinions on that?

mlir/test/Target/LLVMIR/llvmir.mlir

1011–1012

it might make sense to write some of these tests with opaque pointers.

[gysit]

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

548

What I like about the current solution is that the function type of the called function is explicit. If we include the pointer in the arguments of the function type it is not directly related to the type of the called function anymore. Additionally, with the current solution the function type remains the same for direct and indirect calls.

I am thus sightly in favor of the current solution. Happy to hear other opinions.

[nikic]

Do I understand correctly that the additional type argument is needed as an intermediate step to distinguish typed/opaque pointer MLIR right now, but would not be necessary if only opaque pointers were supported?

[zero9178]

In D142884#4091093, @nikic wrote:

Do I understand correctly that the additional type argument is needed as an intermediate step to distinguish typed/opaque pointer MLIR right now, but would not be necessary if only opaque pointers were supported?

No since the pointers also have an address space which has to be printed. Unless there was any restriction on indirect call and those atomic ops that they only work in the default address space.

[ftynse]

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

548

Flattening everything into a single function type will look misleading here, especially during the transition period where we still have typed pointers: llvm.call %1(%0) : (!llvm.ptr, f32) -> () is naively read as calling a function that takes a pointer and an f32, given that the previous syntax would mean exactly that. The syntax with separate function type is okay.

[gysit]

Address reviewer comments and rebase.

Additionally, fix a problem with the export of indirect calls when using opaque
pointers. Instead of using the LLVM function type of the pointer, we now
recompute the LLVM function type given the argument and result types. The
current solution does not support indirect calls of variadic function, which
is not a regression compared to the old solution. Previously the same
LLVM function type reconstruction happened during parsing. A number of
additional tests tries to increase the test coverage for these corner
cases.

[gysit]

Fix comments.

[Dinistro]

LGTM module the comment.

Could you maybe also add a FIXME for the varargs issues you mentioned in the previous comment?

mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp

1088

Didn't we discuss that void should never be explicitly used in the IR? Wouldn't it be cleaner to return an error if there is void to stop further misuse?

[gysit]

Address last review comments.

mlir/include/mlir/Dialect/LLVMIR/LLVMOps.td

def LLVM_LandingpadOp : LLVM_Op<"landingpad"> {
   let hasCustomAssemblyFormat = 1;
   let hasVerifier = 1;
 }
 
 //===----------------------------------------------------------------------===//
 // CallOp
 //===----------------------------------------------------------------------===//
 
+// FIXME: Add a type attribute that carries the LLVM function type to support
+// indirect calls to variadic functions. The type attribute is necessary to
+// distinguish normal and variadic arguments.
 def LLVM_CallOp : LLVM_Op<"call",
                           [DeclareOpInterfaceMethods<FastmathFlagsInterface>,
                            DeclareOpInterfaceMethods<CallOpInterface>,
                            DeclareOpInterfaceMethods<SymbolUserOpInterface>]> {
   let summary = "Call to an LLVM function.";
   let description = [{
     In LLVM IR, functions may return either 0 or 1 value. LLVM IR dialect
     implements this behavior by providing a variadic `call` operation for 0- and
     1-result functions. Even though MLIR supports multi-result functions, LLVM
     IR dialect disallows them.
 
     The `call` instruction supports both direct and indirect calls. Direct calls
     start with a function name (`@`-prefixed) and indirect calls start with an
     SSA value (`%`-prefixed). The direct callee, if present, is stored as a
-    function attribute `callee`. The trailing type of the instruction is always
-    the MLIR function type, which may be different from the indirect callee that
-    has the wrapped LLVM IR function type.
+    function attribute `callee`. The trailing type list contains the optional
+    indirect callee type and the MLIR function type, which differs from the
+    LLVM function type that uses a explicit void type to model functions that do
+    not return a value.
 
     Examples:
 
     ```mlir
     // Direct call without arguments and with one result.
     %0 = llvm.call @foo() : () -> (f32)
 
     // Direct call with arguments and without a result.
     llvm.call @bar(%0) : (f32) -> ()
 
     // Indirect call with an argument and without a result.
-    llvm.call %1(%0) : (f32) -> ()
+    llvm.call %1(%0) : !llvm.ptr, (f32) -> ()
     ```
   }];
 
   let arguments = (ins OptionalAttr<FlatSymbolRefAttr>:$callee,
                        Variadic<LLVM_Type>,
                        DefaultValuedAttr<LLVM_FastmathFlagsAttr,
                                          "{}">:$fastmathFlags,
                        OptionalAttr<ElementsAttr>:$branch_weights);

 
 // Atomic operations.
 //
 
 def LLVM_AtomicRMWType : AnyTypeOf<[LLVM_AnyFloat, AnyInteger]>;
 
 // FIXME: Need to add alignment and syncscope attribute to MLIR atomicrmw
 // operation.
-def LLVM_AtomicRMWOp : LLVM_Op<"atomicrmw"> {
+def LLVM_AtomicRMWOp : LLVM_Op<"atomicrmw", [
+      TypesMatchWith<"result #0 and operand #1 have the same type",
+                     "val", "res", "$_self">]> {
   let arguments = (ins AtomicBinOp:$bin_op,
                    LLVM_PointerTo<LLVM_AtomicRMWType>:$ptr,
                    LLVM_AtomicRMWType:$val, AtomicOrdering:$ordering);
   let results = (outs LLVM_AtomicRMWType:$res);
-  let hasCustomAssemblyFormat = 1;
   let hasVerifier = 1;
+  let assemblyFormat = [{
+    $bin_op $ptr `,` $val $ordering
+    attr-dict `:` qualified(type($ptr)) `,` type($val)
+  }];
   string llvmInstName = "AtomicRMW";
   string llvmBuilder = [{
     $res = builder.CreateAtomicRMW(getLLVMAtomicBinOp($bin_op), $ptr, $val,
                                    llvm::MaybeAlign(),
                                    getLLVMAtomicOrdering($ordering));
   }];
   string mlirBuilder = [{
     auto *atomicInst = cast<llvm::AtomicRMWInst>(inst);
     $res = $_builder.create<LLVM::AtomicRMWOp>($_location, $_resultType,
         getLLVMAtomicBinOp(atomicInst->getOperation()), $ptr, $val,
         getLLVMAtomicOrdering(atomicInst->getOrdering()));
   }];
   // Only $ptr and $val are llvm instruction operands.
   list<int> llvmArgIndices = [-1, 0, 1, -1];
 }
 
 def LLVM_AtomicCmpXchgType : AnyTypeOf<[AnyInteger, LLVM_AnyPointer]>;
-def LLVM_AtomicCmpXchgResultType : Type<And<[
-  LLVM_AnyStruct.predicate,
-  CPred<"$_self.cast<::mlir::LLVM::LLVMStructType>().getBody().size() == 2">,
-  SubstLeaves<"$_self",
-              "$_self.cast<::mlir::LLVM::LLVMStructType>().getBody()[0]",
-              LLVM_AtomicCmpXchgType.predicate>,
-  SubstLeaves<"$_self",
-              "$_self.cast<::mlir::LLVM::LLVMStructType>().getBody()[1]",
-              I1.predicate>]>,
- "an LLVM struct type with any integer or pointer followed by a single-bit "
- "integer">;
 
 // FIXME: Need to add alignment attribute to MLIR cmpxchg operation.
-def LLVM_AtomicCmpXchgOp : LLVM_Op<"cmpxchg"> {
+def LLVM_AtomicCmpXchgOp : LLVM_Op<"cmpxchg", [
+      TypesMatchWith<"operand #1 and operand #2 have the same type",
+                     "val", "cmp", "$_self">,
+      TypesMatchWith<"result #0 has an LLVM struct type consisting of "
+                     "the type of operand #2 and a bool",
+                     "val", "res", "getValAndBoolStructType($_self)">]> {
   let arguments = (ins LLVM_PointerTo<LLVM_AtomicCmpXchgType>:$ptr,
                    LLVM_AtomicCmpXchgType:$cmp, LLVM_AtomicCmpXchgType:$val,
                    AtomicOrdering:$success_ordering,
                    AtomicOrdering:$failure_ordering);
-  let results = (outs LLVM_AtomicCmpXchgResultType:$res);
-  let hasCustomAssemblyFormat = 1;
+  let results = (outs LLVM_AnyStruct:$res);
   let hasVerifier = 1;
+  let assemblyFormat = [{
+    $ptr `,` $cmp `,` $val $success_ordering $failure_ordering
+    attr-dict `:` qualified(type($ptr)) `,` type($val)
+  }];
   string llvmInstName = "AtomicCmpXchg";
   string llvmBuilder = [{
     $res = builder.CreateAtomicCmpXchg($ptr, $cmp, $val, llvm::MaybeAlign(),
                    getLLVMAtomicOrdering($success_ordering),
                    getLLVMAtomicOrdering($failure_ordering));
   }];
   string mlirBuilder = [{
     auto *cmpXchgInst = cast<llvm::AtomicCmpXchgInst>(inst);

mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp

ParseResult StoreOp::parse(OpAsmParser &parser, OperationState &result) {
 
   if (parser.resolveOperand(value, operandType, result.operands) ||
       parser.resolveOperand(addr, type, result.operands))
     return failure();
 
   return success();
 }
 
+//===----------------------------------------------------------------------===//
+// CallOp
+//===----------------------------------------------------------------------===//
+
+void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
+                   StringRef callee, ValueRange args) {
+  build(builder, state, results, builder.getStringAttr(callee), args);
+}
+
+void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
+                   StringAttr callee, ValueRange args) {
+  build(builder, state, results, SymbolRefAttr::get(callee), args, nullptr,
+        nullptr);
+}
+
+void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
+                   FlatSymbolRefAttr callee, ValueRange args) {
+  build(builder, state, results, callee, args, nullptr, nullptr);
+}
+
+void CallOp::build(OpBuilder &builder, OperationState &state, LLVMFuncOp func,
+                   ValueRange args) {
+  SmallVector<Type> results;
+  Type resultType = func.getFunctionType().getReturnType();
+  if (!resultType.isa<LLVM::LLVMVoidType>())
+    results.push_back(resultType);
+  build(builder, state, results, SymbolRefAttr::get(func), args, nullptr,
+        nullptr);
+}
+
+CallInterfaceCallable CallOp::getCallableForCallee() {
+  // Direct call.
+  if (FlatSymbolRefAttr calleeAttr = getCalleeAttr())
+    return calleeAttr;
+  // Indirect call, callee Value is the first operand.
+  return getOperand(0);
+}
+
+Operation::operand_range CallOp::getArgOperands() {
+  return getOperands().drop_front(getCallee().has_value() ? 0 : 1);
+}
+
+LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
+  if (getNumResults() > 1)
+    return emitOpError("must have 0 or 1 result");
+
+  // Type for the callee, we'll get it differently depending if it is a direct
+  // or indirect call.
+  Type fnType;
+
+  bool isIndirect = false;
+
+  // If this is an indirect call, the callee attribute is missing.
+  FlatSymbolRefAttr calleeName = getCalleeAttr();
+  if (!calleeName) {
+    isIndirect = true;
+    if (!getNumOperands())
+      return emitOpError(
+          "must have either a `callee` attribute or at least an operand");
+    auto ptrType = getOperand(0).getType().dyn_cast<LLVMPointerType>();
+    if (!ptrType)
+      return emitOpError("indirect call expects a pointer as callee: ")
+             << getOperand(0).getType();
+
+    if (ptrType.isOpaque())
+      return success();
+
+    fnType = ptrType.getElementType();
+  } else {
+    Operation *callee =
+        symbolTable.lookupNearestSymbolFrom(*this, calleeName.getAttr());
+    if (!callee)
+      return emitOpError()
+             << "'" << calleeName.getValue()
+             << "' does not reference a symbol in the current scope";
+    auto fn = dyn_cast<LLVMFuncOp>(callee);
+    if (!fn)
+      return emitOpError() << "'" << calleeName.getValue()
+                           << "' does not reference a valid LLVM function";
+
+    fnType = fn.getFunctionType();
+  }
+
+  LLVMFunctionType funcType = fnType.dyn_cast<LLVMFunctionType>();
+  if (!funcType)
+    return emitOpError("callee does not have a functional type: ") << fnType;
+
+  // Indirect variadic function calls are not supported since the translation to
+  // LLVM IR reconstructs the LLVM function type from the argument and result
+  // types. An additional type attribute that stores the LLVM function type
+  // would be needed to distinguish normal and variadic function arguments.
+  // TODO: Support indirect calls to variadic function pointers.
+  if (isIndirect && funcType.isVarArg())
+    return emitOpError()
+           << "indirect calls to variadic functions are not supported";
+
+  // Verify that the operand and result types match the callee.
+
+  if (!funcType.isVarArg() &&
+      funcType.getNumParams() != (getNumOperands() - isIndirect))
+    return emitOpError() << "incorrect number of operands ("
+                         << (getNumOperands() - isIndirect)
+                         << ") for callee (expecting: "
+                         << funcType.getNumParams() << ")";
+
+  if (funcType.getNumParams() > (getNumOperands() - isIndirect))
+    return emitOpError() << "incorrect number of operands ("
+                         << (getNumOperands() - isIndirect)
+                         << ") for varargs callee (expecting at least: "
+                         << funcType.getNumParams() << ")";
+
+  for (unsigned i = 0, e = funcType.getNumParams(); i != e; ++i)
+    if (getOperand(i + isIndirect).getType() != funcType.getParamType(i))
+      return emitOpError() << "operand type mismatch for operand " << i << ": "
+                           << getOperand(i + isIndirect).getType()
+                           << " != " << funcType.getParamType(i);
+
+  if (getNumResults() == 0 &&
+      !funcType.getReturnType().isa<LLVM::LLVMVoidType>())
+    return emitOpError() << "expected function call to produce a value";
+
+  if (getNumResults() != 0 &&
+      funcType.getReturnType().isa<LLVM::LLVMVoidType>())
+    return emitOpError()
+           << "calling function with void result must not produce values";
+
+  if (getNumResults() > 1)
+    return emitOpError()
+           << "expected LLVM function call to produce 0 or 1 result";
+
+  if (getNumResults() && getResult().getType() != funcType.getReturnType())
+    return emitOpError() << "result type mismatch: " << getResult().getType()
+                         << " != " << funcType.getReturnType();
+
+  return success();
+}
+
+void CallOp::print(OpAsmPrinter &p) {
+  auto callee = getCallee();
+  bool isDirect = callee.has_value();
+
+  // Print the direct callee if present as a function attribute, or an indirect
+  // callee (first operand) otherwise.
+  p << ' ';
+  if (isDirect)
+    p.printSymbolName(callee.value());
+  else
+    p << getOperand(0);
+
+  auto args = getOperands().drop_front(isDirect ? 0 : 1);
+  p << '(' << args << ')';
+  p.printOptionalAttrDict(processFMFAttr((*this)->getAttrs()), {"callee"});
+
+  p << " : ";
+  if (!isDirect)
+    p << getOperand(0).getType() << ", ";
+
+  // Reconstruct the function MLIR function type from operand and result types.
+  p.printFunctionalType(args.getTypes(), getResultTypes());
+}
+
+/// Parses the type of a call operation and resolves the operands if the parsing
+/// succeeds. Returns failure otherwise.
+static ParseResult parseCallTypeAndResolveOperands(
+    OpAsmParser &parser, OperationState &result, bool isDirect,
+    ArrayRef<OpAsmParser::UnresolvedOperand> operands) {
+  SMLoc trailingTypesLoc = parser.getCurrentLocation();
+  SmallVector<Type> types;
+  if (parser.parseColonTypeList(types))
+    return failure();
+
+  if (isDirect && types.size() != 1)
+    return parser.emitError(trailingTypesLoc,
+                            "expected direct call to have 1 trailing type");
+  if (!isDirect && types.size() != 2)
+    return parser.emitError(trailingTypesLoc,
+                            "expected indirect call to have 2 trailing types");
+
+  auto funcType = types.pop_back_val().dyn_cast<FunctionType>();
+  if (!funcType)
+    return parser.emitError(trailingTypesLoc,
+                            "expected trailing function type");
+  if (funcType.getNumResults() > 1)
+    return parser.emitError(trailingTypesLoc,
+                            "expected function with 0 or 1 result");
+  if (funcType.getNumResults() == 1 &&
+      funcType.getResult(0).isa<LLVM::LLVMVoidType>())
+    return parser.emitError(trailingTypesLoc,
+                            "expected a non-void result type");
+
+  // The head element of the types list matches the callee type for
+  // indirect calls, while the types list is emtpy for direct calls.
+  // Append the function input types to resolve the call operation
+  // operands.
+  llvm::append_range(types, funcType.getInputs());
+  if (parser.resolveOperands(operands, types, parser.getNameLoc(),
+                             result.operands))
+    return failure();
+  if (funcType.getNumResults() != 0)
+    result.addTypes(funcType.getResults());
+
+  return success();
+}
+
+/// Parses an optional function pointer operand before the call argument list
+/// for indirect calls, or stops parsing at the function identifier otherwise.
+static ParseResult parseOptionalCallFuncPtr(
+    OpAsmParser &parser,
+    SmallVectorImpl<OpAsmParser::UnresolvedOperand> &operands) {
+  OpAsmParser::UnresolvedOperand funcPtrOperand;
+  OptionalParseResult parseResult = parser.parseOptionalOperand(funcPtrOperand);
+  if (parseResult.has_value()) {
+    if (failed(*parseResult))
+      return *parseResult;
+    operands.push_back(funcPtrOperand);
+  }
+  return success();
+}
+
+// <operation> ::= `llvm.call` (function-id | ssa-use)`(` ssa-use-list `)`
+//                             attribute-dict? `:` (type `,`)? function-type
+ParseResult CallOp::parse(OpAsmParser &parser, OperationState &result) {
+  SymbolRefAttr funcAttr;
+  SmallVector<OpAsmParser::UnresolvedOperand> operands;
+
+  // Parse a function pointer for indirect calls.
+  if (parseOptionalCallFuncPtr(parser, operands))
+    return failure();
+  bool isDirect = operands.empty();
+
+  // Parse a function identifier for direct calls.
+  if (isDirect)
+    if (parser.parseAttribute(funcAttr, "callee", result.attributes))
+      return failure();
+
+  // Parse the function arguments.
+  if (parser.parseOperandList(operands, OpAsmParser::Delimiter::Paren) ||
+      parser.parseOptionalAttrDict(result.attributes))
+    return failure();
+
+  // Parse the trailing type list and resolve the operands.
+  return parseCallTypeAndResolveOperands(parser, result, isDirect, operands);
+}
+
 ///===---------------------------------------------------------------------===//
 /// LLVM::InvokeOp
 ///===---------------------------------------------------------------------===//
 
 SuccessorOperands InvokeOp::getSuccessorOperands(unsigned index) {
   assert(index < getNumSuccessors() && "invalid successor index");
   return SuccessorOperands(index == 0 ? getNormalDestOperandsMutable()
                                       : getUnwindDestOperandsMutable());
void InvokeOp::print(OpAsmPrinter &p) {
   p << '(' << getOperands().drop_front(isDirect ? 0 : 1) << ')';
   p << " to ";
   p.printSuccessorAndUseList(getNormalDest(), getNormalDestOperands());
   p << " unwind ";
   p.printSuccessorAndUseList(getUnwindDest(), getUnwindDestOperands());
 
   p.printOptionalAttrDict((*this)->getAttrs(),
                           {InvokeOp::getOperandSegmentSizeAttr(), "callee"});
+
   p << " : ";
+  if (!isDirect)
+    p << getOperand(0).getType() << ", ";
   p.printFunctionalType(llvm::drop_begin(getOperandTypes(), isDirect ? 0 : 1),
                         getResultTypes());
 }
 
-/// <operation> ::= `llvm.invoke` (function-id | ssa-use) `(` ssa-use-list `)`
-///                  `to` bb-id (`[` ssa-use-and-type-list `]`)?
-///                  `unwind` bb-id (`[` ssa-use-and-type-list `]`)?
-///                  attribute-dict? `:` function-type
+// <operation> ::= `llvm.invoke` (function-id | ssa-use)
+//                  `(` ssa-use-list `)`
+//                  `to` bb-id (`[` ssa-use-and-type-list `]`)?
+//                  `unwind` bb-id (`[` ssa-use-and-type-list `]`)?
+//                  attribute-dict? `:` (type `,`)? function-type
 ParseResult InvokeOp::parse(OpAsmParser &parser, OperationState &result) {
   SmallVector<OpAsmParser::UnresolvedOperand, 8> operands;
-  FunctionType funcType;
   SymbolRefAttr funcAttr;
-  SMLoc trailingTypeLoc;
   Block *normalDest, *unwindDest;
   SmallVector<Value, 4> normalOperands, unwindOperands;
   Builder &builder = parser.getBuilder();
 
-  // Parse an operand list that will, in practice, contain 0 or 1 operand.  In
-  // case of an indirect call, there will be 1 operand before `(`.  In case of a
-  // direct call, there will be no operands and the parser will stop at the
-  // function identifier without complaining.
-  if (parser.parseOperandList(operands))
+  // Parse a function pointer for indirect calls.
+  if (parseOptionalCallFuncPtr(parser, operands))
     return failure();
   bool isDirect = operands.empty();
 
-  // Optionally parse a function identifier.
+  // Parse a function identifier for direct calls.
   if (isDirect && parser.parseAttribute(funcAttr, "callee", result.attributes))
     return failure();
 
+  // Parse the function arguments.
   if (parser.parseOperandList(operands, OpAsmParser::Delimiter::Paren) ||
       parser.parseKeyword("to") ||
       parser.parseSuccessorAndUseList(normalDest, normalOperands) ||
       parser.parseKeyword("unwind") ||
       parser.parseSuccessorAndUseList(unwindDest, unwindOperands) ||
-      parser.parseOptionalAttrDict(result.attributes) || parser.parseColon() ||
-      parser.getCurrentLocation(&trailingTypeLoc) || parser.parseType(funcType))
-    return failure();
-
-  if (isDirect) {
-    // Make sure types match.
-    if (parser.resolveOperands(operands, funcType.getInputs(),
-                               parser.getNameLoc(), result.operands))
-      return failure();
-    result.addTypes(funcType.getResults());
-  } else {
-    // Construct the LLVM IR Dialect function type that the first operand
-    // should match.
-    if (funcType.getNumResults() > 1)
-      return parser.emitError(trailingTypeLoc,
-                              "expected function with 0 or 1 result");
-
-    Type llvmResultType;
-    if (funcType.getNumResults() == 0) {
-      llvmResultType = LLVM::LLVMVoidType::get(builder.getContext());
-    } else {
-      llvmResultType = funcType.getResult(0);
-      if (!isCompatibleType(llvmResultType))
-        return parser.emitError(trailingTypeLoc,
-                                "expected result to have LLVM type");
-    }
-
-    SmallVector<Type, 8> argTypes;
-    argTypes.reserve(funcType.getNumInputs());
-    for (Type ty : funcType.getInputs()) {
-      if (isCompatibleType(ty))
-        argTypes.push_back(ty);
-      else
-        return parser.emitError(trailingTypeLoc,
-                                "expected LLVM types as inputs");
-    }
+      parser.parseOptionalAttrDict(result.attributes))
 
-    auto llvmFuncType = LLVM::LLVMFunctionType::get(llvmResultType, argTypes);
-    auto wrappedFuncType = LLVM::LLVMPointerType::get(llvmFuncType);
-
-    auto funcArguments = llvm::ArrayRef(operands).drop_front();
-
-    // Make sure that the first operand (indirect callee) matches the wrapped
-    // LLVM IR function type, and that the types of the other call operands
-    // match the types of the function arguments.
-    if (parser.resolveOperand(operands[0], wrappedFuncType, result.operands) ||
-        parser.resolveOperands(funcArguments, funcType.getInputs(),
-                               parser.getNameLoc(), result.operands))
-      return failure();
+  // Parse the trailing type list and resolve the function operands.
+  if (parseCallTypeAndResolveOperands(parser, result, isDirect, operands))
 
-    result.addTypes(llvmResultType);
-  }
   result.addSuccessors({normalDest, unwindDest});
   result.addOperands(normalOperands);
   result.addOperands(unwindOperands);
 
   result.addAttribute(InvokeOp::getOperandSegmentSizeAttr(),
                       builder.getDenseI32ArrayAttr(
                           {static_cast<int32_t>(operands.size()),
                            static_cast<int32_t>(normalOperands.size()),
p << '(' << (isArrayTy ? "filter " : "catch ") << value << " : "
       << value.getType() << ") ";
   }
 
   p.printOptionalAttrDict((*this)->getAttrs(), {"cleanup"});
 
   p << ": " << getType();
 }
 
-/// <operation> ::= `llvm.landingpad` `cleanup`?
-///                 ((`catch` | `filter`) operand-type ssa-use)* attribute-dict?
+// <operation> ::= `llvm.landingpad` `cleanup`?
+//                 ((`catch` | `filter`) operand-type ssa-use)* attribute-dict?
 ParseResult LandingpadOp::parse(OpAsmParser &parser, OperationState &result) {
   // Check for cleanup
   if (succeeded(parser.parseOptionalKeyword("cleanup")))
     result.addAttribute("cleanup", parser.getBuilder().getUnitAttr());
 
   // Parse clauses with types
   while (succeeded(parser.parseOptionalLParen()) &&
          (succeeded(parser.parseOptionalKeyword("filter")) ||
ParseResult LandingpadOp::parse(OpAsmParser &parser, OperationState &result) {
   if (parser.parseColon() || parser.parseType(type))
     return failure();
 
   result.addTypes(type);
   return success();
 }
 
 //===----------------------------------------------------------------------===//
-// CallOp
-//===----------------------------------------------------------------------===//
-
-void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
-                   StringRef callee, ValueRange args) {
-  build(builder, state, results, builder.getStringAttr(callee), args);
-}
-
-void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
-                   StringAttr callee, ValueRange args) {
-  build(builder, state, results, SymbolRefAttr::get(callee), args, nullptr,
-        nullptr);
-}
-
-void CallOp::build(OpBuilder &builder, OperationState &state, TypeRange results,
-                   FlatSymbolRefAttr callee, ValueRange args) {
-  build(builder, state, results, callee, args, nullptr, nullptr);
-}
-
-void CallOp::build(OpBuilder &builder, OperationState &state, LLVMFuncOp func,
-                   ValueRange args) {
-  SmallVector<Type> results;
-  Type resultType = func.getFunctionType().getReturnType();
-  if (!resultType.isa<LLVM::LLVMVoidType>())
-    results.push_back(resultType);
-  build(builder, state, results, SymbolRefAttr::get(func), args, nullptr,
-        nullptr);
-}
-
-CallInterfaceCallable CallOp::getCallableForCallee() {
-  // Direct call.
-  if (FlatSymbolRefAttr calleeAttr = getCalleeAttr())
-    return calleeAttr;
-  // Indirect call, callee Value is the first operand.
-  return getOperand(0);
-}
-
-Operation::operand_range CallOp::getArgOperands() {
-  return getOperands().drop_front(getCallee().has_value() ? 0 : 1);
-}
-
-LogicalResult CallOp::verifySymbolUses(SymbolTableCollection &symbolTable) {
-  if (getNumResults() > 1)
-    return emitOpError("must have 0 or 1 result");
-
-  // Type for the callee, we'll get it differently depending if it is a direct
-  // or indirect call.
-  Type fnType;
-
-  bool isIndirect = false;
-
-  // If this is an indirect call, the callee attribute is missing.
-  FlatSymbolRefAttr calleeName = getCalleeAttr();
-  if (!calleeName) {
-    isIndirect = true;
-    if (!getNumOperands())
-      return emitOpError(
-          "must have either a `callee` attribute or at least an operand");
-    auto ptrType = getOperand(0).getType().dyn_cast<LLVMPointerType>();
-    if (!ptrType)
-      return emitOpError("indirect call expects a pointer as callee: ")
-             << ptrType;
-
-    if (ptrType.isOpaque())
-      return success();
-
-    fnType = ptrType.getElementType();
-  } else {
-    Operation *callee =
-        symbolTable.lookupNearestSymbolFrom(*this, calleeName.getAttr());
-    if (!callee)
-      return emitOpError()
-             << "'" << calleeName.getValue()
-             << "' does not reference a symbol in the current scope";
-    auto fn = dyn_cast<LLVMFuncOp>(callee);
-    if (!fn)
-      return emitOpError() << "'" << calleeName.getValue()
-                           << "' does not reference a valid LLVM function";
-
-    fnType = fn.getFunctionType();
-  }
-
-  LLVMFunctionType funcType = fnType.dyn_cast<LLVMFunctionType>();
-  if (!funcType)
-    return emitOpError("callee does not have a functional type: ") << fnType;
-
-  // Verify that the operand and result types match the callee.
-
-  if (!funcType.isVarArg() &&
-      funcType.getNumParams() != (getNumOperands() - isIndirect))
-    return emitOpError() << "incorrect number of operands ("
-                         << (getNumOperands() - isIndirect)
-                         << ") for callee (expecting: "
-                         << funcType.getNumParams() << ")";
-
-  if (funcType.getNumParams() > (getNumOperands() - isIndirect))
-    return emitOpError() << "incorrect number of operands ("
-                         << (getNumOperands() - isIndirect)
-                         << ") for varargs callee (expecting at least: "
-                         << funcType.getNumParams() << ")";
-
-  for (unsigned i = 0, e = funcType.getNumParams(); i != e; ++i)
-    if (getOperand(i + isIndirect).getType() != funcType.getParamType(i))
-      return emitOpError() << "operand type mismatch for operand " << i << ": "
-                           << getOperand(i + isIndirect).getType()
-                           << " != " << funcType.getParamType(i);
-
-  if (getNumResults() == 0 &&
-      !funcType.getReturnType().isa<LLVM::LLVMVoidType>())
-    return emitOpError() << "expected function call to produce a value";
-
-  if (getNumResults() != 0 &&
-      funcType.getReturnType().isa<LLVM::LLVMVoidType>())
-    return emitOpError()
-           << "calling function with void result must not produce values";
-
-  if (getNumResults() > 1)
-    return emitOpError()
-           << "expected LLVM function call to produce 0 or 1 result";
-
-  if (getNumResults() && getResult().getType() != funcType.getReturnType())
-    return emitOpError() << "result type mismatch: " << getResult().getType()
-                         << " != " << funcType.getReturnType();
-
-  return success();
-}
-
-void CallOp::print(OpAsmPrinter &p) {
-  auto callee = getCallee();
-  bool isDirect = callee.has_value();
-
-  // Print the direct callee if present as a function attribute, or an indirect
-  // callee (first operand) otherwise.
-  p << ' ';
-  if (isDirect)
-    p.printSymbolName(callee.value());
-  else
-    p << getOperand(0);
-
-  auto args = getOperands().drop_front(isDirect ? 0 : 1);
-  p << '(' << args << ')';
-  p.printOptionalAttrDict(processFMFAttr((*this)->getAttrs()), {"callee"});
-
-  // Reconstruct the function MLIR function type from operand and result types.
-  p << " : ";
-  p.printFunctionalType(args.getTypes(), getResultTypes());
-}
-
-// <operation> ::= `llvm.call` (function-id | ssa-use) `(` ssa-use-list `)`
-//                 attribute-dict? `:` function-type
-ParseResult CallOp::parse(OpAsmParser &parser, OperationState &result) {
-  SmallVector<OpAsmParser::UnresolvedOperand, 8> operands;
-  Type type;
-  SymbolRefAttr funcAttr;
-  SMLoc trailingTypeLoc;
-
-  // Parse an operand list that will, in practice, contain 0 or 1 operand.  In
-  // case of an indirect call, there will be 1 operand before `(`.  In case of a
-  // direct call, there will be no operands and the parser will stop at the
-  // function identifier without complaining.
-  if (parser.parseOperandList(operands))
-    return failure();
-  bool isDirect = operands.empty();
-
-  // Optionally parse a function identifier.
-  if (isDirect)
-    if (parser.parseAttribute(funcAttr, "callee", result.attributes))
-      return failure();
-
-  if (parser.parseOperandList(operands, OpAsmParser::Delimiter::Paren) ||
-      parser.parseOptionalAttrDict(result.attributes) || parser.parseColon() ||
-      parser.getCurrentLocation(&trailingTypeLoc) || parser.parseType(type))
-    return failure();
-
-  auto funcType = type.dyn_cast<FunctionType>();
-  if (!funcType)
-    return parser.emitError(trailingTypeLoc, "expected function type");
-  if (funcType.getNumResults() > 1)
-    return parser.emitError(trailingTypeLoc,
-                            "expected function with 0 or 1 result");
-  if (isDirect) {
-    // Make sure types match.
-    if (parser.resolveOperands(operands, funcType.getInputs(),
-                               parser.getNameLoc(), result.operands))
-      return failure();
-    if (funcType.getNumResults() != 0 &&
-        !funcType.getResult(0).isa<LLVM::LLVMVoidType>())
-      result.addTypes(funcType.getResults());
-  } else {
-    Builder &builder = parser.getBuilder();
-    Type llvmResultType;
-    if (funcType.getNumResults() == 0) {
-      llvmResultType = LLVM::LLVMVoidType::get(builder.getContext());
-    } else {
-      llvmResultType = funcType.getResult(0);
-      if (!isCompatibleType(llvmResultType))
-        return parser.emitError(trailingTypeLoc,
-                                "expected result to have LLVM type");
-    }
-
-    SmallVector<Type, 8> argTypes;
-    argTypes.reserve(funcType.getNumInputs());
-    for (int i = 0, e = funcType.getNumInputs(); i < e; ++i) {
-      auto argType = funcType.getInput(i);
-      if (!isCompatibleType(argType))
-        return parser.emitError(trailingTypeLoc,
-                                "expected LLVM types as inputs");
-      argTypes.push_back(argType);
-    }
-    auto llvmFuncType = LLVM::LLVMFunctionType::get(llvmResultType, argTypes);
-    auto wrappedFuncType = LLVM::LLVMPointerType::get(llvmFuncType);
-
-    auto funcArguments =
-        ArrayRef<OpAsmParser::UnresolvedOperand>(operands).drop_front();
-
-    // Make sure that the first operand (indirect callee) matches the wrapped
-    // LLVM IR function type, and that the types of the other call operands
-    // match the types of the function arguments.
-    if (parser.resolveOperand(operands[0], wrappedFuncType, result.operands) ||
-        parser.resolveOperands(funcArguments, funcType.getInputs(),
-                               parser.getNameLoc(), result.operands))
-      return failure();
-
-    if (!llvmResultType.isa<LLVM::LLVMVoidType>())
-      result.addTypes(llvmResultType);
-  }
-
-  return success();
-}
-
-//===----------------------------------------------------------------------===//
 // ExtractValueOp
 //===----------------------------------------------------------------------===//
 
 /// Extract the type at `position` in the LLVM IR aggregate type
 /// `containerType`. Each element of `position` is an index into a nested
 /// aggregate type. Return the resulting type or emit an error.
 static Type getInsertExtractValueElementType(
     function_ref<InFlightDiagnostic(StringRef)> emitError, Type containerType,

 OpFoldResult LLVM::ConstantOp::fold(FoldAdaptor) { return getValue(); }
 
 //===----------------------------------------------------------------------===//
 // Utility functions for parsing atomic ops
 //===----------------------------------------------------------------------===//
 
 // Helper function to parse a keyword into the specified attribute named by
 // `attrName`. The keyword must match one of the string values defined by the
-// AtomicBinOp enum. The resulting I64 attribute is added to the `result`
-// state.
-static ParseResult parseAtomicBinOp(OpAsmParser &parser, OperationState &result,
-                                    StringRef attrName) {
-  SMLoc loc;
-  StringRef keyword;
-  if (parser.getCurrentLocation(&loc) || parser.parseKeyword(&keyword))
-    return failure();
-
-  // Replace the keyword `keyword` with an integer attribute.
-  auto kind = symbolizeAtomicBinOp(keyword);
-  if (!kind) {
-    return parser.emitError(loc)
-           << "'" << keyword << "' is an incorrect value of the '" << attrName
-           << "' attribute";
-  }
-
-  auto value = static_cast<int64_t>(*kind);
-  auto attr = parser.getBuilder().getI64IntegerAttr(value);
-  result.addAttribute(attrName, attr);
-
-  return success();
-}
-
-// Helper function to parse a keyword into the specified attribute named by
-// `attrName`. The keyword must match one of the string values defined by the
 // AtomicOrdering enum. The resulting I64 attribute is added to the `result`
 // state.
 static ParseResult parseAtomicOrdering(OpAsmParser &parser,
                                        OperationState &result,
                                        StringRef attrName) {
   SMLoc loc;
   StringRef ordering;
   if (parser.getCurrentLocation(&loc) || parser.parseKeyword(&ordering))
static ParseResult parseAtomicOrdering(OpAsmParser &parser,
   auto value = static_cast<int64_t>(*kind);
   auto attr = parser.getBuilder().getI64IntegerAttr(value);
   result.addAttribute(attrName, attr);
 
   return success();
 }
 
 //===----------------------------------------------------------------------===//
-// Printer, parser and verifier for LLVM::AtomicRMWOp.
+// Verifier for LLVM::AtomicRMWOp.
 //===----------------------------------------------------------------------===//
 
-void AtomicRMWOp::print(OpAsmPrinter &p) {
-  p << ' ' << stringifyAtomicBinOp(getBinOp()) << ' ' << getPtr() << ", "
-    << getVal() << ' ' << stringifyAtomicOrdering(getOrdering()) << ' ';
-  p.printOptionalAttrDict((*this)->getAttrs(), {"bin_op", "ordering"});
-  p << " : " << getRes().getType();
-}
-
-// <operation> ::= `llvm.atomicrmw` keyword ssa-use `,` ssa-use keyword
-//                 attribute-dict? `:` type
-ParseResult AtomicRMWOp::parse(OpAsmParser &parser, OperationState &result) {
-  Type type;
-  OpAsmParser::UnresolvedOperand ptr, val;
-  if (parseAtomicBinOp(parser, result, "bin_op") || parser.parseOperand(ptr) ||
-      parser.parseComma() || parser.parseOperand(val) ||
-      parseAtomicOrdering(parser, result, "ordering") ||
-      parser.parseOptionalAttrDict(result.attributes) ||
-      parser.parseColonType(type) ||
-      parser.resolveOperand(ptr, LLVM::LLVMPointerType::get(type),
-                            result.operands) ||
-      parser.resolveOperand(val, type, result.operands))
-    return failure();
-
-  result.addTypes(type);
-  return success();
-}
-
 LogicalResult AtomicRMWOp::verify() {
   auto ptrType = getPtr().getType().cast<LLVM::LLVMPointerType>();
   auto valType = getVal().getType();
   if (!ptrType.isOpaque() && valType != ptrType.getElementType())
     return emitOpError("expected LLVM IR element type for operand #0 to "
                        "match type for operand #1");
-  auto resType = getRes().getType();
-  if (resType != valType)
-    return emitOpError(
-        "expected LLVM IR result type to match type for operand #1");
   if (getBinOp() == AtomicBinOp::fadd || getBinOp() == AtomicBinOp::fsub) {
     if (!mlir::LLVM::isCompatibleFloatingPointType(valType))
       return emitOpError("expected LLVM IR floating point type");
   } else if (getBinOp() == AtomicBinOp::xchg) {
     auto intType = valType.dyn_cast<IntegerType>();
     unsigned intBitWidth = intType ? intType.getWidth() : 0;
     if (intBitWidth != 8 && intBitWidth != 16 && intBitWidth != 32 &&
         intBitWidth != 64 && !valType.isa<BFloat16Type>() &&
if (static_cast<unsigned>(getOrdering()) <
     return emitOpError() << "expected at least '"
                          << stringifyAtomicOrdering(AtomicOrdering::monotonic)
                          << "' ordering";
 
   return success();
 }
 
 //===----------------------------------------------------------------------===//
-// Printer, parser and verifier for LLVM::AtomicCmpXchgOp.
+// Verifier for LLVM::AtomicCmpXchgOp.
 //===----------------------------------------------------------------------===//
 
-void AtomicCmpXchgOp::print(OpAsmPrinter &p) {
-  p << ' ' << getPtr() << ", " << getCmp() << ", " << getVal() << ' '
-    << stringifyAtomicOrdering(getSuccessOrdering()) << ' '
-    << stringifyAtomicOrdering(getFailureOrdering());
-  p.printOptionalAttrDict((*this)->getAttrs(),
-                          {"success_ordering", "failure_ordering"});
-  p << " : " << getVal().getType();
-}
-
-// <operation> ::= `llvm.cmpxchg` ssa-use `,` ssa-use `,` ssa-use
-//                 keyword keyword attribute-dict? `:` type
-ParseResult AtomicCmpXchgOp::parse(OpAsmParser &parser,
-                                   OperationState &result) {
-  auto &builder = parser.getBuilder();
-  Type type;
-  OpAsmParser::UnresolvedOperand ptr, cmp, val;
-  if (parser.parseOperand(ptr) || parser.parseComma() ||
-      parser.parseOperand(cmp) || parser.parseComma() ||
-      parser.parseOperand(val) ||
-      parseAtomicOrdering(parser, result, "success_ordering") ||
-      parseAtomicOrdering(parser, result, "failure_ordering") ||
-      parser.parseOptionalAttrDict(result.attributes) ||
-      parser.parseColonType(type) ||
-      parser.resolveOperand(ptr, LLVM::LLVMPointerType::get(type),
-                            result.operands) ||
-      parser.resolveOperand(cmp, type, result.operands) ||
-      parser.resolveOperand(val, type, result.operands))
-    return failure();
-
-  auto boolType = IntegerType::get(builder.getContext(), 1);
-  auto resultType =
-      LLVMStructType::getLiteral(builder.getContext(), {type, boolType});
-  result.addTypes(resultType);
-
-  return success();
+/// Returns an LLVM struct type that contains a value type and a boolean type.
+static LLVMStructType getValAndBoolStructType(Type valType) {
+  auto boolType = IntegerType::get(valType.getContext(), 1);
+  return LLVMStructType::getLiteral(valType.getContext(), {valType, boolType});
 }
 
 LogicalResult AtomicCmpXchgOp::verify() {
   auto ptrType = getPtr().getType().cast<LLVM::LLVMPointerType>();
   if (!ptrType)
     return emitOpError("expected LLVM IR pointer type for operand #0");
-  auto cmpType = getCmp().getType();
   auto valType = getVal().getType();
-  if (cmpType != valType)
-    return emitOpError("expected both value operands to have the same type");
-  if (!ptrType.isOpaque() && cmpType != ptrType.getElementType())
+  if (!ptrType.isOpaque() && valType != ptrType.getElementType())
     return emitOpError("expected LLVM IR element type for operand #0 to "
                        "match type for all other operands");
   auto intType = valType.dyn_cast<IntegerType>();
   unsigned intBitWidth = intType ? intType.getWidth() : 0;
   if (!valType.isa<LLVMPointerType>() && intBitWidth != 8 &&
       intBitWidth != 16 && intBitWidth != 32 && intBitWidth != 64 &&
       !valType.isa<BFloat16Type>() && !valType.isa<Float16Type>() &&
       !valType.isa<Float32Type>() && !valType.isa<Float64Type>())

mlir/lib/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.cpp

convertOperationImpl(Operation &opInst, llvm::IRBuilderBase &builder,
 
   llvm::IRBuilder<>::FastMathFlagGuard fmfGuard(builder);
   if (auto fmf = dyn_cast<FastmathFlagsInterface>(opInst))
     builder.setFastMathFlags(getFastmathFlags(fmf));
 
 #include "mlir/Dialect/LLVMIR/LLVMConversions.inc"
 #include "mlir/Dialect/LLVMIR/LLVMIntrinsicConversions.inc"
 
+  // Helper function to reconstruct the function type for an indirect call given
+  // the result and argument types. The function cannot reconstruct the type of
+  // variadic functions since the call operation does not carry enough
+  // information to distinguish normal and variadic arguments. Supporting
+  // indirect variadic calls requires an additional type attribute on the call
+  // operation that stores the LLVM function type of the callee.
+  // TODO: Support indirect calls to variadic function pointers.
+  auto getCalleeFunctionType = [&](TypeRange resultTypes, ValueRange args) {
+    Type resultType = resultTypes.empty()
+                          ? LLVMVoidType::get(opInst.getContext())
+                          : resultTypes.front();
+    return llvm::cast<llvm::FunctionType>(moduleTranslation.convertType(
+        LLVMFunctionType::get(opInst.getContext(), resultType,
+                              llvm::to_vector(args.getTypes()), false)));
+  };
+
   // Emit function calls.  If the "callee" attribute is present, this is a
   // direct function call and we also need to look up the remapped function
   // itself.  Otherwise, this is an indirect call and the callee is the first
   // operand, look it up as a normal value.
   if (auto callOp = dyn_cast<LLVM::CallOp>(opInst)) {
     auto operands = moduleTranslation.lookupValues(callOp.getOperands());
     ArrayRef<llvm::Value *> operandsRef(operands);
     llvm::CallInst *call;
     if (auto attr = callOp.getCalleeAttr()) {
       call = builder.CreateCall(
           moduleTranslation.lookupFunction(attr.getValue()), operandsRef);
     } else {
-      auto calleeType =
-          callOp->getOperands().front().getType().cast<LLVMPointerType>();
-      auto *calleeFunctionType = cast<llvm::FunctionType>(
-          moduleTranslation.convertType(calleeType.getElementType()));
-      call = builder.CreateCall(calleeFunctionType, operandsRef.front(),
-                                operandsRef.drop_front());
+      call = builder.CreateCall(getCalleeFunctionType(callOp.getResultTypes(),
+                                                      callOp.getArgOperands()),
+                                operandsRef.front(), operandsRef.drop_front());
     }
     llvm::MDNode *branchWeights =
         convertBranchWeights(callOp.getBranchWeights(), moduleTranslation);
     if (branchWeights)
       call->setMetadata(llvm::LLVMContext::MD_prof, branchWeights);
     // If the called function has a result, remap the corresponding value.  Note
     // that LLVM IR dialect CallOp has either 0 or 1 result.
     if (opInst.getNumResults() != 0) {
if (auto invOp = dyn_cast<LLVM::InvokeOp>(opInst)) {
     ArrayRef<llvm::Value *> operandsRef(operands);
     llvm::Instruction *result;
     if (auto attr = opInst.getAttrOfType<FlatSymbolRefAttr>("callee")) {
       result = builder.CreateInvoke(
           moduleTranslation.lookupFunction(attr.getValue()),
           moduleTranslation.lookupBlock(invOp.getSuccessor(0)),
           moduleTranslation.lookupBlock(invOp.getSuccessor(1)), operandsRef);
     } else {
-      auto calleeType =
-          invOp.getCalleeOperands().front().getType().cast<LLVMPointerType>();
-      auto *calleeFunctionType = cast<llvm::FunctionType>(
-          moduleTranslation.convertType(calleeType.getElementType()));
       result = builder.CreateInvoke(
-          calleeFunctionType, operandsRef.front(),
+          getCalleeFunctionType(invOp.getResultTypes(), invOp.getArgOperands()),
+          operandsRef.front(),
           moduleTranslation.lookupBlock(invOp.getSuccessor(0)),
           moduleTranslation.lookupBlock(invOp.getSuccessor(1)),
           operandsRef.drop_front());
     }
     llvm::MDNode *branchWeights =
         convertBranchWeights(invOp.getBranchWeights(), moduleTranslation);
     if (branchWeights)
       result->setMetadata(llvm::LLVMContext::MD_prof, branchWeights);

mlir/test/Conversion/FuncToLLVM/convert-funcs.mlir

 // CHECK-LABEL: llvm.func @body(i32)
 func.func private @body(i32)
 
 // CHECK-LABEL: llvm.func @indirect_const_call
 // CHECK-SAME: (%[[ARG0:.*]]: i32) {
 func.func @indirect_const_call(%arg0: i32) {
 // CHECK-NEXT: %[[ADDR:.*]] = llvm.mlir.addressof @body : !llvm.ptr<func<void (i32)>>
   %0 = constant @body : (i32) -> ()
-// CHECK-NEXT:  llvm.call %[[ADDR]](%[[ARG0:.*]]) : (i32) -> ()
+// CHECK-NEXT:  llvm.call %[[ADDR]](%[[ARG0:.*]]) : !llvm.ptr<func<void (i32)>>, (i32) -> ()
   call_indirect %0(%arg0) : (i32) -> ()
 // CHECK-NEXT:  llvm.return
   return
 }
 
 // CHECK-LABEL: llvm.func @indirect_call(%arg0: !llvm.ptr<func<i32 (f32)>>, %arg1: f32) -> i32 {
 func.func @indirect_call(%arg0: (f32) -> i32, %arg1: f32) -> i32 {
-// CHECK-NEXT:  %0 = llvm.call %arg0(%arg1) : (f32) -> i32
+// CHECK-NEXT:  %0 = llvm.call %arg0(%arg1) : !llvm.ptr<func<i32 (f32)>>, (f32) -> i32
   %0 = call_indirect %arg0(%arg1) : (f32) -> i32
 // CHECK-NEXT:  llvm.return %0 : i32
   return %0 : i32
 }
 
 func.func @variadic_func(%arg0: i32) attributes { "func.varargs" = true } {
   return
 }

mlir/test/Conversion/MemRefToLLVM/memref-to-llvm.mlir

func.func @generic_atomic_rmw(%I : memref<10xi32>, %i : index) {
   %x = memref.generic_atomic_rmw %I[%i] : memref<10xi32> {
     ^bb0(%old_value : i32):
       memref.atomic_yield %old_value : i32
   }
   // CHECK: [[init:%.*]] = llvm.load %{{.*}} : !llvm.ptr<i32>
   // CHECK-NEXT: llvm.br ^bb1([[init]] : i32)
   // CHECK-NEXT: ^bb1([[loaded:%.*]]: i32):
   // CHECK-NEXT: [[pair:%.*]] = llvm.cmpxchg %{{.*}}, [[loaded]], [[loaded]]
-  // CHECK-SAME:                    acq_rel monotonic : i32
+  // CHECK-SAME:                    acq_rel monotonic : !llvm.ptr<i32>, i32
   // CHECK-NEXT: [[new:%.*]] = llvm.extractvalue [[pair]][0]
   // CHECK-NEXT: [[ok:%.*]] = llvm.extractvalue [[pair]][1]
   // CHECK-NEXT: llvm.cond_br [[ok]], ^bb2, ^bb1([[new]] : i32)
   llvm.return
 }
 
 // -----
 

mlir/test/Dialect/LLVMIR/invalid.mlir

 
 func.func @store_malformed_elem_type(%foo: !llvm.ptr, %bar: f32) {
   // expected-error@+1 {{expected non-function type}}
   llvm.store %bar, %foo : !llvm.ptr, "f32"
 }
 
 // -----
 
-func.func @call_non_function_type(%callee : !llvm.func<i8 (i8)>, %arg : i8) {
-  // expected-error@+1 {{expected function type}}
-  llvm.call %callee(%arg) : !llvm.func<i8 (i8)>
+func.func @invalid_call() {
+  // expected-error@+1 {{'llvm.call' op must have either a `callee` attribute or at least an operand}}
+  "llvm.call"() : () -> ()
   llvm.return
 }
 
 // -----
 
-func.func @invalid_call() {
-  // expected-error@+1 {{'llvm.call' op must have either a `callee` attribute or at least an operand}}
-  "llvm.call"() : () -> ()
+func.func @call_missing_ptr_type(%callee : !llvm.func<i8 (i8)>, %arg : i8) {
+  // expected-error@+1 {{expected indirect call to have 2 trailing types}}
+  llvm.call %callee(%arg) : (i8) -> (i8)
+  llvm.return
+}
+
+// -----
+
+func.func private @standard_func_callee()
+
+func.func @call_missing_ptr_type(%arg : i8) {
+  // expected-error@+1 {{expected direct call to have 1 trailing type}}
+  llvm.call @standard_func_callee(%arg) : !llvm.ptr, (i8) -> (i8)
   llvm.return
 }
 
 // -----
 
-func.func @call_non_function_type(%callee : !llvm.func<i8 (i8)>, %arg : i8) {
-  // expected-error@+1 {{expected function type}}
-  llvm.call %callee(%arg) : !llvm.func<i8 (i8)>
+func.func @call_non_pointer_type(%callee : !llvm.func<i8 (i8)>, %arg : i8) {
+  // expected-error@+1 {{indirect call expects a pointer as callee: '!llvm.func<i8 (i8)>'}}
+  llvm.call %callee(%arg) : !llvm.func<i8 (i8)>, (i8) -> (i8)
+  llvm.return
+}
+
+// -----
+
+func.func @call_non_function_type(%callee : !llvm.ptr, %arg : i8) {
+  // expected-error@+1 {{expected trailing function type}}
+  llvm.call %callee(%arg) : !llvm.ptr, !llvm.func<i8 (i8)>
+  llvm.return
+}
+
+// -----
+
+func.func @call_void_result_type(%callee : !llvm.ptr, %arg : i8) {
+  // expected-error@+1 {{expected a non-void result type}}
+  llvm.call %callee(%arg) : !llvm.ptr, (i8) -> (!llvm.void)
   llvm.return
 }
 
 // -----
 
 func.func @call_unknown_symbol() {
   // expected-error@+1 {{'llvm.call' op 'missing_callee' does not reference a symbol in the current scope}}
   llvm.call @missing_callee() : () -> ()
   llvm.return
 }
 
 // -----
 
+func.func @call_variadic(%callee : !llvm.ptr<func<i8 (i8, ...)>>, %arg : i8) {
+  // expected-error@+1 {{indirect calls to variadic functions are not supported}}
+  llvm.call %callee(%arg) : !llvm.ptr<func<i8 (i8, ...)>>, (i8) -> (i8)
+  llvm.return
+}
+
+// -----
+
 func.func private @standard_func_callee()
 
 func.func @call_non_llvm() {
   // expected-error@+1 {{'llvm.call' op 'standard_func_callee' does not reference a valid LLVM function}}
   llvm.call @standard_func_callee() : () -> ()
   llvm.return
 }
 
 // -----
 
-func.func @call_non_llvm_indirect(%arg0 : tensor<*xi32>) {
+func.func @call_non_llvm_arg(%arg0 : tensor<*xi32>) {
   // expected-error@+1 {{'llvm.call' op operand #0 must be LLVM dialect-compatible type}}
   "llvm.call"(%arg0) : (tensor<*xi32>) -> ()
   llvm.return
 }
 
 // -----
 
+func.func @call_non_llvm_res(%callee : !llvm.ptr) {
+  // expected-error@+1 {{'llvm.call' op result #0 must be LLVM dialect-compatible type}}
+  llvm.call %callee() : !llvm.ptr, () -> (tensor<*xi32>)
+  llvm.return
+}
+
+// -----
+
 llvm.func @callee_func(i8) -> ()
 
 func.func @callee_arg_mismatch(%arg0 : i32) {
   // expected-error@+1 {{'llvm.call' op operand type mismatch for operand 0: 'i32' != 'i8'}}
   llvm.call @callee_func(%arg0) : (i32) -> ()
   llvm.return
 }
 

 func.func @indirect_callee_return_mismatch(%callee : !llvm.ptr<func<i8()>>) {
   // expected-error@+1 {{'llvm.call' op result type mismatch: 'i32' != 'i8'}}
   "llvm.call"(%callee) : (!llvm.ptr<func<i8()>>) -> (i32)
   llvm.return
 }
 
 // -----
 
-func.func @call_too_many_results(%callee : () -> (i32,i32)) {
+func.func @call_too_many_results(%callee : !llvm.ptr) {
   // expected-error@+1 {{expected function with 0 or 1 result}}
-  llvm.call %callee() : () -> (i32, i32)
-  llvm.return
-}
-
-// -----
-
-func.func @call_non_llvm_result(%callee : () -> (tensor<*xi32>)) {
-  // expected-error@+1 {{expected result to have LLVM type}}
-  llvm.call %callee() : () -> (tensor<*xi32>)
-  llvm.return
-}
-
-// -----
-
-func.func @call_non_llvm_input(%callee : (tensor<*xi32>) -> (), %arg : tensor<*xi32>) {
-  // expected-error@+1 {{expected LLVM types as inputs}}
-  llvm.call %callee(%arg) : (tensor<*xi32>) -> ()
+  llvm.call %callee() : !llvm.ptr, () -> (i32, i32)
   llvm.return
 }
 
 // -----
 
 llvm.func @void_func_result(%arg0: i32) {
   // expected-error@below {{expected no operands}}
   // expected-note@above {{when returning from function}}
func.func @atomicrmw_expected_ptr(%f32 : f32) {
   %0 = "llvm.atomicrmw"(%f32, %f32) {bin_op=11, ordering=1} : (f32, f32) -> f32
   llvm.return
 }
 
 // -----
 
 func.func @atomicrmw_mismatched_operands(%f32_ptr : !llvm.ptr<f32>, %i32 : i32) {
   // expected-error@+1 {{expected LLVM IR element type for operand #0 to match type for operand #1}}
-  %0 = "llvm.atomicrmw"(%f32_ptr, %i32) {bin_op=11, ordering=1} : (!llvm.ptr<f32>, i32) -> f32
+  %0 = "llvm.atomicrmw"(%f32_ptr, %i32) {bin_op=11, ordering=1} : (!llvm.ptr<f32>, i32) -> i32
   llvm.return
 }
 
 // -----
 
 func.func @atomicrmw_mismatched_operands(%f32_ptr : !llvm.ptr<f32>, %f32 : f32) {
-  // expected-error@+1 {{expected LLVM IR result type to match type for operand #1}}
+  // expected-error@+1 {{op failed to verify that result #0 and operand #1 have the same type}}
   %0 = "llvm.atomicrmw"(%f32_ptr, %f32) {bin_op=11, ordering=1} : (!llvm.ptr<f32>, f32) -> i32
   llvm.return
 }
 
 // -----
 
 func.func @atomicrmw_expected_float(%i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // expected-error@+1 {{expected LLVM IR floating point type}}
-  %0 = llvm.atomicrmw fadd %i32_ptr, %i32 unordered : i32
+  %0 = llvm.atomicrmw fadd %i32_ptr, %i32 unordered : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // -----
 
 func.func @atomicrmw_unexpected_xchg_type(%i1_ptr : !llvm.ptr<i1>, %i1 : i1) {
   // expected-error@+1 {{unexpected LLVM IR type for 'xchg' bin_op}}
-  %0 = llvm.atomicrmw xchg %i1_ptr, %i1 unordered : i1
+  %0 = llvm.atomicrmw xchg %i1_ptr, %i1 unordered : !llvm.ptr<i1>, i1
   llvm.return
 }
 
 // -----
 
 func.func @atomicrmw_expected_int(%f32_ptr : !llvm.ptr<f32>, %f32 : f32) {
   // expected-error@+1 {{expected LLVM IR integer type}}
-  %0 = llvm.atomicrmw max %f32_ptr, %f32 unordered : f32
+  %0 = llvm.atomicrmw max %f32_ptr, %f32 unordered : !llvm.ptr<f32>, f32
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_expected_ptr(%f32_ptr : !llvm.ptr<f32>, %f32 : f32) {
   // expected-error@+1 {{op operand #0 must be LLVM pointer to integer or LLVM pointer type}}
   %0 = "llvm.cmpxchg"(%f32, %f32, %f32) {success_ordering=2,failure_ordering=2} : (f32, f32, f32) -> !llvm.struct<(f32, i1)>
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_mismatched_operands(%i64_ptr : !llvm.ptr<i64>, %i32 : i32) {
   // expected-error@+1 {{expected LLVM IR element type for operand #0 to match type for all other operands}}
   %0 = "llvm.cmpxchg"(%i64_ptr, %i32, %i32) {success_ordering=2,failure_ordering=2} : (!llvm.ptr<i64>, i32, i32) -> !llvm.struct<(i32, i1)>
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_mismatched_value_operands(%ptr : !llvm.ptr, %i32 : i32, %i64 : i64) {
-  // expected-error@+1 {{expected both value operands to have the same type}}
+  // expected-error@+1 {{op failed to verify that operand #1 and operand #2 have the same type}}
   %0 = "llvm.cmpxchg"(%ptr, %i32, %i64) {success_ordering=2,failure_ordering=2} : (!llvm.ptr, i32, i64) -> !llvm.struct<(i32, i1)>
   llvm.return
 }
+
+// -----
+
+func.func @cmpxchg_mismatched_result(%ptr : !llvm.ptr, %i64 : i64) {
+  // expected-error@+1 {{op failed to verify that result #0 has an LLVM struct type consisting of the type of operand #2 and a bool}}
+  %0 = "llvm.cmpxchg"(%ptr, %i64, %i64) {success_ordering=2,failure_ordering=2} : (!llvm.ptr, i64, i64) -> !llvm.struct<(i64, i64)>
+  llvm.return
+}
+
 // -----
 
 func.func @cmpxchg_unexpected_type(%i1_ptr : !llvm.ptr<i1>, %i1 : i1) {
   // expected-error@+1 {{unexpected LLVM IR type}}
-  %0 = llvm.cmpxchg %i1_ptr, %i1, %i1 monotonic monotonic : i1
+  %0 = llvm.cmpxchg %i1_ptr, %i1, %i1 monotonic monotonic : !llvm.ptr<i1>, i1
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_at_least_monotonic_success(%i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // expected-error@+1 {{ordering must be at least 'monotonic'}}
-  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 unordered monotonic : i32
+  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 unordered monotonic : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_at_least_monotonic_failure(%i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // expected-error@+1 {{ordering must be at least 'monotonic'}}
-  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 monotonic unordered : i32
+  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 monotonic unordered : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_failure_release(%i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // expected-error@+1 {{failure ordering cannot be 'release' or 'acq_rel'}}
-  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 acq_rel release : i32
+  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 acq_rel release : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // -----
 
 func.func @cmpxchg_failure_acq_rel(%i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // expected-error@+1 {{failure ordering cannot be 'release' or 'acq_rel'}}
-  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 acq_rel acq_rel : i32
+  %0 = llvm.cmpxchg %i32_ptr, %i32, %i32 acq_rel acq_rel : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // -----
 
 llvm.func @foo(i32) -> i32
 llvm.func @__gxx_personality_v0(...) -> i32
 

mlir/test/Dialect/LLVMIR/roundtrip.mlir

// CHECK-NEXT:  %{{.*}} = llvm.bitcast %[[ALLOCA]] : !llvm.ptr<f64> to !llvm.ptr<i64>
   llvm.store %15, %13 : !llvm.ptr<f64>
   %16 = llvm.bitcast %13 : !llvm.ptr<f64> to !llvm.ptr<i64>
 
 // Function call-related operations.
 //
 // CHECK: %[[STRUCT:.*]] = llvm.call @foo(%[[I32]]) : (i32) -> !llvm.struct<(i32, f64, i32)>
 // CHECK: %[[VALUE:.*]] = llvm.extractvalue %[[STRUCT]][0] : !llvm.struct<(i32, f64, i32)>
 // CHECK: %[[NEW_STRUCT:.*]] = llvm.insertvalue %[[VALUE]], %[[STRUCT]][2] : !llvm.struct<(i32, f64, i32)>
-// CHECK: %[[FUNC:.*]] = llvm.mlir.addressof @foo : !llvm.ptr<func<struct<(i32, f64, i32)> (i32)>>
-// CHECK: %{{.*}} = llvm.call %[[FUNC]](%[[I32]]) : (i32) -> !llvm.struct<(i32, f64, i32)>
+// CHECK: %[[FUNC:.*]] = llvm.mlir.addressof @foo : !llvm.ptr
+// CHECK: %{{.*}} = llvm.call %[[FUNC]](%[[I32]]) : !llvm.ptr, (i32) -> !llvm.struct<(i32, f64, i32)>
   %17 = llvm.call @foo(%arg0) : (i32) -> !llvm.struct<(i32, f64, i32)>
   %18 = llvm.extractvalue %17[0] : !llvm.struct<(i32, f64, i32)>
   %19 = llvm.insertvalue %18, %17[2] : !llvm.struct<(i32, f64, i32)>
-  %20 = llvm.mlir.addressof @foo : !llvm.ptr<func<struct<(i32, f64, i32)> (i32)>>
-  %21 = llvm.call %20(%arg0) : (i32) -> !llvm.struct<(i32, f64, i32)>
-
+  %20 = llvm.mlir.addressof @foo : !llvm.ptr
+  %21 = llvm.call %20(%arg0) : !llvm.ptr, (i32) -> !llvm.struct<(i32, f64, i32)>
 
 // Terminator operations and their successors.
 //
 // CHECK: llvm.br ^[[BB1:.*]]
   llvm.br ^bb1
 
 // CHECK: ^[[BB1]]
 ^bb1:
func.func @null() {
   // CHECK: llvm.mlir.null : !llvm.ptr<i8>
   %0 = llvm.mlir.null : !llvm.ptr<i8>
   // CHECK: llvm.mlir.null : !llvm.ptr<struct<(ptr<func<void (i32, ptr<func<void ()>>)>>, i64)>>
   %1 = llvm.mlir.null : !llvm.ptr<struct<(ptr<func<void (i32, ptr<func<void ()>>)>>, i64)>>
   llvm.return
 }
 
 // CHECK-LABEL: @atomicrmw
-func.func @atomicrmw(%ptr : !llvm.ptr<f32>, %val : f32) {
-  // CHECK: llvm.atomicrmw fadd %{{.*}}, %{{.*}} monotonic : f32
-  %0 = llvm.atomicrmw fadd %ptr, %val monotonic : f32
+func.func @atomicrmw(%ptr : !llvm.ptr, %val : f32) {
+  // CHECK: llvm.atomicrmw fadd %{{.*}}, %{{.*}} monotonic : !llvm.ptr, f32
+  %0 = llvm.atomicrmw fadd %ptr, %val monotonic : !llvm.ptr, f32
   llvm.return
 }
 
 // CHECK-LABEL: @cmpxchg
-func.func @cmpxchg(%ptr : !llvm.ptr<i32>, %cmp : i32, %new : i32) {
-  // CHECK: llvm.cmpxchg %{{.*}}, %{{.*}}, %{{.*}} acq_rel monotonic : i32
-  %0 = llvm.cmpxchg %ptr, %cmp, %new acq_rel monotonic : i32
+func.func @cmpxchg(%ptr : !llvm.ptr, %cmp : i32, %new : i32) {
+  // CHECK: llvm.cmpxchg %{{.*}}, %{{.*}}, %{{.*}} acq_rel monotonic : !llvm.ptr, i32
+  %0 = llvm.cmpxchg %ptr, %cmp, %new acq_rel monotonic : !llvm.ptr, i32
   llvm.return
 }
 
 llvm.mlir.global external constant @_ZTIi() : !llvm.ptr<i8>
 llvm.func @bar(!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>)
 llvm.func @__gxx_personality_v0(...) -> i32
 
 // CHECK-LABEL: @invokeLandingpad
^bb2:
   llvm.return %7 : i32
 
 // CHECK: ^[[BB3:.*]]:
 // CHECK:   llvm.invoke @bar(%[[a8]], %[[a6]], %[[a4]]) to ^[[BB2]] unwind ^[[BB1]] : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> ()
 ^bb3:
   llvm.invoke @bar(%8, %6, %4) to ^bb2 unwind ^bb1 : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> ()
 
 // CHECK: ^[[BB4:.*]]:
-// CHECK:   llvm.return %[[a0]] : i32
+// CHECK: %[[FUNC:.*]] = llvm.mlir.addressof @foo : !llvm.ptr
+// CHECK: %{{.*}} = llvm.invoke %[[FUNC]]{{.*}}: !llvm.ptr,
 ^bb4:
+  %12 = llvm.mlir.addressof @foo : !llvm.ptr
+  %13 = llvm.invoke %12(%7) to ^bb2 unwind ^bb1 : !llvm.ptr, (i32) -> !llvm.struct<(i32, f64, i32)>
+
+// CHECK: ^[[BB5:.*]]:
+// CHECK:   llvm.return %[[a0]] : i32
+^bb5:
   llvm.return %0 : i32
 }
 
 // CHECK-LABEL: @useFreezeOp
 func.func @useFreezeOp(%arg0: i32) {
   // CHECK:  = llvm.freeze %[[ARG0:.*]] : i32
   %0 = llvm.freeze %arg0 : i32
   // CHECK: %[[x:.*]] = llvm.mlir.undef : i8

mlir/test/Dialect/OpenMP/invalid.mlir

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = arith.addf %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 func.func @foo(%lb : index, %ub : index, %step : index, %mem : memref<1xf32>) {
   %c1 = arith.constant 1 : i32
 
   // expected-error @below {{expected accumulator ('memref<1xf32>') to be the same type as reduction declaration ('!llvm.ptr<f32>')}}
   omp.wsloop reduction(@add_f32 -> %mem : memref<1xf32>)

 combiner {
 ^bb1(%arg0: i32, %arg1: i32):
   %1 = arith.addi %arg0, %arg1 : i32
   omp.yield (%1 : i32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<i32>, %arg3: !llvm.ptr<i32>):
   %2 = llvm.load %arg3 : !llvm.ptr<i32>
-  llvm.atomicrmw add %arg2, %2 monotonic : i32
+  llvm.atomicrmw add %arg2, %2 monotonic : !llvm.ptr<i32>, i32
   omp.yield
 }
 
 func.func @omp_task(%mem: memref<1xf32>) {
   // expected-error @below {{op expected accumulator ('memref<1xf32>') to be the same type as reduction declaration ('!llvm.ptr<i32>')}}
   omp.task in_reduction(@add_i32 -> %mem : memref<1xf32>) {
     // CHECK: "test.foo"() : () -> ()
     "test.foo"() : () -> ()

mlir/test/Dialect/OpenMP/ops.mlir

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = arith.addf %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 // CHECK-LABEL: func @wsloop_reduction
 func.func @wsloop_reduction(%lb : index, %ub : index, %step : index) {
   %c1 = arith.constant 1 : i32
   %0 = llvm.alloca %c1 x i32 : (i32) -> !llvm.ptr<f32>
   // CHECK: reduction(@add_f32 -> %{{.+}} : !llvm.ptr<f32>)

mlir/test/Target/LLVMIR/Import/constant.ll

 ; CHECK-LABEL: @function_address_before_def
 define i32 @function_address_before_def() {
   %1 = alloca ptr
   ; CHECK:  %[[FUN:.*]] = llvm.mlir.addressof @callee : !llvm.ptr
   ; CHECK:  llvm.store %[[FUN]], %[[PTR:.*]] : !llvm.ptr, !llvm.ptr
   store ptr @callee, ptr %1
   ; CHECK:  %[[INDIR:.*]] = llvm.load %[[PTR]] : !llvm.ptr -> !llvm.ptr
   %2 = load ptr, ptr %1
-  ; CHECK:  llvm.call %[[INDIR]]()
+  ; CHECK:  llvm.call %[[INDIR]]() : !llvm.ptr, () -> i32
   %3 = call i32 %2()
   ret i32 %3
 }
 
 define i32 @callee() {
   ret i32 42
 }
 
 ; Calling a function that has been defined.
 ; CHECK-LABEL: @function_address_after_def
 define i32 @function_address_after_def() {
   %1 = alloca ptr
   ; CHECK:  %[[FUN:.*]] = llvm.mlir.addressof @callee : !llvm.ptr
   ; CHECK:  llvm.store %[[FUN]], %[[PTR:.*]] : !llvm.ptr, !llvm.ptr
   store ptr @callee, ptr %1
   ; CHECK:  %[[INDIR:.*]] = llvm.load %[[PTR]] : !llvm.ptr -> !llvm.ptr
   %2 = load ptr, ptr %1
-  ; CHECK:  llvm.call %[[INDIR]]()
+  ; CHECK:  llvm.call %[[INDIR]]() : !llvm.ptr, () -> i32
   %3 = call i32 %2()
   ret i32 %3
 }
 
 ; // -----
 
 ; Verify the aggregate constant import.
 

mlir/test/Target/LLVMIR/Import/instructions.ll

 ; // -----
 
 ; CHECK-LABEL: @atomic_rmw
 ; CHECK-SAME:  %[[PTR1:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[VAL1:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[PTR2:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[VAL2:[a-zA-Z0-9]+]]
 define void @atomic_rmw(ptr %ptr1, i32 %val1, ptr %ptr2, float %val2) {
-  ; CHECK:  llvm.atomicrmw xchg %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw xchg %[[PTR1]], %[[VAL1]] acquire
   %1 = atomicrmw xchg ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw add %[[PTR1]], %[[VAL1]] release  : i32
+  ; CHECK:  llvm.atomicrmw add %[[PTR1]], %[[VAL1]] release
   %2 = atomicrmw add ptr %ptr1, i32 %val1 release
-  ; CHECK:  llvm.atomicrmw sub %[[PTR1]], %[[VAL1]] acq_rel  : i32
+  ; CHECK:  llvm.atomicrmw sub %[[PTR1]], %[[VAL1]] acq_rel
   %3 = atomicrmw sub ptr %ptr1, i32 %val1 acq_rel
-  ; CHECK:  llvm.atomicrmw _and %[[PTR1]], %[[VAL1]] seq_cst  : i32
+  ; CHECK:  llvm.atomicrmw _and %[[PTR1]], %[[VAL1]] seq_cst
   %4 = atomicrmw and ptr %ptr1, i32 %val1 seq_cst
-  ; CHECK:  llvm.atomicrmw nand %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw nand %[[PTR1]], %[[VAL1]] acquire
   %5 = atomicrmw nand ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw _or %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw _or %[[PTR1]], %[[VAL1]] acquire
   %6 = atomicrmw or ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw _xor %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw _xor %[[PTR1]], %[[VAL1]] acquire
   %7 = atomicrmw xor ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw max %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw max %[[PTR1]], %[[VAL1]] acquire
   %8 = atomicrmw max ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw min %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw min %[[PTR1]], %[[VAL1]] acquire
   %9 = atomicrmw min ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw umax %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw umax %[[PTR1]], %[[VAL1]] acquire
   %10 = atomicrmw umax ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw umin %[[PTR1]], %[[VAL1]] acquire  : i32
+  ; CHECK:  llvm.atomicrmw umin %[[PTR1]], %[[VAL1]] acquire
   %11 = atomicrmw umin ptr %ptr1, i32 %val1 acquire
-  ; CHECK:  llvm.atomicrmw fadd %[[PTR2]], %[[VAL2]] acquire  : f32
+  ; CHECK:  llvm.atomicrmw fadd %[[PTR2]], %[[VAL2]] acquire
   %12 = atomicrmw fadd ptr %ptr2, float %val2 acquire
-  ; CHECK:  llvm.atomicrmw fsub %[[PTR2]], %[[VAL2]] acquire  : f32
+  ; CHECK:  llvm.atomicrmw fsub %[[PTR2]], %[[VAL2]] acquire
   %13 = atomicrmw fsub ptr %ptr2, float %val2 acquire
   ret void
 }
 
 ; // -----
 
 ; CHECK-LABEL: @atomic_cmpxchg
 ; CHECK-SAME:  %[[PTR1:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[VAL1:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[VAL2:[a-zA-Z0-9]+]]
 define void @atomic_cmpxchg(ptr %ptr1, i32 %val1, i32 %val2) {
-  ; CHECK:  llvm.cmpxchg %[[PTR1]], %[[VAL1]], %[[VAL2]] seq_cst seq_cst : i32
+  ; CHECK:  llvm.cmpxchg %[[PTR1]], %[[VAL1]], %[[VAL2]] seq_cst seq_cst
   %1 = cmpxchg ptr %ptr1, i32 %val1, i32 %val2 seq_cst seq_cst
-  ; CHECK:  llvm.cmpxchg %[[PTR1]], %[[VAL1]], %[[VAL2]] monotonic seq_cst : i32
+  ; CHECK:  llvm.cmpxchg %[[PTR1]], %[[VAL1]], %[[VAL2]] monotonic seq_cst
   %2 = cmpxchg ptr %ptr1, i32 %val1, i32 %val2 monotonic seq_cst
   ret void
 }
 
 ; // -----
 
 ; CHECK: llvm.func @fn(i32) -> f32
 declare float @fn(i32)
 
-; CHECK-LABEL: @call_fn
+; CHECK-LABEL: @direct_call
 ; CHECK-SAME:  %[[ARG1:[a-zA-Z0-9]+]]
-define float @call_fn(i32 %arg1) {
+define float @direct_call(i32 %arg1) {
   ; CHECK:  llvm.call @fn(%[[ARG1]])
   %1 = call float @fn(i32 %arg1)
   ret float %1
 }
 
 ; // -----
 
-; CHECK-LABEL: @call_fn_ptr
+; CHECK-LABEL: @indirect_call
 ; CHECK-SAME:  %[[PTR:[a-zA-Z0-9]+]]
-define void @call_fn_ptr(ptr %fn) {
+define void @indirect_call(ptr addrspace(42) %fn) {
   ; CHECK:  %[[C0:[0-9]+]] = llvm.mlir.constant(0 : i16) : i16
-  ; CHECK:  llvm.call %[[PTR]](%[[C0]])
-  call void %fn(i16 0)
+  ; CHECK:  llvm.call %[[PTR]](%[[C0]]) : !llvm.ptr<42>, (i16) -> ()
+  call addrspace(42) void %fn(i16 0)
   ret void
 }
 
 ; // -----
 
 ; CHECK-LABEL: @gep_static_idx
 ; CHECK-SAME:  %[[PTR:[a-zA-Z0-9]+]]
 define void @gep_static_idx(ptr %ptr) {
   ; CHECK: %[[IDX:.+]] = llvm.mlir.constant(7 : i32)
   ; CHECK: llvm.getelementptr inbounds %[[PTR]][%[[IDX]]] : (!llvm.ptr, i32) -> !llvm.ptr
   %1 = getelementptr inbounds float, ptr %ptr, i32 7
   ret void
 }
 
 ; // -----
 
+; CHECK: @varargs(...)
+declare void @varargs(...)
+
+; CHECK-LABEL: @varargs_call
+; CHECK-SAME:  %[[ARG1:[a-zA-Z0-9]+]]
+define void @varargs_call(i32 %0) {
+  ; CHECK:  llvm.call @varargs(%[[ARG1]]) : (i32) -> ()
+  call void (...) @varargs(i32 %0)
+  ret void
+}
+
+; // -----
+
 %sub_struct = type { i32, i8 }
 %my_struct = type { %sub_struct, [4 x i32] }
 
 ; CHECK-LABEL: @gep_dynamic_idx
 ; CHECK-SAME:  %[[PTR:[a-zA-Z0-9]+]]
 ; CHECK-SAME:  %[[IDX:[a-zA-Z0-9]+]]
 define void @gep_dynamic_idx(ptr %ptr, i32 %idx) {
   ; CHECK: %[[C0:.+]] = llvm.mlir.constant(0 : i32)

mlir/test/Target/LLVMIR/llvmir.mlir

 
 //
 // Indirect function calls
 //
 
 // CHECK-LABEL: define void @indirect_const_call(i64 {{%.*}})
 llvm.func @indirect_const_call(%arg0: i64) {
 // CHECK-NEXT:  call void @body(i64 %0)
-  %0 = llvm.mlir.addressof @body : !llvm.ptr<func<void (i64)>>
-  llvm.call %0(%arg0) : (i64) -> ()
+  %0 = llvm.mlir.addressof @body : !llvm.ptr
+  llvm.call %0(%arg0) : !llvm.ptr, (i64) -> ()
 // CHECK-NEXT:  ret void
   llvm.return
 }
 
-// CHECK-LABEL: define i32 @indirect_call(ptr {{%.*}}, float {{%.*}})
-llvm.func @indirect_call(%arg0: !llvm.ptr<func<i32 (f32)>>, %arg1: f32) -> i32 {
-// CHECK-NEXT:  %3 = call i32 %0(float %1)
-  %0 = llvm.call %arg0(%arg1) : (f32) -> i32
+// CHECK-LABEL: define i32 @indirect_call(ptr addrspace(42) {{%.*}}, float {{%.*}})
+llvm.func @indirect_call(%arg0: !llvm.ptr<42>, %arg1: f32) -> i32 {
+// CHECK-NEXT:  %3 = call addrspace(42) i32 %0(float %1)
+  %0 = llvm.call %arg0(%arg1) : !llvm.ptr<42>, (f32) -> i32
 // CHECK-NEXT:  ret i32 %3
   llvm.return %0 : i32
 }
 
 //
 // Check that we properly construct phi nodes in the blocks that have the same
 // predecessor more than once.
 //

 llvm.func @dereferenceableattr_ret_decl() -> (!llvm.ptr {llvm.dereferenceable = 32 : i64})
 
 // CHECK-LABEL: declare dereferenceable_or_null(16) ptr @dereferenceableornullattr_ret_decl()
 llvm.func @dereferenceableornullattr_ret_decl() -> (!llvm.ptr {llvm.dereferenceable_or_null = 16 : i64})
 
 // CHECK-LABEL: declare inreg ptr @inregattr_ret_decl()
 llvm.func @inregattr_ret_decl() -> (!llvm.ptr {llvm.inreg})
 
-// CHECK-LABEL: @llvm_varargs(...)
-llvm.func @llvm_varargs(...)
+// CHECK-LABEL: @varargs(...)
+llvm.func @varargs(...)
+
+// CHECK-LABEL: define void @varargs_call
+llvm.func @varargs_call(%arg0 : i32) {
+// CHECK:  call void (...) @varargs(i32 %{{.*}})
+  llvm.call @varargs(%arg0) : (i32) -> ()
+  llvm.return
+}
 
 llvm.func @intpointerconversion(%arg0 : i32) -> i32 {
 // CHECK:      %2 = inttoptr i32 %0 to ptr
 // CHECK-NEXT: %3 = ptrtoint ptr %2 to i32
   %1 = llvm.inttoptr %arg0 : i32 to !llvm.ptr<i32>
   %2 = llvm.ptrtoint %1 : !llvm.ptr<i32> to i32
   llvm.return %2 : i32
 }
llvm.func @elements_constant_3d_array() -> !llvm.array<2 x array<2 x array<2 x i32>>> {
   llvm.return %0 : !llvm.array<2 x array<2 x array<2 x i32>>>
 }
 
 // CHECK-LABEL: @atomicrmw
 llvm.func @atomicrmw(
     %f32_ptr : !llvm.ptr<f32>, %f32 : f32,
     %i32_ptr : !llvm.ptr<i32>, %i32 : i32) {
   // CHECK: atomicrmw fadd ptr %{{.*}}, float %{{.*}} monotonic
-  %0 = llvm.atomicrmw fadd %f32_ptr, %f32 monotonic : f32
+  %0 = llvm.atomicrmw fadd %f32_ptr, %f32 monotonic : !llvm.ptr<f32>, f32
   // CHECK: atomicrmw fsub ptr %{{.*}}, float %{{.*}} monotonic
-  %1 = llvm.atomicrmw fsub %f32_ptr, %f32 monotonic : f32
+  %1 = llvm.atomicrmw fsub %f32_ptr, %f32 monotonic : !llvm.ptr<f32>, f32
   // CHECK: atomicrmw xchg ptr %{{.*}}, float %{{.*}} monotonic
-  %2 = llvm.atomicrmw xchg %f32_ptr, %f32 monotonic : f32
+  %2 = llvm.atomicrmw xchg %f32_ptr, %f32 monotonic : !llvm.ptr<f32>, f32
   // CHECK: atomicrmw add ptr %{{.*}}, i32 %{{.*}} acquire
-  %3 = llvm.atomicrmw add %i32_ptr, %i32 acquire : i32
+  %3 = llvm.atomicrmw add %i32_ptr, %i32 acquire : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw sub ptr %{{.*}}, i32 %{{.*}} release
-  %4 = llvm.atomicrmw sub %i32_ptr, %i32 release : i32
+  %4 = llvm.atomicrmw sub %i32_ptr, %i32 release : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw and ptr %{{.*}}, i32 %{{.*}} acq_rel
-  %5 = llvm.atomicrmw _and %i32_ptr, %i32 acq_rel : i32
+  %5 = llvm.atomicrmw _and %i32_ptr, %i32 acq_rel : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw nand ptr %{{.*}}, i32 %{{.*}} seq_cst
-  %6 = llvm.atomicrmw nand %i32_ptr, %i32 seq_cst : i32
+  %6 = llvm.atomicrmw nand %i32_ptr, %i32 seq_cst : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw or ptr %{{.*}}, i32 %{{.*}} monotonic
-  %7 = llvm.atomicrmw _or %i32_ptr, %i32 monotonic : i32
+  %7 = llvm.atomicrmw _or %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw xor ptr %{{.*}}, i32 %{{.*}} monotonic
-  %8 = llvm.atomicrmw _xor %i32_ptr, %i32 monotonic : i32
+  %8 = llvm.atomicrmw _xor %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw max ptr %{{.*}}, i32 %{{.*}} monotonic
-  %9 = llvm.atomicrmw max %i32_ptr, %i32 monotonic : i32
+  %9 = llvm.atomicrmw max %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw min ptr %{{.*}}, i32 %{{.*}} monotonic
-  %10 = llvm.atomicrmw min %i32_ptr, %i32 monotonic : i32
+  %10 = llvm.atomicrmw min %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw umax ptr %{{.*}}, i32 %{{.*}} monotonic
-  %11 = llvm.atomicrmw umax %i32_ptr, %i32 monotonic : i32
+  %11 = llvm.atomicrmw umax %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   // CHECK: atomicrmw umin ptr %{{.*}}, i32 %{{.*}} monotonic
-  %12 = llvm.atomicrmw umin %i32_ptr, %i32 monotonic : i32
+  %12 = llvm.atomicrmw umin %i32_ptr, %i32 monotonic : !llvm.ptr<i32>, i32
   llvm.return
 }
 
 // CHECK-LABEL: @cmpxchg
 llvm.func @cmpxchg(%ptr : !llvm.ptr<i32>, %cmp : i32, %val: i32) {
   // CHECK: cmpxchg ptr %{{.*}}, i32 %{{.*}}, i32 %{{.*}} acq_rel monotonic
-  %0 = llvm.cmpxchg %ptr, %cmp, %val acq_rel monotonic : i32
+  %0 = llvm.cmpxchg %ptr, %cmp, %val acq_rel monotonic : !llvm.ptr<i32>, i32
   // CHECK: %{{[0-9]+}} = extractvalue { i32, i1 } %{{[0-9]+}}, 0
   %1 = llvm.extractvalue %0[0] : !llvm.struct<(i32, i1)>
   // CHECK: %{{[0-9]+}} = extractvalue { i32, i1 } %{{[0-9]+}}, 1
   %2 = llvm.extractvalue %0[1] : !llvm.struct<(i32, i1)>
   llvm.return
 }
 
 llvm.mlir.global external constant @_ZTIi() : !llvm.ptr<i8>

mlir/test/Target/LLVMIR/openmp-reduction.mlir

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = llvm.fadd %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 // CHECK-LABEL: @simple_reduction
 llvm.func @simple_reduction(%lb : i64, %ub : i64, %step : i64) {
   %c1 = llvm.mlir.constant(1 : i32) : i32
   %0 = llvm.alloca %c1 x i32 : (i32) -> !llvm.ptr<f32>
   omp.parallel {

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = llvm.fadd %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 // When the same reduction declaration is used several times, its regions
 // are translated several times, which shouldn't lead to value/block
 // remapping assertions.
 // CHECK-LABEL: @reuse_declaration
 llvm.func @reuse_declaration(%lb : i64, %ub : i64, %step : i64) {

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = llvm.fadd %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 // It's okay not to reference the reduction variable in the body.
 // CHECK-LABEL: @missing_omp_reduction
 llvm.func @missing_omp_reduction(%lb : i64, %ub : i64, %step : i64) {
   %c1 = llvm.mlir.constant(1 : i32) : i32
   %0 = llvm.alloca %c1 x i32 : (i32) -> !llvm.ptr<f32>

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = llvm.fadd %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 // It's okay to refer to the same reduction variable more than once in the
 // body.
 // CHECK-LABEL: @double_reference
 llvm.func @double_reference(%lb : i64, %ub : i64, %step : i64) {
   %c1 = llvm.mlir.constant(1 : i32) : i32

 combiner {
 ^bb1(%arg0: f32, %arg1: f32):
   %1 = llvm.fadd %arg0, %arg1 : f32
   omp.yield (%1 : f32)
 }
 atomic {
 ^bb2(%arg2: !llvm.ptr<f32>, %arg3: !llvm.ptr<f32>):
   %2 = llvm.load %arg3 : !llvm.ptr<f32>
-  llvm.atomicrmw fadd %arg2, %2 monotonic : f32
+  llvm.atomicrmw fadd %arg2, %2 monotonic : !llvm.ptr<f32>, f32
   omp.yield
 }
 
 omp.reduction.declare @mul_f32 : f32
 init {
 ^bb0(%arg: f32):
   %0 = llvm.mlir.constant(1.0 : f32) : f32
   omp.yield (%0 : f32)