[mlir] Ability to build CAPI dylibs from out of tree projects against installed LLVM.

[stellaraccident]

Summary

• Incorporates a reworked version of D106419 (which I have closed but has comments on it).
• Extends the standalone example to include a minimal CAPI (for registering its dialect) and a test which, from out of tree, creates an aggregate dylib and links a little sample program against it. This will likely only work today in *static* MLIR builds (until the TypeID fiasco is finally put to bed). It should work on all platforms, though (including Windows - albeit I haven't tried this exact incarnation there).
• This is the biggest pre-requisite to being able to build out of tree MLIR Python-based projects from an installed MLIR/LLVM.
• I am rather nauseated by the CMake shenanigans I had to endure to get this working. The primary complexity, above and beyond the previous patch is because (with no reason given), it is impossible to export target properties that contain generator expressions... because, of course it isn't. In this case, the primary reason we use generator expressions on the individual embedded libraries is to support arbitrary ordering. Since that need doesn't apply to out of tree (which import everything via FindPackage at the outset), we fall back to a more imperative way of doing the same thing if we detect that the target was imported. Gross, but I don't expect it to need a lot of maintenance.
• There should be a relatively straight-forward path from here to rebase libMLIR.so on top of this facility and also make it include the CAPI.

[stellaraccident]

Fix test

[stephenneuendorffer]

mlir/cmake/modules/AddMLIR.cmake

63

Can you document what ENABLE_AGGREGATION does here? is there a reason why we shouldn't always do this (other than that it requires an OBJLIB?)

[stephenneuendorffer]

mlir/cmake/modules/AddMLIR.cmake

282–283

Need some checks here in case the library was not exported with the right information.

[stellaraccident]

Better detection for import and comments.

[stellaraccident]

Error check.

[stellaraccident]

mlir/cmake/modules/AddMLIR.cmake

63

Done and added a TODO to just make it a default. I had originally tried to do that, but there is cruft in a number of the in-tree libraries that keep them from being built with an object library. I'd like to do a second pass and clean all of those up and then switch this to a DISABLE flag (i.e. may want to disable when just building a single-use runtime library, for example -- need to futz with that a bit).

[stephenneuendorffer]

I think this can go in tree now.

[stellaraccident]

Rebase

[jpienaar]

This seems to be causing some build failures: https://github.com/google/mlir-hs/runs/3894495306?check_suite_focus=true

[stellaraccident]

That is weird. It appears to be something odd in the debug logging the new
rule does (which I left enabled for everyone). I think we can just comment
that out with a note that it can be enabled if needed.

I don't know much about this project. If you have it buildable, would you
mind testing my theory. Comment out the two commands here:
https://github.com/llvm/llvm-project/blob/b6c218d4fdb74c0ee467e078721438c3396dc599/mlir/cmake/modules/AddMLIR.cmake#L342

[kwk]

mlir/cmake/modules/AddMLIR.cmake

366

@stellaraccident is this really needed for anything? I mean which part is supposed to consume these *.o files?

[stellaraccident]

mlir/cmake/modules/AddMLIR.cmake

366

The add_mlir_aggregate uses them when constructing a shared library for a slice of the dependencies. Installing them supports out of tree projects (building from an installed LLVM) and allows them to build such things in the same way that we do in-tree. This is used today for downstream Python and Haskell bindings, both of which build a subset of the MLIR dependencies to bundle for their native package manager. In the future, it could allow out of tree projects to roll their own libMLIR.so equivalent, which is a feature that keeps getting asked for (i.e. as far as I understand it, a lot of folks have workflows where one team/CI produces the LLVM installs for all of their architectures and then multiple "downstream" teams build MLIR based things on top of them).

I assume you are asking because this universally adds bulk to the installs. I had considered gating this with a CMake option (i.e. MLIR_INSTALL_AGGREGATE_OBJECTS), which would let us scope the cost to just the folks who have this use case.

[kwk]

mlir/cmake/modules/AddMLIR.cmake

366

Thank you for the explanation.

I assume you are asking because this universally adds bulk to the installs.

That is correct. All of a sudden /usr/lib64/objects-RelWithDebInfo/obj.MLIRCAPI* files appeared that were installed but not packaged. I've asked my question because I'd like to understand where to put the files.

I had considered gating this with a CMake option (i.e. MLIR_INSTALL_AGGREGATE_OBJECTS), which would let us scope the cost to just the folks who have this use case.

Sure, then packagers can gradually opt in to this kind of extra files. I'd suggest to make the default true though and add a very good explanation what it does. Then we have one build failing and you're forced to make a decision as a packager. I'd rather have this than going silent without ever knowing about these extra set of files. Make sense?

[stellaraccident]

mlir/cmake/modules/AddMLIR.cmake

366

Makes sense (and also thank you for work/attention to packaging).

Just to restate before rolling a patch:

• Add a new MLIR_INSTALL_AGGREGATE_OBJECTS defaulting to true
• Export this to the installed CMake config so we can check it and issue an appropriate error if mlir_add_aggregate is called on an install that doesn't have it.
• Packagers (you) will flip this to false until/if they want to include these assets?

[stellaraccident]

Followup in https://reviews.llvm.org/D112090

mlir/cmake/modules/AddMLIR.cmake

 
 # Declare an mlir library which can be compiled in libMLIR.so
 # In addition to everything that llvm_add_librar accepts, this
 # also has the following option:
 # EXCLUDE_FROM_LIBMLIR
 #   Don't include this library in libMLIR.so.  This option should be used
 #   for test libraries, executable-specific libraries, or rarely used libraries
 #   with large dependencies.
+# ENABLE_AGGREGATION
+#   Forces generation of an OBJECT library, exports additional metadata,
+#   and installs additional object files needed to include this as part of an
+#   aggregate shared library.
+#   TODO: Make this the default for all MLIR libraries once all libraries
+#   are compatible with building an object library.
 function(add_mlir_library name)
   cmake_parse_arguments(ARG
-    "SHARED;INSTALL_WITH_TOOLCHAIN;EXCLUDE_FROM_LIBMLIR;DISABLE_INSTALL"
+    "SHARED;INSTALL_WITH_TOOLCHAIN;EXCLUDE_FROM_LIBMLIR;DISABLE_INSTALL;ENABLE_AGGREGATION"
     ""
     "ADDITIONAL_HEADERS;DEPENDS;LINK_COMPONENTS;LINK_LIBS"
     ${ARGN})
   set(srcs)
   if(MSVC_IDE OR XCODE)
     # Add public headers
     file(RELATIVE_PATH lib_path
       ${MLIR_SOURCE_DIR}/lib/
function(add_mlir_library name)
   endif(MSVC_IDE OR XCODE)
   if(srcs OR ARG_ADDITIONAL_HEADERS)
     set(srcs
       ADDITIONAL_HEADERS
       ${srcs}
       ${ARG_ADDITIONAL_HEADERS} # It may contain unparsed unknown args.
       )
   endif()
+
+  # Is an object library needed.
+  set(NEEDS_OBJECT_LIB OFF)
+  if(ARG_ENABLE_AGGREGATION)
+    set(NEEDS_OBJECT_LIB ON)
+  endif()
+
+  # Determine type of library.
   if(ARG_SHARED)
     set(LIBTYPE SHARED)
   else()
     # llvm_add_library ignores BUILD_SHARED_LIBS if STATIC is explicitly set,
     # so we need to handle it here.
     if(BUILD_SHARED_LIBS)
       set(LIBTYPE SHARED)
     else()
       set(LIBTYPE STATIC)
     endif()
-    if(NOT XCODE)
-      # The Xcode generator doesn't handle object libraries correctly.
-      list(APPEND LIBTYPE OBJECT)
-    endif()
     # Test libraries and such shouldn't be include in libMLIR.so
     if(NOT ARG_EXCLUDE_FROM_LIBMLIR)
+      set(NEEDS_OBJECT_LIB ON)
       set_property(GLOBAL APPEND PROPERTY MLIR_STATIC_LIBS ${name})
       set_property(GLOBAL APPEND PROPERTY MLIR_LLVM_LINK_COMPONENTS ${ARG_LINK_COMPONENTS})
       set_property(GLOBAL APPEND PROPERTY MLIR_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
     endif()
   endif()
 
+  if(NEEDS_OBJECT_LIB AND NOT XCODE)
+    # The Xcode generator doesn't handle object libraries correctly.
+    # We special case xcode when building aggregates.
+    list(APPEND LIBTYPE OBJECT)
+  endif()
+
   # MLIR libraries uniformly depend on LLVMSupport.  Just specify it once here.
   list(APPEND ARG_LINK_COMPONENTS Support)
 
   # LINK_COMPONENTS is necessary to allow libLLVM.so to be properly
   # substituted for individual library dependencies if LLVM_LINK_LLVM_DYLIB
   # Perhaps this should be in llvm_add_library instead?  However, it fails
   # on libclang-cpp.so
   get_property(llvm_component_libs GLOBAL PROPERTY LLVM_COMPONENT_LIBS)
if(TARGET ${name})
     if(NOT ARG_DISABLE_INSTALL)
       add_mlir_library_install(${name})
     endif()
   else()
     # Add empty "phony" target
     add_custom_target(${name})
   endif()
   set_target_properties(${name} PROPERTIES FOLDER "MLIR libraries")
+
+  # Setup aggregate.
+  if(ARG_ENABLE_AGGREGATION)
+    # Compute and store the properties needed to build aggregates.
+    set(AGGREGATE_OBJECTS)
+    set(AGGREGATE_OBJECT_LIB)
+    set(AGGREGATE_DEPS)
+    if(XCODE)
+      # XCode has limited support for object libraries. Instead, add dep flags
+      # that force the entire library to be embedded.
+      list(APPEND AGGREGATE_DEPS "-force_load" "${name}")
+    else()
+      list(APPEND AGGREGATE_OBJECTS "$<TARGET_OBJECTS:obj.${name}>")
+      list(APPEND AGGREGATE_OBJECT_LIB "obj.${name}")
+    endif()
+
+    # For each declared dependency, transform it into a generator expression
+    # which excludes it if the ultimate link target is excluding the library.
+    set(NEW_LINK_LIBRARIES)
+    get_target_property(CURRENT_LINK_LIBRARIES  ${name} LINK_LIBRARIES)
+    get_mlir_filtered_link_libraries(NEW_LINK_LIBRARIES ${CURRENT_LINK_LIBRARIES})
+    set_target_properties(${name} PROPERTIES LINK_LIBRARIES "${NEW_LINK_LIBRARIES}")
+    list(APPEND AGGREGATE_DEPS ${NEW_LINK_LIBRARIES})
+    set_target_properties(${name} PROPERTIES
+      EXPORT_PROPERTIES "MLIR_AGGREGATE_OBJECT_LIB_IMPORTED;MLIR_AGGREGATE_DEP_LIBS_IMPORTED"
+      MLIR_AGGREGATE_OBJECTS "${AGGREGATE_OBJECTS}"
+      MLIR_AGGREGATE_DEPS "${AGGREGATE_DEPS}"
+      MLIR_AGGREGATE_OBJECT_LIB_IMPORTED "${AGGREGATE_OBJECT_LIB}"
+      MLIR_AGGREGATE_DEP_LIBS_IMPORTED "${CURRENT_LINK_LIBRARIES}"
+    )
+
+    # In order for out-of-tree projects to build aggregates of this library,
+    # we need to install the OBJECT library.
+    if(NOT ARG_DISABLE_INSTALL)
+      add_mlir_library_install(obj.${name})
+    endif()
+  endif()
 endfunction(add_mlir_library)
 
+# Sets a variable with a transformed list of link libraries such individual
+# libraries will be dynamically excluded when evaluated on a final library
+# which defines an MLIR_AGGREGATE_EXCLUDE_LIBS which contains any of the
+# libraries. Each link library can be a generator expression but must not
+# resolve to an arity > 1 (i.e. it can be optional).
+function(get_mlir_filtered_link_libraries output)
+  set(_results)
+  foreach(linklib ${ARGN})
+    # In English, what this expression does:
+    # For each link library, resolve the property MLIR_AGGREGATE_EXCLUDE_LIBS
+    # on the context target (i.e. the executable or shared library being linked)
+    # and, if it is not in that list, emit the library name. Otherwise, empty.
+    list(APPEND _results
+      "$<$<NOT:$<IN_LIST:${linklib},$<GENEX_EVAL:$<TARGET_PROPERTY:MLIR_AGGREGATE_EXCLUDE_LIBS>>>>:${linklib}>"
+    )
+  endforeach()
+  set(${output} "${_results}" PARENT_SCOPE)
+endfunction(get_mlir_filtered_link_libraries)
+
+# Declares an aggregate library. Such a library is a combination of arbitrary
+# regular add_mlir_library() libraries with the special feature that they can
+# be configured to statically embed some subset of their dependencies, as is
+# typical when creating a .so/.dylib/.dll or a mondo static library.
+#
+# It is always safe to depend on the aggregate directly in order to compile/link
+# against the superset of embedded entities and transitive deps.
+#
+# Arguments:
+#   PUBLIC_LIBS: list of dependent libraries to add to the
+#     INTERFACE_LINK_LIBRARIES property, exporting them to users. This list
+#     will be transitively filtered to exclude any EMBED_LIBS.
+#   EMBED_LIBS: list of dependent libraries that should be embedded directly
+#     into this library. Each of these must be an add_mlir_library() library
+#     without DISABLE_AGGREGATE.
+#
+# Note: This is a work in progress and is presently only sufficient for certain
+# non nested cases involving the C-API.
+function(add_mlir_aggregate name)
+  cmake_parse_arguments(ARG
+    "SHARED;STATIC"
+    ""
+    "PUBLIC_LIBS;EMBED_LIBS"
+    ${ARGN})
+  set(_libtype)
+  if(ARG_STATIC)
+    list(APPEND _libtype STATIC)
+  endif()
+  if(ARG_SHARED)
+    list(APPEND _libtype SHARED)
+  endif()
+  set(_debugmsg)
+
+  set(_embed_libs)
+  set(_objects)
+  set(_deps)
+  foreach(lib ${ARG_EMBED_LIBS})
+    # We have to handle imported vs in-tree differently:
+    #   in-tree: To support arbitrary ordering, the generator expressions get
+    #     set on the dependent target when it is constructed and then just
+    #     eval'd here. This means we can build an aggregate from targets that
+    #     may not yet be defined, which is typical for in-tree.
+    #   imported: Exported properties do not support generator expressions, so
+    #     we imperatively query and manage the expansion here. This is fine
+    #     because imported targets will always be found/configured first and
+    #     do not need to support arbitrary ordering. If CMake every supports
+    #     exporting generator expressions, then this can be simplified.
+    set(_is_imported OFF)
+    if(TARGET ${lib})
+      get_target_property(_is_imported ${lib} IMPORTED)
+    endif()
+
+    if(NOT _is_imported)
+      # Evaluate the in-tree generator expressions directly (this allows target
+      # order independence, since these aren't evaluated until the generate
+      # phase).
+      # What these expressions do:
+      # In the context of this aggregate, resolve the list of OBJECTS and DEPS
+      # that each library advertises and patch it into the whole.
+      set(_local_objects $<TARGET_GENEX_EVAL:${name},$<TARGET_PROPERTY:${lib},MLIR_AGGREGATE_OBJECTS>>)
+      set(_local_deps $<TARGET_GENEX_EVAL:${name},$<TARGET_PROPERTY:${lib},MLIR_AGGREGATE_DEPS>>)
+    else()
+      # It is an imported target, which can only have flat strings populated
+      # (no generator expressions).
+      # Rebuild the generator expressions from the imported flat string lists.
+      get_target_property(_imp_local_object_lib ${lib} MLIR_AGGREGATE_OBJECT_LIB_IMPORTED)
+      get_target_property(_imp_dep_libs ${lib} MLIR_AGGREGATE_DEP_LIBS_IMPORTED)
+      set(_local_objects)
+      if(_imp_local_object_lib)
+        set(_local_objects "$<TARGET_OBJECTS:${_imp_local_object_lib}>")
+      endif()
+      # We should just be able to do this:
+      #   get_mlir_filtered_link_libraries(_local_deps ${_imp_dep_libs})
+      # However, CMake complains about the unqualified use of the one-arg
+      # $<TARGET_PROPERTY> expression. So we do the same thing but use the
+      # two-arg form which takes an explicit target.
+      foreach(_imp_dep_lib ${_imp_dep_libs})
+        # In English, what this expression does:
+        # For each link library, resolve the property MLIR_AGGREGATE_EXCLUDE_LIBS
+        # on the context target (i.e. the executable or shared library being linked)
+        # and, if it is not in that list, emit the library name. Otherwise, empty.
+        list(APPEND _local_deps
+          "$<$<NOT:$<IN_LIST:${_imp_dep_lib},$<GENEX_EVAL:$<TARGET_PROPERTY:${name},MLIR_AGGREGATE_EXCLUDE_LIBS>>>>:${_imp_dep_lib}>"
+        )
+      endforeach()
+    endif()
+
+    list(APPEND _embed_libs ${lib})
+    list(APPEND _objects ${_local_objects})
+    list(APPEND _deps ${_local_deps})
+
+    string(APPEND _debugmsg
+      ": EMBED_LIB ${lib}:\n"
+      "    OBJECTS = ${_local_objects}\n"
+      "    DEPS = ${_local_deps}\n\n")
+  endforeach()
+
+  # Unfortunately need to compile at least one source file, which is hard
+  # to guarantee, so just always generate one. We generate one vs using the
+  # LLVM common dummy.cpp because it works better out of tree.
+  set(_empty_src "${CMAKE_CURRENT_BINARY_DIR}/${name}__empty.cpp")
+  file(WRITE "${_empty_src}" "typedef int dummy;")
+
+  add_mlir_library(${name}
+    ${_libtype}
+    ${ARG_UNPARSED_ARGUMENTS}
+    PARTIAL_SOURCES_INTENDED
+    EXCLUDE_FROM_LIBMLIR
+    "${_empty_src}"
+    LINK_LIBS PRIVATE
+    ${_deps}
+    ${ARG_PUBLIC_LIBS}
+  )
+  target_sources(${name} PRIVATE ${_objects})
+  # TODO: Should be transitive.
+  set_target_properties(${name} PROPERTIES
+    MLIR_AGGREGATE_EXCLUDE_LIBS "${_embed_libs}")
+  if(MSVC)
+    set_property(TARGET ${name} PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS ON)
+  endif()
+  string(APPEND _debugmsg
+    ": MAIN LIBRARY:\n"
+    "    OBJECTS = ${_objects}\n"
+    "    SOURCES = $<TARGET_GENEX_EVAL:${name},$<TARGET_PROPERTY:${name},SOURCES>>\n"
+    "    DEPS = ${_deps}\n"
+    "    LINK_LIBRARIES = $<TARGET_GENEX_EVAL:${name},$<TARGET_PROPERTY:${name},LINK_LIBRARIES>>\n"
+    "    MLIR_AGGREGATE_EXCLUDE_LIBS = $<TARGET_GENEX_EVAL:${name},$<TARGET_PROPERTY:${name},MLIR_AGGREGATE_EXCLUDE_LIBS>>\n"
+  )
+  file(GENERATE OUTPUT
+    "${CMAKE_CURRENT_BINARY_DIR}/${name}.aggregate_debug.txt"
+    CONTENT "${_debugmsg}"
+  )
+endfunction(add_mlir_aggregate)
+
 # Adds an MLIR library target for installation.
 # This is usually done as part of add_mlir_library but is broken out for cases
 # where non-standard library builds can be installed.
 function(add_mlir_library_install name)
   if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
   get_target_export_arg(${name} MLIR export_to_mlirtargets UMBRELLA mlir-libraries)
   install(TARGETS ${name}
     COMPONENT ${name}
     ${export_to_mlirtargets}
     LIBRARY DESTINATION lib${LLVM_LIBDIR_SUFFIX}
     ARCHIVE DESTINATION lib${LLVM_LIBDIR_SUFFIX}
-    RUNTIME DESTINATION bin)
+    RUNTIME DESTINATION bin
+    # Note that CMake will create a directory like:
+    #   objects-${CMAKE_BUILD_TYPE}/obj.LibName
+    # and put object files there.
+    OBJECTS DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+  )
 
   if (NOT LLVM_ENABLE_IDE)
     add_llvm_install_targets(install-${name}
                             DEPENDS ${name}
                             COMPONENT ${name})
   endif()
   set_property(GLOBAL APPEND PROPERTY MLIR_ALL_LIBS ${name})
   endif()
   set_property(GLOBAL APPEND PROPERTY MLIR_EXPORTS ${name})
 endfunction()
 
 # Declare an mlir library which is part of the public C-API.
 function(add_mlir_public_c_api_library name)
   add_mlir_library(${name}
     ${ARGN}
-    # NOTE: Generates obj.${name} which is used for shared library building.
-    OBJECT
     EXCLUDE_FROM_LIBMLIR
+    ENABLE_AGGREGATION
     ADDITIONAL_HEADER_DIRS
     ${MLIR_MAIN_INCLUDE_DIR}/mlir-c
   )
   # API libraries compile with hidden visibility and macros that enable
   # exporting from the DLL. Only apply to the obj lib, which only affects
   # the exports via a shared library.
   set_target_properties(obj.${name}
     PROPERTIES

mlir/cmake/modules/AddMLIRPython.cmake

 #   EMBED_LIBS: Additional libraries to embed (must be built with OBJECTS and
 #     have an "obj.${name}" object library associated).
 function(add_mlir_python_common_capi_library name)
   cmake_parse_arguments(ARG
     ""
     "INSTALL_COMPONENT;INSTALL_DESTINATION;OUTPUT_DIRECTORY;RELATIVE_INSTALL_ROOT"
     "DECLARED_SOURCES;EMBED_LIBS"
     ${ARGN})
-  # TODO: Upgrade to the aggregate utility in https://reviews.llvm.org/D106419
-  # once ready.
-
   # Collect all explicit and transitive embed libs.
   set(_embed_libs ${ARG_EMBED_LIBS})
   _flatten_mlir_python_targets(_all_source_targets ${ARG_DECLARED_SOURCES})
   foreach(t ${_all_source_targets})
     get_target_property(_local_embed_libs ${t} PYTHON_EMBED_CAPI_LINK_LIBS)
     if(_local_embed_libs)
       list(APPEND _embed_libs ${_local_embed_libs})
     endif()
   endforeach()
   list(REMOVE_DUPLICATES _embed_libs)
 
-  foreach(lib ${_embed_libs})
-    if(XCODE)
-      # Xcode doesn't support object libraries, so we have to trick it into
-      # linking the static libraries instead.
-      list(APPEND _deps "-force_load" ${lib})
-    else()
-      list(APPEND _objects $<TARGET_OBJECTS:obj.${lib}>)
-    endif()
-    # Accumulate transitive deps of each exported lib into _DEPS.
-    list(APPEND _deps $<TARGET_PROPERTY:${lib},LINK_LIBRARIES>)
-  endforeach()
-
-  add_mlir_library(${name}
-    PARTIAL_SOURCES_INTENDED
+  # Generate the aggregate .so that everything depends on.
+  add_mlir_aggregate(${name}
     SHARED
     DISABLE_INSTALL
-    ${_objects}
-    EXCLUDE_FROM_LIBMLIR
-    LINK_LIBS
-    ${_deps}
+    EMBED_LIBS ${_embed_libs}
   )
+
   if(MSVC)
     set_property(TARGET ${name} PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS ON)
   endif()
   set_target_properties(${name} PROPERTIES
     LIBRARY_OUTPUT_DIRECTORY "${ARG_OUTPUT_DIRECTORY}"
     BINARY_OUTPUT_DIRECTORY "${ARG_OUTPUT_DIRECTORY}"
     # Needed for windows (and don't hurt others).
     RUNTIME_OUTPUT_DIRECTORY "${ARG_OUTPUT_DIRECTORY}"

mlir/examples/standalone/.gitignore

+build/

mlir/examples/standalone/include/Standalone-c/Dialects.h

+//===- Dialects.h - CAPI for dialects -----------------------------*- C -*-===//
+//
+// This file is licensed under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef STANDALONE_C_DIALECTS_H
+#define STANDALONE_C_DIALECTS_H
+
+#include "mlir-c/Registration.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+MLIR_DECLARE_CAPI_DIALECT_REGISTRATION(Standalone, standalone);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif // STANDALONE_C_DIALECTS_H

mlir/examples/standalone/lib/CAPI/CMakeLists.txt

+add_mlir_public_c_api_library(StandaloneCAPI
+  Dialects.cpp
+  LINK_LIBS PUBLIC
+  MLIRStandalone
+)

mlir/examples/standalone/lib/CAPI/Dialects.cpp

+//===- Dialects.cpp - CAPI for dialects -----------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "Standalone-c/Dialects.h"
+
+#include "Standalone/StandaloneDialect.h"
+#include "mlir/CAPI/Registration.h"
+
+MLIR_DEFINE_CAPI_DIALECT_REGISTRATION(Standalone, standalone,
+                                      mlir::standalone::StandaloneDialect)

mlir/examples/standalone/lib/CMakeLists.txt

+add_subdirectory(CAPI)
 add_subdirectory(Standalone)

mlir/examples/standalone/test/CAPI/CMakeLists.txt

+# Here we create a single aggregate shared library with the parts of the CAPI
+# that we want to bundle together. Then we link a simple C executable
+# against it to demonstrate that it does have the fully self contained
+# core MLIR library and our own standalone dialect.
+add_mlir_aggregate(StandaloneCAPITestLib
+  SHARED
+  EMBED_LIBS
+  MLIRCAPIIR
+  MLIRCAPIRegistration
+  StandaloneCAPI
+)
+
+add_llvm_executable(standalone-capi-test
+  standalone-capi-test.c
+)
+llvm_update_compile_flags(standalone-capi-test)
+target_link_libraries(standalone-capi-test
+  PRIVATE StandaloneCAPITestLib)

mlir/examples/standalone/test/CAPI/lit.local.cfg

+config.suffixes.add('.c')

mlir/examples/standalone/test/CAPI/standalone-capi-test.c

+//===- standalone-cap-demo.c - Simple demo of C-API -----------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM
+// Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// RUN: standalone-capi-test 2>&1 | FileCheck %s
+
+#include <stdio.h>
+
+#include "mlir-c/IR.h"
+#include "Standalone-c/Dialects.h"
+
+int main(int argc, char **argv) {
+  MlirContext ctx = mlirContextCreate();
+  // TODO: Create the dialect handles for the builtin dialects and avoid this.
+  // This adds dozens of MB of binary size over just the standalone dialect.
+  mlirRegisterAllDialects(ctx);
+  mlirDialectHandleRegisterDialect(mlirGetDialectHandle__standalone__(), ctx);
+
+  MlirModule module = mlirModuleCreateParse(
+      ctx, mlirStringRefCreateFromCString("%0 = constant 2 : i32\n"
+                                          "%1 = standalone.foo %0 : i32\n"));
+  if (mlirModuleIsNull(module)) {
+    printf("ERROR: Could not parse.\n");
+    mlirContextDestroy(ctx);
+    return 1;
+  }
+  MlirOperation op = mlirModuleGetOperation(module);
+
+  // CHECK: %[[C:.*]] = constant 2 : i32
+  // CHECK: standalone.foo %[[C]] : i32
+  mlirOperationDump(op);
+
+  mlirModuleDestroy(module);
+  mlirContextDestroy(ctx);
+  return 0;
+}

mlir/examples/standalone/test/CMakeLists.txt

 configure_lit_site_cfg(
         ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
         ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
         MAIN_CONFIG
         ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
 )
 
 set(STANDALONE_TEST_DEPENDS
         FileCheck count not
+        standalone-capi-test
         standalone-opt
         standalone-translate
         )
 
 add_lit_testsuite(check-standalone "Running the standalone regression tests"
         ${CMAKE_CURRENT_BINARY_DIR}
         DEPENDS ${STANDALONE_TEST_DEPENDS}
         )
 set_target_properties(check-standalone PROPERTIES FOLDER "Tests")
 
 add_lit_testsuites(STANDALONE ${CMAKE_CURRENT_SOURCE_DIR} DEPENDS ${STANDALONE_TEST_DEPENDS})
+
+add_subdirectory(CAPI)

mlir/examples/standalone/test/lit.cfg.py

 config.test_exec_root = os.path.join(config.standalone_obj_root, 'test')
 config.standalone_tools_dir = os.path.join(config.standalone_obj_root, 'bin')
 
 # Tweak the PATH to include the tools dir.
 llvm_config.with_environment('PATH', config.llvm_tools_dir, append_path=True)
 
 tool_dirs = [config.standalone_tools_dir, config.llvm_tools_dir]
 tools = [
+    'standalone-capi-test',
     'standalone-opt',
     'standalone-translate'
 ]
 
 llvm_config.add_tool_substitutions(tools, tool_dirs)

mlir/test/Examples/standalone/test.toy

 # RUN: %cmake %mlir_src_root/examples/standalone -DCMAKE_CXX_COMPILER=%host_cxx -DCMAKE_C_COMPILER=%host_cc -DLLVM_ENABLE_LIBCXX=%enable_libcxx -DMLIR_DIR=%mlir_cmake_dir ; %cmake --build . --target check-standalone | tee %t | FileCheck %s
 
-# CHECK: Passed: 3
+# CHECK: Passed: 4
 # UNSUPPORTED: windows, android