[MLIR] Add support for libMLIR.so

[stephenneuendorffer]

Summary

Putting this up mainly for discussion on
how this should be done. I am interested in MLIR from
the Julia side and we currently have a strong preference
to dynamically linking against the LLVM shared library,
and would like to have a MLIR shared library.

This patch adds a new cmake function add_mlir_library()
which accumulates a list of targets to be compiled into
libMLIR.so. Note that not all libraries make sense to
be compiled into libMLIR.so. In particular, we want
to avoid libraries which primarily exist to support
certain tools (such as mlir-opt and mlir-cpu-runner).

Note that the resulting libMLIR.so depends on LLVM, but
does not contain any LLVM components. As a result, it
is necessary to link with libLLVM.so to avoid linkage
errors. So, libMLIR.so requires LLVM_BUILD_LLVM_DYLIB=on

FYI, Currently it appears that LLVM_LINK_LLVM_DYLIB is broken
because mlir-tblgen is linked against libLLVM.so and
and independent LLVM components.

Previous version of this patch broke depencies on TableGen
targets. This appears to be because it compiled all
libraries to OBJECT libraries (probably because cmake
is generating different target names). Avoiding object
libraries results in correct dependencies.

(updated by Stephen Neuendorffer)

[vchuravy]

llvm/CMakeLists.txt

20 ↗

(On Diff #239404)

This is to work around https://reviews.llvm.org/D72585

[merge_guards_bot]

Unit tests: fail. 62080 tests passed, 1 failed and 784 were skipped.

failed: Clang.Driver/cc-print-options.c

clang-tidy: unknown.

clang-format: pass.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[stephenneuendorffer]

Thanks for looking into this!

mlir/tools/mlir-shlib/CMakeLists.txt

12

I think the right thing to do is to add an add_mlir_library command, similar to D72554 which is used in the other CMakeFiles, so this list does not need to be maintained separately from where the libraries are declared.

[vchuravy]

Properly implement shared library similar to clang-shlib

[stephenneuendorffer]

mlir/cmake/modules/AddMLIR.cmake

40–41

comment is obsolete?

51

Worth a comment? The variable name here seems a little weird. Maybe "MLIR_LIBRARY_LIBS"?

mlir/lib/Dialect/VectorOps/CMakeLists.txt.rej

1 ↗

(On Diff #243422)

Seems like an unintentional addition?

mlir/lib/TableGen/CMakeLists.txt

1 ↗

(On Diff #243422)

See note below.

mlir/test/Dialect/SPIRV/CMakeLists.txt

1 ↗

(On Diff #243422)

Should not be in libMLIR.so

mlir/test/lib/IR/CMakeLists.txt

1 ↗

(On Diff #243422)

Should not be in libMLIR.so

mlir/test/lib/Pass/CMakeLists.txt

1 ↗

(On Diff #243422)

Should not be in libMLIR.so

mlir/test/lib/TestDialect/CMakeLists.txt

14 ↗

(On Diff #243422)

Should not be in libMLIR.so

mlir/test/lib/Transforms/CMakeLists.txt

1 ↗

(On Diff #243422)

Should not be in libMLIR.so

mlir/tools/mlir-opt/CMakeLists.txt

14–16

There's a few libraries which probably shouldn't be linked into libMLIR.so, and I think this is one of them.

mlir/tools/mlir-shlib/CMakeLists.txt

17

I think it's probably better to call add_llvm_library than add_mlir_library and then remove it later.

[mehdi_amini]

Do we have a path forward if we were to use MLIR in LLVM?

[stephenneuendorffer]

mlir/tools/mlir-shlib/CMakeLists.txt

41–42

I suspect that these libraries need to be added with whole_archive_link, to ensure that static initializers are called appropriately. (see the mlir-opt build rules and AddMLIR.cmake)

[stephenneuendorffer]

mlir/cmake/modules/AddMLIR.cmake

47–50

This breaks the dependencies between .cpp files and the tablegen'd .inc files they depend on. I believe the problem is that the CMakeFiles.txt are setup to use:
add_dependencies(someLib someRuleIncGen)

However, if OBJECT is specified, then llvm_add_library creates new targets which cannot replicate those dependencies.

The best solution is probably to mass-convert all the CMakeFiles.txt to use a DEPEND line when calling add_mlir_library. This would get passed through to llvm_add_library and enable the newly created
targets (named obj.${name}) to be created with the correct dependencies.

[vchuravy]

mlir/cmake/modules/AddMLIR.cmake

40–41

I copied this from the clang shlib, so if this is indeed obsolete
we should remove it there as well.

mlir/tools/mlir-shlib/CMakeLists.txt

7

This is copied as well from the clang shlib and https://reviews.llvm.org/D73156 so it would be great if we can fix that generally,
but I don't have easy access to a MSVC environment. I build my windows binaries with mingw.

mlir/CMakeLists.txt

 set(MLIR_CUDA_RUNNER_ENABLED 0 CACHE BOOL "Enable building the mlir CUDA runner")
 set(MLIR_VULKAN_RUNNER_ENABLED 0 CACHE BOOL "Enable building the mlir Vulkan runner")
 
 include_directories( "include")
 include_directories( ${MLIR_INCLUDE_DIR})
 
 add_subdirectory(include/mlir)
 add_subdirectory(lib)
-add_subdirectory(tools)
 add_subdirectory(unittests)
 add_subdirectory(test)
+# Tools needs to come late to ensure that MLIR_ALL_LIBS is populated.
+# Generally things after this point may depend on MLIR_ALL_LIBS or libMLIR.so.
+add_subdirectory(tools)
 
 if( LLVM_INCLUDE_EXAMPLES )
   add_subdirectory(examples)
 endif()
 
 option(MLIR_INCLUDE_DOCS "Generate build targets for the MLIR docs."
   ${LLVM_INCLUDE_DOCS})
 if (MLIR_INCLUDE_DOCS)

mlir/cmake/modules/AddMLIR.cmake

function(add_mlir_dialect dialect dialect_doc_filename)
   set(LLVM_TARGET_DEFINITIONS ${dialect}.td)
   mlir_tablegen(${dialect}.h.inc -gen-op-decls)
   mlir_tablegen(${dialect}.cpp.inc -gen-op-defs)
   add_public_tablegen_target(MLIR${dialect}IncGen)
   add_dependencies(mlir-headers MLIR${dialect}IncGen)
 
   # Generate Dialect Documentation
   set(LLVM_TARGET_DEFINITIONS ${dialect_doc_filename}.td)
   tablegen(MLIR ${dialect_doc_filename}.md -gen-op-doc "-I${MLIR_MAIN_SRC_DIR}" "-I${MLIR_INCLUDE_DIR}")
   set(GEN_DOC_FILE ${MLIR_BINARY_DIR}/docs/Dialects/${dialect_doc_filename}.md)
   add_custom_command(
           OUTPUT ${GEN_DOC_FILE}
           COMMAND ${CMAKE_COMMAND} -E copy
                   ${CMAKE_CURRENT_BINARY_DIR}/${dialect_doc_filename}.md
                   ${GEN_DOC_FILE}
           DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${dialect_doc_filename}.md)
   add_custom_target(${dialect_doc_filename}DocGen DEPENDS ${GEN_DOC_FILE})
   add_dependencies(mlir-doc ${dialect_doc_filename}DocGen)
 endfunction()
 
+# Declare a library which can be compiled in libMLIR.so
+macro(add_mlir_library name)
+  set_property(GLOBAL APPEND PROPERTY MLIR_ALL_LIBS ${name})
+  add_llvm_library(${ARGV})
+endmacro(add_mlir_library)
+
 # Declare the library associated with a dialect.
 function(add_mlir_dialect_library name)
   set_property(GLOBAL APPEND PROPERTY MLIR_DIALECT_LIBS ${name})
-  add_llvm_library(${ARGV})
+  add_mlir_library(${ARGV})
 endfunction(add_mlir_dialect_library)
 
 # Declare the library associated with a conversion.
 function(add_mlir_conversion_library name)
   set_property(GLOBAL APPEND PROPERTY MLIR_CONVERSION_LIBS ${name})
-  add_llvm_library(${ARGV})
+  add_mlir_library(${ARGV})
 endfunction(add_mlir_conversion_library)

mlir/lib/Analysis/CMakeLists.txt

 set(LLVM_OPTIONAL_SOURCES
   AffineAnalysis.cpp
   AffineStructures.cpp
   CallGraph.cpp
   Dominance.cpp
   InferTypeOpInterface.cpp
   Liveness.cpp
   LoopAnalysis.cpp
   NestedMatcher.cpp
   SliceAnalysis.cpp
   Utils.cpp
   Verifier.cpp
   )
 
-add_llvm_library(MLIRAnalysis
+add_mlir_library(MLIRAnalysis
   CallGraph.cpp
   InferTypeOpInterface.cpp
   Liveness.cpp
   SliceAnalysis.cpp
   Dominance.cpp
   Verifier.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis
 
   DEPENDS
   MLIRCallOpInterfacesIncGen
   MLIRTypeInferOpInterfaceIncGen
 
   LINK_LIBS
   MLIRAffineOps
   MLIRIR
   MLIRLoopOps
   MLIRSupport
   LLVMSupport
   )
 
-add_llvm_library(MLIRLoopAnalysis
+add_mlir_library(MLIRLoopAnalysis
   AffineAnalysis.cpp
   AffineStructures.cpp
   LoopAnalysis.cpp
   NestedMatcher.cpp
   Utils.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis

mlir/lib/Dialect/CMakeLists.txt

 add_subdirectory(AffineOps)
 add_subdirectory(FxpMathOps)
 add_subdirectory(GPU)
 add_subdirectory(Linalg)
 add_subdirectory(LLVMIR)
 add_subdirectory(LoopOps)
 add_subdirectory(OpenMP)
 add_subdirectory(QuantOps)
 add_subdirectory(SDBM)
 add_subdirectory(Shape)
 add_subdirectory(SPIRV)
 add_subdirectory(StandardOps)
 add_subdirectory(VectorOps)
 
-
 set(LLVM_OPTIONAL_SOURCES
   Traits.cpp
 )
 
-add_llvm_library(MLIRDialect
+add_mlir_library(MLIRDialect
   Traits.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect
 
   LINK_LIBS
   MLIRIR
   LLVMSupport
   )

mlir/lib/EDSC/CMakeLists.txt

 set(LLVM_OPTIONAL_SOURCES
   Builders.cpp
   CoreAPIs.cpp
   )
 
-add_llvm_library(MLIREDSC
+add_mlir_library(MLIREDSC
   Builders.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/EDSC
 
   LINK_LIBS
   MLIRIR
   MLIRSupport
   LLVMSupport
   )
 
-add_llvm_library(MLIREDSCInterface
+add_mlir_library(MLIREDSCInterface
   CoreAPIs.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/EDSC
 
   LINK_LIBS
   MLIRIR
   MLIRSupport
   MLIRParser
   LLVMSupport
   )

mlir/lib/ExecutionEngine/CMakeLists.txt

 set(LLVM_OPTIONAL_SOURCES
   CRunnerUtils.cpp
   ExecutionEngine.cpp
   RunnerUtils.cpp
   OptUtils.cpp
   )
 
 llvm_map_components_to_libnames(outlibs "nativecodegen" "IPO")
-add_llvm_library(MLIRExecutionEngine
+add_mlir_library(MLIRExecutionEngine
   ExecutionEngine.cpp
   OptUtils.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/ExecutionEngine
 
   LINK_LIBS
   MLIRIR

mlir/lib/IR/CMakeLists.txt

 file(GLOB globbed *.c *.cpp)
-add_llvm_library(MLIRIR
+add_mlir_library(MLIRIR
   ${globbed}
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/IR
 
   DEPENDS
   MLIRCallOpInterfacesIncGen
   MLIROpAsmInterfacesIncGen
 
   LINK_LIBS
   MLIRSupport
   LLVMSupport
   ${LLVM_PTHREAD_LIB}
   )

mlir/lib/Parser/CMakeLists.txt

-add_llvm_library(MLIRParser
+add_mlir_library(MLIRParser
   Lexer.cpp
   Parser.cpp
   Token.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Parser
 
   LINK_LIBS
   MLIRIR
   MLIRAnalysis
   LLVMSupport
   )

mlir/lib/Pass/CMakeLists.txt

 file(GLOB globbed *.c *.cpp)
-add_llvm_library(MLIRPass
+add_mlir_library(MLIRPass
   ${globbed}
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Pass
 
   LINK_LIBS
   MLIRAnalysis
   MLIRIR
   MLIRSupport
   LLVMIRReader
   LLVMSupport
   )

mlir/lib/Quantizer/CMakeLists.txt

 # Support.
-add_llvm_library(MLIRQuantizerSupport
+add_mlir_library(MLIRQuantizerSupport
   Support/Configuration.cpp
   Support/ConstraintAnalysisGraph.cpp
   Support/Metadata.cpp
   Support/Statistics.cpp
   Support/TypeUtils.cpp
   Support/UniformConstraints.cpp
   Support/UniformSolvers.cpp
 
   ADDITIONAL_HEADER_DIRS
 
   LINK_LIBS
   MLIRIR
   MLIRQuantOps
   MLIRSupport
   MLIRStandardOps
   LLVMSupport
   )
 
 # Configurations.
-add_llvm_library(MLIRQuantizerFxpMathConfig
+add_mlir_library(MLIRQuantizerFxpMathConfig
   Configurations/FxpMathConfig.cpp
 
   ADDITIONAL_HEADER_DIRS
 
   DEPENDS
   MLIRFxpMathOpsIncGen
 
   LINK_LIBS
   MLIRIR
   MLIRFxpMathOps
   MLIRQuantOps
   MLIRQuantizerSupport
   MLIRStandardOps
   MLIRSupport
   LLVMSupport
   )
 
 # Transforms.
-add_llvm_library(MLIRQuantizerTransforms
+add_mlir_library(MLIRQuantizerTransforms
   Transforms/AddDefaultStatsTestPass.cpp
   Transforms/InferQuantizedTypesPass.cpp
   Transforms/RemoveInstrumentationPass.cpp
 
   ADDITIONAL_HEADER_DIRS
 
   LINK_LIBS
   MLIRIR
   MLIRQuantizerFxpMathConfig
   MLIRQuantizerSupport
   MLIRQuantOps
   MLIRPass
   MLIRTransforms
   MLIRTransformUtils
   LLVMSupport
   )

mlir/lib/Support/CMakeLists.txt

 set(LLVM_OPTIONAL_SOURCES
   FileUtilities.cpp
   JitRunner.cpp
   MlirOptMain.cpp
   StorageUniquer.cpp
   ToolUtilities.cpp
   TranslateClParser.cpp
 )
 
-add_llvm_library(MLIRSupport
+add_mlir_library(MLIRSupport
   FileUtilities.cpp
   StorageUniquer.cpp
   ToolUtilities.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Support
 
   LINK_LIBS
   LLVMSupport
   ${LLVM_PTHREAD_LIB}
   )
 
-add_llvm_library(MLIROptLib
+add_mlir_library(MLIROptLib
   MlirOptMain.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Support
 
   LINK_LIBS
   MLIRIR
   MLIRPass
   MLIRParser
   MLIRSupport
   LLVMSupport
   )
 
-add_llvm_library(MLIRTranslateClParser
+add_mlir_library(MLIRTranslateClParser
   TranslateClParser.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Support
 
   LINK_LIBS
   LLVMSupport
   MLIRAnalysis

mlir/lib/Target/CMakeLists.txt

-add_llvm_library(MLIRTargetLLVMIRModuleTranslation
+add_mlir_library(MLIRTargetLLVMIRModuleTranslation
   LLVMIR/DebugTranslation.cpp
   LLVMIR/ModuleTranslation.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Target/LLVMIR
 
   DEPENDS
   intrinsics_gen
 
   LINK_LIBS
   MLIRLLVMIR
   LLVMCore
   LLVMIRReader
   LLVMSupport
   LLVMTransformUtils
   MLIRIR
   MLIRSupport
   MLIRTranslation
   )
 
-add_llvm_library(MLIRTargetLLVMIR
+add_mlir_library(MLIRTargetLLVMIR
   LLVMIR/ConvertFromLLVMIR.cpp
   LLVMIR/ConvertToLLVMIR.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Target/LLVMIR
 
   LINK_LIBS
   MLIRIR
   MLIRLLVMIR
   MLIRSupport
   MLIRTargetLLVMIRModuleTranslation
   MLIRTranslation
   LLVMCore
   LLVMIRReader
   LLVMSupport
   )
-add_llvm_library(MLIRTargetNVVMIR
+
+add_mlir_library(MLIRTargetNVVMIR
   LLVMIR/ConvertToNVVMIR.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Target/LLVMIR
 
   DEPENDS
   intrinsics_gen
 
   LINK_LIBS
   MLIRGPU
   MLIRIR
   MLIRLLVMIR
   MLIRNVVMIR
   MLIRTargetLLVMIRModuleTranslation
   MLIRTranslation
   LLVMCore
   LLVMSupport
   )
 
-add_llvm_library(MLIRTargetROCDLIR
+add_mlir_library(MLIRTargetROCDLIR
   LLVMIR/ConvertToROCDLIR.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Target/LLVMIR
 
   DEPENDS
   intrinsics_gen
 

mlir/lib/Transforms/CMakeLists.txt

 add_subdirectory(Utils)
 
-add_llvm_library(MLIRTransforms
+add_mlir_library(MLIRTransforms
   AffineDataCopyGeneration.cpp
   AffineLoopInvariantCodeMotion.cpp
   Canonicalizer.cpp
   CSE.cpp
   DialectConversion.cpp
   Inliner.cpp
   LocationSnapshot.cpp
   LoopCoalescing.cpp

mlir/lib/Transforms/Utils/CMakeLists.txt

-add_llvm_library(MLIRTransformUtils
+add_mlir_library(MLIRTransformUtils
   FoldUtils.cpp
   GreedyPatternRewriteDriver.cpp
   InliningUtils.cpp
   LoopFusionUtils.cpp
   LoopUtils.cpp
   RegionUtils.cpp
   Utils.cpp
 

mlir/lib/Translation/CMakeLists.txt

-add_llvm_library(MLIRTranslation
+add_mlir_library(MLIRTranslation
   Translation.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Translation
 
   LINK_LIBS
   LLVMSupport
 )

mlir/tools/CMakeLists.txt

 add_subdirectory(mlir-cuda-runner)
 add_subdirectory(mlir-cpu-runner)
 add_subdirectory(mlir-opt)
 add_subdirectory(mlir-tblgen)
 add_subdirectory(mlir-translate)
 add_subdirectory(mlir-vulkan-runner)
+add_subdirectory(mlir-shlib)

mlir/tools/mlir-opt/CMakeLists.txt

 set(LLVM_OPTIONAL_SOURCES
   null.cpp
 )
 
 set(LIB_LIBS
   MLIRAnalysis
+  MLIRIR
   MLIRLLVMIR
   MLIROptLib
   MLIRParser
   MLIRPass
   MLIRTransforms
   MLIRSupport
   LLVMSupport
   )
 add_llvm_library(MLIRMlirOptMain
   mlir-opt.cpp
 
   LINK_LIBS
   ${LIB_LIBS}
 )
 
 get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS)

mlir/tools/mlir-shlib/CMakeLists.txt

+# Building libmlir-cpp.so fails if LLVM_ENABLE_PIC=Off
+if (NOT LLVM_ENABLE_PIC)
+  return()
+endif()
+
+# Building libmlir-cpp.so may not work on MSVC
+if (MSVC)
+  return()
+endif()
+
+get_property(mlir_libs GLOBAL PROPERTY MLIR_ALL_LIBS)
+list(REMOVE_DUPLICATES mlir_libs)
+
+foreach (lib ${mlir_libs})
+  if(XCODE)
+    # Xcode doesn't support object libraries, so we have to trick it into
+    # linking the static libraries instead.
+    list(APPEND _DEPS "-force_load" ${lib})
+  else()
+    list(APPEND _OBJECTS $<TARGET_OBJECTS:obj.${lib}>)
+  endif()
+  list(APPEND _DEPS $<TARGET_PROPERTY:${lib},LINK_LIBRARIES>)
+endforeach ()
+
+if(MLIR_LINK_MLIR_DYLIB)
+  set(INSTALL_WITH_TOOLCHAIN INSTALL_WITH_TOOLCHAIN)
+endif()
+
+# libMLIR.so depends on LLVM components.  To avoid multiple
+# copies of those LLVM components, libMLIR.so depends on libLLVM.so.
+# This probably won't work if some LLVM components are not included
+# in libLLVM.so.
+if(LLVM_BUILD_LLVM_DYLIB)
+  add_llvm_library(MLIR
+    SHARED
+    ${INSTALL_WITH_TOOLCHAIN}
+
+    mlir-shlib.cpp
+    )
+  target_link_libraries(MLIR PRIVATE LLVM ${LLVM_PTHREAD_LIB})
+  whole_archive_link(MLIR ${mlir_libs})
+endif()

mlir/tools/mlir-shlib/mlir-shlib.cpp

+// Intentionally empty source file to make CMake happy