Support LLVM_BUILD_USER_FORTRAN_LIBS=[shared|static|all] cmake option

[pscoro]

Summary

The goal of this patch is to enable the user to have more control over the linkage type of the fortran libraries used by user executables. We want to be able to, for example, have all the LLVM libs build and link statically in flang-new, but at the same time retain the ability to have the Fortran user libs (FortranDecimal, FortranRuntime) also be built and linked dynamically for executables made using flang-new.

In order to do this, a new CMake option LLVM_BUILD_USER_FORTRAN_LIBS=[shared|static|all] can be passed. This option works on top of the decisions made by BUILD_SHARED_LIBS/BUILD_STATIC_LIBS by appending additional linkage types for just the user fortran libs. For the example above, this could be done by passing BUILD_SHARED_LIBS and also LLVM_BUILD_USER_FORTRAN_LIBS=shared. The BUILD_SHARED_LIBS will guarantee that all the LLVM libs get built and linked into flang-new statically. But on top of this, the new optoin thats passed will cause cmake to also build FortranDecimal and FortranRuntime as shared. By default, flang-new tried to dynamically link these 2 libraries into user executables. A future patch will introduce a flang-new driver option to allow the end user to choose between static and shared linking of these user fortran libs.

In the case that both linkage types are built by cmake, duplicate names are avoided by appending "_static" to the lib name. This is a feature that already existed in LLVM's cmake. There are some code changes in this patch that are needed to support/expect these "_static" named libs.

[DavidTruby]

Hi, thanks for the patch! I wonder if you could help me understand what's happening here as I'm not sure I fully get it.

Am I right in thinking that currently whether e.g. libFortranRuntime gets built as a shared or static library is solely dependent on BUILD_SHARED_LIBS? i.e. if you have BUILD_SHARED_LIBS set for all of LLVM you get a shared runtime library for flang, and no static one?
And so this patch is trying to allow you to control whether you get a fortran RTL as static or shared independently of the BUILD_SHARED_LIBS used for all of LLVM.

Is my understanding above correct?

[pscoro]

Am I right in thinking that currently whether e.g. libFortranRuntime gets built as a shared or static library is solely dependent on BUILD_SHARED_LIBS? i.e. if you have BUILD_SHARED_LIBS set for all of LLVM you get a shared runtime library for flang, and no static one?

So the thing that complicates the process here is that a different linkage type may be wanted for flang-new than for user executables. For example in our downstream build of flang, we are trying to have flang-new statically link all the llvm libs, including fortran libs like FortranDecimal. But at the same time, we want user fortran libs to get built as shared as well, so that the shared versions can be linked into user executables built by flang-new. To answer your question, BUILD_SHARED_LIBS=true will guarantee that e.g. libFortranRuntime gets built as shared, so if you have BUILD_SHARED_LIBS set you will get a shared runtime lib. However, additional linkage types for the user fortran libs can be also built with the new LLVM_USER_FORTRAN_LIBS flag. Lets say you did have BUILD_SHARED_LIBS set but you want the fortran libs also built statically (lets say that the flang-new driver option to specify --static libs exists), then you can additionally specify LLVM_BUILD_USER_FORTRAN_LIBS=static (or all) and this will build the lib statically in addition. The catch is that when both linkage types get built, the static version gets named e.g. FortranRuntime_static instead.

And so this patch is trying to allow you to control whether you get a fortran RTL as static or shared independently of the BUILD_SHARED_LIBS used for all of LLVM.

Not independently, but in addition to. I dont think this applies to FortranRuntime (I dont think it gets linked into flang-new?), but FortranDecimal needs to be linked in both flang-new and user executables, and we want to have the option to have the linkage types be different.

This entire patch is very open to design changes.. We had a specific goal of making it possible to build flang-new with static linkage while retaining shared linkage of user fortran libs in user executables (or more broadly, make it possible to choose static/shared linking for both flang-new/user executables). We wanted to achieve this in 2 steps:

1. by offering a cmake option that gives us more control over which linkage types are built for the user fortran libs (talking about libFortranRuntime and libFortranDecimal specifically) separately from the other LLVM libs, while retaining the current behavior of using BUILD_SHARED_LIBS to determine which link type actually gets linked into flang-new.
2. (next patch) by offering a new flang-new driver option to allow user to choose between static and shared fortran libs linkage into user executables. But in order for a user to, for example, specify --static libs for their executables, those fortran libs need to be built statically. In the scenario that flang-new is being built with shared linking, this patch provides the ability for those user fortran libs to also be built statically so that they can be provided as static libs for the end user.

@DavidTruby

[pscoro]

Also in case there is still some confusion, note the distinction between choosing which linkage types get built vs choosing which linkage types actually get linked into what. This patch aims to not change how cmake chooses which link type gets linked into flang-new (behaviour determined by BUILD_SHARED_LIBS). This patch enables someone to change which link types get built for certain libs. The next patch will enable someone to change which link types get linked into user executables, but in order for that to be possible, you first need to be able to control which link types get built for those libs, as provided by this patch.

[klausler]

In D153426#4438113, @pscoro wrote:

Not independently, but in addition to. I dont think this applies to FortranRuntime (I dont think it gets linked into flang-new?), but FortranDecimal needs to be linked in both flang-new and user executables, and we want to have the option to have the linkage types be different.

That's right. The same binary<->decimal conversion routines in FortranDecimal are used by both the compiler and the runtime. The runtime support library FortranRuntime is not linked to the compiler, although several of its headers are included by lowering source files.

[PeteSteinfeld]

Thanks for working on this. Once the design is set, could you please document this new build functionality? A good place to do so might be https://github.com/llvm/llvm-project/blob/main/flang/docs/GettingStarted.md.

[pscoro]

Thanks for working on this. Once the design is set, could you please document this new build functionality?

Sure thing. I will wait for a couple days as I expect that major design changes could be suggested by the community (since this touches some fundamental cmake code).

[awarzynski]

Thanks for working on this!

In D153426#4438645, @pscoro wrote:

Thanks for working on this. Once the design is set, could you please document this new build functionality?

Sure thing. I will wait for a couple days as I expect that major design changes could be suggested by the community (since this touches some fundamental cmake code).

It would be good to advertise this on Discourse for better visibility. I imagine that this will have a pretty large impact on folks creating LLVM packages.

I would also appreciate an overview of what the end goal is (you mentioned e.g. -static). And, for example, the interaction between:

• BUILD_SHARED_LIBS, and
• LLVM_BUILD_USER_FORTRAN_LIBS

Perhaps you could add a table in the summary that would show what happens when toggling these flags? And the impact on the runtime libs? (i.e. what libs are built)

[DavidTruby]

I have a couple of minor comments on the cmake here but I also agree we might want to discuss this in an RFC as well to hammer out exactly the behaviour we want. Thanks!

flang/CMakeLists.txt

9

Do we need to have _static added to the name? I think they can continue to have the same name as the file extensions will be different

12

This variable should be called FLANG_<something> as it is specific to the flang subproject. I also find this name a bit confusing as to me it suggests whether you're going to build the libraries or not, but the libraries will always need to be built. I also think RUNTIME would be clearer than USER as a term here.
I'd propose something like FLANG_RUNTIME_LIB_TYPES but am open to other suggestions.

flang/cmake/modules/AddFlang.cmake

112–132

can you split all this out into a function and call it in both places, rather than duplicating all the code?

[pscoro]

flang/CMakeLists.txt

9

I am doing some further investigating and am preparing a more detailed discourse post at the moment where this can be discussed more thoroughly but iirc the code that adds "_static" is actually in AddLLVM.cmake and is not added by this patch. The file extension is not enough to distinguish the libs in CMake. If there are 2 add_library calls with the same name, then later when that lib name is used in CMake code (eg, when its added as a dependency for a target) it is unclear which lib (static or shared) is being linked.

In actuality I think this behavior is already not very clear. I've been looking into the possibility of another variable (something like LLVM_LINK_TYPE or FLANG_LINK_TYPE, haven't decided on the scope). The issue is that when libs are built as both static AND shared, by default flang-new was linking the shared libs because they are the ones with the unmodified lib name (without _static). Currently, this patch reverses this for the fortran runtime libraries (FortranRuntime, FortranDecimal), but I'm not sure if this is the best way to handle this. My thinking is that if a variable like FLANG_LINK_TYPE=shared is passed when *all* lib types are being built (ignored if lib type != all), then the person building the compiler can choose whether the static or shared libs get linked into flang.

12

I also find this name a bit confusing as to me it suggests whether you're going to build the libraries or not, but the libraries will always need to be built

I understand that the libs always need to be built but the reason I added the word build in here is because im trying to make the distinction between which lib types are built and which ones are actually linked into flang-new. I like the name FLANG_RUNTIME_LIB_TYPES except for that the distinction is no longer made, if I say BUILD_SHARED_LIBS and FLANG_RUNTIME_LIB_TYPES=static, what this patch is arguing should happen is that *all* libs get built as shared and linked as shared into flang-new, but then in addition, the runtime libs get built as static (and thats it, they do not get linked into anything during the compiler building stage, they are just used by the user later). Because of this, I think if we just use FLANG_RUNTIME_LIB_TYPES I'm not sure its obvious that we are just building additional linkage types.

Based on this, what do you think of the name FLANG_BUILD_RUNTIME_LIBS=shared|static|all?

flang/cmake/modules/AddFlang.cmake

112–132

Will do

[pscoro]

The discourse thread for this patch is now open here: https://discourse.llvm.org/t/rfc-support-cmake-option-to-control-link-type-built-for-flang-runtime-libraries/71602

I want to note that while doing some further investigating and writing up that post, I began to doubt the need for a BUILD_STATIC_LIBS variable (this variable was being introduced in this patch and never used elsewhere). I will likely remove any reference to this variable.

[DavidTruby]

flang/CMakeLists.txt

9

Right, I actually hit something on Windows today that made me realise why we need to do this anyway; on Windows .lib files are generated for both static and dynamic builds of libraries, so they do need to have different names. That's why AddLLVM.cmake uses _static

12

Based on this, what do you think of the name FLANG_BUILD_RUNTIME_LIBS=shared|static|all?

Works for me :)

flang/CMakeLists.txt

 cmake_minimum_required(VERSION 3.20.0)
 
 if(NOT DEFINED LLVM_COMMON_CMAKE_UTILS)
   set(LLVM_COMMON_CMAKE_UTILS ${CMAKE_CURRENT_SOURCE_DIR}/../cmake)
 endif()
 include(${LLVM_COMMON_CMAKE_UTILS}/Modules/CMakePolicy.cmake
   NO_POLICY_SCOPE)
 
+# Cases where "all" lib types are built will cause libs with "_static"
+# appended to the lib name to be built.
+if ((BUILD_SHARED_LIBS AND BUILD_STATIC_LIBS) OR
+    (LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "all") OR
+    (BUILD_SHARED_LIBS AND LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "static") OR
+    (BUILD_STATIC_LIBS AND LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "shared"))
+  set_property(GLOBAL PROPERTY FORTRAN_STATIC_LIB_NAME TRUE)
+elseif()
+  set_property(GLOBAL PROPERTY FORTRAN_STATIC_LIB_NAME FALSE)
+endif()
+
 set(CMAKE_BUILD_WITH_INSTALL_NAME_DIR ON)
 
 # Flang requires C++17.
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED TRUE)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
 if (LLVM_ENABLE_EH)

flang/cmake/modules/AddFlang.cmake

else()
     if (BUILD_SHARED_LIBS AND NOT ARG_STATIC)
       set(LIBTYPE SHARED OBJECT)
     else()
       set(LIBTYPE STATIC OBJECT)
     endif()
     set_property(GLOBAL APPEND PROPERTY FLANG_STATIC_LIBS ${name})
   endif()
 
+  # Append additional linkage build types for fortran libraries used by users.
+  if (LLVM_BUILD_USER_FORTRAN_LIBS)
+    set(USER_FORTRAN_LIBS "FortranDecimal" "FortranRuntime")
+    if (${name} IN_LIST USER_FORTRAN_LIBS)
+      if (LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "all")
+        list(APPEND LIBTYPE SHARED STATIC)
+      elseif (LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "shared")
+        list(APPEND LIBTYPE SHARED)
+      elseif (LLVM_BUILD_USER_FORTRAN_LIBS STREQUAL "static")
+        list(APPEND LIBTYPE STATIC)
+      else()
+        message(FATAL_ERROR
+            "LLVM_BUILD_USER_FORTRAN_LIBS must be one of [static|shared|all].")
+      endif()
+    endif()
+  endif()
+
   llvm_add_library(${name} ${LIBTYPE} ${ARG_UNPARSED_ARGUMENTS} ${srcs})
 
   clang_target_link_libraries(${name} PRIVATE ${ARG_CLANG_LIBS})
 
+  set(name_static "${name}_static")
+  # Both name and name_static can exist at the same time.
+  if (TARGET ${name} OR TARGET ${name_static})
     if (TARGET ${name})
-
       if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY OR ${name} STREQUAL "libflang"
           OR ARG_INSTALL_WITH_TOOLCHAIN)
         get_target_export_arg(${name} Flang export_to_flangtargets UMBRELLA flang-libraries)
         install(TARGETS ${name}
           COMPONENT ${name}
           ${export_to_flangtargets}
           LIBRARY DESTINATION lib${LLVM_LIBDIR_SUFFIX}
           ARCHIVE DESTINATION lib${LLVM_LIBDIR_SUFFIX}
           RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}")
 
         if (NOT LLVM_ENABLE_IDE)
           add_llvm_install_targets(install-${name}
                                    DEPENDS ${name}
                                    COMPONENT ${name})
         endif()
 
         set_property(GLOBAL APPEND PROPERTY FLANG_LIBS ${name})
       endif()
       set_property(GLOBAL APPEND PROPERTY FLANG_EXPORTS ${name})
+    endif()
+    if (TARGET ${name_static})
+      if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY OR ${name_static} STREQUAL "libflang"
+          OR ARG_INSTALL_WITH_TOOLCHAIN)
+        get_target_export_arg(${name_static} Flang export_to_flangtargets UMBRELLA flang-libraries)
+        install(TARGETS ${name_static}
+          COMPONENT ${name_static}
+          ${export_to_flangtargets}
+          LIBRARY DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+          ARCHIVE DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+          RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}")
+
+        if (NOT LLVM_ENABLE_IDE)
+          add_llvm_install_targets(install-${name_static}
+                                   DEPENDS ${name_static}
+                                   COMPONENT ${name_static})
+        endif()
+
+        set_property(GLOBAL APPEND PROPERTY FLANG_LIBS ${name_static})
+      endif()
+      set_property(GLOBAL APPEND PROPERTY FLANG_EXPORTS ${name_static})
+    endif()
   else()
     # Add empty "phony" target
     add_custom_target(${name})
   endif()
 
   set_target_properties(${name} PROPERTIES FOLDER "Flang libraries")
   set_flang_windows_version_resource_properties(${name})
 endfunction(add_flang_library)

flang/lib/Evaluate/CMakeLists.txt

if(LIBPGMATH)
     else()
       message(STATUS "Libpgmath will not be used because C99 complex is not supported.")
     endif()
   else()
     message(STATUS "Libpgmath not found in: ${LIBPGMATH_DIR}")
   endif()
 endif()
 
+# If the user fortran libraries are built both shared and static,
+# FortranDecimal becomes FortranDecimal_static. If we want to build flang-new
+# statically, we must check if FortranDecimal_static exists.
+get_property(STATIC_NAME GLOBAL PROPERTY FORTRAN_STATIC_LIB_NAME)
+if(STATIC_NAME)
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal_static)
+else()
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal)
+endif()
+
 add_flang_library(FortranEvaluate
   call.cpp
   characteristics.cpp
   check-expression.cpp
   common.cpp
   complex.cpp
   constant.cpp
   expression.cpp
add_flang_library(FortranEvaluate
   static-data.cpp
   target.cpp
   tools.cpp
   type.cpp
   variable.cpp
 
   LINK_LIBS
   FortranCommon
-  FortranDecimal
+  ${USER_FORTRAN_LIBS}
   FortranParser
   ${LIBPGMATH}
 
   LINK_COMPONENTS
   Support
 
   DEPENDS
   acc_gen
   omp_gen
+  ${USER_FORTRAN_LIBS}
 )

flang/runtime/CMakeLists.txt

 
 # Disable libstdc++/libc++ assertions, even in an LLVM_ENABLE_ASSERTIONS build,
 # to avoid an unwanted dependency on libstdc++/libc++.so.
 add_definitions(-U_GLIBCXX_ASSERTIONS)
 add_definitions(-U_LIBCPP_ENABLE_ASSERTIONS)
 
 add_subdirectory(FortranMain)
 
+# If the user fortran libraries are built both shared and static,
+# FortranDecimal becomes FortranDecimal_static. If we want to build flang-new
+# statically, we must check if FortranDecimal_static exists.
+get_property(STATIC_NAME GLOBAL PROPERTY FORTRAN_STATIC_LIB_NAME)
+if(STATIC_NAME)
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal_static)
+else()
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal)
+endif()
+
 add_flang_library(FortranRuntime
   ISO_Fortran_binding.cpp
   allocatable.cpp
   array-constructor.cpp
   assign.cpp
   buffer.cpp
   command.cpp
   complex-powi.cpp
add_flang_library(FortranRuntime
   transformational.cpp
   type-code.cpp
   type-info.cpp
   unit.cpp
   unit-map.cpp
   utf.cpp
 
   LINK_LIBS
-  FortranDecimal
+  ${USER_FORTRAN_LIBS}
 
   INSTALL_WITH_TOOLCHAIN
 )

flang/tools/flang-driver/CMakeLists.txt

 # Infrastructure to build flang driver entry point. Flang driver depends on
 # LLVM libraries.
 
 # Set your project compile flags.
 link_directories(${LLVM_LIBRARY_DIR})
 
 set( LLVM_LINK_COMPONENTS
   ${LLVM_TARGETS_TO_BUILD}
   Option
   Support
   TargetParser
 )
 
+# If the user fortran libraries are built both shared and static, the static
+# libs get "_static" appended to the name. If we want to build flang-new
+# statically, we must check if the "_static" libraries exist.
+get_property(STATIC_NAME GLOBAL PROPERTY FORTRAN_STATIC_LIB_NAME)
+if(STATIC_NAME)
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal_static FortranRuntime_static)
+else()
+  list(APPEND USER_FORTRAN_LIBS FortranDecimal FortranRuntime)
+endif()
+
 add_flang_tool(flang-new
   driver.cpp
   fc1_main.cpp
 
   DEPENDS
   # These libraries are used in the linker invocation generated by the driver
   # (i.e. when constructing the linker job). Without them the driver would be
   # unable to generate executables.
-  FortranRuntime
-  FortranDecimal
+  ${USER_FORTRAN_LIBS}
   Fortran_main
 )
 
 target_link_libraries(flang-new
   PRIVATE
   flangFrontend
   flangFrontendTool
 )