[RFC][OpenMP][Doc] No backward compatible for libomptarget and plugins

[tianshilei1992]

Summary

Now we state that backward compatibility is not guaranteed in the document.

[JonChesterfield]

I'm not sure they were ever backwards compatible. We definitely didn't test mixing pieces across releases. Perhaps we should phrase it as "they are not backwards compatible", dropping the "no longer" part?

[tianshilei1992]

In D133277#3769403, @JonChesterfield wrote:

I'm not sure they were ever backwards compatible. We definitely didn't test mixing pieces across releases. Perhaps we should phrase it as "they are not backwards compatible", dropping the "no longer" part?

I think in the past we were trying to make it backward compatible. For example, we keep all existing interfaces unchanged, and their behaviors unchanged. Also we only add new interfaces and never remove old ones. However, as you said, we didn't promise it, and didn't test that either. I can remove it from the description.

[tianshilei1992]

fix comment

[tianshilei1992]

fix

[JonChesterfield]

LG, thanks!

[jhuber6]

This is emergent from being based on LLVM which isn't compatible between versions I'm assuming. We should probably discuss this at the multi-company meeting as it would have some other ramifications. For example, we wouldn't need to keep the legacy interfaces, and having version fields wouldn't be necessary as we can strictly control the version without an installation.

[tianshilei1992]

In D133277#3769421, @jhuber6 wrote:

This is emergent from being based on LLVM which isn't compatible between versions I'm assuming. We should probably discuss this at the multi-company meeting as it would have some other ramifications. For example, we wouldn't need to keep the legacy interfaces, and having version fields wouldn't be necessary as we can strictly control the version without an installation.

Yes, I already put the two patches at the top of the discussion doc.

[dreachem]

openmp/docs/SupportAndFAQ.rst

340

[dreachem]

@tianshilei1992 Can you explain a bit more what is the motivation here? Does requiring a new entry point rather than changing the interface for a pre-existing entry point create problems? The Cray compiler doesn't use libomptarget, but it does use the interface. It is also designed to support interoperability with other LLVM-based compilers, which may be based on a newer version of LLVM. Backwards compatibility with the libomptarget interface is needed for that to work (not necessary for the plugins).

[jhuber6]

In D133277#3774954, @dreachem wrote:

@tianshilei1992 Can you explain a bit more what is the motivation here? Does requiring a new entry point rather than changing the interface for a pre-existing entry point create problems? The Cray compiler doesn't use libomptarget, but it does use the interface. It is also designed to support interoperability with other LLVM-based compilers, which may be based on a newer version of LLVM. Backwards compatibility with the libomptarget interface is needed for that to work (not necessary for the plugins).

Currently we attempt to maintain backwards compatibility with old programs being run with a new libomptarget build. We do this by keeping around the legacy function interfaces, even if they are slower now. This was never an explicitly stated goal guarantee , but it has some benefits for users on clusters switching versions of LLVM without needing to recompile. Now that we have made libomptarget and the plugins an LLVM library we use versioned libraries, e.g. libomptarget.so is a symbolic link to libomptarget.so.16. LLVM isn't backwards compatible between versions, so if users require an older version they can stick with the versioned library that they require. The argument is that we should do the same with libomptarget as this would allow us to make useful changes to the binary and remove some legacy interfaces. We could keep the old entries into libomptarget, but there are some we'd like to change. For example, I would like to remove the omp.register_requires function and replace it with a new interface. This would be a breaking change if we expect backward compatibility as the old function would either not exist or be a no-op. I would like the ability to make changes like this, I'm not exactly sure what CUDA's backwards compatibility method is between CUDA releases, we may be able to compare policies for what users expect.

[tianshilei1992]

In D133277#3774954, @dreachem wrote:

@tianshilei1992 Can you explain a bit more what is the motivation here? Does requiring a new entry point rather than changing the interface for a pre-existing entry point create problems? The Cray compiler doesn't use libomptarget, but it does use the interface. It is also designed to support interoperability with other LLVM-based compilers, which may be based on a newer version of LLVM. Backwards compatibility with the libomptarget interface is needed for that to work (not necessary for the plugins).

Like @jhuber6 mentioned, the libraries are not backward compatible any more, thus there are pointless to maintain the interfaces compatibility. It will be more convenient and more efficient.

However, I got your point. You still hope applications generated by Cray compiler can be executed with upstream libomptarget. That is a good design. Does it make sense for Cray to just support the interoperability with specific version of libomptarget? For example, say CCE x.y.z is compatible with libomptarget.so.16, and CCE k.m.n is compatibly with libomptarget.so.17, and so on. That will require Cray compiler to follow the interface changes in upstream. After all, libomptarget never claims to be backward compatibility and API stable. In the past we just "tried" to maintain it, but in fact never tested that. IIRC, we never expect users to used the mixed versions of libraries.

[tianshilei1992]

rewrite

[RaviNarayanaswamy]

openmp/docs/SupportAndFAQ.rst

341–342

I don't think we should say this. We don't say what user can do, we already say they should not mix libraries and compiler.

[dreachem]

openmp/docs/SupportAndFAQ.rst

338–342

[tianshilei1992]

fix comments

[tianshilei1992]

@dreachem Does it look good to you now?

[dreachem]

Looks good.

openmp/docs/SupportAndFAQ.rst

 ``LIBOMPTARGET_STACK_SIZE`` can be used to increase the stack size if this
 occurs.
 
 Q: Can OpenMP offloading compile for multiple architectures?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Since LLVM version 15.0, OpenMP offloading supports offloading to multiple
 architectures at once. This allows for executables to be run on different
 targets, such as offloading to AMD and NVIDIA GPUs simultaneously, as well as
 multiple sub-architectures for the same target. Additionally, static libraries
 will only extract archive members if an architecture is used, allowing users to
 create generic libraries.
 
 The architecture can either be specified manually using ``--offload-arch=``. If
 ``--offload-arch=`` is present no ``-fopenmp-targets=`` flag is present then the
 targets will be inferred from the architectures. Conversely, if
 ``--fopenmp-targets=`` is present with no ``--offload-arch``  then the target
 architecture will be set to a default value, usually the architecture supported
 by the system LLVM was built on.
 
 For example, an executable can be built that runs on AMDGPU and NVIDIA hardware

 ``--offload-link``. Additionally, ``-fgpu-rdc`` must be used to create a
 linkable device image.
 
 .. code-block:: shell
 
    clang++ openmp.cpp -fopenmp --offload-arch=sm_80 -c
    clang++ cuda.cu --offload-new-driver --offload-arch=sm_80 -fgpu-rdc -c
    clang++ openmp.o cuda.o --offload-link -o app
+
+Q: Are libomptarget and plugins backward compatible?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+No. libomptarget and plugins are now built as LLVM libraries starting from LLVM
+15. Because LLVM libraries are not backward compatible, libomptarget and plugins
+are not as well. Given that fact, the interfaces between 1) the Clang compiler
+and libomptarget, 2) the Clang compiler and device runtime library, and
+3) libomptarget and plugins are not guaranteed to be compatible with an earlier
+version. Users are responsible for ensuring compatibility when not using the
+Clang compiler and runtime libraries from the same build. Nevertheless, in order
+to better support third-party libraries and toolchains that depend on existing
+libomptarget entry points, contributors are discouraged from making
+modifications to them.