This is an archive of the discontinued LLVM Phabricator instance.

[Libomptarget] Add support for offloading binaries in libomptarget
ClosedPublic

Authored by jhuber6 on Jun 9 2022, 12:17 PM.

Details

Summary

The previous path changed the linker wrapper to embed the offloading
binary format inside the target image instead. This will allow us to
more generically bundle metadata with these images, such as requires
clauses or the target architecture it was compiled for.

I wasn't sure how to handle this best, so I introduced a new type that
replaces the old __tgt_device_image struct that we can expand inside
the runtime library. I made the new __tgt_device_binary struct pretty
much the same for now. In the future we could change this struct to
pretty much be the OffloadBinary class in the future.

Diff Detail

Event Timeline

jhuber6 created this revision.Jun 9 2022, 12:17 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 9 2022, 12:17 PM
jhuber6 requested review of this revision.Jun 9 2022, 12:17 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 9 2022, 12:17 PM
jhuber6 updated this revision to Diff 435905.Jun 10 2022, 7:09 AM

Fixing strcmp.

jhuber6 updated this revision to Diff 435924.Jun 10 2022, 8:26 AM

Fix problem with using pointers to thing in a vector, need to reserve the space beforehand so we don't realloc.

Instead of changing the __tgt_device_image, do you think it would be better to just to make a new type that only contains the extracted metadata? We could just pass a pointer to the into struct to the is_valid_binary, might be much cleaner. Then we would make a new __tgt_device_image that points to the actual ELF and use that as we do currently.

tianshilei1992 added inline comments.Jun 10 2022, 9:49 AM
openmp/libomptarget/include/omptarget.h
143

Is it better to define it as follows?

struct __tgt_device_binary {
  __tgt_device_image Image;
  __tgt_image_info Info;
}
jhuber6 added inline comments.Jun 10 2022, 9:55 AM
openmp/libomptarget/include/omptarget.h
143

I did it that way to somewhat reduce noise when changing to the new format. But now that I'm thinking about it, where else will we need this metadata? It might be fine to just parse it out and create new images so we don't change them, e.g.

__tgt_image_info info = getInfoFromImage(image); // Gets the strings out from the offload binary type in the __tgt_device_image`.
__tgt_device_image image = getELFFromImage(image); //Gets the ELF out and makes a new image with begin / end set correctly (Need to make a new one, the pointers are in RO memory).
is_binary_valid(image, info); // Plugin uses info struct if necessary, every other interface is the same.

WDYT?

openmp/libomptarget/include/omptarget.h
143

That looks neater.
In addition, we also want to have a clear definition for front end as well. I'm not saying the front end is gonna include the header. At least we have a clear definition with proper documentation, instead of nested definition in a function.

jhuber6 updated this revision to Diff 436009.Jun 10 2022, 12:18 PM

Chaning to use an additional struct instead. Also implemented the class to parse these.

Let me know if there are runtime considerations here, with the unique pointer allocations and string conversions for the map. The usage of __tgt_is_valid_binary is supposed to be mostly lightweight without invoking the runtime, so let me know if this is too expensive.

Is it necessary to create __tgt_image_info in libomptarget? If the front end emits a binary compatible with OffloadBinary, all the information a plugin needs is in it right? Can we just include the header and read those information in plugin instead of in libomptarget such that we don't need to change the interface?

openmp/libomptarget/include/OffloadBinary.h
71

you may want to make it private

72

Does it make more sense to return nullptr instead of "" here?

75

you may want to make it private

openmp/libomptarget/src/rtl.cpp
13–16

I'd do the following.

287–288

plus clang-format of course.

Is it necessary to create __tgt_image_info in libomptarget? If the front end emits a binary compatible with OffloadBinary, all the information a plugin needs is in it right? Can we just include the header and read those information in plugin instead of in libomptarget such that we don't need to change the interface?

I was originally thinking about doing that, but it's much noisier since we'd need to change a lot of code that actually loads the binaries. It would free us from reallocating these structs however. Otherwise we'd need to change each usage of the Image, this just extracts the image at the start and everything else stays the same. I don't know which is the superior approach. I find this is a "cleaner" way to augment the existing functionality while maintaining backwards compatibility, but maybe there's a need for everything else to have access to the metadata in the plugin.

openmp/libomptarget/include/OffloadBinary.h
71

It doesn't modify any content so

72

It's supposed to be a string, so an empty string is more appropriate than a null pointer I feel.

75

It should be private, I'll see if I can do that considering the static create functions calls it.

jhuber6 updated this revision to Diff 436384.Jun 13 2022, 7:35 AM

Making suggested changes.

jdoerfert added inline comments.Jun 13 2022, 7:44 AM
openmp/libomptarget/include/omptargetplugin.h
33

I am unsure if we should/need to create a new api here.

jhuber6 updated this revision to Diff 436400.Jun 13 2022, 8:18 AM

Changing the is_valid_binary function to be a new interface. This allows the plguins to optionally define the interface that uses the additional information struct, and we will use it if availible.

jhuber6 updated this revision to Diff 437244.Jun 15 2022, 10:48 AM

Changing the vector to a list so we always have stable pointers. Reserving a vector doesn't work because of shared libraries, also storing the extract information in the list as well.

jhuber6 updated this revision to Diff 445034.Jul 15 2022, 9:49 AM

A previous revision proposes making libomptarget link with the LLVM libraries. This allows us to use LLVMObject directly without duplicating code. Update this revision with that in mind.

JonChesterfield accepted this revision.Jul 15 2022, 1:59 PM

Looks ok to me. Does propagating const into __tgt_device_image (and possibly further) work to remove the const cast? Iirc the libelf interfaces take void* and libllvm take const, so it's possible we'll be able to gradually move const all the way through the stack.

Strictly I suppose const fields in the type are an abi change, we could static_assert that making them const doesn't change offsetof if feeling paranoid.

This revision is now accepted and ready to land.Jul 15 2022, 1:59 PM

Looks ok to me. Does propagating const into __tgt_device_image (and possibly further) work to remove the const cast? Iirc the libelf interfaces take void* and libllvm take const, so it's possible we'll be able to gradually move const all the way through the stack.

Strictly I suppose const fields in the type are an abi change, we could static_assert that making them const doesn't change offsetof if feeling paranoid.

Right now that would require const_cast somewhere else. I believe there are calls to libelf in the x86_64 version of the plugin which requires a char * input. We will probably be able to transition away from those if we are allowed to use LLVM libraries. The data that these images point to are global constants, so writing to them will result in whatever happens when you write to RO memory in an ELF.

This revision was landed with ongoing or failed builds.Jul 21 2022, 10:20 AM
This revision was automatically updated to reflect the committed changes.
openmp/libomptarget/src/rtl.cpp
457

This turns out to be a bad thing. unregisterLib can be called by the host application exit, at which point the plugin may have been unloaded. Is_valid_binary happens to work ok anyway, though I think that's sketchy, but is_valid_binary_info will call into cuda / hsa after they've been torn down and blow up.

jhuber6 added inline comments.Jul 28 2022, 5:30 AM
openmp/libomptarget/src/rtl.cpp
457

Is there a reason to even call this? We already determined the valid binaries before. We should only need to traverse a list of the valid ones here withotu checking it again.

openmp/libomptarget/src/rtl.cpp
528

I'm doubtful that calling into the plugin to run destructors is any safer than is_valid_binary_info. I think code is working because libomptarget happens to be torn down before the plugins but nothing particularly ensures that will be the case.

This is however a good place to call a destructor function that we add to the plugins.

So we can add tgt_rtl_init_plugin() and tgt_rtl_dtor_plugin() functions, replace the global state constructors in the plugins with those, call init shortly before init_device and call the dtor from here. That removes this undefined behaviour and fixes the segfault Joseph just noticed.