This is an archive of the discontinued LLVM Phabricator instance.

[ThinLTO] Consolidate cache key computation between new/old LTO APIs
ClosedPublic

Authored by tejohnson on Nov 16 2018, 10:00 AM.

Details

Summary

The old legacy LTO API had a separate cache key computation, which was
a subset of the cache key computation in the new LTO API (from what I
can tell this is largely just because certain features such as CFI,
dsoLocal, etc are only utilized via the new LTO API). However, having
separate computations is unnecessary (much of the code is duplicated),
and can lead to bugs when adding new optimizations if both cache
computation algorithms aren't updated properly - it's much easier to
maintain if we have a single facility.

This patch refactors the old LTO API code to use the cache key
computation from the new LTO API. To do this, we set up an lto::Config
object and fill in the fields that the old LTO was hashing (the others
will just use the defaults).

There are two notable changes:

  • I added a Freestanding flag to the LTO Config. Currently this is only

used by the legacy LTO API. In the patch that added it (D30791) I had
asked about adding it to the new LTO API, but it looks like that was not
addressed. This should probably be discussed as a follow up to this
change, as it is orthogonal.

  • The legacy LTO API had some code that was hashing the GUID of all

preserved symbols defined in the module. I looked back at the history of
this (which was added with the original hashing in the legacy LTO API in
D18494), and there is a comment in the review thread that it was added
in preparation for future internalization. We now do the internalization
of course, and that is handled in the new LTO API cache key computation
by hashing the recorded linkage type of all defined globals. Therefore I
didn't try to move over and keep the preserved symbols handling.

Diff Detail

Event Timeline

tejohnson created this revision.Nov 16 2018, 10:00 AM

I wonder freestanding is still an issue after r316819. There is a function attribute for that now so it should be respected if the module is built with -ffreestanding and it should create different hash. I will do some digging to see if I can find the original report and how they setup the build. @mehdi_amini, do you remember the original problem?

I also need to do a bit more thinking on the preserve symbol. I think the new LTO API went a completely different route from the old API.

I wonder freestanding is still an issue after r316819. There is a function attribute for that now so it should be respected if the module is built with -ffreestanding and it should create different hash. I will do some digging to see if I can find the original report and how they setup the build. @mehdi_amini, do you remember the original problem?

There is some background on the bug (https://bugs.llvm.org/show_bug.cgi?id=30403). Per comment 2, the problem was a different issue than what is solved by the function attribute (I checked, the application of the nobuiltin attribute predates D30791). But since there should be a function attribute applied, that presumably would affect the hash as you noted. Hopefully @pcc can comment as he did some investigation on the original bug.

I also need to do a bit more thinking on the preserve symbol. I think the new LTO API went a completely different route from the old API.

steven_wu accepted this revision.Nov 26 2018, 10:06 AM

I did some experiment and put some more thought into this patch. Not hashing PreservedSymbol passed to the C API is fine because the reason you mentioned but freestanding/no-builtin bits stills need to be handled separately in current status. That should be a different discussion so this patch is LGTM.

This revision is now accepted and ready to land.Nov 26 2018, 10:06 AM
This revision was automatically updated to reflect the committed changes.