This is an archive of the discontinued LLVM Phabricator instance.

[ThinLTO] Fix internalization decisions for weak/linkonce ODR
ClosedPublic

Authored by tejohnson on Jun 1 2023, 8:57 PM.

Details

Summary

This fixes a runtime error that occurred due to incorrect
internalization of linkonce_odr functions where function pointer
equality was broken. This was hit because the prevailing copy was in a
native object, so the IR copies were not exported, and the existing code
internalized all of the IR copies. It could be fixed by guarding this
internalization on whether the defs are (local_)unnamed_addr, meaning
that their address is not significant (which we have in the summary
currently for linkonce_odr via the CanAutoHide flag). Or we can
propagate reference attributes as we do when determining whether a
global variable is read or write-only (reference edges are annotated
with whether they are read-only, write-only, or neither, and taking the
address of a function would result in a reference edge to the function
that is not read or write-only).

However, this exposed a larger issue with the internalization handling.
Looking at test cases, it appears the intent is to internalize when
there is a single definition of a linkonce/weak ODR symbol (that isn't
exported). This makes sense in the case of functions, because the
inliner can apply its last call to static heuristic when appropriate. In
the case where there is no prevailing copy in IR, internalizing all of
the IR copies of a linkonce_odr, even if legal, just increases binary
size. In that case it is better to fall back to the normal handling of
converting all non-prevailing copies to available_externally so that
they are eliminated after inlining.

In the case of variables, the existing code was attempting to
internalize the non-exported linkonce/weak ODR variables if they were
read or write-only. While this is legal (we propagate reference
attributes to determine this information), we don't even need to
internalize these here as there is later separate handling that
internalizes read and write-only variables when we process the module at
the start of the ThinLTO backend (processGlobalForThinLTO). Instead, we
can also internalize any non-exported variable when there is only one
(IR) definition, which is prevailing. And in that case, we don't need to
require that it is read or write-only, since we are guaranteed that all
uses must use that single definition.

In the new LTO API, if there are multiple defs of a linkonce or weak ODR
it will be marked exported, but it isn't clear that this will always be
true for the legacy LTO API. Therefore, require that there is only a
single (non-local) def, and that it is prevailing.

The test cases changes are both to reflect the change in the handling of
linkonce_odr IR copies where the prevailing def is not in IR (the main
correctness bug fix here), and to reflect the more aggressive
internalization of variables when there is only a single def, it is in
IR, and not exported.

I've also added some additional testing via the new LTO API.

Diff Detail

Event Timeline

tejohnson created this revision.Jun 1 2023, 8:57 PM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptJun 1 2023, 8:57 PM
Herald added a reviewer: Restricted Project. · View Herald Transcript
tejohnson requested review of this revision.Jun 1 2023, 8:57 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 1 2023, 8:57 PM
steven_wu accepted this revision.Jun 2 2023, 10:29 AM

I think this should work for legacy API. ld64 always picks the prevailing one to be from native object file and ask LTO to reserve the symbol (so LTO cannot internalize them) so it doesn't really hit the bug mentioned. It is not the ideal solution but it is the best can be done when no symbol resolution information is available from API.

It should be safe to assume atom only gets referenced once if there is only one visible and it is not in the reserved symbol list. I don't think preserved symbols will turn into prevailing, but I don't think they can be internalized in this case too. Would be good to add a test case to make sure if the weak/linkonce in the preserve GUID will get the correct behavior.

LGTM.

This revision is now accepted and ready to land.Jun 2 2023, 10:29 AM

I think this should work for legacy API. ld64 always picks the prevailing one to be from native object file and ask LTO to reserve the symbol (so LTO cannot internalize them) so it doesn't really hit the bug mentioned. It is not the ideal solution but it is the best can be done when no symbol resolution information is available from API.

It should be safe to assume atom only gets referenced once if there is only one visible and it is not in the reserved symbol list. I don't think preserved symbols will turn into prevailing, but I don't think they can be internalized in this case too. Would be good to add a test case to make sure if the weak/linkonce in the preserve GUID will get the correct behavior.

The only way I can think of to test that is to use llvm-lto with --exported-symbol=, which preserves that symbol via ThinLTOCodeGenerator::preserveSymbol. If that's what you mean, there is already a check of this case in llvm/test/ThinLTO/X86/weak_resolution.ll:

; When exported, we always preserve a linkonce
; RUN: llvm-lto -thinlto-action=promote %t.bc -thinlto-index=%t3.bc -o - --exported-symbol=_linkonceodrfuncInSingleModule | llvm-dis -o - | FileCheck %s --check-prefix=EXPORTED
...
; EXPORTED: define weak_odr void @linkonceodrfuncInSingleModule()

Let me know if you meant something else.

LGTM.

The only way I can think of to test that is to use llvm-lto with --exported-symbol=, which preserves that symbol via ThinLTOCodeGenerator::preserveSymbol.

So the new API users don't use the exported symbol function? Then current test case looks fine to me.

The only way I can think of to test that is to use llvm-lto with --exported-symbol=, which preserves that symbol via ThinLTOCodeGenerator::preserveSymbol.

So the new API users don't use the exported symbol function? Then current test case looks fine to me.

Right. The new LTO expects the linker (or llvm-lto2) to give the exact resolution of every non-internal symbol defined or referenced in each module (i.e. whether prevailing, exported, etc).

ychen added a subscriber: ychen.Jun 2 2023, 2:27 PM
This revision was landed with ongoing or failed builds.Jun 2 2023, 3:34 PM
This revision was automatically updated to reflect the committed changes.