This is an archive of the discontinued LLVM Phabricator instance.

[LTO] Parse deplibs sections for possible libcalls.
AbandonedPublic

Authored by mysterymath on Jun 15 2022, 11:16 AM.

Details

Summary

[LTO] Parse deplibs sections for possible libcalls.

LTO codegen can add symbol references resolved by object files with
.deplibs sections. These object files will be added to the link by
symbol resolution, but parseFile() will never be called on the object
file, so the .deplibs will never be resolved. This can lead to undefined
references.

This change scans the deplibs sections of such object files before LTO
occurs and summarily adds these libraries to the link. Shared libraries
are added as if --as-needed were specified, since they may not end up
actually being required. Similarly, shared libraries will only be
extracted if something actually ends up referencing the newly added
symbols.

Since the problem was first observed on AArch64-specific libcalls, this
also adds an ELF lld target interface for obtaining target-specific
libcalls (those corresponding to setLibcallName() calls in target
lowering). These are filled in for AArch64.

Diff Detail

Event Timeline

mysterymath created this revision.Jun 15 2022, 11:16 AM
Herald added a project: Restricted Project. · View Herald Transcript

Add test case.

Fix requires.

Add comment.

Move parse loop earlier.

Update description.

Expand the summary.

mysterymath retitled this revision from Candidate fix. to [LTO] Parse input files added after LTO codegen..Jun 15 2022, 4:17 PM
mysterymath edited the summary of this revision. (Show Details)
mysterymath added reviewers: phosek, mcgrathr.
mysterymath published this revision for review.Jun 15 2022, 4:17 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 15 2022, 4:17 PM
MaskRay requested changes to this revision.Jun 15 2022, 4:29 PM

This breaks the symbol resolution model of LTO. The symbol resolution intends to know all symbols and no new symbol can be added after LTO compiling.

This revision now requires changes to proceed.Jun 15 2022, 4:29 PM

This breaks the symbol resolution model of LTO. The symbol resolution intends to know all symbols and no new symbol can be added after LTO compiling.

Does that apply to symbols like __aarch64_ldadd4_relax? AFAICT, these are only discovered after LTO completes, which is the origin of the issue.
There's a bit of logic to handleLibcalls() before the link, but only if the libcalls are supplied via bitcode themselves, but not if they're provided via object files, as in the test case.

I've created https://github.com/llvm/llvm-project/issues/56070 to track the general issue and provide more background on the issue's diagnosis.

I've created https://github.com/llvm/llvm-project/issues/56070 to track the general issue and provide more background on the issue's diagnosis.

Thanks and I tagged folks who may want to participate in the discussion. Sending the patch without tagging appropriate folks motivated me to click "Request Changes".

Extract deplibs before LTO, as discussed in #56070.

This substantially reworks the change to partially parse deplibs sections from
object files that provide libcalls. A target interface is also added to supply
target-specific libcall names; this is filled out for AArch64.

Any found shared libraries are added as-needed, since the libcalls that might
require them may never be emitted. Static libraries are added lazily, as usual.

mysterymath retitled this revision from [LTO] Parse input files added after LTO codegen. to [LTO] Parse deplibs sections for possible libcalls..Jun 23 2022, 5:19 PM
mysterymath edited the summary of this revision. (Show Details)
mysterymath abandoned this revision.Mar 15 2023, 4:36 PM

Since there wasn't a clear resolution around the ideal semantics, I'll abandon this change.