Page MenuHomePhabricator

[WIP] [ELF] Implement --copy-dt-needed-entries
Needs ReviewPublic

Authored by mgorny on Jan 13 2019, 3:19 AM.

Details

Summary

This is my proof-of-concept on making --copy-dt-needed-entries work. Apparently, i've been able to hack on it hard enough to make it actually emit DT_NEEDED for all symbols, recursively:

$ clang -fuse-ld=lld  -Wl,--copy-dt-needed-entries test.c  -lgit2
$ readelf -d a.out 

Dynamic section at offset 0x3010 contains 40 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libgit2.so.27]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libcurl.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libcares.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libnghttp2.so.14]
 0x0000000000000001 (NEEDED)             Shared library: [libidn2.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libunistring.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.1.1]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.1.1]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libhttp_parser.so.2.9]
 0x0000000000000001 (NEEDED)             Shared library: [libssh2.so.1]
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000007 (RELA)               0x200620
 0x0000000000000008 (RELASZ)             48 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000017 (JMPREL)             0x200650
 0x0000000000000002 (PLTRELSZ)           72 (bytes)
 0x0000000000000003 (PLTGOT)             0x202010
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000006 (SYMTAB)             0x2002b0
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000005 (STRTAB)             0x200488
 0x000000000000000a (STRSZ)              404 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x200400
 0x0000000000000004 (HASH)               0x200428
 0x0000000000000019 (INIT_ARRAY)         0x203008
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x203000
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000000000000c (INIT)               0x201200
 0x000000000000000d (FINI)               0x201218
 0x000000006ffffff0 (VERSYM)             0x2003b8
 0x000000006ffffffe (VERNEED)            0x2003d0
 0x000000006fffffff (VERNEEDNUM)         1
 0x0000000000000000 (NULL)               0x0

Can't say the code is nice but before I try to make it cleanly, I'd appreciate some tips on whether I'm even starting in the right direction.

FWICS, GNU ld's --copy-dt-needed-entries adds lot less libraries — it seems as if it implicitly forced --as-needed, and --no-as-needed doesn't seem to make a difference.

Diff Detail

Event Timeline

mgorny created this revision.Jan 13 2019, 3:19 AM
joerg added a subscriber: joerg.Jan 14 2019, 1:09 PM

As first step, it goes into the right direction. I would explicitly set --as-needed for all those indirectly loaded objects. If people want to retain the questionable behavior of newer GNU tools, it could be a separate flag so that a final round can warn if an indirectly pulled library is necessary, but that behavior doesn't IMO make much sense. Full version has to look at DT_RUNPATH/DT_RPATH and also --rpath-link.

grimar added a subscriber: grimar.Jan 15 2019, 2:36 AM
Herald added a project: Restricted Project. · View Herald TranscriptMar 3 2019, 11:02 AM
Herald added a subscriber: MaskRay. · View Herald Transcript
MaskRay added a comment.EditedMar 3 2019, 7:29 PM

I read your https://blog.netbsd.org/tnf/entry/the_first_report_on_lld :) The chapter "Handling of indirect shared library dependencies" summarizes the transitive DT_NEEDED problem well.

But do we have a convincing list of applications that really need this option to work? According my understanding, lld doesn't try to be fully ld.bfd compatible. It only implements reasonable ld.bfd features that are still used a lot. lld is enabled on FreeBSD 12 and is capable to link nearly all applications (with ~10 exceptions; but nothing uses --copy-dt-needed-entries) (https://bugs.llvm.org/show_bug.cgi?id=23214#c59)

gold doesn't support --copy-dt-needed-entries https://sourceware.org/git/?p=binutils-gdb.git;a=blob;hb=b09e83c1f814a6f218519abeb94aaf45869ba03c;f=gold/object.cc#l3236 It doesn't support --rpath-link either.

In the article,

However, GNU gold, LLD and newer versions of GNU ld use different logic and permit only symbols provided by the direct dependencies.

I just checked that ld.bfd doesn't enable this option by default. It looks like a historical remnant then :)

NetBSD patches GNU linker to behave in the original way. This behavior is mandated from lld as well in order to treat it as a drop-in replacement.

Preference here and of other lld characteristics is so strong that the NetBSD community in case of rejected patches will continue to maintain downstream patches.