This is an archive of the discontinued LLVM Phabricator instance.

[ELF] - Implemented support of default/non-default symbols versions
ClosedPublic

Authored by grimar on Jun 24 2016, 3:50 AM.

Details

Summary

It is possible to create new version of symbol instead of depricated one
using combination of version script and asm commands. For example:

__asm__(".symver b_1,b@LIBSAMPLE_1.0");
int b_1() { return 10; }
__asm__(".symver b_2,b@@LIBSAMPLE_2.0");
int b_2() { return 20; }

This code makes b_2() to be default implementation for b().
b_1() is used for compatibility with binaries compiled against
library of older version LIBSAMPLE_1.0.

This patch implements support for above functionality in lld.

Diff Detail

Repository
rL LLVM

Event Timeline

grimar updated this revision to Diff 61769.Jun 24 2016, 3:50 AM
grimar retitled this revision from to [ELF] - Implemented support of default/non-default symbols versions.
grimar updated this object.
grimar added reviewers: ruiu, rafael.
grimar added subscribers: llvm-commits, grimar.
ruiu edited edge metadata.Jun 24 2016, 4:34 AM

This may be an expensive feature because it works for each symbol. Can you run benchmarks with this change?

grimar added a comment.EditedJun 24 2016, 4:37 AM
In D21681#466350, @ruiu wrote:

This may be an expensive feature because it works for each symbol. Can you run benchmarks with this change?

Sure. But I think it should not be expensive. For symbols that does not have non-default versions almost nothing changed. (except single search of @ in name, not sure how much that is expensive).
I think there are no that many symbols that are affected by this change. Do you have suggestions for what to use for benchmark ?

ruiu added a comment.Jun 24 2016, 4:41 AM

Maybe a clang and lld link times?

Specifically, I'm wondering whether Name.find("@") is expensive or not.

In D21681#466355, @ruiu wrote:

Maybe a clang and lld link times?

Specifically, I'm wondering whether Name.find("@") is expensive or not.

Yes, I just updated my comment about this. That is the only change for them.

ruiu added inline comments.Jun 24 2016, 5:04 AM
ELF/SymbolTable.cpp
167 ↗(On Diff #61769)

Before running benchmarks, please change this to Name.find('@'). find(char) should be faster than find(StringRef).

grimar added inline comments.Jun 24 2016, 5:14 AM
ELF/SymbolTable.cpp
167 ↗(On Diff #61769)

Ok. Btw we probably can avoid doing that search if not linking shared or have no Config->SymbolVersions.
I think we can assume that there is no versioning an that case.

grimar updated this revision to Diff 61783.EditedJun 24 2016, 6:31 AM
grimar edited edge metadata.

Minor changes:

  1. "@" -> '@'
  2. Removed excessive empty lines in testcase.
  3. Rebased.

So below is my benchmark results. In short - there is no any perfomance hit observed.
I tried to link clang-3.9 with and w/o this patch using next commandline. (Iook it from clang -v output).

sudo schedtool -F -p 99 -a 0x1 -e perf stat -r 50 /home/umb/LLVM/llvm-build/bin/ld.lld -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o bin/clang-3.9 /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1 -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../.. -L/home/umb/LLVM/llvm-build/bin/../lib -L/lib -L/usr/lib -allow-shlib-undefined --export-dynamic -O3 tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o lib/libLLVMAArch64CodeGen.a lib/libLLVMAArch64AsmPrinter.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMAMDGPUAsmPrinter.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmPrinter.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMDisassembler.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFAsmPrinter.a lib/libLLVMBPFDesc.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMHexagonDisassembler.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmPrinter.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsInfo.a lib/libLLVMMipsDisassembler.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430AsmPrinter.a lib/libLLVMMSP430Desc.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXAsmPrinter.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmPrinter.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCInfo.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmPrinter.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcInfo.a lib/libLLVMSparcDisassembler.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmPrinter.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZInfo.a lib/libLLVMSystemZDisassembler.a lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a lib/libLLVMX86Disassembler.a lib/libLLVMXCoreCodeGen.a lib/libLLVMXCoreAsmPrinter.a lib/libLLVMXCoreDesc.a lib/libLLVMXCoreInfo.a lib/libLLVMXCoreDisassembler.a lib/libLLVMAnalysis.a lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a lib/libLLVMScalarOpts.a lib/libLLVMSupport.a lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a -lpthread lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangFrontendTool.a lib/libLLVMGlobalISel.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64AsmPrinter.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUAsmPrinter.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMDesc.a lib/libLLVMARMAsmPrinter.a lib/libLLVMARMInfo.a lib/libLLVMBPFAsmPrinter.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMMipsAsmPrinter.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430AsmPrinter.a lib/libLLVMNVPTXAsmPrinter.a lib/libLLVMPowerPCAsmPrinter.a lib/libLLVMPowerPCInfo.a lib/libLLVMSparcAsmPrinter.a lib/libLLVMSparcInfo.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZAsmPrinter.a lib/libLLVMSystemZInfo.a lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a lib/libLLVMXCoreAsmPrinter.a lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a lib/libLLVMXCoreInfo.a lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMCoverage.a lib/libLLVMObject.a lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a lib/libclangDriver.a lib/libLLVMOption.a lib/libLLVMProfileData.a lib/libclangParse.a lib/libLLVMMCParser.a lib/libclangSerialization.a lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -lpthread -lz -lm -rpath "\$ORIGIN/../lib" -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtend.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crtn.o

Without this patch result was:

     277.083272      task-clock (msec)         #    0.999 CPUs utilized            ( +-  0.23% )
              0      context-switches          #    0.000 K/sec                    ( +- 69.99% )
              0      cpu-migrations            #    0.000 K/sec                  
         48,403      page-faults               #    0.175 M/sec                    ( +-  0.04% )
<not supported>      cycles                   
<not supported>      stalled-cycles-frontend  
<not supported>      stalled-cycles-backend   
<not supported>      instructions             
<not supported>      branches                 
<not supported>      branch-misses            

    0.277259064 seconds time elapsed                                          ( +-  0.23% )

With this patch result was:

     276.607353      task-clock (msec)         #    0.999 CPUs utilized            ( +-  0.23% )
              0      context-switches          #    0.000 K/sec                    ( +- 69.99% )
              0      cpu-migrations            #    0.000 K/sec                  
         48,383      page-faults               #    0.175 M/sec                    ( +-  0.06% )
<not supported>      cycles                   
<not supported>      stalled-cycles-frontend  
<not supported>      stalled-cycles-backend   
<not supported>      instructions             
<not supported>      branches                 
<not supported>      branch-misses            

    0.276840515 seconds time elapsed                                          ( +-  0.23% )
emaste added a subscriber: emaste.Jun 24 2016, 6:40 AM
ruiu added inline comments.Jun 24 2016, 9:41 PM
ELF/SymbolTable.cpp
188 ↗(On Diff #61783)

This is O(n)*O(m) where n is the number of symbols with "@" and m is the number of symbols in version scripts. This is probably too inefficient. Please use a hash table.

211 ↗(On Diff #61783)

I don't see a reason to do this right here in insert(). You can add a new scan* path, no?

grimar added inline comments.Jun 25 2016, 12:31 AM
ELF/SymbolTable.cpp
188 ↗(On Diff #61783)

I think you're mistaken. Point here that I am iterating over symbol versions, not over all globals for each version.
So it is sure O(n)*O(m), but m is number of versions, not the number of symbols in version script. That should be very fast.

I think there should be little amount of both versions and symbols with @.

211 ↗(On Diff #61783)

scanVersionScript() iterates over globals of version script file, not iterates through all symtab.
We should set Sym->VersionedName flag somewhere. If I move this code to scanVersionScript(),
I`ll need to iterate over all symtab symbols to check the names, I do not think it is acceptable.

Also please look closer on a code from method above:

size_t I = 2;
  for (elf::Version &V : Config->SymbolVersions) {
    if (V.Name == Version) {

I am iterating over SymbolVersions here and check if versioned name has the same version.
Imagine that name of symbols are "foo@version1" and "foo@@version2".
Appropriate version script can be:

version0{  }; 
version1{  } version1; 
version2{ global: foo; } version2;

If we have only script, like we have in scanVersionScript(), we can imagine that
foo can also be foo@version0. I do not think we want to bruteforce name before
doing search in symtab, so again we return to the iterating over all symtab then.

So I think doing it here is just a faster and probably fastest way.

ruiu added inline comments.Jun 25 2016, 1:43 AM
ELF/SymbolTable.cpp
166 ↗(On Diff #61783)

You can make this function side-effect-free.

static uint16_t getVersionId(Symbol *Sym, StringRef Name)

which returns a version ID. Then use it in insert() like this.

Sym->VersionId = getVersionId(Sym, Name);
grimar added inline comments.Jun 25 2016, 9:55 AM
ELF/SymbolTable.cpp
166 ↗(On Diff #61783)

I can, but what about VersionedName flag ?
I need to set it too, I think the only way to do that without second search of '@' would be:

Sym->VersionId = getVersionId(Sym, Name); //search is inside
Sym->VersionedName = Sym->VersionId != VER_NDX_LOCAL && 
                                            Sym->VersionId != VER_NDX_GLOBAL;

Looks ok ?

ruiu added inline comments.Jun 26 2016, 2:07 AM
ELF/SymbolTable.cpp
166 ↗(On Diff #61783)

Well, if you can compute VersionedName from VersionId, you don't need to store it to a symbol. Instead, you can define bool isVersioned().

grimar updated this revision to Diff 61939.Jun 27 2016, 1:12 AM
  • Addressed review comments.
ELF/SymbolTable.cpp
166 ↗(On Diff #61783)

No, generally I can't compute it.
Only in this place it is possible to set this flag, because getVersionId() can return version > than VER_NDX_GLOBAL only if
version was extracted from symbol name, so it is versioned name. Later, scanVersionScript() will assign
versions to regular symbols and it will not be possible to compute if name is versioned or not.
So I need this flag I think.

ruiu added inline comments.Jun 27 2016, 1:21 AM
ELF/SymbolTable.cpp
166 ↗(On Diff #61939)

This function needs a function comment as it is not obvious what this is for.

189 ↗(On Diff #61939)

I wouldn't strip '@' and following characters because it is what users want to know.

191 ↗(On Diff #61939)

Since this is a dummy value, return something meaningless such as just 0.

ruiu added inline comments.Jun 27 2016, 1:34 AM
ELF/SymbolTable.cpp
172 ↗(On Diff #61939)

Remove Version variable and inline Config->VersionScriptGlobalByDefault ? ... here.

grimar updated this revision to Diff 61944.Jun 27 2016, 1:50 AM
grimar marked 4 inline comments as done.
  • Addressed review comments.
  • Reformatted comment.
grimar added inline comments.Jun 27 2016, 1:52 AM
ELF/SymbolTable.cpp
166 ↗(On Diff #61939)

Done.

172 ↗(On Diff #61939)

Done.

189 ↗(On Diff #61939)

Done.

191 ↗(On Diff #61939)

Done.

ruiu added inline comments.Jun 27 2016, 7:31 PM
ELF/SymbolTable.cpp
166 ↗(On Diff #61944)
// A symbol version may be included in a symbol name as a prefix after '@'.
// This function parses that part and returns a version ID number.
ELF/Symbols.cpp
106 ↗(On Diff #61944)

N is usually a number, so name this S.

109 ↗(On Diff #61944)

"@" -> '@'

grimar updated this revision to Diff 62059.Jun 27 2016, 11:15 PM
  • Addressed review comments.
ruiu accepted this revision.Jun 27 2016, 11:32 PM
ruiu edited edge metadata.

LGTM

This revision is now accepted and ready to land.Jun 27 2016, 11:32 PM
This revision was automatically updated to reflect the committed changes.
ruiu added a comment.Jul 15 2016, 3:55 PM

Unfortunately this change have slowed down the linker. Without this change, the linker was able to link LLD in 6.3426 seconds. With this, it takes 6.6993 seconds. It's not negligible. I'll investigate how to speed this up.

In D21681#486066, @ruiu wrote:

Unfortunately this change have slowed down the linker. Without this change, the linker was able to link LLD in 6.3426 seconds. With this, it takes 6.6993 seconds. It's not negligible. I'll investigate how to speed this up.

Forgot to reply here for history. So my comments are on page of your change (D22433).