This is an archive of the discontinued LLVM Phabricator instance.

Handle versioned symbols efficiently.
ClosedPublic

Authored by ruiu on Jul 15 2016, 6:54 PM.

Details

Summary

Versions can be assigned to symbols in two different ways.
One is the usual version scripts, and the other is special
symbol suffix '@'. If a symbol contains '@', the string after
that is considered to specify a version name.

Previously, we look for '@' for all symbols.

Anything that works on every symbol can be expensive because
the linker has to handle a lot of symbols. The search for '@'
was not an exception.

In this patch, I made two optimizations.

The first optimization is to handle '@' only when at least one
version is defined. If no versions are defined, no versions can
be assigned to any symbols, so it's waste of time to search for '@'.

The second optimization is to scan only suffixes of symbol names
instead of entire symbol names. Symbol names can be very long, but
symbol versions are usually short, so scanning entire symbol names
is waste of time, too.

There are some error cases which we no longer be able to detect
with this patch. I don't think it's a major drawback because they
are minor errors. Speed is more important.

This change improves LLD with debug info self-link time from
6.6993 seconds to 6.3426 seconds (or -5.3%).

Diff Detail

Repository
rL LLVM

Event Timeline

ruiu updated this revision to Diff 64219.Jul 15 2016, 6:54 PM
ruiu retitled this revision from to Handle versioned symbols efficiently..
ruiu updated this object.
ruiu added a reviewer: grimar.
ruiu added a subscriber: llvm-commits.
grimar edited edge metadata.EditedJul 16 2016, 2:19 AM

Interesting. What is wierd for me that I think I did perfomance tests that showed that there is no difference with/without the search. (https://reviews.llvm.org/D21681#466406). Since that time I was really sure that search of @ in symbol names is really quick operation.

I also have a little consern about this patch. If I do not missing anything, it should break the https://llvm.org/bugs/show_bug.cgi?id=28359 again ? Not sure do we really want that ?

grimar edited edge metadata.Jul 16 2016, 2:20 AM
grimar added a subscriber: emaste.
ruiu added a comment.Jul 16 2016, 11:40 AM

No, it won't bring back the issue as this code works only when you pass a version script. I verified that using the reproduce file that Ed posted.

grimar accepted this revision.Jul 16 2016, 11:29 PM
grimar edited edge metadata.
In D22433#486255, @ruiu wrote:

No, it won't bring back the issue as this code works only when you pass a version script. I verified that using the reproduce file that Ed posted.

Right, I see. LGTM with few nits.

ELF/SymbolTable.cpp
598 ↗(On Diff #64219)

I think return size_t would be more clean.

626 ↗(On Diff #64219)

This required the next change for me to compile using msvs2015:

for (elf::VersionDefinition &V : Config->VersionDefinitions) {
  if (V.Name == Verstr)
    return{ Name, IsDefault ? V.Id : (V.Id | VERSYM_HIDDEN) };
}
This revision is now accepted and ready to land.Jul 16 2016, 11:29 PM
ruiu added inline comments.Jul 16 2016, 11:58 PM
ELF/SymbolTable.cpp
598 ↗(On Diff #64219)

I agree, but it's a little bit tricky. At line 609, we are using the result to pass std::max. Since we want to compare the result of the expression against 0, we want to get a signed result. So I avoided using an unsigned value.

This revision was automatically updated to reflect the committed changes.
ruiu added a comment.Jul 17 2016, 10:31 AM

Submitted as r275711. Can you run your benchmark again so that we find what was wrong?

ELF/SymbolTable.cpp
626 ↗(On Diff #64219)

Fixed.

In D22433#486515, @ruiu wrote:

Submitted as r275711. Can you run your benchmark again so that we find what was wrong?

Sure, I`ll do that tomorrow.

In D22433#486515, @ruiu wrote:

Submitted as r275711. Can you run your benchmark again so that we find what was wrong?

By the way, do you want me to check the latest revision, or (what I guess) to retest the change performed by my patch initially ?

In D22433#486515, @ruiu wrote:

Submitted as r275711. Can you run your benchmark again so that we find what was wrong?

I probably missing something but I can't realize how you have so large numbers like 6-7 seconds in your benchmark.
The last my attemp was:

  1. I prepared release lld build. (I suppose you probably used debug build here ?)
  1. I configured another build with

-DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_PARALLEL_COMPILE_JOBS=8 -DLLVM_ENABLE_THREADS=true -DCMAKE_CXX_FLAGS="-fPIC -std=c++11 -fuse-ld=lld" -DCMAKE_C_FLAGS="-fPIC -fuse-ld=lld" -DCMAKE_C_COMPILER=/home/umb/LLVM/llvm-build/bin/clang -DCMAKE_CXX_COMPILER=/home/umb/LLVM/llvm-build/bin/clang++
So it uses everything from step 1. And builded it.

  1. After that I am trying to just relink the lld configured at step 2 with lld from step 1.

sudo schedtool -F -p 99 -a 0x1 -e perf stat -r 1 /home/umb/LLVM/llvm-build/bin/ld.lld -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o bin/lld /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtbegin.o -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1 -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../.. -L/home/umb/LLVM/llvm-build/bin/../lib -L/lib -L/usr/lib -allow-shlib-undefined -O3 --gc-sections tools/lld/tools/lld/CMakeFiles/lld.dir/lld.cpp.o -lpthread lib/liblldDriver.a lib/liblldCOFF.a lib/liblldELF.a lib/libLLVMSupport.a lib/liblldMachO.a lib/liblldReaderWriter.a lib/liblldYAML.a lib/liblldCore.a lib/libLLVMLibDriver.a lib/liblldConfig.a lib/libLLVMOption.a lib/libLLVMAArch64CodeGen.a lib/libLLVMGlobalISel.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64AsmPrinter.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUAsmPrinter.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDisassembler.a lib/libLLVMARMDesc.a lib/libLLVMARMAsmPrinter.a lib/libLLVMARMInfo.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFDesc.a lib/libLLVMBPFAsmPrinter.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDisassembler.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsAsmPrinter.a lib/libLLVMMipsDisassembler.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430Desc.a lib/libLLVMMSP430AsmPrinter.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXAsmPrinter.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCAsmPrinter.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMPowerPCInfo.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcAsmPrinter.a lib/libLLVMSparcDisassembler.a lib/libLLVMSparcInfo.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDisassembler.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZAsmPrinter.a lib/libLLVMSystemZInfo.a lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a lib/libLLVMXCoreCodeGen.a lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMSelectionDAG.a lib/libLLVMXCoreDesc.a lib/libLLVMXCoreAsmPrinter.a lib/libLLVMXCoreDisassembler.a lib/libLLVMXCoreInfo.a lib/libLLVMMCDisassembler.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a lib/libLLVMipo.a lib/libLLVMObject.a lib/libLLVMMCParser.a lib/libLLVMLinker.a lib/libLLVMIRReader.a lib/libLLVMBitReader.a lib/libLLVMAsmParser.a lib/libLLVMScalarOpts.a lib/libLLVMVectorize.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMSupport.a -lrt -ldl -lpthread -lz -lm -rpath "\$ORIGIN/../lib" -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-linux-gnu/5.3.1/crtend.o /usr/lib/gcc/x86_64-linux-gnu/5.3.1/../../../x86_64-linux-gnu/crtn.o

My relink time is still about 0.2-0.3 seconds then:

222.831375      task-clock (msec)         #    0.999 CPUs utilized          
         0      context-switches          #    0.000 K/sec                  
         0      cpu-migrations            #    0.000 K/sec                  
    35,448      page-faults               #    0.159 M/sec

So I see we do the benchmark differently. That I guess the reason of why I had no difference in results here: https://reviews.llvm.org/D21681#466406.

lld/trunk/test/ELF/verdef-defaultver.s