Page MenuHomePhabricator

[NOT FOR SUBMISSION] Fragmented DWARF prototype
Needs ReviewPublic

Authored by jhenderson on Oct 12 2020, 3:56 AM.

Details

Reviewers
espindola
MaskRay
Summary

This is the linker patch and python script I used to create and link the packages for my fragmented DWARF experiment. See the LLVM Dev 2020 "Fragmented DWARF" for full details.

To use the script, first create a linker reproducer package (optional, but easiest for reproducing the results), extract it, then run the following command:

python fragment_package.py -o <output directory> <input package directory> --bindir <directory containing llvm-dwarfdump and llvm-ar>

This will copy the input package to the output directory, fragmenting all ELF object files (including those in archives) along the way. It will also create a file called <something>.success next to each input file, after it has fragmented it, indicating to the script to skip the file next time, in case the process is aborted. Deleting these files or using --force-split will cause the script to not skip the files.

It's worth noting that at this time, the script does not attempt to patch up DWARF length fields, meaning the output DWARF will not be readable.

The linker changes are necessary to get the output to be what is expected, due to the slightly abused method of using SHF_LINK_ORDER in the script currently. Longer term this should be replaced with the use of SHT_GROUP sections instead. Note that these changes disable SHF_LINK_ORDER's ordering effect.

The linker can be used to link the output of the script as normal. If someone wishes to repeat the experiment I did to gather the figures in the presentation for their own code base, simply run the linker with and without --gc-sections. --mark-live-pc allows turning up the "aggressiveness" of GC-ing by skipping some of the relocations during liveness analysis. It takes a decimal from 0 to 1 (1 == default, normal GC), which represents the fraction of relocations to use per file. It's not an entirely accurate modelling (IIRC, the first relocation in each file is always used, for example), but gives some approximate controls.

Diff Detail

Unit TestsFailed

TimeTest
240 mslinux > lld.ELF::gc-sections-linkorder-err.s
Script: -- : 'RUN: at line 5'; /mnt/disks/ssd0/agent/llvm-project/build/bin/llvm-mc -filetype=obj -triple=x86_64 /mnt/disks/ssd0/agent/llvm-project/lld/test/ELF/gc-sections-linkorder-err.s -o /mnt/disks/ssd0/agent/llvm-project/build/tools/lld/test/ELF/Output/gc-sections-linkorder-err.s.tmp.o
170 mslinux > lld.ELF::linkorder-mixed.s
Script: -- : 'RUN: at line 4'; /mnt/disks/ssd0/agent/llvm-project/build/bin/llvm-mc -filetype=obj -triple=x86_64 /mnt/disks/ssd0/agent/llvm-project/lld/test/ELF/linkorder-mixed.s -o /mnt/disks/ssd0/agent/llvm-project/build/tools/lld/test/ELF/Output/linkorder-mixed.s.tmp.o
110 mslinux > lld.ELF::linkorder-script.s
Script: -- : 'RUN: at line 2'; /mnt/disks/ssd0/agent/llvm-project/build/bin/llvm-mc --triple=x86_64 -filetype=obj /mnt/disks/ssd0/agent/llvm-project/lld/test/ELF/linkorder-script.s -o /mnt/disks/ssd0/agent/llvm-project/build/tools/lld/test/ELF/Output/linkorder-script.s.tmp.o
130 mslinux > lld.ELF/linkerscript::linkorder-linked-to.s
Script: -- : 'RUN: at line 7'; /mnt/disks/ssd0/agent/llvm-project/build/bin/llvm-mc -filetype=obj --triple=x86_64 /mnt/disks/ssd0/agent/llvm-project/lld/test/ELF/linkerscript/linkorder-linked-to.s -o /mnt/disks/ssd0/agent/llvm-project/build/tools/lld/test/ELF/linkerscript/Output/linkorder-linked-to.s.tmp.o
130 mslinux > lld.ELF/linkerscript::linkorder.s
Script: -- : 'RUN: at line 2'; /mnt/disks/ssd0/agent/llvm-project/build/bin/llvm-mc -filetype=obj -triple=x86_64-pc-linux /mnt/disks/ssd0/agent/llvm-project/lld/test/ELF/linkerscript/linkorder.s -o /mnt/disks/ssd0/agent/llvm-project/build/tools/lld/test/ELF/linkerscript/Output/linkorder.s.tmp.o
View Full Test Results (15 Failed)

Event Timeline

jhenderson created this revision.Oct 12 2020, 3:56 AM
jhenderson requested review of this revision.Oct 12 2020, 3:56 AM