The partitioning feature was proposed here:
http://lists.llvm.org/pipermail/llvm-dev/2019-February/130583.html
This is mostly just documentation. The feature itself will be contributed
in subsequent patches.
Paths
| Differential D60242
Add IR support, ELF section and user documentation for partitioning feature. ClosedPublic Authored by pcc on Apr 3 2019, 5:34 PM.
Details
Summary The partitioning feature was proposed here: This is mostly just documentation. The feature itself will be contributed
Diff Detail
Event TimelineComment Actions I've put some suggestions for the documentation. Feel free to ignore them if you prefer the original. I can't see anything wrong with the LLVM changes at a cursory look.
pcc marked 11 inline comments as done. Comment Actions
Comment Actions Hi Peter, I really like the concept. I had some nebulous concerns when I read the RFC; but, I didn't comment as I couldn't think of any alternatives to what you were proposing. What I am writing here is still not fully formed.. so feel free to ignore if it is unhelpful. My main concern is that this is supporting a rare usecase and I don't really like the idea of adding complexity to the core tools for niche cases. I feel like it would be better if this could be implemented via post-processing in some way; outside of the core tools. Some aspect of this, like the linker being able to report all of the functions reachable from a given set of entry points, are of general utility though. I wonder if there is any scope for implementing something different along the lines of:
Comment Actions
Hi Ben, thanks for the feedback. I shared a similar concern around complexity, but I don't see any reasonable alternatives to implementing this directly in the linker. Your idea of using orderfiles is interesting, but I see at least two showstopper problems with it. The first is that it is incompatible with range extension thunks. Imagine that you have one main partition and two loadable partitions. An orderfile is used to sort the main partition's sections before partition 1's sections before partition 2's sections. We're going to split partition 1 and partition 2 into separate files, but the linker doesn't know that, so it may place a range extension thunk in the middle of partition 1's sections and have partition 2 call it. Now at runtime we will crash if partition 2 is loaded without partition 1 and it tries to call one of the range extension thunks in partition 1. We might be able to work around this issue somehow by telling the linker where the partition boundaries are, but at that point since the linker needs to know how you're going to split the file up anyway it might as well actually split it up for you. The second is that replacing parts of the file with zeros and compressing it isn't always useful from a size perspective; in many cases it's the uncompressed size that matters because that's what takes up storage space on the device. One might consider uncompressing the code at program startup but that leads to other problems: it slows down startup and prevents the kernel from paging out the pages containing the uncompressed code. I also see other problems that aren't showstoppers but are still significant:
pcc added a child revision: D60353: ELF: Add basic partition data structures and behaviours..Apr 5 2019, 7:09 PM Comment Actions
Thanks for considering this carefully. IMO most of your concerns could be accepted as reasonable limitations to keep the feature out of the linker. We could probably deal with the points that you raised with the compression scheme somehow (e.g. some horrible scheme that adjusts the existing program headers and adds an additional bss one, similar to your original proposal but done by post processing). However, I think your point about thunks is a hard showstopper. We would need a much more expressive means of controlling the linkers placement of things (than the current ordering files, linker scripts, --section-start etc..) in order to create a layout that would be amenable to post processing; and, as you hinted at, this would be just as much work (or more) than just getting the linker to do the splitting up. Given the above, I am happy with the implementation. Comment Actions Some inline comment requests and if you wouldn't mind changing the type allocator separately it'd be great. Otherwise LGTM. Thanks!
This revision is now accepted and ready to land.May 28 2019, 6:41 PM Comment Actions
Closed by commit rL361923: Add IR support, ELF section and user documentation for partitioning feature. (authored by pcc). · Explain WhyMay 28 2019, 8:27 PM This revision was automatically updated to reflect the committed changes. pcc marked an inline comment as done.
Revision Contents
Diff 198335 lld/docs/Partitions.rst
lld/docs/index.rst
lld/docs/partitions.dot
lld/docs/partitions.svg
llvm/docs/Extensions.rst
llvm/include/llvm/BinaryFormat/ELF.h
llvm/include/llvm/IR/GlobalValue.h
llvm/lib/AsmParser/LLLexer.cpp
llvm/lib/AsmParser/LLParser.cpp
llvm/lib/AsmParser/LLToken.h
llvm/lib/Bitcode/Reader/BitcodeReader.cpp
llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
llvm/lib/IR/AsmWriter.cpp
llvm/lib/IR/Globals.cpp
llvm/lib/IR/LLVMContextImpl.h
llvm/lib/IR/Type.cpp
llvm/lib/MC/MCParser/ELFAsmParser.cpp
llvm/lib/MC/MCSectionELF.cpp
llvm/lib/Object/ELF.cpp
llvm/test/Bitcode/compatibility.ll
llvm/test/CodeGen/X86/partition.ll
llvm/test/MC/ELF/section.s
llvm/test/Object/X86/irsymtab.ll
|
Program is an interesting choice of word here. From the initial RFC I think this is largely aimed at executables. I don't think that it would be impossible to make this work with shared-libraries, however I'd expect the constraints would make it difficult to use effectively. I'm thinking of an executable calling a function in the shared library that happens to be in a partition not loaded.
Maybe replace program with executable if that is the only option.