This is an archive of the discontinued LLVM Phabricator instance.

[LLD] New ELF implementation
ClosedPublic

Authored by Bigcheese on Jul 14 2015, 11:27 AM.

Details

Summary

This is a direct port of the new PE/COFF linker to ELF. It can link simple object files, but doesn't do relocations yet.

The plan for this is to implement gnu-ld semantics and see what code can be shared with the new PE/COFF implementation, if any.

Diff Detail

Repository
rL LLVM

Event Timeline

Bigcheese updated this revision to Diff 29688.Jul 14 2015, 11:27 AM
Bigcheese retitled this revision from to [LLD] New ELF implementation.
Bigcheese updated this object.
Bigcheese added a subscriber: llvm-commits.
atanasyan edited edge metadata.Jul 14 2015, 1:36 PM

Just a few nits.

The patch contains some code copied from COFF especially in the driver. Do you plan to cleanup it before commit?

ELF/Chunks.h
44–46 ↗(On Diff #29688)

Make these methods const

73–74 ↗(On Diff #29688)

Make these methods const

ELF/Driver.cpp
57 ↗(On Diff #29688)

Does the exe extension has a sense for ELF file?

84 ↗(On Diff #29688)

s/obj/o/

97 ↗(On Diff #29688)

s/obj/o/

120 ↗(On Diff #29688)

This code is not applicable for ELF

124 ↗(On Diff #29688)

/nodefaultlib is not applicable for ELF.

135 ↗(On Diff #29688)

The code below are from the COFF linker. Do we really need to commit it to ELF?

ELF/InputFiles.cpp
33 ↗(On Diff #29688)

Is this code equal to llvm::sys::path::filename?

137 ↗(On Diff #29688)

Will this function return something else except "success" code?

ELF/Symbols.h
166 ↗(On Diff #29688)

I guess it is a COFF equivalent for ELF Weak symbol?

ruiu edited edge metadata.Jul 14 2015, 2:14 PM

I want you to strip this patch even more. Especially I'd like you to remove LTO completely from this initial patch. LTO code is fairly large but totally dead and untested. We can restore that when we actually start working on that.

I'd also want you to remove Windows-specific code as I left comments.

Overall, the code looks good -- if I'm eligible to say that. This is really a direct translation of my COFF linker to ELF, and of course I didn't find any difficulty to read and understand the code. The duplication of code with COFF is a bit concerning, but I'd imagine that if we try to merge this with COFF, it would become very messy because so many little things are different in many places. This is what I learned from the old LLD. We'd have eventually had to add tons of hooks and virtual function calls everywhere. So I think this is practical approach to port the COFF linker to ELF. We should keep COFF and ELF in sync as much as possible, and factor out common code to libObject or other LLVM library to remove code from COFF or ELF directories.

Regarding Windows-ness of the COFF linker which has been directly copied to this ELF port. The symbol resolution semantics in particular is not compatible with ELF. As I described in COFF/README.md, the symbol table and SymbolBodies implement Windows symbol resolution semantics. As a result, this ELF linker would behave as if all input files are wrapped with --start-group and --end-group. This would be more efficient than the regular Unix linker because the Unix semantics is too dumb, but it's not fully compatible with other linkers (although it's mostly compatible IMO). So there's a tradeoff between "new, efficient but incompatible way" and "dumb but compatible way".

I'm totally on the side to keep this new semantics in this ELF linker. --start-gropu and --end-group is a hack to deal with Unix linker's dumb semantics, and if we can replace that with better semantics, we should do that using this chance to rewrite a ELF linker. This is much better than before, and it open the possibility of significant optimization. I also think that we can implement backward-compatible mode to this ELF linker without too much trouble. My ambition is that ultimately we would provide "better and mostly compatible" and "slow and fully compatible" options to the user and let users to choose the former. I imagine that that can even be a strong reason to choose LLD over other ELF linkers in the future.

ELF/CMakeLists.txt
17 ↗(On Diff #29688)

Can you remove LTO from this patch? Unless we start working on LTO right now, it's going to be dead code. I'd like you to strip down as much as you can.

ELF/Chunks.cpp
27 ↗(On Diff #29688)

You can remove "elfv2::".

ELF/Chunks.h
20 ↗(On Diff #29688)

I'd name elf2 instead of elfv2 because it's shorter and consistent with lldELF2.

ELF/Driver.cpp
53–58 ↗(On Diff #29688)

Let's remove this function and make -o option mandatory for now. This function is really Windows-specific.

84–103 ↗(On Diff #29688)

Ditto

127 ↗(On Diff #29688)

Same for findFile, doFindLib and findLib.

136–142 ↗(On Diff #29688)

If you remove LTO, you can remove these function calls.

ELF/InputFiles.cpp
123 ↗(On Diff #29688)

Replace "auto" with its real type.

127 ↗(On Diff #29688)

Ditto

142 ↗(On Diff #29688)

Ditto

148 ↗(On Diff #29688)

Ditto

ELF/SymbolTable.cpp
79–88 ↗(On Diff #29688)

Does this also make sense for ELF? If in doubt, I'd remove for now.

ELF/SymbolTable.h
66–67 ↗(On Diff #29688)

I'd remove this for now. This maybe Windows-specific, and we can bring it back if we need it.

ELF/Symbols.h
197–201 ↗(On Diff #29688)

Remove this if you don't use this.

ELF/Writer.cpp
26 ↗(On Diff #29688)

Remove

54–55 ↗(On Diff #29688)

Move this above "static const int PageSize...".

143–148 ↗(On Diff #29688)

I think you don't actually need this function for ELF.

joerg added a subscriber: joerg.Jul 14 2015, 3:07 PM

I don't understand how forcing everything to be grouped can make things more efficient, given that it is strictly required to do more work. It doesn't even matter for plain object files, just for libraries. In the case of libraries, there are subtle error cases involving weak symbols, so please do *not* change the resolution algorithm.

Bigcheese marked 18 inline comments as done.Jul 14 2015, 3:15 PM
Bigcheese added inline comments.
ELF/InputFiles.cpp
137 ↗(On Diff #29688)

It will. The intent is for this to be the one place we scan the entire section table, which can find broken files.

ELF/Symbols.h
166 ↗(On Diff #29688)

This is for archive files.

Bigcheese updated this revision to Diff 29721.Jul 14 2015, 3:15 PM
Bigcheese edited edge metadata.

Address review comments.

ELF/Chunks.cpp
27 ↗(On Diff #29688)

ObjectFile is also a name in llvm::object.

ruiu accepted this revision.Jul 14 2015, 3:42 PM
ruiu edited edge metadata.

LGTM

Please leave this for a while so that other people can review this patch as well.

ELF/Symbols.h
201 ↗(On Diff #29721)

You can now remove this member.

This revision is now accepted and ready to land.Jul 14 2015, 3:42 PM
davide accepted this revision.Jul 14 2015, 4:24 PM
davide edited edge metadata.

This seems good to me (assuming you'll remove all the COFF'isms)

rafael edited edge metadata.Jul 22 2015, 7:29 AM

I am currently rebasing this on trunk.

test/elfv2/basic.test
1 ↗(On Diff #29721)

Rename the directory to elf2

You can test this with llvm-mc. Please don't use yaml2obj when it is not needed.

A patch that builds on trunk is attached.

I have also fixed it to build when using shared libraries.

There is still a *lot* of stuff that should be deleted before this
goes in. Please update phab with this patch (and then change the test
to use llvm-mc).

Bigcheese updated this revision to Diff 30404.Jul 22 2015, 2:43 PM
Bigcheese edited edge metadata.

Rafael's changes.

Bigcheese updated this revision to Diff 30415.Jul 22 2015, 3:52 PM

Address review comments.

.s test.
Rename test dir.

Bigcheese marked an inline comment as done.Jul 22 2015, 3:53 PM
rafael added inline comments.Jul 22 2015, 4:21 PM
ELF/CMakeLists.txt
17 ↗(On Diff #30415)

Please delete the LTO dependency. I added it to CMakeList to fix the build, but it should be removed and the code changed to avoid the dependency.

ELF/Chunks.cpp
84 ↗(On Diff #30415)

This test doesn't need common symbols. Delete this. It should be added in a followup patch. This is also copying over COFF specific code and comments.

79 ↗(On Diff #29721)

You don't need this on the first patch.

ELF/Chunks.h
90 ↗(On Diff #30415)

Delete the garbage collector. It should be added in another patch and the terminology for ELF is different anyway.

ELF/DriverUtils.cpp
111 ↗(On Diff #30415)

Delete this. It should be added when we add a test that uses response files.

ELF/InputFiles.cpp
46 ↗(On Diff #30415)

Handle the error in here instead of returning it.

test/elf2/basic.test
7 ↗(On Diff #30415)

The type is not needed.

Bigcheese updated this revision to Diff 30433.Jul 22 2015, 5:30 PM
Bigcheese marked 6 inline comments as done.

Remove GC and LTO remnants.

This comment was removed by Bigcheese.

The attached patch looks good to me.

It is your patch, but I have:

  • removed code that had COFF logic.
  • removed untested code.
  • added a few more tests.
  • simplified error handling.
  • removed unused dependencies.

Even with the extra tests this patch adds 1255 lines. Yours was 1995.

Cheers,
Rafael

emaste added a subscriber: emaste.Jul 24 2015, 7:40 AM
This revision was automatically updated to reflect the committed changes.