This is an archive of the discontinued LLVM Phabricator instance.

[lld] LinkDriver, lld-link: introduce shim.
AbandonedPublic

Authored by martell on Aug 14 2015, 2:16 AM.

Details

Reviewers
compnerd
ruiu
Summary

lld-link is intended to be a link.exe compatible utility
This is intended to become a shim that converts link commands to ld commands.

This was previously suggested by Saleem here
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20150608/280890.html

I would like to implement this in 4 stages.

  1. Introduce this LinkDriver.
  2. Migrate the driver in COFF to the LinkDriver
  3. Unify the COFF and ELF linker to have a unified interface via ld
  4. Change LinkDriver from an interface to an argument conversion shim.

I believe this would be the best way todo this because the product will still be complete at all stages with a fully working linker for all targets
We can then weigh in the pros and cons of changing LinkDriver to a conversion shim after stage 3.

If this proves successful we can do a Ld64Driver similar to this.

I also want to note that this was designed with llvm-lib as a reference so that we can be consistent between the llvm projects

Diff Detail

Event Timeline

martell updated this revision to Diff 32139.Aug 14 2015, 2:16 AM
martell retitled this revision from to [lld] LinkDriver, llvm-link: introduce shim..
martell updated this object.
martell added reviewers: compnerd, ruiu.
martell updated this object.
martell retitled this revision from [lld] LinkDriver, llvm-link: introduce shim. to [lld] LinkDriver, lld-link: introduce shim..
martell added subscribers: llvm-commits, lld, silvas, meadori.
martell updated this object.Aug 14 2015, 2:19 AM
martell updated this object.Aug 14 2015, 2:49 AM
ruiu edited edge metadata.Aug 14 2015, 3:14 AM
ruiu added a subscriber: ruiu.

Defining a universal driver and making COFF linker to be a shim to that is
not going to work. Such translation is needlessly complex. Unix and Windows
semantics are significantly different if you take a closer look. A lot of
information is going to be lost in a straightforward translation. So both
shim and the universal driver are going to be smart and complex.

Please take a look at COFF/Driver.cpp. The diver drives many moving parts
such as the writer, symbol table etc. You don't want to mix that code with
other drivers which are also complicated in different ways.

They are pretty different, so let's not mix them. Keep each driver to focus
on only one thing.
2015/08/14 18:49 "Martell Malone" <martellmalone@gmail.com>:

martell updated the summary for this revision.

http://reviews.llvm.org/D12029

martell updated this revision to Diff 32147.Aug 14 2015, 4:28 AM
martell edited edge metadata.

Updated to Stage 2

In D12029#224385, @ruiu wrote:

Defining a universal driver and making COFF linker to be a shim to that is
not going to work. Such translation is needlessly complex. Unix and Windows
semantics are significantly different if you take a closer look. A lot of
information is going to be lost in a straightforward translation. So both
shim and the universal driver are going to be smart and complex.

Okay that is fine if you think turning it into a shim at the end is needlessly complex.
We probably shouldn't go down the route of parsing and converting args then.

Please take a look at COFF/Driver.cpp. The diver drives many moving parts
such as the writer, symbol table etc. You don't want to mix that code with
other drivers which are also complicated in different ways.

They are pretty different, so let's not mix them. Keep each driver to focus
on only one thing.

We should at least extract COFF/Driver to Lib/LinkDriver so that we can have 2 drivers for COFF.
A GNU driver along with the link driver that are separate.

My thoughts are that

  1. The GNU COFF Driver should be in COFF/Driver just like it is in ELF/Driver.
  2. The Link Driver should be separate and structured much like llvm-lib. -> Lib/LinkDriver

Your thoughts?

compnerd edited edge metadata.Aug 15 2015, 9:43 AM

If we are fine with adding custom flags to the link command line, then aliases would be sufficient I think. The idea is that you want to preserve the semantics of PE/COFF (which you called the semantics of Windows). The difference is that the linker invocation should be similar to ld's, but continue to provide the current semantics. There are a few extensions that are useful (which are compatible with the PE/COFF semantics), but the binaries that are generated by the alternate interface are meant to run on a Windows system, so losing the semantics of PE/COFF would be problematic.

Just because the driver is written on/for unix, doesn't mean that the linker should provide unix semantics. The semantics are that of PE/COFF because that is the target. Its similar to how clang provides a GCC compatible interface which can still be used to generate a proper COFF object, even though ELF and COFF semantics are quite different.

rnk added a subscriber: rnk.Aug 17 2015, 10:54 AM

BFD ld has some options that link.exe doesn't have. If we do aliasing and argument translation, what will we do for such options? Invent our own link.exe-style options? Or try to pass through the BFD option?

Based on our experience with clang vs clang-cl, I think we want to implement a driver that consumes all options, aliases everything that can be aliased, and treats the ld-style flags as first class citizens. In retrospect, I wish that clang-cl used a blacklist approach, where incompatible GCC-style flags are blacklisted, rather than whitelisting core options one by one. If there aren't any conflicts between link.exe and gnu ld flags, we can go ahead and do this.

We can avoid duplicating the common options between ELF and COFF by generating a single option table and adding COFF / ELF flags like we do for clang / clang-cc1.

Based on our experience with clang vs clang-cl, I think we want to
implement a driver that consumes all options, aliases everything that can
be aliased, and treats the ld-style flags as first class citizens. In
retrospect, I wish that clang-cl used a blacklist approach, where
incompatible GCC-style flags are blacklisted, rather than whitelisting core
options one by one. If there aren't any conflicts between link.exe and gnu
ld flags, we can go ahead and do this.

We can avoid duplicating the common options between ELF and COFF by
generating a single option table and adding COFF / ELF flags like we do for
clang / clang-cc1.

I think this approach would probably be the best in the long run.
At the same time agree with Rui on getting something simple into the
project.

I would like to avoid using a python script and have something within lld
in c/c++ itself just for consistency.
What are your thoughts on a patch like this (keep in mind that it is not
finished and i haven't added the ld options yet)

https://gist.github.com/martell/0f510546ebb01a2fb04e

I created 2 new files GNUDriverUtils.cpp and GNUDriver.cpp and from within
the universal Driver checked if it was using ld front end with a windows pe
target and got it to use that new driver if that was the case.
With this it keeps the 2 drivers separate but simple and we can start
filling out the ld options and build in the relevant alias's.

This seems like the simplest get support in to me.
We could then match up the options and once we are happy with them merge
them into the one options/driver file that consumes everything?

Thoughts?

alexr added a subscriber: alexr.Aug 17 2015, 5:33 PM

Saleem's original referenced proposal sounds to me a lot more like the original plan for lld's driver that was removed.

@rnk, fwiw, that sounds very much in line with what I was trying to push for. AFAIK, the options don't conflict.

I don't think a pure aliasing or wrapping approach will work long term.
There are things like -rdynamic that we want to support but they have no
link.exe analogue.

Sent from phone

martell abandoned this revision.Jun 4 2017, 11:38 AM
COFF/SymbolTable.cpp