This is an archive of the discontinued LLVM Phabricator instance.

[lld][ELF] --start-lib should be treated as archives when using --whole-archives
Needs RevisionPublic

Authored by christylee on Feb 23 2022, 5:06 PM.

Details

Reviewers
ayermolo
MaskRay
Summary

--start-lib starts "a grouping of objects that should be treated as if they
were together in an archive", we should follow the archive semantics and
create object files instead of lazy files when --whole-archive is used.

Diff Detail

Event Timeline

christylee created this revision.Feb 23 2022, 5:06 PM
christylee requested review of this revision.Feb 23 2022, 5:06 PM
MaskRay added a comment.EditedFeb 23 2022, 5:31 PM

The interaction of --start-lib and --whole-archive is unclear.
You may argue that --whole-archive --start-lib a.o should behave like a relocatable object file, but
another person may argue that --start-lib --whole-archive a.a should use archive semantics by applying the --whole-archive semantics first then applying the --start-lib semantics.

I tend to think we should only apply one of them, not both.
It's simpler to think that way.
You may need to fix your build system to avoid this situation.
(For Bazel I have filed a related issue: https://github.com/bazelbuild/bazel/issues/12706)

MaskRay requested changes to this revision.Feb 23 2022, 5:32 PM
This revision now requires changes to proceed.Feb 23 2022, 5:32 PM
MaskRay added a comment.EditedFeb 23 2022, 5:34 PM

I have thought reporting an error for such nested usage, but it needs some research that no build system relies on the behavior.
I'll be happy to check whether Bazel has nested usage. Perhaps you can do the same for your build system?

I have thought reporting an error for such nested usage, but it needs some research that no build system relies on the behavior.
I'll be happy to check whether Bazel has nested usage. Perhaps you can do the same for your build system?

We use Buck, we are quite fortunate that we do not rely on nested usage. We use --start-lib to get archive semantics without needing to run ar,
for libraries with only one file, this saves us a lot of build time.

If --start-lib is designed to treat object files like they are in an archive, then for consistency, I feel that --whole-archive should probably have
precedent. But thinking about this a little more, what should we do in the case of --start-lib a.o --whole-archive b.o? Should we create lazy file
for a.o but object file for b.o?

Maybe logic should be to apply operators from inside out, and disallow complex cases for sanity sake? Outer most wins out?
So for --whole-archive --start-lib a.o --end-lib --no-whole-archive behavior is whole-archive on .a.
For --start-lib --whole-archive a.a, we end up were we started with an archive.

Maybe logic should be to apply operators from inside out, and disallow complex cases for sanity sake? Outer most wins out?
So for --whole-archive --start-lib a.o --end-lib --no-whole-archive behavior is whole-archive on .a.
For --start-lib --whole-archive a.a, we end up were we started with an archive.

--start-lib --start-lib is currently an error.

I think different people may interpret --start-lib --whole-archive and --whole-archive --start-lib differently.
I did not add one just because unsure whether anyone abuses the current behavior.
Since you bring it up and can test, perhaps it's time to add an error.

The issue I'm facing is that if I use --whole-archive with --start-lib A.o --end-lib, the symbols that are included are not the same as those
if I were to just include libA.a. I would have expected that --start-lib A.o --end-lib would behave like archives, and that they would result
in the same symbol resolution.

I've attached a tiny repro here. First I compile the files:

$ clang++ -c A.cpp B.cpp C.cpp

If I make package A.o into an archive and then link it with C.o into a shared library, the symbol _Z16get_value_from_av from A.o is included as
expected.

$ ar rcs libA.a A.o
$ /home/christylee/local/server-llvm/llvm-build/bin/ld.lld --shared --whole-archive libA.a  C.o
$ nm a.out
00000000000023b0 d _DYNAMIC
0000000000001370 T _Z16get_value_from_av
                 U _Z16get_value_from_bv
0000000000001380 T _Z16get_value_from_cv

If, however, I included A.o via --start-lib and --end-lib, then the symbol does not get included:

$/home/christylee/local/server-llvm/llvm-build/bin/ld.lld --shared --whole-archive --start-lib A.o --end-lib  C.o
$ nm a.out
00000000000022a0 d _DYNAMIC
0000000000001290 T _Z16get_value_from_cv
MaskRay added a comment.EditedFeb 24 2022, 2:05 PM

In --whole-archive --start-lib A.o --end-lib, I am unsure --whole-archive needs to apply to the virtual archive. --whole-archive can be defined to apply to only regular and thin archives.
For your use case, use A.o in place of --whole-archive --start-lib A.o --end-lib.