This is an archive of the discontinued LLVM Phabricator instance.

[llvm-objcopy] Support --{enable,disable}-deterministic-archives
ClosedPublic

Authored by rupprecht on Oct 30 2018, 5:03 PM.

Details

Summary

ar and objcopy/strip all support configuring whether archives are written deterministically (timestamps/UIDs/GIDs/etc zero'd). This has been ported to llvm-ar (the U/D modifiers) but not yet to llvm-objcopy/strip.

Diff Detail

Event Timeline

rupprecht created this revision.Oct 30 2018, 5:03 PM
ruiu added a subscriber: ruiu.Oct 30 2018, 5:08 PM

I feel it should be enabled by default, as deterministic build is a very nice property.

I feel it should be enabled by default, as deterministic build is a very nice property.

It was before this patch and it still is. This is providing the option to make it not default (if anyone wants that), and also won't break if someone is explicitly running "objcopy --enable-deterministic-archives" and we drop llvm-objcopy in place of that.
I can add a test case for that, actually.

rupprecht updated this revision to Diff 171842.Oct 30 2018, 5:14 PM
  • Add test case that deterministic is the default
ruiu added a comment.Oct 30 2018, 5:15 PM

It was before this patch and it still is. This is providing the option to make it not default (if anyone wants that), and also won't break if someone is explicitly running "objcopy --enable-deterministic-archives" and we drop llvm-objcopy in place of that.

Oh sorry I didn't notice that. What was the motivation for you to add a new flag so that you can choose deterministic and non-deterministic? (Just wondering.)

It was before this patch and it still is. This is providing the option to make it not default (if anyone wants that), and also won't break if someone is explicitly running "objcopy --enable-deterministic-archives" and we drop llvm-objcopy in place of that.

Oh sorry I didn't notice that. What was the motivation for you to add a new flag so that you can choose deterministic and non-deterministic? (Just wondering.)

These flags exist in GNU strip/objcopy/ar (see [man strip]), so it isn't really "new".
If we convert users of strip to llvm-strip, they will break if they are specifying --enable-determinstic-archives and this flag is not defined.

ruiu added a comment.Oct 30 2018, 5:24 PM

I do not see any problem of adding these flags for the sake of compatibility with GNU. What I wondered is whether or not you really have a need to make it nondeterministic. I thought that we might be able to just ignore these flags.

I do not see any problem of adding these flags for the sake of compatibility with GNU. What I wondered is whether or not you really have a need to make it nondeterministic. I thought that we might be able to just ignore these flags.

Personally, I think that if the default is to make them deterministic, then I don't see any reason not to allow overriding this default, as the code does not become significantly more complex with this change. However, I could be persuaded that nobody has this use case.

test/tools/llvm-objcopy/deterministic-archive.test
23–26

Missing llvm-strip equivalents?

29–30

Ditto.

tools/llvm-objcopy/CopyConfig.cpp
348–349

Is this what GNU objcopy does? I'd actually expect this to be a "last one wins" scenario (similar to various --[no-]some-option in LLD etc.

425–426

Same comment as above.

rupprecht marked 3 inline comments as done.
  • Implement last-one-wins for -D/-U
rupprecht marked an inline comment as done.Oct 31 2018, 10:52 AM

I do not see any problem of adding these flags for the sake of compatibility with GNU. What I wondered is whether or not you really have a need to make it nondeterministic. I thought that we might be able to just ignore these flags.

Personally, I think that if the default is to make them deterministic, then I don't see any reason not to allow overriding this default, as the code does not become significantly more complex with this change. However, I could be persuaded that nobody has this use case.

Ditto -- sorry for not understanding the comment earlier.

I only have a use case for explicitly setting deterministic archives. I don't have a use case for non-deterministic archives, although I could imagine some build systems maybe want it (for preserving timestamps). At any rate, it's trivial to provide. I think the onus is on someone providing evidence that exposing this option is *harmful*.

tools/llvm-objcopy/CopyConfig.cpp
348–349

I thought it was simpler to just error instead of match objcopy, but it actually comes to fewer lines implementing last-one-wins. Done.

jhenderson added inline comments.Nov 1 2018, 2:26 AM
tools/llvm-objcopy/CopyConfig.cpp
348–349

You don't need to use the filtered and loop method at all, you can just use the hasFlag method, which takes both a positive and negative option:

I think this is right (various LLD options are implemented this way):

Config->DeterministicArchives(OOBJCOPY_enable_deterministic_archvies, OBJCOPY_disable_deterministic_archives, /*default*/true);
tools/llvm-objcopy/ObjcopyOpts.td
42

Somebody pointed out in an llvm-objdump review (see D53804) that Alias options are not in the help text by default (they are hidden), but they really should be. This might want to be a wider issue for us to fix at some point though, but in the meantime, perhaps you should add these particular ones? What do you think?

rupprecht updated this revision to Diff 172138.Nov 1 2018, 8:50 AM
rupprecht marked an inline comment as done.
  • Use better flag parsing, and include alias in --help
rupprecht added inline comments.Nov 1 2018, 8:54 AM
tools/llvm-objcopy/CopyConfig.cpp
348–349

Oh, I didn't see that method. Even shorter now, thanks! :)

tools/llvm-objcopy/ObjcopyOpts.td
42

Agreed that --help is terrible. It also bothers me that we print help for both "--foo=value" and "--foo value". Fixed for these ones.

jhenderson accepted this revision.Nov 1 2018, 10:05 AM

LGTM.

Might want to wait to see if @ruiu or anybody else has any other comments though.

This revision is now accepted and ready to land.Nov 1 2018, 10:05 AM
ruiu added a comment.Nov 1 2018, 10:13 AM

You don't need to wait for my LGTM for this, but it is LGTM. :)

This revision was automatically updated to reflect the committed changes.