This is an archive of the discontinued LLVM Phabricator instance.

[ELF] - Basic versioned symbols support implemented.
ClosedPublic

Authored by grimar on Jun 6 2016, 5:05 AM.

Diff Detail

Repository
rL LLVM

Event Timeline

grimar updated this revision to Diff 59706.Jun 6 2016, 5:05 AM
grimar retitled this revision from to [ELF] - Basic versioned symbols support implemented..
grimar updated this object.
grimar added reviewers: ruiu, rafael.
grimar added subscribers: llvm-commits, grimar.
grimar updated this revision to Diff 60014.Jun 8 2016, 4:01 AM

After that change, I can link the reproduce if add the --allow-multiple-definition flag. Flag helps here as this patch does not support comparing versioned vs unversioned symbols yet. So for example

acl_compat.So contains acl_add_perm@FBSD_1.0
acl_perm.So contains acl_add_perm

Resolving fails without flag with duplicate symbol error now.

ruiu edited edge metadata.Jun 8 2016, 8:56 AM

Here is my first round of review.

ELF/InputFiles.cpp
343 ↗(On Diff #60014)

Why do you need this?

ELF/SymbolListFile.cpp
84 ↗(On Diff #60014)

A version definition without a version name is not actually a version definition in the sense that they controls only visibility and have nothing to do with symbol versioning. I think it is better to handle that case completely differently from symbol versioning.

121 ↗(On Diff #60014)

Since you are not using this member at all, please remove Parent member from the struct and just discard the next token here.

ELF/SymbolListFile.h
23–27 ↗(On Diff #60014)

Move this definition to Config.h and make version definitions a member of Config class.

grimar updated this revision to Diff 60151.Jun 9 2016, 3:47 AM
grimar edited edge metadata.
grimar marked 4 inline comments as done.
  • Addressed review comments.
grimar added a comment.Jun 9 2016, 3:48 AM
In D21018#452308, @ruiu wrote:

Here is my first round of review.

Thanks for review ! My response to comments are below.

ELF/InputFiles.cpp
343 ↗(On Diff #60014)

Right, for this patch and its trivial versionscript and cases this is excessive. That left from my experiments with bit more complex cases.
Removed.

ELF/SymbolListFile.cpp
84 ↗(On Diff #60014)

Ok. I was thinking about that as about a some virtual "base version of the library".

Spec says: "Symbols defined in the library which aren't specifically bound to a version node are effectively bound to an unspecified base version of the library. " (http://ftp.gnu.org/old-gnu/Manuals/ld-2.9.1/html_node/ld_25.html). That is not exactly our case probably, but it is sounds relative.
I reimplemented code according to your comment though, lets see how it goes.

121 ↗(On Diff #60014)

Done.

ELF/SymbolListFile.h
23–27 ↗(On Diff #60014)

Done.

ruiu added a comment.Jun 9 2016, 12:23 PM

For some reason, I received no email notifications on this review thread. George, do you have an idea why?

I haven't fully understand the core part of the patch yet, so this review is mostly on stylistic things. I'll read it again later today or tomorrow. (I'm traveling for a few days until Sunday.)

ELF/OutputSections.h
246 ↗(On Diff #60151)

Using a platform-dependent integral type for a ELF flag is a bit alarming. Can you use uint32_t?

You can format like this

uint32_t Flags; // Version definition flags
uint32_t Hash;  // ...
ELF/SymbolListFile.cpp
106–108 ↗(On Diff #60151)

Does this work?

Config->SymbolVersions.push_back({next()});
111 ↗(On Diff #60151)

You don't need to cache the result of peek() -- it's a cheap function, and this function is not in a performance-critical path.

ruiu added inline comments.Jun 9 2016, 12:23 PM
ELF/OutputSections.h
254 ↗(On Diff #60151)

Add a blank line.

grimar added a subscriber: ruiu.Jun 10 2016, 12:23 AM
In D21018#453725, @ruiu wrote:

For some reason, I received no email notifications on this review thread. George, do you have an idea why?

I think the only difference here from my usual style was that I quoted the part of your message and posted responce "Thanks for review ! My response to comments are below." in the same message with responces on review comments. Usually I think I do not do that and comments just auto merged to the last (patch updated) post. I am always using web form for that, so no much ways to do almost the same here exist.

Not sure that it can be the reason, I added you as a subscriber just in case as well, hope that helps.

I haven't fully understand the core part of the patch yet, so this review is mostly on stylistic things. I'll read it again later today or tomorrow. (I'm traveling for a few days until Sunday.)

Ok, just in case I think I`ll also be able to address any comments for this in next 2 days if any.

grimar updated this revision to Diff 60328.Jun 10 2016, 2:54 AM
grimar marked 4 inline comments as done.
  • Addressed review comments.
ELF/OutputSections.h
246 ↗(On Diff #60151)

Done.

254 ↗(On Diff #60151)

Done.

ELF/SymbolListFile.cpp
106–108 ↗(On Diff #60151)

Actually this does not. Though it should be valid С++ construction and works in msvs/gcc,
as far I know clang produces an error if you do not specify all members in aggregate specialization.
So this might be:

Config->SymbolVersions.push_back({next(), {}});

to work. And I do not like this, because if you change the members of struct (amount of them),
you have to be sure to check and update all such places to support clang.

I updated this place, since it is single, but above was the reason why I did not do that initially (even was thinking about adding constructor for Name to Version struct).

111 ↗(On Diff #60151)

Done.

ruiu added a comment.Jun 10 2016, 11:01 AM

Ahh, I got the reason why it didn't appear in my inbox. It went to a spam folder. I'm sorry about that. If you have sent me mails and did not get responses, please re-send.

ELF/SymbolListFile.cpp
106–108 ↗(On Diff #60328)

Then how about defining a constructor that takes a StringRef?

grimar updated this revision to Diff 60437.Jun 11 2016, 8:50 AM
grimar marked an inline comment as done.
  • Addressed review comment.
ELF/SymbolListFile.cpp
106–108 ↗(On Diff #60328)

Done. That is not very consistent with other lld code, but solves the problem.
(actually I just do not understand why it can not be solved(fixed) in clang, as this feature of c++ language looks
pretty cool for me).

pcc added a subscriber: pcc.Jun 13 2016, 1:37 PM
pcc added inline comments.
ELF/OutputSections.cpp
1432 ↗(On Diff #60437)

Can we rename our implementation of the ELF hash function to something like hashElf? I previously wasn't aware that we had one.

1437 ↗(On Diff #60437)

This is a well-known problem, we probably don't need to call out every instance of it.

1445 ↗(On Diff #60437)

Grammar nitpick: "to the number of definitions"

1457 ↗(On Diff #60437)

It isn't deterministic to iterate over a DenseMap like that.

1532 ↗(On Diff #60437)

Grammar nitpick: "if it exists"

1535 ↗(On Diff #60437)

The std::max seems redundant because getVerDefNum() cannot return a value less than 2 here.

ELF/OutputSections.h
245–248 ↗(On Diff #60437)

I don't think you need either of the flags or hash fields here. The first one is never anything but zero in your implementation, and you can compute the second one as you produce the contents of the .gnu.version_d section.

251 ↗(On Diff #60437)

Instead of having this DenseMap, could you move VerDefEntry's fields to Version?

262 ↗(On Diff #60437)

Please update this comment to cover version definitions.

ELF/Symbols.h
142 ↗(On Diff #60437)

This field belongs on Symbol, as it is a property of a symbol name (and this won't work correctly if a bitcode file defines a versioned symbol). Can you please move this to Symbol and add a test showing that LTO works correctly with this feature?

On that point, I'm not sure that we want all 3 of SharedSymbol::VersionId, Symbol::VersionScriptGlobal and this new field. Did you think about unifying them somehow?

rafael added inline comments.Jun 16 2016, 6:22 AM
ELF/SymbolListFile.cpp
80 ↗(On Diff #60437)

Do we really need to duplicate the parsing? We should be able to have a single parser and take different actions based on the name, no?

grimar marked 9 inline comments as done.Jun 16 2016, 12:37 PM

Hi Peter ! Thanks for usefull review. I hope to update this one soon. As soon as solve unifying problem you're mentioned.

ELF/OutputSections.cpp
1432 ↗(On Diff #60437)

I have nothing against that if Rui and Rafael also support the new naming. That can be committed separatelly.

1437 ↗(On Diff #60437)

Ok, I did not know about that.

1445 ↗(On Diff #60437)

Thanks !

ELF/Symbols.h
142 ↗(On Diff #60437)

Probably yes. I am working on unifying.

grimar updated this revision to Diff 61078.Jun 17 2016, 3:34 AM
grimar retitled this revision from [ELF] - Basic versioned symbols support implemented. to [ELF/WIP] - Basic versioned symbols support implemented..
grimar marked 3 inline comments as done.
  • Addressed review comments (all except creating test for LTO, I am working on it as never did such ones yet I think, will update patch later today I think).

Note: script parsing code in this version based on Rafael patch "[PATCH] Make local: optional.", to show how new functionality fits.

grimar updated this revision to Diff 61091.Jun 17 2016, 6:27 AM
  • Added LTO test.
  • Fixed issue: sh_addralign field was not set for .gnu.version_d
ruiu added inline comments.Jun 17 2016, 12:05 PM
ELF/Config.h
38 ↗(On Diff #61091)
Version(llvm::StringRef Name) : Name(Name) {}
ELF/OutputSections.cpp
566 ↗(On Diff #61091)

Maybe the comment is too detailed and didn't actually say what it is doing? I'd say "Returns the number of version definition entries. Because the first entry is for the version definition itself, it is the number of versioned symbols plus one. Note that we don't support multiple versions yet."

1430 ↗(On Diff #61091)

Other sections finalize() functions add strings to .dymstr. Is there any reason you didn't choose to do this in finalize()?

1455 ↗(On Diff #61091)

You are writing two different types of records in this loop -- one for the first entry and the other for the remaining entries. It is better to separate the first entry from the others. Then you can remove all the IsBase dispatches from this loop.

ELF/SymbolListFile.cpp
77 ↗(On Diff #61091)

Please rebase.

ELF/SymbolTable.cpp
528 ↗(On Diff #61091)

Use range-based for loop.

ELF/Writer.cpp
913 ↗(On Diff #61091)

Instead of adding isRequired() method, don't instantiate Out<ELFT>::VerSym if not needed. It's what we usually do for other sections.

pcc added inline comments.Jun 17 2016, 12:13 PM
ELF/OutputSections.cpp
1431 ↗(On Diff #61091)

I think this should be the soname, not the output file name.

1462 ↗(On Diff #61091)

Same here

1499 ↗(On Diff #61091)

Fold this variable into its use.

ELF/OutputSections.h
256 ↗(On Diff #61091)

Either .gnu.version_r or .gnu.version_d.

ELF/Symbols.cpp
269 ↗(On Diff #61091)

This is the only place where we are using the VerDef and VersionScriptGlobal fields. Can you replace (VersionScriptGlobal || body()->symbol()->VerDef) here with VersionId != VER_NDX_LOCAL and remove both of those fields?

(Also, body()->symbol()-> is unnecessary here, as it just takes you back to the Symbol you started with.)

Thank you for reviews ! I`ll address your comments during my tomorrow.

grimar updated this revision to Diff 61163.Jun 18 2016, 4:33 AM
grimar marked 12 inline comments as done.
  • Addressed review comments.
grimar added inline comments.Jun 18 2016, 4:34 AM
ELF/Config.h
38 ↗(On Diff #61091)

Done.

ELF/OutputSections.cpp
566 ↗(On Diff #61091)

Changed.

1430 ↗(On Diff #61091)

Moved to finalize(). I think there was some problem, not really remember what exactly in very first iteration of this patch (createDefs did more work that time). Now it is redundant.

1431 ↗(On Diff #61091)

Checked that, both ld/gold use soname, and if it is not set then output file name is taken. Corrected the logic to match.

1455 ↗(On Diff #61091)

Done.

1462 ↗(On Diff #61091)

Fixed.

1499 ↗(On Diff #61091)

Done.

ELF/OutputSections.h
256 ↗(On Diff #61091)

Done.

ELF/SymbolListFile.cpp
77 ↗(On Diff #61091)

Done.

ELF/SymbolTable.cpp
528 ↗(On Diff #61091)

Done.

ELF/Symbols.cpp
269 ↗(On Diff #61091)

Done.

ELF/Writer.cpp
913 ↗(On Diff #61091)

Done.

ruiu added inline comments.Jun 18 2016, 5:46 PM
ELF/OutputSections.cpp
567 ↗(On Diff #61163)

Make this a function comment.

1453 ↗(On Diff #61163)

unsigned -> size_t.

1477 ↗(On Diff #61163)

Just pass 1 instead of I.

ELF/SymbolListFile.cpp
119–121 ↗(On Diff #61163)

So you are constructing a Version object in two steps -- first by passing only Version parameter to the constructor and then add Cur to its Globals array, right? That's a bit hard to read. Is there any way to do this in one step?

ELF/SymbolTable.cpp
531 ↗(On Diff #61163)

size_t should be more accurate type (in practice they don't differ though.)

533 ↗(On Diff #61163)

Increment this at end of the loop -- that's easier to read.

ELF/Symbols.h
420 ↗(On Diff #61163)

Does unsigned VersionId : 16 save space?

ELF/Writer.cpp
121–123 ↗(On Diff #61163)

So, instead of instantiating them unconditionally, can you instantiate them only when needed?

grimar updated this revision to Diff 61194.Jun 19 2016, 3:02 AM
grimar marked 8 inline comments as done.
  • Addressed review comments.
grimar added inline comments.Jun 19 2016, 3:03 AM
ELF/OutputSections.cpp
567 ↗(On Diff #61163)

Done.

1453 ↗(On Diff #61163)

Done.

1477 ↗(On Diff #61163)

Done.

ELF/SymbolListFile.cpp
119–121 ↗(On Diff #61163)

Oh. There was a bug here. It created new elf::Version for each global. Though that did not affect whole functionality, it is defenitely not what I wanted.
Fixed.

ELF/SymbolTable.cpp
531 ↗(On Diff #61163)

Probably yes, but we also have VersionId of type uint16_t, so I am not sure how much it can be called accurate in total. Changed to size_t.

533 ↗(On Diff #61163)

Done.

ELF/Symbols.h
420 ↗(On Diff #61163)

72 vs 80, yes.
Moved it under uint8_t Binding; The same result but looks a bit nicer I think ?

ELF/Writer.cpp
121–123 ↗(On Diff #61163)

Ah, sorry, I misunderstood at first. Fixed for Verdef (I think because of VerNeed it is not possible to perform that change for other 2, as Out<ELFT>::VerNeed->addSymbol(SS) called after creation of sections).

grimar retitled this revision from [ELF/WIP] - Basic versioned symbols support implemented. to [ELF] - Basic versioned symbols support implemented..Jun 19 2016, 3:15 AM
ruiu accepted this revision.Jun 19 2016, 10:43 PM
ruiu edited edge metadata.

LGTM

ELF/SymbolListFile.h
14 ↗(On Diff #61194)

Remove.

ELF/SymbolTable.cpp
535 ↗(On Diff #61194)

I'm wondering if this could be I++?

ELF/Writer.cpp
915–916 ↗(On Diff #61194)

Add doesn't do anything if an argument is null. So you can remove the if guard.

This revision is now accepted and ready to land.Jun 19 2016, 10:43 PM
In D21018#461946, @ruiu wrote:

LGTM

What a good morning :) Thanks for review, I'll commit in a few hours then.

ELF/SymbolTable.cpp
535 ↗(On Diff #61194)

No, because each Version can have a list of globals. And all of them should receive the same VersionId.

This revision was automatically updated to reflect the committed changes.