Implementation marks non-prevailing symbols as not live in the summary.
Then them are dropped in backends.
Details
Diff Detail
Event Timeline
@grimar Can't internalization be skipped during backend processing rather than changing linkage in summary index?. At the very first glance it is possible to check if def is prevailing or not in MustPreserveGV (see thinLTOInternalizeModule)
We don't have linker info in the backends, so that code is just looking at what was recorded in the index during the thin link (where we have the linker info).
lib/LTO/LTO.cpp | ||
---|---|---|
333 | We also need to change importing to not allow import of a non-prevailing symbol. Then this patch should be changed to assert if it was non-prevailing but exported. |
No - we communicate via the index to support distributed ThinLTO backends.
To drop non-prevailing symbols completely in the backends, as discussed in D41988, we will need to either add a bit to the summary flags, or possibly we can utilize the isLive flag (which currently is only used to communicate from the compile step through the thin link (to perform whole program DCE). Right now we mark dead symbols in the index with internal linkage and rely on the back end to update the linkage (in the code you were looking at) and then GlobalDCE to remove them. We can handle the dropping of non-prevailing and globally dead symbols the same way perhaps - mark as not live in the summary and have a mechanism to just drop them without going through internalization etc.
I am trying to implement this (hope I am doing right thing).
This approach looks better atm, though patch is not yet finished, I have 3 testcases failing atm, working on them:
LLVM :: LTO/Resolution/X86/comdat-mixed-lto.ll LLVM :: LTO/Resolution/X86/lowertypetests.ll LLVM :: LTO/Resolution/X86/setting-dso-local.ll
Going to upload current WIP version I have in a few minutes and continue tomorrow.
- WIP version that tries to leave non-prevealing symbols as dead and then drop them in backend.
- Finished implementation. Now all LLD\LTO and llvm\test\LTO tests pass.
Implementation marks non-prevailing symbols as "dead" in the summary.
And then drops them in backend.
include/llvm/Transforms/IPO/FunctionImport.h | ||
---|---|---|
113 | s/isPrevaling/isPrevailing/ | |
113 | Clarify that non-prevailing means that the prevailing copy is not anywhere in IR. I.e as opposed to the non-prevailing copy which has a prevailing copy in another bitcode module. In that case we won't drop, which is fine for the purposes of this bug. | |
116 | s/isPrevaling/isPrevailing/ | |
lib/LTO/ThinLTOCodeGenerator.cpp | ||
626 | I guess this is because we don't have linker resolution info. Not sure what ld64 does if it has the same situation. Maybe add a comment that this interface can't do any better in the case where the prevailing symbol is in a native object because it doesn't provide linker info? | |
649 | suggest keeping the original name, which should be fine since it will just get overloaded. it matches the comment and the behavior better. | |
lib/Transforms/IPO/FunctionImport.cpp | ||
501 | s/isPrevaling/isPrevailing/ | |
501 | Add comment about what bool represents. | |
546 | s/isPrevaling/isPrevailing/ | |
546 | Unclear to me what is going on here and why IsLiveRoot needs to be tracked and passed down (its more like "is this reached via a live root", so maybe ReachedByLiveRoot?). What does it mean for something to be a live root but not prevailing, which seems to be the case requiring these changes? | |
564 | Interesting - is this a bug that was there before? It seems like an oversight not to have done this before. Or is it necessitated by your changes? If a separate bug, would be better to commit this separately. | |
565 | Don't even need to check if Summary is an alias. Just set Base live unconditionally - either it is the aliasee or equal to Summary. | |
631 | I think this is a hole. I.e. if the non-prevailing symbol is an alias, we can still do the incorrect internalization/inlining, right? Ok to fix in a follow-on patch though. |
- Addressed review comments.
lib/LTO/ThinLTOCodeGenerator.cpp | ||
---|---|---|
626 | Yes, we do not have resolution info. It is consistent with: Updated the comment. | |
649 | It needs either defining new static helper in llvm namespace too (where main computeDeadSymbols lives), May be we could rename the helper to something like stripDeadSymbols or doSummaryBasedGC | |
lib/Transforms/IPO/FunctionImport.cpp | ||
501 | Updated comment for computeDeadSymbols with description of isPrevailing return value. | |
546 | Yes, ReachedByLiveRoot describes the intention better. I did it for comdat-mixed-lto.ll case: Otherwise it would drop __cxx_global_var_init which is @llvm.global_ctors. | |
564 | Bug was before, currently Aliasee does not get Live flag, while Alias does. For example following code used to compute imports: I was need this change because introduced dropDeadSymbols which iterates over DefinedGlobals and can drop aliasee | |
631 | I'll investigate this case separatelly. |
LGTM. Just a few minor suggestions below.
include/llvm/Transforms/IPO/FunctionImport.h | ||
---|---|---|
112–119 | s/anywere/anywhere/ | |
114 | s/has no any/has no/ | |
lib/LTO/ThinLTOCodeGenerator.cpp | ||
649 | I see. Maybe computeDeadSymbolsInIndex then. | |
lib/Transforms/IPO/FunctionImport.cpp | ||
501 | I meant on the declaration of Worklist (the bool in the pair). It would be good to have a comment there indicating what the bool represents (i.e. whether it was reached by a value marked as a live root in the summary). | |
546 | Ok, just add a comment that values already marked as live roots in the summary are typically llvm internal global values that the linker doesn't know about, therefore we can't look at whether the linker decided it was prevailing. | |
564 | Ok, I think I can see why that wouldn't manifest as an issue before. |
Thanks for your comments, Teresa !
I am sorry, unfortunately I was not aware that ThinLTO has testcases in llvm/test/ThinLTO,
and not only in llvm/test/LTO. After running check-llvm I see few more testcases
failtures and going to investigate them now.
- Addressed comments.
- Updated patch to fix previously broken tests.
Now both check-llvm, and check-llvm-tools-gold pass.
Main change to fix testcases was following.
I had to check isDSOLocal in computeDeadSymbols and keep DSOLocal
symbols to be live. An example where it needed:
\test\ThinLTO\X86\deadstrip.ll
(https://github.com/llvm-mirror/llvm/blob/master/test/ThinLTO/X86/deadstrip.ll#L116)
it has bar_internal which is reachable but not prevailing. Linker does not see them.
Comments on other testcases changes:
- \test\ThinLTO\X86\deadstrip.ll: define was changed to declare because dead_func is dead, was dropped and converted to declaration.
- \test\ThinLTO\X86\internalize.ll: bar and linkonce_func were unreachable and were dropped. Made them be reachable.
- test\tools\gold\X86\global_with_section.ll: changes caused by converting dead dropped things to declarations.
LGTM with minor comment nit and missing change noted below.
Thanks - this looks cleaner using the enum!
include/llvm/Transforms/IPO/FunctionImport.h | ||
---|---|---|
111–112 | Suggest changing "that are not anywhere in IR are normally dead" to something like "are symbols without a prevailing copy anywhere in IR and are normally dead". I.e. a non-prevailing symbol here can have a def in IR, it is just not prevailing. | |
lib/Transforms/IPO/FunctionImport.cpp | ||
501 | I think you lost the corresponding change to the header declaration in this version of the patch. |
Thanks, Teresa !
Addressed last comments, rebased after rL323444,
and going to commit this in my tomorrow
(want to watch for bots to ensure all of them are happy).
include/llvm/Transforms/IPO/FunctionImport.h | ||
---|---|---|
111–112 | Done. | |
lib/Transforms/IPO/FunctionImport.cpp | ||
501 | I think it is still there, right below text change you suggested :) /// Compute all the symbols that are "dead": i.e these that can't be reached /// in the graph from any of the given symbols listed in /// \p GUIDPreservedSymbols. Non-prevailing symbols are symbols without a /// prevailing copy anywhere in IR and are normally dead, \p isPrevailing /// predicate returns status of symbol. I removed comment done on the declaration of Worklist though, because after introducing enum |
(Note that the change to the declaration of computeDeadSymbols in ModuleSummaryIndex.h is still missing from the uploaded version of the patch)
No worries - I have done the same thing myself (broken lld bots with LTO changes since I don't always run them before commit).
lib/LTO/LTO.cpp | ||
---|---|---|
429 | This assumes that the non-prevailing asm symbol with the name is seen here before the prevailing symbol without a name (which does always seem to currently be the case). Otherwise, the non-prevailing symbol would come along and fill in the name since it will be empty. Suggest moving this block up above the previous one that sets the Prevailing flag, then add an assert that if !Sym.getIRName.empty() that Res.Prevailing is not already true. I.e. that we don't overwrite the IRName when we already set it from the Prevailing copy. |
lib/LTO/LTO.cpp | ||
---|---|---|
429 | Yes, currently in a single module asm symbols looks always placed before IR ones seems. If we have 2 modules, where
it would not work, because handing of resolution from module B would trigger assertion failture. Seems what we can do is never update IRName if symbol is already known to be prevailing (so always prefer IR name of prevailing symbol). |
Suggest changing "that are not anywhere in IR are normally dead" to something like "are symbols without a prevailing copy anywhere in IR and are normally dead". I.e. a non-prevailing symbol here can have a def in IR, it is just not prevailing.