This is an archive of the discontinued LLVM Phabricator instance.

[rfc] [lld] Do not ignore comdats when parsing LTO objects
AbandonedPublic

Authored by christylee on May 19 2020, 3:49 PM.

Details

Summary

We are trying to migrate from gold to lld, and we found that when building with thinlto, lld does not de-duplicate .debug_types. De-duplication is successful with monolithic lto. Repro: clang -flto=thin -fdebug-types-section -fuse-ld=lld -Wl,-plugin-opt=-generate-type-units

If we set ignoreComdats=false for lto builds, then the de-duplication is successful again. Looking at the thinlto code, it looks like there is no comdat folding at thinlto time.

For more context, see https://reviews.llvm.org/D62884

Diff Detail

Event Timeline

christylee created this revision.May 19 2020, 3:49 PM
christylee edited the summary of this revision. (Show Details)May 19 2020, 3:53 PM
christylee added reviewers: ruiu, hoy, wenlei, sbc100, dblaikie.

If you are going to make the change here can you make the same change in the wasm linker too (in SymbolTable::addCombinedLTOObject)?

Also, if we land this then we can completely remove the argument ignoreComdats argument to parse.

Addressed comments. 1). made the same change to wasm. 2). removed the argument ignoreComdats.

christylee retitled this revision from [RFC] [LLD] Do not ignore comdats when parsing LTO objects to [lld] Do not ignore comdats when parsing LTO objects.May 20 2020, 10:38 AM
christylee retitled this revision from [lld] Do not ignore comdats when parsing LTO objects to [rfc] [lld] Do not ignore comdats when parsing LTO objects.May 20 2020, 3:43 PM

I tried building a more complex workload and the build failed due to some undefined hidden symbol. For thinlto, is the linker or codegen supposed to handle comdat folding?

ruiu added a comment.May 20 2020, 9:49 PM

I tried building a more complex workload and the build failed due to some undefined hidden symbol. For thinlto, is the linker or codegen supposed to handle comdat folding?

How did it work with gold with the LTO plugin?

How did it work with gold with the LTO plugin?

As far as I can tell, the llvm gold-plugin.cpp takes the compiled files and pass to gold for normal linking, where comdat de-duplication happens in Sized_relobj_file<size, big_endian>::include_section_group() https://kernel.googlesource.com/pub/scm/linux/kernel/git/hjl/binutils/+/hjl/secondary/gold/object.cc?autodive=0%2F#856. The only difference I see is that for gold, if the first word of a SHT_GROUP section contains flags then it treats the section as a normal section, whereas lld would continue and not create a new section for it. However, the "undefined hidden symbols" come from SHT_GROUP sections so I don't think they are related.

christylee abandoned this revision.May 28 2020, 2:06 PM

@hoy and I figured out the issue, the problem is caused by de-duplicating twice in thinlto, once in thin link, once in final link. Abandoning this in favor of a new approach. New diff coming up soon.