[LLD][RFC] Deduplicate type units with local ThinLTO

[ayermolo]

Summary

I ran into an issue where LLD wasn't de-duplicating type units (TU) with local
ThinLTO. A case is where one TU came from bitcode file build with local ThinLTO,
and another TU came from pre-build ELF object.
With distributed ThinLTO everything was de-duplicated as expected since inputs to
final link are just ELF files. This is a variation of if both type units come
from ThinLTO.

I took a look at https://reviews.llvm.org/D80765, which seems to go nowhere and
primarily just dealt with type units that come from ThinLTO. Due to it's
implementation it wouldn't work for mix mode.

Here I propose a more localized fix for this that just deals with type units. It
deals with a case where all type units come from ThinlTO, and with mix mode
case, and makes it so that behavior is the same between regular and distributed
ThinLTO.

P.S. I will add tests after it's clear that this isn't a dead end.

[ayermolo]

a bit of cleanup

[ayermolo]

a bit of cleanup

[ayermolo]

@MaskRay ping.

lld/ELF/InputFiles.cpp

 
 template <class ELFT>
 uint32_t ObjFile<ELFT>::getSectionIndex(const Elf_Sym &sym) const {
   return CHECK(
       this->getObj().getSectionIndex(sym, getELFSyms<ELFT>(), shndxTable),
       this);
 }
 
+
+/// For DWARF4 the Type Units are in their own seperate .debug_types section.
+/// For DWARF5 they are part of the .debug_info section.
+/// Returns true if link group contains either of those sections.
+template <class ELFT, class Elf_Word>
+static bool isDebugGroupSection(ObjFile<ELFT> &objFile,
+                                ArrayRef<Elf_Word> &entries,
+                                StringRef &shstrtab) {
+  ArrayRef<Elf_Word> ndxArray = entries.slice(1);
+  object::ELFFile<ELFT> obj = objFile.getObj();
+  for (uint32_t ndx : ndxArray) {
+    Expected<const typename ELFT::Shdr *> grpSection = obj.getSection(ndx);
+    if (!grpSection) {
+      warn(toString(&objFile) + " : " +
+           toString(std::move(grpSection.takeError())));
+      break;
+    }
+    StringRef name = check(obj.getSectionName(**grpSection, shstrtab));
+    if ((name.find("debug_types") != StringRef::npos) ||
+        (name.find("debug_info") != StringRef::npos))
+      return true;
+  }
+  return false;
+}
+
 template <class ELFT> void ObjFile<ELFT>::parse(bool ignoreComdats) {
   object::ELFFile<ELFT> obj = this->getObj();
   // Read a section table. justSymbols is usually false.
   if (this->justSymbols) {
     initializeJustSymbols();
     initializeSymbols(obj);
     return;
   }
for (size_t i = 0; i != size; ++i) {
 
     if (sec.sh_type != SHT_GROUP)
       continue;
     StringRef signature = getShtGroupSignature(objSections, sec);
     ArrayRef<Elf_Word> entries =
         CHECK(obj.template getSectionContentsAsArray<Elf_Word>(sec), this);
     if (entries.empty())
       fatal(toString(this) + ": empty SHT_GROUP");
-
     Elf_Word flag = entries[0];
     if (flag && flag != GRP_COMDAT)
       fatal(toString(this) + ": unsupported SHT_GROUP format");
-
+    bool isDebugSection = isDebugGroupSection<ELFT, Elf_Word>(*this, entries, shstrtab);
     bool keepGroup =
-        (flag & GRP_COMDAT) == 0 || ignoreComdats ||
+        (flag & GRP_COMDAT) == 0 || (ignoreComdats && !isDebugSection) ||
         symtab.comdatGroups.try_emplace(CachedHashStringRef(signature), this)
             .second;
     if (keepGroup) {
       if (config->relocatable)
         this->sections[i] = createInputSection(
             i, sec, check(obj.getSectionName(sec, shstrtab)));
       continue;
     }