[ELF] Parallelize input section initialization

[MaskRay]

Summary

This implements the last step of
https://discourse.llvm.org/t/parallel-input-file-parsing/60164 for the ELF port.

For an ELF object file, we previously did: parse, (parallel) initializeLocalSymbols, (parallel) postParseObjectFile.
Now we do: parse, (parallel) initSectionsAndLocalSyms, (parallel) postParseObjectFile.

initSectionsAndLocalSyms does most of input section initialization.
The sequential parse does SHT_ARM_ATTRIBUTES/SHT_RISCV_ATTRIBUTES/SHT_GROUP initialization for now.

Performance linking some programs with --threads=8 (glibc 2.33 malloc and mimalloc):

• clang: 1.05x as fast with glibc malloc, 1.03x as fast with mimalloc
• chrome: 1.04x as fast with glibc malloc, 1.03x as fast with mimalloc
• internal search program: 1.08x as fast with glibc malloc, 1.05x as fast with mimalloc

[MaskRay]

reduce diff

[peter.smith]

Will take a look as soon as I can. Have been on holiday the past week so I'm a bit behind schedule.

[peter.smith]

lld/ELF/Driver.cpp

2389

I don't think any caller has used the ignoreComdats default parameter, is it worth removing the = false ?

lld/ELF/InputFiles.cpp

438

Worth a comment along the lines of:
"Handle dependent libraries and selection of ELF Section Groups as these cannot be done in parallel."

639

Out of curiosity, is there a reason this needs to be moved from the previous place? No objections to moving it, just curious.

902

IIUC createInputSection can be called in parallel now from initializeSections will be worth a comment as a warning to what is safe to be done within the function and any functions called from it.

LLD's handling of Arm build attributes is somewhat threadbare at the moment. I expect that Arm will want to send in some patches to improve this in the future. When they do I think this will need to be moved to avoid threading problems. The current implementation gets away with it as I don't think it ever reads from the global state, just writes to it.

919

I'm guessing make_unique is thread safe in this context?

973

Will be worth checking readAndFeatures as they read and update a global value.

lld/include/lld/Common/Memory.h

65

Apologies in advance, I'm not too familiar with LLVM in this area:

• As I understand it, we'll be creating a new allocator per thread, and storing it in TLS?
• I'm assuming that if the underlying threads are destroyed then the allocator and all the memory it points to will be destroyed as well?
• I'm assuming that we're relying on parallelForEach never destroying threads during the lifetime of a link? Is this safe to rely on?

Would be useful to document the assumptions, as we may have to implement a different solution if parallelForEach ever changes policy in the future.

79

I suggest makeThreadLocal rather than makeTL as it isn't obvious from the call site that TL is Thread Local and there are not that many calls to it. Not a strong opinion though.

[MaskRay]

address comments

[MaskRay]

lld/ELF/InputFiles.cpp

639

Moved back. The reason is that I have had a very old patch and just recently fixed some issues and posted it here. The code move was due to playing with some parallelism schemes and was not needed.

902

Thanks for the heads-up. I have to move the attributes code to ObjFile<ELFT>::parse as otherwise in.attributes == nullptr is racy, though benign, would likely fail thread sanitizer.

973

Thx. readAndFeatures is thread-safe.

lld/include/lld/Common/Memory.h

65

Good catch. Yes to all 3 questions. I've added a comment to document the assumption.

[peter.smith]

Thanks for the update, looks good to me, I don't have any more comments.

[MaskRay]

Moved makeThreadLocal to a separate commit to reduce diff and make the patch more focused.

lld/ELF/Driver.cpp

static uint32_t getAndFeatures() {
 
   // Force enable Shadow Stack.
   if (config->zShstk)
     ret |= GNU_PROPERTY_X86_FEATURE_1_SHSTK;
 
   return ret;
 }
 
-static void initializeLocalSymbols(ELFFileBase *file) {
+static void initSectionsAndLocalSyms(ELFFileBase *file,
+                                     bool ignoreComdats = false) {
   switch (config->ekind) {
   case ELF32LEKind:
-    cast<ObjFile<ELF32LE>>(file)->initializeLocalSymbols();
+    cast<ObjFile<ELF32LE>>(file)->initSectionsAndLocalSyms(ignoreComdats);
     break;
   case ELF32BEKind:
-    cast<ObjFile<ELF32BE>>(file)->initializeLocalSymbols();
+    cast<ObjFile<ELF32BE>>(file)->initSectionsAndLocalSyms(ignoreComdats);
     break;
   case ELF64LEKind:
-    cast<ObjFile<ELF64LE>>(file)->initializeLocalSymbols();
+    cast<ObjFile<ELF64LE>>(file)->initSectionsAndLocalSyms(ignoreComdats);
     break;
   case ELF64BEKind:
-    cast<ObjFile<ELF64BE>>(file)->initializeLocalSymbols();
+    cast<ObjFile<ELF64BE>>(file)->initSectionsAndLocalSyms(ignoreComdats);
     break;
   default:
     llvm_unreachable("");
   }
 }
 
 static void postParseObjectFile(ELFFileBase *file) {
   switch (config->ekind) {
if (!ctx->bitcodeFiles.empty())
     for (auto *s : lto::LTO::getRuntimeLibcallSymbols())
       handleLibcall(s);
 
   // Archive members defining __wrap symbols may be extracted.
   std::vector<WrappedSymbol> wrapped = addWrappedSymbols(args);
 
   // No more lazy bitcode can be extracted at this point. Do post parse work
   // like checking duplicate symbols.
-  parallelForEach(ctx->objectFiles, initializeLocalSymbols);
+  parallelForEach(ctx->objectFiles, [](ELFFileBase *file) {
+    initSectionsAndLocalSyms(file, /*ignoreComdats=*/false);
+  });
   parallelForEach(ctx->objectFiles, postParseObjectFile);
   parallelForEach(ctx->bitcodeFiles,
                   [](BitcodeFile *file) { file->postParse(); });
   for (auto &it : ctx->nonPrevailingSyms) {
     Symbol &sym = *it.first;
     sym.replace(Undefined{sym.file, sym.getName(), sym.binding, sym.stOther,
                           sym.type, it.second});
     cast<Undefined>(sym).nonPrevailing = true;
void LinkerDriver::link(opt::InputArgList &args) {
 
   // Bail out if normal linked output is skipped due to LTO.
   if (skipLinkedOutput)
     return;
 
   // compileBitcodeFiles may have produced lto.tmp object files. After this, no
   // more file will be added.
   auto newObjectFiles = makeArrayRef(ctx->objectFiles).slice(numObjsBeforeLTO);
-  parallelForEach(newObjectFiles, initializeLocalSymbols);
+  parallelForEach(newObjectFiles, [](ELFFileBase *file) {
+    initSectionsAndLocalSyms(file, /*ignoreComdats=*/true);
+  });
   parallelForEach(newObjectFiles, postParseObjectFile);
   for (const DuplicateSymbol &d : ctx->duplicates)
     reportDuplicate(*d.sym, d.file, d.section, d.value);
 
   // Handle --exclude-libs again because lto.tmp may reference additional
   // libcalls symbols defined in an excluded archive. This may override
   // versionId set by scanVersionScript().
   if (args.hasArg(OPT_exclude_libs))

lld/ELF/InputFiles.h

public:
 
   // True if the file defines functions compiled with -fsplit-stack,
   // but had one or more functions with the no_split_stack attribute.
   bool someNoSplitStack = false;
 
   // Get cached DWARF information.
   DWARFCache *getDwarf();
 
-  void initializeLocalSymbols();
+  void initSectionsAndLocalSyms(bool ignoreComdats);
   void postParse();
 
 private:
   void initializeSections(bool ignoreComdats,
                           const llvm::object::ELFFile<ELFT> &obj);
   void initializeSymbols(const llvm::object::ELFFile<ELFT> &obj);
   void initializeJustSymbols();
 

lld/ELF/InputFiles.cpp

uint32_t ObjFile<ELFT>::getSectionIndex(const Elf_Sym &sym) const {
   return CHECK(
       this->getObj().getSectionIndex(sym, getELFSyms<ELFT>(), shndxTable),
       this);
 }
 
 template <class ELFT> void ObjFile<ELFT>::parse(bool ignoreComdats) {
   object::ELFFile<ELFT> obj = this->getObj();
   // Read a section table. justSymbols is usually false.
-  if (this->justSymbols)
+  if (this->justSymbols) {
     initializeJustSymbols();
-  else
-    initializeSections(ignoreComdats, obj);
+    initializeSymbols(obj);
+    return;
+  }
+
+  ArrayRef<Elf_Shdr> objSections = getELFShdrs<ELFT>();
+  StringRef shstrtab = CHECK(obj.getSectionStringTable(objSections), this);
+  uint64_t size = objSections.size();
+  sections.resize(size);
+  for (size_t i = 0; i != size; ++i) {
+    const Elf_Shdr &sec = objSections[i];
+    if (sec.sh_type == SHT_LLVM_DEPENDENT_LIBRARIES && !config->relocatable) {
+      StringRef name = check(obj.getSectionName(sec, shstrtab));
+      ArrayRef<char> data = CHECK(
+          this->getObj().template getSectionContentsAsArray<char>(sec), this);
+      if (!data.empty() && data.back() != '\0') {
+        error(
+            toString(this) +
+            ": corrupted dependent libraries section (unterminated string): " +
+            name);
+      } else {
+        for (const char *d = data.begin(), *e = data.end(); d < e;) {
+          StringRef s(d);
+          addDependentLibrary(s, this);
+          d += s.size() + 1;
+        }
+      }
+      this->sections[i] = &InputSection::discarded;
+      continue;
+    }
+
+    if (sec.sh_type != SHT_GROUP)
+      continue;
+    StringRef signature = getShtGroupSignature(objSections, sec);
+    ArrayRef<Elf_Word> entries =
+        CHECK(obj.template getSectionContentsAsArray<Elf_Word>(sec), this);
+    if (entries.empty())
+      fatal(toString(this) + ": empty SHT_GROUP");
+
+    Elf_Word flag = entries[0];
+    if (flag && flag != GRP_COMDAT)
+      fatal(toString(this) + ": unsupported SHT_GROUP format");
+
+    bool keepGroup =
+        (flag & GRP_COMDAT) == 0 || ignoreComdats ||
+        symtab->comdatGroups.try_emplace(CachedHashStringRef(signature), this)
+            .second;
+    if (keepGroup) {
+      if (config->relocatable)
+        this->sections[i] = createInputSection(
+            i, sec, check(obj.getSectionName(sec, shstrtab)));
+      continue;
+    }
+
+    // Otherwise, discard group members.
+    for (uint32_t secIndex : entries.slice(1)) {
+      if (secIndex >= size)
+        fatal(toString(this) +
+              ": invalid section index in group: " + Twine(secIndex));
+      this->sections[secIndex] = &InputSection::discarded;
+    }
+  }
 
   // Read a symbol table.
   initializeSymbols(obj);
 }
 
 // Sections with SHT_GROUP and comdat bits define comdat section groups.
 // They are identified and deduplicated by group name. This function
 // returns a group name.

 }
 
 template <class ELFT>
 void ObjFile<ELFT>::initializeSections(bool ignoreComdats,
                                        const llvm::object::ELFFile<ELFT> &obj) {
   ArrayRef<Elf_Shdr> objSections = getELFShdrs<ELFT>();
   StringRef shstrtab = CHECK(obj.getSectionStringTable(objSections), this);
   uint64_t size = objSections.size();
-  this->sections.resize(size);
-
-  std::vector<ArrayRef<Elf_Word>> selectedGroups;
-
+  SmallVector<ArrayRef<Elf_Word>, 0> selectedGroups;
   for (size_t i = 0; i != size; ++i) {
     if (this->sections[i] == &InputSection::discarded)
       continue;
     const Elf_Shdr &sec = objSections[i];
 
     // SHF_EXCLUDE'ed sections are discarded by the linker. However,
     // if -r is given, we'll let the final link discard such sections.
     // This is compatible with GNU.
if ((sec.sh_flags & SHF_EXCLUDE) && !config->relocatable) {
                "(likely created using objcopy or ld -r)");
       }
       this->sections[i] = &InputSection::discarded;
       continue;
     }
 
     switch (sec.sh_type) {
     case SHT_GROUP: {
-      // De-duplicate section groups by their signatures.
-      StringRef signature = getShtGroupSignature(objSections, sec);
-      this->sections[i] = &InputSection::discarded;
-
+      if (!config->relocatable)
+        sections[i] = &InputSection::discarded;
+      StringRef signature =
+          cantFail(this->getELFSyms<ELFT>()[sec.sh_info].getName(stringTable));
       ArrayRef<Elf_Word> entries =
-          CHECK(obj.template getSectionContentsAsArray<Elf_Word>(sec), this);
-      if (entries.empty())
-        fatal(toString(this) + ": empty SHT_GROUP");
-
-      Elf_Word flag = entries[0];
-      if (flag && flag != GRP_COMDAT)
-        fatal(toString(this) + ": unsupported SHT_GROUP format");
-
-      bool keepGroup =
-          (flag & GRP_COMDAT) == 0 || ignoreComdats ||
-          symtab->comdatGroups.try_emplace(CachedHashStringRef(signature), this)
-              .second;
-      if (keepGroup) {
-        if (config->relocatable)
-          this->sections[i] = createInputSection(
-              i, sec, check(obj.getSectionName(sec, shstrtab)));
+          cantFail(obj.template getSectionContentsAsArray<Elf_Word>(sec));
+      if ((entries[0] & GRP_COMDAT) == 0 || ignoreComdats ||
+          symtab->comdatGroups.find(CachedHashStringRef(signature))->second ==
+              this)
         selectedGroups.push_back(entries);
-        continue;
-      }
-
-      // Otherwise, discard group members.
-      for (uint32_t secIndex : entries.slice(1)) {
-        if (secIndex >= size)
-          fatal(toString(this) +
-                ": invalid section index in group: " + Twine(secIndex));
-        this->sections[secIndex] = &InputSection::discarded;
-      }
       break;
     }
     case SHT_SYMTAB_SHNDX:
       shndxTable = CHECK(obj.getSHNDXTable(sec, objSections), this);
       break;
     case SHT_SYMTAB:
     case SHT_STRTAB:
     case SHT_REL:
     case SHT_RELA:
     case SHT_NULL:
       break;
     case SHT_LLVM_SYMPART:
       ctx->hasSympart.store(true, std::memory_order_relaxed);
       LLVM_FALLTHROUGH;
     default:
       this->sections[i] =
           createInputSection(i, sec, check(obj.getSectionName(sec, shstrtab)));
     }
   }
 
+  for (ArrayRef<Elf_Word> entries : selectedGroups)
+    handleSectionGroup<ELFT>(this->sections, entries);
+
   // We have a second loop. It is used to:
   // 1) handle SHF_LINK_ORDER sections.
   // 2) create SHT_REL[A] sections. In some cases the section header index of a
   //    relocation section may be smaller than that of the relocated section. In
   //    such cases, the relocation section would attempt to reference a target
   //    section that has not yet been created. For simplicity, delay creation of
   //    relocation sections until now.
   for (size_t i = 0; i != size; ++i) {
if (sec.sh_type == SHT_REL || sec.sh_type == SHT_RELA) {
         continue;
 
       // ELF spec allows mergeable sections with relocations, but they are rare,
       // and it is in practice hard to merge such sections by contents, because
       // applying relocations at end of linking changes section contents. So, we
       // simply handle such sections as non-mergeable ones. Degrading like this
       // is acceptable because section merging is optional.
       if (auto *ms = dyn_cast<MergeInputSection>(s)) {
-        s = make<InputSection>(ms->file, ms->flags, ms->type, ms->alignment,
+        s = makeTL<InputSection>(ms->file, ms->flags, ms->type, ms->alignment,
                                  ms->data(), ms->name);
         sections[info] = s;
       }
 
       if (s->relSecIdx != 0)
         error(
             toString(s) +
             ": multiple relocation sections to one section are not supported");
       s->relSecIdx = i;
 
       // Relocation sections are usually removed from the output, so return
       // `nullptr` for the normal case. However, if -r or --emit-relocs is
       // specified, we need to copy them to the output. (Some post link analysis
       // tools specify --emit-relocs to obtain the information.)
       if (config->copyRelocs) {
-        auto *isec = make<InputSection>(
+        auto *isec = makeTL<InputSection>(
             *this, sec, check(obj.getSectionName(sec, shstrtab)));
         // If the relocated section is discarded (due to /DISCARD/ or
         // --gc-sections), the relocation section should be discarded as well.
         s->dependentSections.push_back(isec);
         sections[i] = isec;
       }
       continue;
     }
for (size_t i = 0; i != size; ++i) {
     // discarded.
     InputSection *isec = cast<InputSection>(this->sections[i]);
     linkSec->dependentSections.push_back(isec);
     if (!isa<InputSection>(linkSec))
       error("a section " + isec->name +
             " with SHF_LINK_ORDER should not refer a non-regular section: " +
             toString(linkSec));
   }
-
-  for (ArrayRef<Elf_Word> entries : selectedGroups)
-    handleSectionGroup<ELFT>(this->sections, entries);
 }
 
 // For ARM only, to set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD
 // flag in the ELF Header we need to look at Tag_ABI_VFP_args to find out how
 // the input objects have been compiled.
 static void updateARMVFPArgs(const ARMAttributeParser &attributes,
                              const InputFile *f) {
   Optional<unsigned> attr =
error(toString(this) + Twine(": relocation section (index ") + Twine(idx) +
         ") has invalid sh_info (" + Twine(info) + ")");
   return nullptr;
 }
 
 template <class ELFT>
 InputSectionBase *ObjFile<ELFT>::createInputSection(uint32_t idx,
                                                     const Elf_Shdr &sec,
                                                     StringRef name) {
   if (sec.sh_type == SHT_ARM_ATTRIBUTES && config->emachine == EM_ARM) {
     ARMAttributeParser attributes;
     ArrayRef<uint8_t> contents = check(this->getObj().getSectionContents(sec));
     if (Error e = attributes.parse(contents, config->ekind == ELF32LEKind
                                                  ? support::little
                                                  : support::big)) {
       auto *isec = make<InputSection>(*this, sec, name);
       warn(toString(isec) + ": " + llvm::toString(std::move(e)));
     } else {
       updateSupportedARMFeatures(attributes);
       updateARMVFPArgs(attributes, this);
 
       // FIXME: Retain the first attribute section we see. The eglibc ARM
       // dynamic loaders require the presence of an attribute section for dlopen
       // to work. In a full implementation we would merge all attribute
       // sections.
       if (in.attributes == nullptr) {
         in.attributes = std::make_unique<InputSection>(*this, sec, name);
         return in.attributes.get();
       }
       return &InputSection::discarded;
     }
   }
 
   if (sec.sh_type == SHT_RISCV_ATTRIBUTES && config->emachine == EM_RISCV) {
     RISCVAttributeParser attributes;
if (Error e = attributes.parse(contents, support::little)) {
       if (in.attributes == nullptr) {
         in.attributes = std::make_unique<InputSection>(*this, sec, name);
         return in.attributes.get();
       }
       return &InputSection::discarded;
     }
   }
 
-  if (sec.sh_type == SHT_LLVM_DEPENDENT_LIBRARIES && !config->relocatable) {
-    ArrayRef<char> data =
-        CHECK(this->getObj().template getSectionContentsAsArray<char>(sec), this);
-    if (!data.empty() && data.back() != '\0') {
-      error(toString(this) +
-            ": corrupted dependent libraries section (unterminated string): " +
-            name);
-      return &InputSection::discarded;
-    }
-    for (const char *d = data.begin(), *e = data.end(); d < e;) {
-      StringRef s(d);
-      addDependentLibrary(s, this);
-      d += s.size() + 1;
-    }
-    return &InputSection::discarded;
-  }
-
   if (name.startswith(".n")) {
     // The GNU linker uses .note.GNU-stack section as a marker indicating
     // that the code in the object file does not expect that the stack is
     // executable (in terms of NX bit). If all input files have the marker,
     // the GNU linker adds a PT_GNU_STACK segment to tells the loader to
     // make the stack non-executable. Most object files have this section as
     // of 2017.
     //
if (name.startswith(".n")) {
     // Object files that use processor features such as Intel Control-Flow
     // Enforcement (CET) or AArch64 Branch Target Identification BTI, use a
     // .note.gnu.property section containing a bitfield of feature bits like the
     // GNU_PROPERTY_X86_FEATURE_1_IBT flag. Read a bitmap containing the flag.
     //
     // Since we merge bitmaps from multiple object files to create a new
     // .note.gnu.property containing a single AND'ed bitmap, we discard an input
     // file's .note.gnu.property section.
     if (name == ".note.gnu.property") {
       this->andFeatures = readAndFeatures<ELFT>(InputSection(*this, sec, name));
       return &InputSection::discarded;
     }
 
     // Split stacks is a feature to support a discontiguous stack,
     // commonly used in the programming language Go. For the details,
     // see https://gcc.gnu.org/wiki/SplitStacks. An object file compiled
     // for split stack will include a .note.GNU-split-stack section.
if (name.startswith(".n")) {
     if (name == ".note.gnu.build-id")
       return &InputSection::discarded;
   }
 
   // The linker merges EH (exception handling) frames and creates a
   // .eh_frame_hdr section for runtime. So we handle them with a special
   // class. For relocatable outputs, they are just passed through.
   if (name == ".eh_frame" && !config->relocatable)
-    return make<EhInputSection>(*this, sec, name);
+    return makeTL<EhInputSection>(*this, sec, name);
 
   if ((sec.sh_flags & SHF_MERGE) && shouldMerge(sec, name))
-    return make<MergeInputSection>(*this, sec, name);
-  return make<InputSection>(*this, sec, name);
+    return makeTL<MergeInputSection>(*this, sec, name);
+  return makeTL<InputSection>(*this, sec, name);
 }
 
 // Initialize this->Symbols. this->Symbols is a parallel array as
 // its corresponding ELF symbol table.
 template <class ELFT>
 void ObjFile<ELFT>::initializeSymbols(const object::ELFFile<ELFT> &obj) {
   SymbolTable &symtab = *elf::symtab;
 
for (unsigned i : undefineds) {
     Symbol *sym = symbols[i];
     sym->resolve(Undefined{this, StringRef(), eSym.getBinding(), eSym.st_other,
                            eSym.getType()});
     sym->isUsedInRegularObj = true;
     sym->referenced = true;
   }
 }
 
-template <class ELFT> void ObjFile<ELFT>::initializeLocalSymbols() {
+template <class ELFT>
+void ObjFile<ELFT>::initSectionsAndLocalSyms(bool ignoreComdats) {
+  if (!justSymbols)
+    initializeSections(ignoreComdats, getObj());
+
   if (!firstGlobal)
     return;
   localSymStorage = std::make_unique<SymbolUnion[]>(firstGlobal);
   SymbolUnion *locals = localSymStorage.get();
 
   ArrayRef<Elf_Sym> eSyms = this->getELFSyms<ELFT>();
   for (size_t i = 0, end = firstGlobal; i != end; ++i) {
     const Elf_Sym &eSym = eSyms[i];

lld/ELF/InputSection.cpp

 std::string InputSectionBase::getObjMsg(uint64_t off) {
   std::string filename = std::string(file->getName());
 
   std::string archive;
   if (!file->archiveName.empty())
     archive = (" in archive " + file->archiveName).str();
 
   // Find a symbol that encloses a given location. getObjMsg may be called
-  // before ObjFile::initializeLocalSymbols where local symbols are initialized.
+  // before ObjFile::initSectionsAndLocalSyms where local symbols are
+  // initialized.
   for (Symbol *b : file->getSymbols())
     if (auto *d = dyn_cast_or_null<Defined>(b))
       if (d->section == this && d->value <= off && off < d->value + d->size)
         return filename + ":(" + toString(*d) + ")" + archive;
 
   // If there's no symbol, print out the offset in the section.
   return (filename + ":(" + name + "+0x" + utohexstr(off) + ")" + archive)
       .str();

lld/include/lld/Common/Memory.h

 
 // Creates new instances of T off a (almost) contiguous arena/object pool. The
 // instances are destroyed whenever lldMain() goes out of scope.
 template <typename T, typename... U> T *make(U &&... args) {
   return new (getSpecificAllocSingleton<T>().Allocate())
       T(std::forward<U>(args)...);
 }
 
+template <typename T>
+inline llvm::SpecificBumpPtrAllocator<T> &
+getSpecificAllocSingletonThreadLocal() {
+  thread_local SpecificAlloc<T> instance;
+  return instance.alloc;
+}
+
+// Create a new instance of T off a thread-local SpecificAlloc, used by code
+// like parallel input section initialization.
+//
+// Note: Some ports (e.g. ELF) have lots of global states which are currently
+// infeasible to remove, and context() just adds overhead with no benefit. The
+// allocation performance is of higher importance, so we simply use thread_local
+// allocators instead of doing context indirection and pthread_getspecific.
+template <typename T, typename... U> T *makeTL(U &&...args) {
+  return new (getSpecificAllocSingletonThreadLocal<T>().Allocate())
+      T(std::forward<U>(args)...);
+}
+
 } // namespace lld
 
 #endif