ELF: Create unique SHF_GNU_RETAIN sections for llvm.used global objects

[MaskRay]

Summary

If a global object is listed in @llvm.used, place it in a unique section with
the SHF_GNU_RETAIN flag. The section is a GC root under ld --gc-sections
with LLD>=13 or GNU ld>=2.36.

For front ends which do not expect to see multiple sections of the same name,
consider emitting @llvm.compiler.used instead of @llvm.used.

SHF_GNU_RETAIN is restricted to ELFOSABI_GNU and ELFOSABI_FREEBSD in
binutils. We don't do the restriction - see the rationale in D95749.

The integrated assembler has supported SHF_GNU_RETAIN since D95730.
GNU as>=2.36 supports section flag 'R'.
We don't need to worry about GNU ld support because older GNU ld just ignores
the unknown SHF_GNU_RETAIN.

With this change, __attribute__((retain)) functions/variables emitted
by clang will get the SHF_GNU_RETAIN flag.

[rnk]

This will mostly only affect users that are using the integrated assembler, or who are using clang -fno-integrated-as -fbinutils-version=2.36+. I think that means all the compatibility concerns about the new R section flag are addressed. Nice.

Looks good to me.

[nikic]

Just FYI this patch exposes a bug in at least GNU gold (GNU Binutils for Ubuntu 2.34) 1.16 when used in conjunction with section(".init_array").

; RUN: clang -fuse-ld=gold %s
#include <stdio.h>
static void do_init(void) { printf("Foobar\n"); }

__attribute__((used, retain, section(".init_array")))
static void (*init)(void) = &do_init;

int main() { return 0; }

The symbol is not placed between __init_array_start/__init_array_end and thus the init function is not run.

Following your recommendation, we switched the #[used] attribute in rust to use llvm.compiler.used instead of llvm.used. Hopefully that won't have other fallout.

[MaskRay]

In D97448#2933400, @nikic wrote:

Just FYI this patch exposes a bug in at least GNU gold (GNU Binutils for Ubuntu 2.34) 1.16 when used in conjunction with section(".init_array").

; RUN: clang -fuse-ld=gold %s
#include <stdio.h>
static void do_init(void) { printf("Foobar\n"); }

__attribute__((used, retain, section(".init_array")))
static void (*init)(void) = &do_init;

int main() { return 0; }

The symbol is not placed between __init_array_start/__init_array_end and thus the init function is not run.

The gold bug is fixed in 2.36 (https://sourceware.org/bugzilla/show_bug.cgi?id=27039)

Following your recommendation, we switched the #[used] attribute in rust to use llvm.compiler.used instead of llvm.used. Hopefully that won't have other fallout.

I informed Rust folks in March. https://github.com/rust-lang/rust/issues/40289#issuecomment-788567604
Switching to llvm.compiler.used improves compatibility with GCC/Clang.

[nikic]

Thanks for the reference! The part that wasn't obvious to me (both from the patch summary here and the gold bug) is that using SHF_GNU_RETAIN with an older binutils version will not only ignore the retain semantics (which is entirely expected), but will lead to silent miscompiles unrelated to retain semantics. From the description here and rnk's comment, it sounded like there aren't any toolchain compatibility concerns.

It would be good to update LangRef in this regard. Currently the llvm.used documentation says:

This is commonly used to represent references from inline asms and other things the compiler cannot “see”, and corresponds to “attribute((used))” in GNU C.

And the llvm.compiler.used documentation says:

This is a rare construct that should only be used in rare circumstances, and should not be exposed to source languages.

While now the recommendation seems to be the other way around: Frontends should map "used"-like attributes to llvm.compiler.used rather than llvm.used, and using the latter may be dangerous if the frontend doesn't control the used toolchain.

[MaskRay]

In D97448#2933741, @nikic wrote:

Thanks for the reference! The part that wasn't obvious to me (both from the patch summary here and the gold bug) is that using SHF_GNU_RETAIN with an older binutils version will not only ignore the retain semantics (which is entirely expected), but will lead to silent miscompiles unrelated to retain semantics. From the description here and rnk's comment, it sounded like there aren't any toolchain compatibility concerns.

It would be good to update LangRef in this regard. Currently the llvm.used documentation says:

This is commonly used to represent references from inline asms and other things the compiler cannot “see”, and corresponds to “attribute((used))” in GNU C.

LangRef has mentioned the linker behavior since 2009 (rGae76db5edd9251d76af488d601be01d903805f8e), despite the fact that ELF did not really distinguish llvm.used and llvm.compiler.used before this patch.

GCC never adds linker semantics to __attribute__((used)).
The armcc documentation mentions the linker behavior "Data marked with attribute((used)) is tagged in the object file to avoid removal by linker unused section removal."
I don't have access to armcc to check whether/how this is actually satisfied.
Clang Mach-O may use the semantics for a long time.

And the llvm.compiler.used documentation says:

This is a rare construct that should only be used in rare circumstances, and should not be exposed to source languages.

While now the recommendation seems to be the other way around: Frontends should map "used"-like attributes to llvm.compiler.used rather than llvm.used, and using the latter may be dangerous if the frontend doesn't control the used toolchain.

Using is the latter is not more dangerous:/ The compiler just did not do what LangRef promised on ELF for a long time.
Rust likely misused llvm.used...

That said, sounds good to make llvm.compiler.used more supported, since the GCC/Clang/(Rust?) behavior has become more popular.

llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h

 //==- llvm/CodeGen/TargetLoweringObjectFileImpl.h - Object Info --*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements classes used to handle lowerings specific to common
 // object file formats.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CODEGEN_TARGETLOWERINGOBJECTFILEIMPL_H
 #define LLVM_CODEGEN_TARGETLOWERINGOBJECTFILEIMPL_H
 
+#include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/BinaryFormat/XCOFF.h"
 #include "llvm/Target/TargetLoweringObjectFile.h"
 
 namespace llvm {
 
 class GlobalValue;
 class MachineModuleInfo;
 class MachineFunction;
 class MCContext;
 class MCExpr;
 class MCSection;
 class MCSymbol;
 class Module;
 class TargetMachine;
 
 class TargetLoweringObjectFileELF : public TargetLoweringObjectFile {
   bool UseInitArray = false;
   mutable unsigned NextUniqueID = 1;  // ID 0 is reserved for execute-only sections
+  SmallPtrSet<GlobalObject *, 2> Used;
 
 protected:
   MCSymbolRefExpr::VariantKind PLTRelativeVariantKind =
       MCSymbolRefExpr::VK_None;
 
 public:
   TargetLoweringObjectFileELF();
   ~TargetLoweringObjectFileELF() override = default;
 
   void Initialize(MCContext &Ctx, const TargetMachine &TM) override;
 
+  void getModuleMetadata(Module &M) override;
+
   /// Emit Obj-C garbage collection and linker options.
   void emitModuleMetadata(MCStreamer &Streamer, Module &M) const override;
 
   void emitPersonalityValue(MCStreamer &Streamer, const DataLayout &DL,
                             const MCSymbol *Sym) const override;
 
   /// Given a constant with the SectionKind, return a section that it should be
   /// placed in.

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

if (isPositionIndependent()) {
       TTypeEncoding = dwarf::DW_EH_PE_absptr;
     }
     break;
   default:
     break;
   }
 }
 
+void TargetLoweringObjectFileELF::getModuleMetadata(Module &M) {
+  SmallVector<GlobalValue *, 4> Vec;
+  collectUsedGlobalVariables(M, Vec, false);
+  for (GlobalValue *GV : Vec)
+    if (auto *GO = dyn_cast<GlobalObject>(GV))
+      Used.insert(GO);
+}
+
 void TargetLoweringObjectFileELF::emitModuleMetadata(MCStreamer &Streamer,
                                                      Module &M) const {
   auto &C = getContext();
 
   if (NamedMDNode *LinkerOptions = M.getNamedMetadata("llvm.linker.options")) {
     auto *S = C.getELFSection(".linker-options", ELF::SHT_LLVM_LINKER_OPTIONS,
                               ELF::SHF_EXCLUDE);
 
MCSection *TargetLoweringObjectFileELF::getExplicitSectionGlobal(
   }
 
   unsigned EntrySize = getEntrySizeForKind(Kind);
 
   // A section can have at most one associated section. Put each global with
   // MD_associated in a unique section.
   unsigned UniqueID = MCContext::GenericSectionID;
   const MCSymbolELF *LinkedToSym = getLinkedToSymbol(GO, TM);
-  if (GO->getMetadata(LLVMContext::MD_associated)) {
+  const bool Associated = GO->getMetadata(LLVMContext::MD_associated);
+  const bool Retain = Used.count(GO);
+  if (Associated || Retain) {
     UniqueID = NextUniqueID++;
+    if (Associated)
       Flags |= ELF::SHF_LINK_ORDER;
+    if (Retain && (getContext().getAsmInfo()->useIntegratedAssembler() ||
+                   getContext().getAsmInfo()->binutilsIsAtLeast(2, 36)))
+      Flags |= ELF::SHF_GNU_RETAIN;
   } else {
     if (getContext().getAsmInfo()->useIntegratedAssembler() ||
         getContext().getAsmInfo()->binutilsIsAtLeast(2, 35)) {
       // Symbols must be placed into sections with compatible entry
       // sizes. Generate unique sections for symbols that have not
       // been assigned to compatible sections.
       if (Flags & ELF::SHF_MERGE) {
         auto maybeID = getContext().getELFUniqueIDForEntsize(SectionName, Flags,
if (Kind.isExecuteOnly())
     UniqueID = 0;
   return Ctx.getELFSection(Name, getELFSectionType(Name, Kind), Flags,
                            EntrySize, Group, IsComdat, UniqueID,
                            AssociatedSymbol);
 }
 
 static MCSection *selectELFSectionForGlobal(
     MCContext &Ctx, const GlobalObject *GO, SectionKind Kind, Mangler &Mang,
-    const TargetMachine &TM, bool EmitUniqueSection,  unsigned Flags,
-    unsigned *NextUniqueID) {
+    const TargetMachine &TM, bool Retain, bool EmitUniqueSection,
+    unsigned Flags, unsigned *NextUniqueID) {
   const MCSymbolELF *LinkedToSym = getLinkedToSymbol(GO, TM);
   if (LinkedToSym) {
     EmitUniqueSection = true;
     Flags |= ELF::SHF_LINK_ORDER;
   }
+  if (Retain && (Ctx.getAsmInfo()->useIntegratedAssembler() ||
+                 Ctx.getAsmInfo()->binutilsIsAtLeast(2, 36))) {
+    EmitUniqueSection = true;
+    Flags |= ELF::SHF_GNU_RETAIN;
+  }
 
   MCSectionELF *Section = selectELFSectionForGlobal(
       Ctx, GO, Kind, Mang, TM, EmitUniqueSection, Flags,
       NextUniqueID, LinkedToSym);
   assert(Section->getLinkedToSymbol() == LinkedToSym);
   return Section;
 }
 
 MCSection *TargetLoweringObjectFileELF::SelectSectionForGlobal(
     const GlobalObject *GO, SectionKind Kind, const TargetMachine &TM) const {
   unsigned Flags = getELFSectionFlags(Kind);
 
   // If we have -ffunction-section or -fdata-section then we should emit the
   // global value to a uniqued section specifically for it.
   bool EmitUniqueSection = false;
   if (!(Flags & ELF::SHF_MERGE) && !Kind.isCommon()) {
     if (Kind.isText())
       EmitUniqueSection = TM.getFunctionSections();
     else
       EmitUniqueSection = TM.getDataSections();
   }
   EmitUniqueSection |= GO->hasComdat();
   return selectELFSectionForGlobal(getContext(), GO, Kind, getMangler(), TM,
-                                   EmitUniqueSection, Flags, &NextUniqueID);
+                                   Used.count(GO), EmitUniqueSection, Flags,
+                                   &NextUniqueID);
 }
 
 MCSection *TargetLoweringObjectFileELF::getUniqueSectionForFunction(
     const Function &F, const TargetMachine &TM) const {
   SectionKind Kind = SectionKind::getText();
   unsigned Flags = getELFSectionFlags(Kind);
-  return selectELFSectionForGlobal(getContext(), &F, Kind, getMangler(), TM,
-                                   /* EmitUniqueSection = */ true, Flags,
-                                   &NextUniqueID);
+  return selectELFSectionForGlobal(
+      getContext(), &F, Kind, getMangler(), TM, Used.count(&F),
+      /*EmitUniqueSection=*/true, Flags, &NextUniqueID);
 }
 
 MCSection *TargetLoweringObjectFileELF::getSectionForJumpTable(
     const Function &F, const TargetMachine &TM) const {
   // If the function can be removed, produce a unique section so that
   // the table doesn't prevent the removal.
   const Comdat *C = F.getComdat();
   bool EmitUniqueSection = TM.getFunctionSections() || C;

llvm/lib/Target/RISCV/RISCVTargetObjectFile.cpp

MCSection *RISCVELFTargetObjectFile::SelectSectionForGlobal(
   if (Kind.isData() && isGlobalInSmallSection(GO, TM))
     return SmallDataSection;
 
   // Otherwise, we work the same as ELF.
   return TargetLoweringObjectFileELF::SelectSectionForGlobal(GO, Kind, TM);
 }
 
 void RISCVELFTargetObjectFile::getModuleMetadata(Module &M) {
+  TargetLoweringObjectFileELF::getModuleMetadata(M);
   SmallVector<Module::ModuleFlagEntry, 8> ModuleFlags;
   M.getModuleFlagsMetadata(ModuleFlags);
 
   for (const auto &MFE : ModuleFlags) {
     StringRef Key = MFE.Key->getString();
     if (Key == "SmallDataLimit") {
       SSThreshold = mdconst::extract<ConstantInt>(MFE.Val)->getZExtValue();
       break;

llvm/test/CodeGen/PowerPC/func-addr-consts.ll

 ; RUN: llc < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64--linux"
 
 @g = internal constant i8* bitcast (void ()* @f to i8*), section "gsection", align 8
 @h = constant i8* bitcast (void ()* @f to i8*), section "hsection", align 8
 @llvm.used = appending global [2 x i8*] [i8* bitcast (i8** @g to i8*), i8* bitcast (i8** @h to i8*)], section "llvm.metadata"
 
 ; Function Attrs: nounwind uwtable
 define internal void @f() {
 entry:
   ret void
 }
 
-; CHECK: .section gsection,"aw",@progbits
-; CHECK: .section hsection,"aw",@progbits
+; CHECK: .section gsection,"awR",@progbits
+; CHECK: .section hsection,"awR",@progbits

llvm/test/CodeGen/PowerPC/no-dead-strip.ll

 ; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-linux-gnu < %s | FileCheck %s
 
-; CHECK: .section  .bss,"aw",@nobits
+; CHECK: .section  .bss.X,"awR",@nobits
 ; CHECK: .weak X
 ; CHECK-LABEL: X:
 ; CHECK: .long 0
 ; CHECK: .size X, 4
 
 @X = weak global i32 0          ; <i32*> [#uses=1]
 @.str = internal constant [4 x i8] c"t.c\00", section "llvm.metadata"          ; <[4 x i8]*> [#uses=1]
 @llvm.used = appending global [1 x i8*] [ i8* bitcast (i32* @X to i8*) ], section "llvm.metadata"       ; <[1 x i8*]*> [#uses=0]
 

llvm/test/CodeGen/X86/elf-retain.ll

+;; Place a global object in the llvm.used list in a unique section with the SHF_GNU_RETAIN flag.
+; RUN: llc -mtriple=x86_64 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64 -data-sections=1 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64 -no-integrated-as -binutils-version=2.36 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64 -no-integrated-as -binutils-version=2.35 < %s | FileCheck %s --check-prefix=OLDGAS
+
+; RUN: llc -mtriple=x86_64 -data-sections=1 -unique-section-names=0 < %s | FileCheck %s --check-prefix=NOUNIQUE
+
[e-mail] = appending global [10 x i8*] [
+  i8* bitcast (void ()* @fa to i8*), i8* bitcast (void ()* @fb to i8*), i8* bitcast (void ()* @fc to i8*),
+  i8* bitcast (i32* @ga to i8*), i8* bitcast (i32* @gb to i8*), i8* bitcast (i32* @gc to i8*), i8* bitcast (i32* @gd to i8*), i8* bitcast (i32* @ge to i8*),
+  i8* bitcast (i32* @aa to i8*), i8* bitcast (i32* @ab to i8*) ], section "llvm.metadata"
+
+; CHECK:    .section .text.fa,"axR",@progbits{{$}}
+; OLDGAS-NOT: .section .text
+; NOUNIQUE: .section .text,"axR",@progbits,unique,1
+define dso_local void @fa() {
+entry:
+  ret void
+}
+
+; CHECK:    .section .text.fb,"axR",@progbits{{$}}
+; NOUNIQUE: .section .text,"axR",@progbits,unique,2
+define internal void @fb() {
+entry:
+  ret void
+}
+
+;; Explicit section.
+; CHECK:    .section ccc,"axR",@progbits,unique,1
+; OLDGAS:   .section ccc,"ax",@progbits,unique,1
+; NOUNIQUE: .section ccc,"axR",@progbits,unique,3
+define dso_local void @fc() section "ccc" {
+entry:
+  ret void
+}
+
+; CHECK:    .section .bss.ga,"awR",@nobits{{$}}
+; OLDGAS:   .bss{{$}}
+; NOUNIQUE: .section .bss,"awR",@nobits,unique,4
+@ga = global i32 0
+
+; CHECK:    .section .data.gb,"awR",@progbits{{$}}
+; OLDGAS:   .data{{$}}
+; NOUNIQUE: .section .data,"awR",@progbits,unique,5
+@gb = internal global i32 2
+
+; CHECK:    .section .rodata.gc,"aR",@progbits{{$}}
+; OLDGAS:   .section .rodata,"a",@progbits{{$}}
+; NOUNIQUE: .section .rodata,"aR",@progbits,unique,6
+@gc = constant i32 3
+
+;; Explicit section.
+; CHECK:    .section ddd,"awR",@progbits,unique,2
+; OLDGAS:   .section ddd,"aw",@progbits,unique,2
+; NOUNIQUE: .section ddd,"awR",@progbits,unique,7
+@gd = global i32 1, section "ddd"
+
+;; Used together with !associated.
+; CHECK:    .section .data.ge,"awoR",@progbits,gc
+; OLDGAS:   .section .data.ge,"awo",@progbits,gc
+; NOUNIQUE: .section .data,"awoR",@progbits,gc,unique,8
+@ge = global i32 1, !associated !0
+
+;; Aliases in llvm.used are ignored.
+; CHECK:    .section fff,"aw",@progbits{{$}}
+; OLDGAS:   .section fff,"aw",@progbits{{$}}
+; NOUNIQUE: .section fff,"aw",@progbits{{$}}
+@gf = global i32 1, section "fff"
+
+@aa = alias i32, i32* @gf
+@ab = internal alias i32, i32* @gf
+
+!0 = !{i32* @gc}