[PowerPC] Add .ref in backend for AIX XCOFF to support `-bcdtors:csect` linker option

[tingwang]

Summary

On AIX -bcdtors:csect linker option requires ".ref" pseudo-op to indicate the connection among static variables (static global variable, static class member etc.) and static init/term functions, so that linker can generate the correct output.

Here in IR we borrow ‘associated’ metadata to indicate the relationship for ".ref". There are three types of association need be addressed:
(1) variable to init/term functions (required to add functions to _cdtors array if the variable is included in linker output)
(2) dtor function to term function (required to correctly handle atexit/unatexit registration/unregistration)
(3) program code csect to internal variable classified as Data or ThreadData

This is the first half change to support -bcdtors:csect linker option, it adds ".ref" in case (3), and emit ".ref" in assembly output. The other two items will be handled in clang.

[shchenz]

llvm/include/llvm/MC/MCStreamer.h

625 ↗

(On Diff #417140)

Can we only use one parameter const MCSymbol *Symbol for this function? And update other callers accordingly. These two parameters are kind of duplicate.

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2335 ↗

(On Diff #417140)

However .ref does not support BS mapping class.

This seems weird. Should be "csect with BS storage mapping class can not contain .ref"?

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2532

can we change the GVkind here for static globals with associated metadata? We use BSS kind for normal static globals, maybe we can change it to RW, so that we may don't need to change MCSectionXCOFF::printSwitchToSection and TargetLoweringObjectFileXCOFF::SelectSectionForGlobal?

2558

nit: !GV->getMetadata(LLVMContext::MD_associated) seems more common

[shchenz]

llvm/lib/MC/MCAsmStreamer.cpp

950

Maybe we can add an assert here that we are not adding .ref in a section with BS/UC storage mapping class.

[tingwang]

Update based on Chen Zheng's comments:
(1) Corrected emitXCOFFRefDirective argument, the original one should be fine.
(2) Add assert check to guard against BS and UC mapping class.
(3) Updated code comment and code style.

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2532

My initial thought was the same. After looking into TargetLoweringObjectFile::getKindForGlobal(), I didn't find any point for platform specific logic to change the decision, and thus I turned into our platform dependent code to make changes here. (If there is already one hook, I will be happy to use that. However at this moment, I don't see any for this purpose.)

[shchenz]

llvm/lib/MC/MCAsmStreamer.cpp

949

can we use getCurrentSectionOnly()?

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2532

Is it possible to custom the GVKind in current function? This function is AIX specific?

[tingwang]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2532

If I'm understanding correct, this is customize in the middle (my original approach is customize in the end). The trace log shows that getKindForGlobal gets called from different places, for example: GlobalMerge.cpp:::GlobalMerge::doInitialization,
PPCAsmPrinter.cpp:::PPCAIXAsmPrinter::doInitialization, TargetLoweringObjectFile.cpp:::TargetLoweringObjectFile::SectionForGlobal, TargetLoweringObjectFileImpl.cpp:::TargetLoweringObjectFileXCOFF::getTargetSymbol. Customizing in the middle means we have to cover all above code path for correct behavior, and we also have to take care of any more upcoming code changes regarding this API usage. Probably the original approach is simpler.

[tingwang]

Add assert to guard MCSection

[tingwang]

During test some case revealed another issue regarding BSSLocal: there maybe some function associated with variable which has storage class BSSLocal (zero initialized). Unfortunately the variable is the association target, and there is no associated metadata attached to the variable. I have to work out a solution for this, incorporating Zheng's comments here. I will update code once the solution is formed.

[tingwang]

Use opaque-pointer in testcase aix-xcoff-ref.ll

[tingwang]

Rebase && drop TLS related .ref for now

[tingwang]

Rebase.

[stephenpeckham]

I'm not familiar with a lot of this code, but I don't see any obvious errors.

[tingwang]

In D122198#4185848, @stephenpeckham wrote:

I'm not familiar with a lot of this code, but I don't see any obvious errors.

Thank you for help!

[tingwang]

Rebase.

llvm/lib/MC/MCAsmStreamer.cpp

 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstPrinter.h"
 #include "llvm/MC/MCObjectFileInfo.h"
 #include "llvm/MC/MCObjectWriter.h"
 #include "llvm/MC/MCPseudoProbe.h"
 #include "llvm/MC/MCRegister.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSectionMachO.h"
+#include "llvm/MC/MCSectionXCOFF.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSymbolXCOFF.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/LEB128.h"
if (C == DQ)
       OS << DQ;
     OS << C;
   }
   OS << DQ;
   EmitEOL();
 }
 
 void MCAsmStreamer::emitXCOFFRefDirective(const MCSymbol *Symbol) {
+  assert([&]() -> bool {
+    MCSectionXCOFF *CS = cast<MCSectionXCOFF>(getCurrentSectionOnly());
+    return CS && CS->getCSectType() == XCOFF::XTY_SD &&
+           CS->getMappingClass() != XCOFF::XMC_BS &&
+           CS->getMappingClass() != XCOFF::XMC_UC;
+  }() && "The .ref pseudo-op cannot be included in a csect with a storage "
+         "mapping class of BS or UC");
+  assert(
+      !(Symbol->getName().endswith_insensitive("[bs]") ||
+        Symbol->getName().endswith_insensitive("[uc]")) &&
+      ".ref operand cannot be a name with storage mapping class of BS or UC");
   OS << "\t.ref ";
   Symbol->print(OS, MAI);
   EmitEOL();
 }
 
 void MCAsmStreamer::emitXCOFFExceptDirective(const MCSymbol *Symbol,
                                              const MCSymbol *Trap,
                                              unsigned Lang,

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

protected:
   // entry as a TLSGD entry so we can add the @m relocation:
   //   .tc .i[TC],i[TL]@m
   // By default, VK_None is used for the VariantKind.
   MapVector<std::pair<const MCSymbol *, MCSymbolRefExpr::VariantKind>,
             MCSymbol *>
       TOC;
   const PPCSubtarget *Subtarget = nullptr;
 
+  virtual void trackTOCRef(const MCSymbol *Sym) {}
+
 public:
   explicit PPCAsmPrinter(TargetMachine &TM,
                          std::unique_ptr<MCStreamer> Streamer)
       : AsmPrinter(TM, std::move(Streamer)) {}
 
   StringRef getPassName() const override { return "PowerPC Assembly Printer"; }
 
   enum TOCEntryType {
private:
   /// sinit/sterm function names.
   std::string FormatIndicatorAndUniqueModId;
 
   // Record a list of GlobalAlias associated with a GlobalObject.
   // This is used for AIX's extra-label-at-definition aliasing strategy.
   DenseMap<const GlobalObject *, SmallVector<const GlobalAlias *, 1>>
       GOAliasMap;
 
+  // To support `-bcdtors:csect` on AIX, clang handles .ref regarding variable
+  // and corresponding init/term functions. In addition, we need to generate
+  // .ref from program code to internal global variables for safety reason
+  // consider following case: one internal global variable with init function
+  // which produces side effect, but that variable is not referenced by any
+  // function; without .ref from program code to the variable, the side effect
+  // could be missed after link. We put all target internal GV in
+  // InternalGlobalRW, keep track of function touched GV in FuncBodyTouchedTOC,
+  // and finally generate .ref for those not touched.
+  SmallVector<const MCSymbolXCOFF *, 8> InternalGlobalRW;
+  SmallSet<const MCSymbolXCOFF *, 8> FuncBodyTouchedTOC;
+  bool IsTextEmpty = true;
+  bool IsFuncBodyTOC = false;
+
   uint16_t getNumberOfVRSaved();
   void emitTracebackTable();
 
   SmallVector<const GlobalVariable *, 8> TOCDataGlobalVars;
 
   void emitGlobalVariableHelper(const GlobalVariable *);
 
   // Get the offset of an alias based on its AliaseeObject.
   uint64_t getAliasOffset(const Constant *C);
 
+  void emitAssociatedRefs(const GlobalObject &);
+
+  void trackTOCRef(const MCSymbol *Sym) override {
+    if (IsFuncBodyTOC)
+      if (const auto *S = dyn_cast<const MCSymbolXCOFF>(Sym))
+        FuncBodyTouchedTOC.insert(S);
+  }
+
 public:
   PPCAIXAsmPrinter(TargetMachine &TM, std::unique_ptr<MCStreamer> Streamer)
       : PPCAsmPrinter(TM, std::move(Streamer)) {
     if (MAI->isLittleEndian())
       report_fatal_error(
           "cannot create AIX PPC Assembly Printer for a little-endian target");
   }
 
   StringRef getPassName() const override { return "AIX PPC Assembly Printer"; }
 
   bool doInitialization(Module &M) override;
 
   void emitXXStructorList(const DataLayout &DL, const Constant *List,
                           bool IsCtor) override;
 
   void SetupMachineFunction(MachineFunction &MF) override;
 
   void emitGlobalVariable(const GlobalVariable *GV) override;
 
   void emitFunctionDescriptor() override;
 
   void emitFunctionEntryLabel() override;
 
+  void emitFunctionBodyStart() override { IsFuncBodyTOC = true; }
+
   void emitFunctionBodyEnd() override;
 
   void emitPGORefs();
 
   void emitEndOfAsmFile(Module &) override;
 
   void emitLinkage(const GlobalValue *GV, MCSymbol *GVSym) const override;
 

 /// the TOC entry.
 MCSymbol *
 PPCAsmPrinter::lookUpOrCreateTOCEntry(const MCSymbol *Sym, TOCEntryType Type,
                                       MCSymbolRefExpr::VariantKind Kind) {
   // If this is a new TOC entry add statistics about it.
   if (!TOC.contains({Sym, Kind}))
     collectTOCStats(Type);
 
+  trackTOCRef(Sym);
   MCSymbol *&TOCEntry = TOC[{Sym, Kind}];
   if (!TOCEntry)
     TOCEntry = createTempSymbol("C");
   return TOCEntry;
 }
 
 void PPCAsmPrinter::LowerSTACKMAP(StackMaps &SM, const MachineInstr &MI) {
   unsigned NumNOPBytes = MI.getOperand(1).getImm();
for (unsigned Reg = PPC::V20; Reg <= PPC::V31; ++Reg)
       if (MRI.isPhysRegModified(Reg))
         // Number of VRs saved.
         return PPC::V31 - Reg + 1;
   }
   return 0;
 }
 
 void PPCAIXAsmPrinter::emitFunctionBodyEnd() {
+  IsTextEmpty = false;
+  if (TM.getFunctionSections()) {
+    for (auto I = InternalGlobalRW.begin(), E = InternalGlobalRW.end(); I != E;
+         ++I)
+      if (!FuncBodyTouchedTOC.contains(*I))
+        OutStreamer->emitXCOFFRefDirective((*I));
+    FuncBodyTouchedTOC.clear();
+  }
+  IsFuncBodyTOC = false;
 
   if (!TM.getXCOFFTracebackTable())
     return;
 
   emitTracebackTable();
 
   // If ShouldEmitEHBlock returns true, then the eh info table
   // will be emitted via `AIXException::endFunction`. Otherwise, we
void PPCAIXAsmPrinter::emitGlobalVariableHelper(const GlobalVariable *GV) {
 
   MCSymbolXCOFF *GVSym = cast<MCSymbolXCOFF>(getSymbol(GV));
 
   if (GV->isDeclarationForLinker()) {
     emitLinkage(GV, GVSym);
     return;
   }
 
   SectionKind GVKind = getObjFileLowering().getKindForGlobal(GV, TM);
   if (!GVKind.isGlobalWriteableData() && !GVKind.isReadOnly() &&
       !GVKind.isThreadLocal()) // Checks for both ThreadData and ThreadBSS.
     report_fatal_error("Encountered a global variable kind that is "
                        "not supported yet.");
 
   // Print GV in verbose mode
   if (isVerbose()) {
     if (GV->hasInitializer()) {
void PPCAIXAsmPrinter::emitGlobalVariableHelper(const GlobalVariable *GV) {
   // Switch to the containing csect.
   OutStreamer->switchSection(Csect);
 
   const DataLayout &DL = GV->getParent()->getDataLayout();
 
   // Handle common and zero-initialized local symbols.
   if (GV->hasCommonLinkage() || GVKind.isBSSLocal() ||
       GVKind.isThreadBSSLocal()) {
     Align Alignment = GV->getAlign().value_or(DL.getPreferredAlign(GV));
     uint64_t Size = DL.getTypeAllocSize(GV->getValueType());
     GVSym->setStorageClass(
         TargetLoweringObjectFileXCOFF::getStorageClassForGlobal(GV));
 
     if (GVKind.isBSSLocal() || GVKind.isThreadBSSLocal())
       OutStreamer->emitXCOFFLocalCommonSymbol(
           OutContext.getOrCreateSymbol(GVSym->getSymbolTableName()), Size,
           GVSym, Alignment);
     else
       OutStreamer->emitCommonSymbol(GVSym, Size, Alignment);
     return;
   }
 
   MCSymbol *EmittedInitSym = GVSym;
 
+  emitAssociatedRefs(*GV);
+
   // Emit linkage for the global variable and its aliases.
   emitLinkage(GV, EmittedInitSym);
   for (const GlobalAlias *GA : GOAliasMap[GV])
     emitLinkage(GA, getSymbol(GA));
 
   emitAlignment(getGVAlignment(GV, DL), GV);
 
   // When -fdata-sections is enabled, every GlobalVariable will
void PPCAIXAsmPrinter::emitGlobalVariableHelper(const GlobalVariable *GV) {
   for (const GlobalAlias *GA : GOAliasMap[GV])
     AliasList[getAliasOffset(GA->getAliasee())].push_back(GA);
 
   // Emit alias label and element value for global variable.
   emitGlobalConstant(GV->getParent()->getDataLayout(), GV->getInitializer(),
                      &AliasList);
 }
 
+void PPCAIXAsmPrinter::emitAssociatedRefs(const GlobalObject &GO) {
+  MDNode *MD = GO.getMetadata(LLVMContext::MD_associated);
+  if (!MD)
+    return;
+
+  for (unsigned i = 0, e = MD->getNumOperands(); i != e; ++i) {
+    const MDOperand &Op = MD->getOperand(i);
+    if (!Op.get())
+      continue;
+
+    const auto *VM = dyn_cast<ValueAsMetadata>(Op);
+    if (!VM)
+      report_fatal_error("MD_associated operand is not ValueAsMetadata");
+
+    if (const auto *OtherGV = dyn_cast<GlobalValue>(VM->getValue()))
+      if (OtherGV != &GO)
+        OutStreamer->emitXCOFFRefDirective(getSymbol(OtherGV));
+  }
+}
+
 void PPCAIXAsmPrinter::emitFunctionDescriptor() {
   const DataLayout &DL = getDataLayout();
   const unsigned PointerSize = DL.getPointerSizeInBits() == 64 ? 8 : 4;
 
   MCSectionSubPair Current = OutStreamer->getCurrentSection();
   // Emit function descriptor.
   OutStreamer->switchSection(
       cast<MCSymbolXCOFF>(CurrentFnDescSym)->getRepresentedCsect());
 
   // Emit aliasing label for function descriptor csect.
   for (const GlobalAlias *Alias : GOAliasMap[&MF->getFunction()])
     OutStreamer->emitLabel(getSymbol(Alias));
 
+  // Emit .ref from dtor to term function
+  emitAssociatedRefs(MF->getFunction());
+
   // Emit function entry point address.
   OutStreamer->emitValue(MCSymbolRefExpr::create(CurrentFnSym, OutContext),
                          PointerSize);
   // Emit TOC base address.
   const MCSymbol *TOCBaseSym =
       cast<MCSectionXCOFF>(getObjFileLowering().getTOCBaseSection())
           ->getQualNameSymbol();
   OutStreamer->emitValue(MCSymbolRefExpr::create(TOCBaseSym, OutContext),
bool PPCAIXAsmPrinter::doInitialization(Module &M) {
   for (const auto &Alias : M.aliases()) {
     const GlobalObject *Base = Alias.getAliaseeObject();
     if (!Base)
       report_fatal_error(
           "alias without a base object is not yet supported on AIX");
     GOAliasMap[Base].push_back(&Alias);
   }
 
+  // Collect all LGLOBAL located inside {RW} mapping class for .ref
+  for (const auto &G : M.globals())
+    if (G.getLinkage() == GlobalValue::InternalLinkage ||
+        G.getLinkage() == GlobalValue::PrivateLinkage) {
+      SectionKind GVKind = getObjFileLowering().getKindForGlobal(&G, TM);
+      if (GVKind.isData() || GVKind.isReadOnlyWithRel())
+        InternalGlobalRW.push_back(cast<const MCSymbolXCOFF>(getSymbol(&G)));
+    }
+
   return Result;
 }
 
 void PPCAIXAsmPrinter::emitInstruction(const MachineInstr *MI) {
   switch (MI->getOpcode()) {
   default:
     break;
   case PPC::TW:

 bool PPCAIXAsmPrinter::doFinalization(Module &M) {
   // Do streamer related finalization for DWARF.
   if (!MAI->usesDwarfFileAndLocDirectives() && MMI->hasDebugInfo())
     OutStreamer->doFinalizationAtSectionEnd(
         OutStreamer->getContext().getObjectFileInfo()->getTextSection());
 
   for (MCSymbol *Sym : ExtSymSDNodeSymbols)
     OutStreamer->emitSymbolAttribute(Sym, MCSA_Extern);
+
+  // Do not generate .ref for empty .text{PR}
+  if (!TM.getFunctionSections() && !IsTextEmpty) {
+    for (auto I = InternalGlobalRW.begin(), E = InternalGlobalRW.end(); I != E;
+         ++I)
+      if (!FuncBodyTouchedTOC.contains(*I))
+        OutStreamer->emitXCOFFRefDirective((*I));
+  }
+  InternalGlobalRW.clear();
+  FuncBodyTouchedTOC.clear();
+
   return PPCAsmPrinter::doFinalization(M);
 }
 
 static unsigned mapToSinitPriority(int P) {
   if (P < 0 || P > 65535)
     report_fatal_error("invalid init priority");
 
   if (P <= 20)

llvm/lib/Target/TargetLoweringObjectFile.cpp

SectionKind TargetLoweringObjectFile::getKindForGlobal(const GlobalObject *GO,
 
   // Variables with common linkage always get classified as common.
   if (GVar->hasCommonLinkage())
     return SectionKind::getCommon();
 
   // Most non-mergeable zero data can be put in the BSS section unless otherwise
   // specified.
   if (isSuitableForBSS(GVar) && !TM.Options.NoZerosInBSS) {
-    if (GVar->hasLocalLinkage())
+    if (GVar->hasLocalLinkage()) {
+      // AIX XCOFF with associated metadata is part of .ref pseudo-op, should be
+      // classified as Data.
+      if (TM.getTargetTriple().isOSAIX() &&
+          GVar->getMetadata(LLVMContext::MD_associated))
+        return SectionKind::getData();
       return SectionKind::getBSSLocal();
-    else if (GVar->hasExternalLinkage())
+    } else if (GVar->hasExternalLinkage())
       return SectionKind::getBSSExtern();
     return SectionKind::getBSS();
   }
 
   // Global variables with '!exclude' should get the exclude section kind if
   // they have an explicit section and no other metadata.
   if (GVar->hasSection())
     if (MDNode *MD = GVar->getMetadata(LLVMContext::MD_exclude))

llvm/test/CodeGen/PowerPC/aix-xcoff-ref-empty-text.ll

+; RUN: llc -mtriple powerpc-ibm-aix-xcoff -mcpu=pwr9 --filetype=asm < %s | FileCheck %s --check-prefix=CHECK
+
+@a = internal global i32 3, align 4
+@b = internal global i32 6, align 4
+@c = internal global i32 9, align 4
+
+; CHECK:      .csect .text[PR],5
+; CHECK-NOT:  .ref

llvm/test/CodeGen/PowerPC/aix-xcoff-ref.ll

+; RUN: llc -mtriple powerpc-ibm-aix-xcoff -mcpu=pwr9 --function-sections --filetype=asm < %s | FileCheck %s --check-prefixes=CHECK-FS,CHECK
+; RUN: llc -mtriple powerpc-ibm-aix-xcoff -mcpu=pwr9 --filetype=asm < %s | FileCheck %s --check-prefixes=CHECK-NFS,CHECK
+
+@a = internal global i32 0, align 4, !associated !0
+@b = internal global i32 3, align 4
+@c = internal global i32 0, align 4
+@d = internal global i32 3, align 4, !associated !2
+
+define i32 @foo() {
+entry:
+  %0 = load i32, ptr @a, align 4
+  ret i32 %0
+}
+
+define void @bar() {
+entry:
+  ret void
+}
+
+define void @baz() !associated !1 {
+entry:
+  ret void
+}
+
+!0 = !{i32 ()* @foo, void ()* @bar}
+!1 = !{void ()* @bar}
+!2 = distinct !{null}
+
+; CHECK-FS:         .csect .foo[PR],5
+; CHECK-FS:         .csect .foo[PR],5
+; CHECK-FS-NEXT:    # %bb.0:                                # %entry
+; CHECK-FS-NEXT:    lwz 3, L..C0(2)                         # @a
+; CHECK-FS-NEXT:    lwz 3, 0(3)
+; CHECK-FS-NEXT:    blr
+; CHECK-FS-NEXT:    .ref b[RW]
+; CHECK-FS-NEXT:    .ref d[RW]
+
+; CHECK-FS:         .csect .bar[PR],5
+; CHECK-FS:         .csect .bar[PR],5
+; CHECK-FS-NEXT:    # %bb.0:                                # %entry
+; CHECK-FS-NEXT:    blr
+; CHECK-FS-NEXT:    .ref a[RW]
+; CHECK-FS-NEXT:    .ref b[RW]
+; CHECK-FS-NEXT:    .ref d[RW]
+
+; CHECK-FS:         .csect baz[DS],2
+; CHECK-FS-NEXT:    .ref bar[DS]                            # @baz
+; CHECK-FS:         .csect .baz[PR],5
+; CHECK-FS-NEXT:    # %bb.0:                                # %entry
+; CHECK-FS-NEXT:    blr
+; CHECK-FS-NEXT:    .ref a[RW]
+; CHECK-FS-NEXT:    .ref b[RW]
+; CHECK-FS-NEXT:    .ref d[RW]
+
+; CHECK-NFS:        .csect baz[DS],2
+; CHECK-NFS-NEXT:   .ref bar[DS]                            # @baz
+
+; The whole .text{PR} has one .ref points to b[RW] and d[RW]
+; CHECK-NFS:        .byte   "baz"                           # Function Name
+; CHECK-NFS-NEXT:                                           # -- End function
+; CHECK-NFS-NEXT:   .ref b[RW]
+; CHECK-NFS-NEXT:   .ref d[RW]
+
+; CHECK:            .csect a[RW],2
+; CHECK-NEXT:       .ref foo[DS]                            # @a
+; CHECK-NEXT:       .ref bar[DS]
+
+; CHECK:            .lcomm  c,4,c[BS],2                     # @c
+
+; CHECK:           .csect d[RW],2
+; CHECK:           .lglobl d[RW]                           # @d