[AIX][TLS] Generate TLS variables in assembly files

[NeHuang]

Summary

This patch allows generating TLS variables in assembly files on AIX.
Initialized and external uninitialized variables are generated with the
.csect pseudo-op and local uninitialized variables are generated with
the .comm/.lcomm pseudo-ops. The patch also adds a check to
explicitly say that TLS is not yet supported on AIX.

[bsaleil]

Add support for variables with internal linkage.

[jasonliu]

LGTM with minor nit.

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2224

nit: reformat as the suggestion.

llvm/lib/MC/MCSectionXCOFF.cpp

41

nit:

[amyk]

llvm/lib/MC/MCSectionXCOFF.cpp

41

Would it be helpful to mention in the comment that the initialized data are generated with the .csect pseudo-op?

[daltenty]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

14 ↗

(On Diff #322203)

nit: technically we emit into separate csects. not XCOFF sections.

15 ↗

(On Diff #322203)

same as above

[daltenty]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

21 ↗

(On Diff #322203)

It's out of scope for this patch, but there is definitely a delta here between how we emit thread-local (to .tbss) and non-thread local uninitialized data (i.e. to regular data section instead of bss) beyond just the thread-local part. Not sure if that's something we should address.

[daltenty]

LGTM, with minor nit

[daltenty]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

21 ↗

(On Diff #322203)

As a follow on, as discussed offline, using common for uninitialized data may have linkage considerations. We should consider if this applies to .tbss data as well before using .comm directives.

[bsaleil]

Generate uninitialized external TLS data with the .csect directive instead of .comm

[sfertile]

LGTM. Please wait until David has given it the OK too though.

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2216

Maybe if you add a comment before : XCOFF::XMC_UL; clang-format will end up keeping a similar layout to what you already have. If not we should follow clang-format even though its inferior in this instance 😦 .

[daltenty]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2105

!GV->hasExternalLinkage() doesn't actually imply the symbol is local, we probably want something more like isLocalLinkage()

We should add some testcases for other non-local linkages like weak.

[NeHuang]

@daltenty As per discussion offline

• Update the code to check isLocalLinkage
• More test coverage for weak linkage and update test case
• clangformat the patch

[daltenty]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

53 ↗

(On Diff #324783)

We're missing the csect directives here

122 ↗

(On Diff #324783)

Was this supposed to be initialized?

[NeHuang]

• Fix the missing directives printing for weak linkage.
• Update the test case accordingly.

[NeHuang]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

122 ↗

(On Diff #324783)

Yes, updated.

[lei]

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

111 ↗

(On Diff #325422)

it would be easier to verify these checks if they were placed right before the declaration of the variable they are checking. Similar to what is done when we write checks for multiple functions.

[daltenty]

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2215

Remove the second sentence here. Where other things get mapped depends on data-sections and other factors that aren't relevant to this block.

2217

I'm thinking this condition is important enough for us to provide a convince function. Something like isEligibleForCommonCsect and use it in the other three places where this exact condition should appear.

llvm/lib/MC/MCObjectFileInfo.cpp

890

We should initialize and use the TLSBSSSection where appropriate.

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

52 ↗

(On Diff #325422)

Should probably be UL, since this is uninitialized

[daltenty]

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2280

We should have another case for Kind.isThreadBSS, where we should probably use either a csect with XMC_UL or TLSBSSSection depending on TM.getDataSections()

[NeHuang]

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2280

Agree, we should add the coverage for non-initialized cases using Kind.isThreadBSS()
Should we use Kind.isThreadData() check instead of Kind.isThreadLocal() for the initialized cases?

llvm/lib/MC/MCSectionXCOFF.cpp

70

We would also need isEligibleForCommonCsect check to be aligned here.

96

We need to cover the other TLS scenario here which is not eligible for commonCsect - to print CsectDirective.

llvm/test/CodeGen/PowerPC/aix-tls-variables.ll

111 ↗

(On Diff #325422)

Sure. Will also alternate the test cases as below
non-tls variable
tls variable
...
...

[NeHuang]

Rebased the patch to be aligned with interface change of function getXCOFFSection

[NeHuang]

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

2105

Update the comments to explain all the scenarios.

2112

check if "GV->hasInternalLinkage()" and "GVKind.isThreadBSS() && GlobalValue::isLocalLinkage(GV->getLinkage())" are equivalent

[daltenty]

LGTM with minor comment and naming fixes

llvm/test/CodeGen/PowerPC/aix-tls-variables-ppc32.ll

24

As discussed offline, we should rename these from TL/UL to something a bit more descriptive

[daltenty]

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

2289

As we recently found out XCOFF::XMC_UL implies XCOFF::XTY_CM in the assembler.

Since we may have external linkage symbols inside this section, we cannot use XCOFF::XTY_CM, so we must use XMC_TL here.

2292

Same as earlier, we cannot emit this to TLSBSSSection. Any external linkage symbols inside it will make the assembler unhappy.

llvm/lib/MC/MCObjectFileInfo.cpp

898

Based on testing XMC_UL implies XMC_CM so this isn't actually a legal combination. We should probably add an assert to that extent in a follow on patch.

Also, we can't actually use this section as is in this patch anymore, so I suggest we leave out the initialization and move it to a follow on patch.

[NeHuang]

llvm/lib/MC/MCSectionXCOFF.cpp

41

here should be initialized TLS data

[NeHuang]

Addressed the review comments to

• Fix the storaging map class for zero-initialized TLS data with external/weak linkage and update the test cases.
• Renaming the test cases with proper description (zero-initialized, val-initialized and uninitialized
• Add more comments for the implementation and correct some minors.

llvm/include/llvm/MC/SectionKind.h

/// to during program runtime.
        /// be thread local!
 
            /// ThreadBSS - Zero-initialized TLS data objects.
            ThreadBSS,
 
            /// ThreadData - Initialized TLS data objects.
            ThreadData,
 
+           /// ThreadBSSLocal - Zero-initialized TLS data objects with local linkage.
+           ThreadBSSLocal,
+
        /// GlobalWriteableData - Writeable data that is global (not thread
        /// local).
 
            /// BSS - Zero initialized writeable data.
            BSS,
 
                /// BSSLocal - This is BSS (zero initialized and writable) data
                /// which has local linkage.
public:
   bool isMergeableConst16() const { return K == MergeableConst16; }
   bool isMergeableConst32() const { return K == MergeableConst32; }
 
   bool isWriteable() const {
     return isThreadLocal() || isGlobalWriteableData();
   }
 
   bool isThreadLocal() const {
-    return K == ThreadData || K == ThreadBSS;
+    return K == ThreadData || K == ThreadBSS || K == ThreadBSSLocal;
   }
 
-  bool isThreadBSS() const { return K == ThreadBSS; }
+  bool isThreadBSS() const { return K == ThreadBSS || K == ThreadBSSLocal; }
   bool isThreadData() const { return K == ThreadData; }
+  bool isThreadBSSLocal() const { return K == ThreadBSSLocal; }
 
   bool isGlobalWriteableData() const {
     return isBSS() || isCommon() || isData() || isReadOnlyWithRel();
   }
 
   bool isBSS() const { return K == BSS || K == BSSLocal || K == BSSExtern; }
   bool isBSSLocal() const { return K == BSSLocal; }
   bool isBSSExtern() const { return K == BSSExtern; }
static SectionKind getMergeable4ByteCString() {
     return get(Mergeable4ByteCString);
   }
   static SectionKind getMergeableConst4() { return get(MergeableConst4); }
   static SectionKind getMergeableConst8() { return get(MergeableConst8); }
   static SectionKind getMergeableConst16() { return get(MergeableConst16); }
   static SectionKind getMergeableConst32() { return get(MergeableConst32); }
   static SectionKind getThreadBSS() { return get(ThreadBSS); }
   static SectionKind getThreadData() { return get(ThreadData); }
+  static SectionKind getThreadBSSLocal() { return get(ThreadBSSLocal); }
   static SectionKind getBSS() { return get(BSS); }
   static SectionKind getBSSLocal() { return get(BSSLocal); }
   static SectionKind getBSSExtern() { return get(BSSExtern); }
   static SectionKind getCommon() { return get(Common); }
   static SectionKind getData() { return get(Data); }
   static SectionKind getReadOnlyWithRel() { return get(ReadOnlyWithRel); }
 };
 
 } // end namespace llvm
 
 #endif

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

if (GO->isDeclarationForLinker())
       return cast<MCSectionXCOFF>(getSectionForExternalReference(GO, TM))
           ->getQualNameSymbol();
 
     SectionKind GOKind = getKindForGlobal(GO, TM);
     if (GOKind.isText())
       return cast<MCSectionXCOFF>(
                  getSectionForFunctionDescriptor(cast<Function>(GO), TM))
           ->getQualNameSymbol();
-    if ((TM.getDataSections() && !GO->hasSection()) || GOKind.isCommon() ||
-        GOKind.isBSSLocal())
+    if ((TM.getDataSections() && !GO->hasSection()) || GO->hasCommonLinkage() ||
+        GOKind.isBSSLocal() || GOKind.isThreadBSSLocal())
       return cast<MCSectionXCOFF>(SectionForGlobal(GO, GOKind, TM))
           ->getQualNameSymbol();
   }
 
   // For all other cases, fall back to getSymbol to return the unqualified name.
   return nullptr;
 }
 

 MCSection *TargetLoweringObjectFileXCOFF::getSectionForExternalReference(
     const GlobalObject *GO, const TargetMachine &TM) const {
   assert(GO->isDeclarationForLinker() &&
          "Tried to get ER section for a defined global.");
 
   SmallString<128> Name;
   getNameWithPrefix(Name, GO, TM);
 
+  XCOFF::StorageMappingClass SMC =
+      isa<Function>(GO) ? XCOFF::XMC_DS : XCOFF::XMC_UA;
+  if (GO->isThreadLocal())
+    SMC = XCOFF::XMC_UL;
+
   // Externals go into a csect of type ER.
   return getContext().getXCOFFSection(
       Name, SectionKind::getMetadata(),
-      XCOFF::CsectProperties(isa<Function>(GO) ? XCOFF::XMC_DS : XCOFF::XMC_UA,
-                             XCOFF::XTY_ER));
+      XCOFF::CsectProperties(SMC, XCOFF::XTY_ER));
 }
 
 MCSection *TargetLoweringObjectFileXCOFF::SelectSectionForGlobal(
     const GlobalObject *GO, SectionKind Kind, const TargetMachine &TM) const {
   // Common symbols go into a csect with matching name which will get mapped
   // into the .bss section.
-  if (Kind.isBSSLocal() || Kind.isCommon()) {
+  // Zero-initialized local TLS symbols go into a csect with matching name which
+  // will get mapped into the .tbss section.
+  if (Kind.isBSSLocal() || GO->hasCommonLinkage() || Kind.isThreadBSSLocal()) {
     SmallString<128> Name;
     getNameWithPrefix(Name, GO, TM);
+    XCOFF::StorageMappingClass SMC = Kind.isBSSLocal() ? XCOFF::XMC_BS
+                                     : Kind.isCommon() ? XCOFF::XMC_RW
+                                                       : XCOFF::XMC_UL;
     return getContext().getXCOFFSection(
-        Name, Kind,
-        XCOFF::CsectProperties(
-            Kind.isBSSLocal() ? XCOFF::XMC_BS : XCOFF::XMC_RW, XCOFF::XTY_CM));
+        Name, Kind, XCOFF::CsectProperties(SMC, XCOFF::XTY_CM));
   }
 
   if (Kind.isMergeableCString()) {
     Align Alignment = GO->getParent()->getDataLayout().getPreferredAlign(
         cast<GlobalVariable>(GO));
 
     unsigned EntrySize = getEntrySizeForKind(Kind);
     std::string SizeSpec = ".rodata.str" + utostr(EntrySize) + ".";
     SmallString<128> Name;
if (TM.getDataSections()) {
       getNameWithPrefix(Name, GO, TM);
       return getContext().getXCOFFSection(
           Name, SectionKind::getReadOnly(),
           XCOFF::CsectProperties(XCOFF::XMC_RO, XCOFF::XTY_SD));
     }
     return ReadOnlySection;
   }
 
+  // External/weak TLS data and initialized local TLS data are not eligible
+  // to be put into common csect. If data sections are enabled, thread
+  // data are emitted into separate sections. Otherwise, thread data
+  // are emitted into the .tdata section.
+  if (Kind.isThreadLocal()) {
+    if (TM.getDataSections()) {
+      SmallString<128> Name;
+      getNameWithPrefix(Name, GO, TM);
+      return getContext().getXCOFFSection(
+          Name, Kind, XCOFF::CsectProperties(XCOFF::XMC_TL, XCOFF::XTY_SD));
+    }
+    return TLSDataSection;
+  }
+
   report_fatal_error("XCOFF other section types not yet implemented.");
 }
 
 MCSection *TargetLoweringObjectFileXCOFF::getSectionForJumpTable(
     const Function &F, const TargetMachine &TM) const {
   assert (!F.getComdat() && "Comdat not supported on XCOFF.");
 
   if (!TM.getFunctionSections())

llvm/lib/MC/MCObjectFileInfo.cpp

DataSection = Ctx->getXCOFFSection(
       ".data", SectionKind::getData(),
       XCOFF::CsectProperties(XCOFF::StorageMappingClass::XMC_RW, XCOFF::XTY_SD),
       /* MultiSymbolsAllowed*/ true);
 
   ReadOnlySection = Ctx->getXCOFFSection(
       ".rodata", SectionKind::getReadOnly(),
       XCOFF::CsectProperties(XCOFF::StorageMappingClass::XMC_RO, XCOFF::XTY_SD),
       /* MultiSymbolsAllowed*/ true);
 
+  TLSDataSection = Ctx->getXCOFFSection(
+      ".tdata", SectionKind::getThreadData(),
+      XCOFF::CsectProperties(XCOFF::StorageMappingClass::XMC_TL, XCOFF::XTY_SD),
+      /* MultiSymbolsAllowed*/ true);
+
   TOCBaseSection = Ctx->getXCOFFSection(
       "TOC", SectionKind::getData(),
       XCOFF::CsectProperties(XCOFF::StorageMappingClass::XMC_TC0,
                              XCOFF::XTY_SD));
 
   // The TOC-base always has 0 size, but 4 byte alignment.
   TOCBaseSection->setAlignment(Align(4));
 
   LSDASection = Ctx->getXCOFFSection(
       ".gcc_except_table", SectionKind::getReadOnly(),
       XCOFF::CsectProperties(XCOFF::StorageMappingClass::XMC_RO,

llvm/lib/MC/MCSectionXCOFF.cpp

void MCSectionXCOFF::PrintSwitchToSection(const MCAsmInfo &MAI, const Triple &T,
 
   if (getKind().isReadOnly()) {
     if (getMappingClass() != XCOFF::XMC_RO)
       report_fatal_error("Unhandled storage-mapping class for .rodata csect.");
     printCsectDirective(OS);
     return;
   }
 
+  // Initialized TLS data.
+  if (getKind().isThreadData()) {
+    // We only expect XMC_TL here for initialized TLS data.
+    if (getMappingClass() != XCOFF::XMC_TL)
+      report_fatal_error("Unhandled storage-mapping class for .tdata csect.");
+    printCsectDirective(OS);
+    return;
+  }
+
   if (getKind().isData()) {
     switch (getMappingClass()) {
     case XCOFF::XMC_RW:
     case XCOFF::XMC_DS:
       printCsectDirective(OS);
       break;
     case XCOFF::XMC_TC:
     case XCOFF::XMC_TE:
       break;
     case XCOFF::XMC_TC0:
       OS << "\t.toc\n";
       break;
     default:
       report_fatal_error(
           "Unhandled storage-mapping class for .data csect.");
     }
     return;
   }
 
-  if (getKind().isBSSLocal() || getKind().isCommon()) {
+  // Common csect type (uninitialized storage) does not have to print csect
+  // directive for section switching.
+  if (getCSectType() == XCOFF::XTY_CM) {
     assert((getMappingClass() == XCOFF::XMC_RW ||
-            getMappingClass() == XCOFF::XMC_BS) &&
-           "Generated a storage-mapping class for a common/bss csect we don't "
+            getMappingClass() == XCOFF::XMC_BS ||
+            getMappingClass() == XCOFF::XMC_UL) &&
+           "Generated a storage-mapping class for a common/bss/tbss csect we "
+           "don't "
            "understand how to switch to.");
-    assert(getCSectType() == XCOFF::XTY_CM &&
-           "wrong csect type for .bss csect");
-    // Don't have to print a directive for switching to section for commons.
-    // '.comm' and '.lcomm' directives for the variable will create the needed
-    // csect.
+    // Common symbols and local zero-initialized symbols for TLS and Non-TLS are
+    // eligible for .bss/.tbss csect, getKind().isThreadBSS() is used to cover
+    // TLS common and zero-initialized local symbols since linkage type (in the
+    // GlobalVariable) is not accessible in this class.
+    assert((getKind().isBSSLocal() || getKind().isCommon() ||
+            getKind().isThreadBSS()) &&
+           "wrong symbol type for .bss/.tbss csect");
+    // Don't have to print a directive for switching to section for commons and
+    // zero-initialized TLS data. The '.comm' and '.lcomm' directives of the
+    // variable will create the needed csect.
+    return;
+  }
+
+  // Zero-initialized TLS data with weak or external linkage are not eligible to
+  // be put into common csect.
+  if (getKind().isThreadBSS()) {
+    printCsectDirective(OS);
     return;
   }
 
   report_fatal_error("Printing for this SectionKind is unimplemented.");
 }
 
 bool MCSectionXCOFF::UseCodeAlign() const { return getKind().isText(); }
 
 bool MCSectionXCOFF::isVirtualSection() const {
   assert(isCsect() && "Only csect section can be virtual!");
   return XCOFF::XTY_CM == CsectProp->Type;
 }

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

private:
   /// Symbols lowered from ExternalSymbolSDNodes, we will need to emit extern
   /// linkage for them in AIX.
   SmallPtrSet<MCSymbol *, 8> ExtSymSDNodeSymbols;
 
   /// A format indicator and unique trailing identifier to form part of the
   /// sinit/sterm function names.
   std::string FormatIndicatorAndUniqueModId;
 
-  static void ValidateGV(const GlobalVariable *GV);
   // Record a list of GlobalAlias associated with a GlobalObject.
   // This is used for AIX's extra-label-at-definition aliasing strategy.
   DenseMap<const GlobalObject *, SmallVector<const GlobalAlias *, 1>>
       GOAliasMap;
 
   void emitTracebackTable();
 
 public:
if (ExtensionTableFlag & ExtendedTBTableFlag::TB_EH_INFO) {
     OutStreamer->AddComment("EHInfo Table");
     OutStreamer->emitValue(Exp, DL.getPointerSize());
   }
 
 #undef GENBOOLCOMMENT
 #undef GENVALUECOMMENT
 }
 
-void PPCAIXAsmPrinter::ValidateGV(const GlobalVariable *GV) {
-  // Early error checking limiting what is supported.
-  if (GV->isThreadLocal())
-    report_fatal_error("Thread local not yet supported on AIX.");
-
-  if (GV->hasComdat())
-    report_fatal_error("COMDAT not yet supported by AIX.");
-}
-
 static bool isSpecialLLVMGlobalArrayToSkip(const GlobalVariable *GV) {
   return GV->hasAppendingLinkage() &&
          StringSwitch<bool>(GV->getName())
              // TODO: Linker could still eliminate the GV if we just skip
              // handling llvm.used array. Skipping them for now until we or the
              // AIX OS team come up with a good solution.
              .Case("llvm.used", true)
              // It's correct to just skip llvm.compiler.used array here.

 
 void PPCAIXAsmPrinter::emitGlobalVariable(const GlobalVariable *GV) {
   // Special LLVM global arrays have been handled at the initialization.
   if (isSpecialLLVMGlobalArrayToSkip(GV) || isSpecialLLVMGlobalArrayForStaticInit(GV))
     return;
 
   assert(!GV->getName().startswith("llvm.") &&
          "Unhandled intrinsic global variable.");
-  ValidateGV(GV);
+
+  if (GV->hasComdat())
+    report_fatal_error("COMDAT not yet supported by AIX.");
 
   MCSymbolXCOFF *GVSym = cast<MCSymbolXCOFF>(getSymbol(GV));
 
   if (GV->isDeclarationForLinker()) {
     emitLinkage(GV, GVSym);
     return;
   }
 
   SectionKind GVKind = getObjFileLowering().getKindForGlobal(GV, TM);
-  if (!GVKind.isGlobalWriteableData() && !GVKind.isReadOnly())
+  if (!GVKind.isGlobalWriteableData() && !GVKind.isReadOnly() &&
+      !GVKind.isThreadLocal()) // Checks for both ThreadData and ThreadBSS.
     report_fatal_error("Encountered a global variable kind that is "
                        "not supported yet.");
 
   MCSectionXCOFF *Csect = cast<MCSectionXCOFF>(
       getObjFileLowering().SectionForGlobal(GV, GVKind, TM));
 
   // Switch to the containing csect.
   OutStreamer->SwitchSection(Csect);
 
   const DataLayout &DL = GV->getParent()->getDataLayout();
 
-  // Handle common symbols.
-  if (GVKind.isCommon() || GVKind.isBSSLocal()) {
+  // Handle common and zero-initialized local symbols.
+  if (GV->hasCommonLinkage() || GVKind.isBSSLocal() ||
+      GVKind.isThreadBSSLocal()) {
     Align Alignment = GV->getAlign().getValueOr(DL.getPreferredAlign(GV));
     uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
     GVSym->setStorageClass(
         TargetLoweringObjectFileXCOFF::getStorageClassForGlobal(GV));
 
-    if (GVKind.isBSSLocal())
+    if (GVKind.isBSSLocal() || GVKind.isThreadBSSLocal())
       OutStreamer->emitXCOFFLocalCommonSymbol(
           OutContext.getOrCreateSymbol(GVSym->getSymbolTableName()), Size,
           GVSym, Alignment.value());
     else
       OutStreamer->emitCommonSymbol(GVSym, Size, Alignment.value());
     return;
   }
 
   MCSymbol *EmittedInitSym = GVSym;

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

SDValue PPCTargetLowering::LowerBlockAddress(SDValue Op,
   getLabelAccessInfo(IsPIC, Subtarget, MOHiFlag, MOLoFlag);
   SDValue TgtBAHi = DAG.getTargetBlockAddress(BA, PtrVT, 0, MOHiFlag);
   SDValue TgtBALo = DAG.getTargetBlockAddress(BA, PtrVT, 0, MOLoFlag);
   return LowerLabelRef(TgtBAHi, TgtBALo, IsPIC, DAG);
 }
 
 SDValue PPCTargetLowering::LowerGlobalTLSAddress(SDValue Op,
                                               SelectionDAG &DAG) const {
+  if (Subtarget.isAIXABI())
+    report_fatal_error("TLS is not yet supported on AIX.");
   // FIXME: TLS addresses currently use medium model code sequences,
   // which is the most useful form.  Eventually support for small and
   // large models could be added if users need it, at the cost of
   // additional complexity.
   GlobalAddressSDNode *GA = cast<GlobalAddressSDNode>(Op);
   if (DAG.getTarget().useEmulatedTLS())
     return LowerToTLSEmulatedModel(GA, DAG);
 

llvm/lib/Target/TargetLoweringObjectFile.cpp

SectionKind TargetLoweringObjectFile::getKindForGlobal(const GlobalObject *GO,
   if (isa<BasicBlock>(GO))
     return SectionKind::getText();
 
   // Global variables require more detailed analysis.
   const auto *GVar = cast<GlobalVariable>(GO);
 
   // Handle thread-local data first.
   if (GVar->isThreadLocal()) {
-    if (isSuitableForBSS(GVar) && !TM.Options.NoZerosInBSS)
+    if (isSuitableForBSS(GVar) && !TM.Options.NoZerosInBSS) {
+      // Zero-initialized TLS variables with local linkage always get classified
+      // as ThreadBSSLocal.
+      if (GVar->hasLocalLinkage()) {
+        return SectionKind::getThreadBSSLocal();
+      }
       return SectionKind::getThreadBSS();
+    }
     return SectionKind::getThreadData();
   }
 
   // Variables with common linkage always get classified as common.
   if (GVar->hasCommonLinkage())
     return SectionKind::getCommon();
 
   // Most non-mergeable zero data can be put in the BSS section unless otherwise

llvm/test/CodeGen/PowerPC/aix-tls-checks.ll

+; RUN: not --crash llc -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec \
+; RUN:   -mtriple powerpc-ibm-aix-xcoff < %s - 2>&1 | FileCheck %s
+; RUN: not --crash llc -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec \
+; RUN:   -mtriple powerpc64-ibm-aix-xcoff < %s - 2>&1 | FileCheck %s
+
+; CHECK: TLS is not yet supported on AIX
+
+@tls1 = thread_local global i32 0, align 4
+
+define i32* @getTls1Addr() {
+entry:
+  ret i32* @tls1
+}

llvm/test/CodeGen/PowerPC/aix-tls-variables-ppc32.ll

+; This file tests 32 bit TLS variable generation.
+
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec -mtriple \
+; RUN:      powerpc-ibm-aix-xcoff < %s | FileCheck %s
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec -mtriple \
+; RUN:      powerpc-ibm-aix-xcoff -data-sections=false < %s | FileCheck %s \
+; RUN:      --check-prefix=NODATASEC
+
+; When data-sections is true (default), we emit data into separate sections.
+; When data-sections is false, we emit data into the .data / .tdata sections.
+
+; Int global variable, TLS/Non-TLS, local/external/weak/common linkage
+; CHECK:           .csect  global_int_external_val_initialized[RW],2
+; CHECK-NEXT:      .globl  global_int_external_val_initialized[RW]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 1
+
+; NODATASEC:       .csect  .data[RW],3
+; NODATASEC-NEXT:  .globl  global_int_external_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:global_int_external_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 1
+@global_int_external_val_initialized = global i32 1, align 4
+
+; CHECK-NEXT:      .csect  global_int_external_zero_initialized[RW],2
+; CHECK-NEXT:      .globl  global_int_external_zero_initialized[RW]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 0
+
+; NODATASEC-NEXT:  .globl  global_int_external_zero_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:global_int_external_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+@global_int_external_zero_initialized = global i32 0, align 4
+
+; CHECK-NEXT:      .csect  tls_global_int_external_val_initialized[TL],2
+; CHECK-NEXT:      .globl  tls_global_int_external_val_initialized[TL]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 1
+
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .globl  tls_global_int_external_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:tls_global_int_external_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 1
+@tls_global_int_external_val_initialized = thread_local global i32 1, align 4
+
+; CHECK-NEXT:      .csect  tls_global_int_external_zero_initialized[TL],2
+; CHECK-NEXT:      .globl  tls_global_int_external_zero_initialized[TL]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 0
+
+; NODATASEC-NEXT:  .globl  tls_global_int_external_zero_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:tls_global_int_external_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+@tls_global_int_external_zero_initialized = thread_local global i32 0, align 4
+
+; CHECK-NEXT:      .csect  global_int_local_val_initialized[RW],2
+; CHECK-NEXT:      .lglobl global_int_local_val_initialized[RW]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 2
+
+; NODATASEC-NEXT:  .csect .data[RW],3
+; NODATASEC-NEXT:  .lglobl global_int_local_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:global_int_local_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 2
+@global_int_local_val_initialized = internal global i32 2, align 4
+
+; CHECK-NEXT:      .csect  tls_global_int_local_val_initialized[TL],2
+; CHECK-NEXT:      .lglobl tls_global_int_local_val_initialized[TL]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 2
+
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .lglobl tls_global_int_local_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:tls_global_int_local_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 2
+@tls_global_int_local_val_initialized = internal thread_local global i32 2, align 4
+
+; CHECK-NEXT:      .lcomm  global_int_local_zero_initialized,4,global_int_local_zero_initialized[BS],2
+; NODATASEC-NEXT:  .lcomm  global_int_local_zero_initialized,4,global_int_local_zero_initialized[BS],2
+@global_int_local_zero_initialized = internal global i32 0, align 4
+
+; CHECK-NEXT:      .lcomm  tls_global_int_local_zero_initialized,4,tls_global_int_local_zero_initialized[UL],2
+; NODATASEC-NEXT:  .lcomm  tls_global_int_local_zero_initialized,4,tls_global_int_local_zero_initialized[UL],2
+@tls_global_int_local_zero_initialized = internal thread_local global i32 0, align 4
+
+; CHECK-NEXT:      .csect  global_int_weak_zero_initialized[RW],2
+; CHECK-NEXT:      .weak   global_int_weak_zero_initialized[RW]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 0
+
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_int_weak_zero_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:global_int_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+@global_int_weak_zero_initialized = weak global i32 0, align 4
+
+; CHECK-NEXT:      .csect  tls_global_int_weak_zero_initialized[TL],2
+; CHECK-NEXT:      .weak   tls_global_int_weak_zero_initialized[TL]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 0
+
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_int_weak_zero_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:tls_global_int_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+@tls_global_int_weak_zero_initialized = weak thread_local global i32 0, align 4
+
+; CHECK-NEXT:      .comm   global_int_common_zero_initialized[RW],4,2
+; NODATASEC-NEXT:  .comm   global_int_common_zero_initialized[RW],4,2
+@global_int_common_zero_initialized = common global i32 0, align 4
+
+; CHECK-NEXT:      .comm   tls_global_int_common_zero_initialized[UL],4,2
+; NODATASEC-NEXT:  .comm   tls_global_int_common_zero_initialized[UL],4,2
+@tls_global_int_common_zero_initialized = common thread_local global i32 0, align 4
+
+; CHECK-NEXT:      .csect  global_int_weak_val_initialized[RW],2
+; CHECK-NEXT:      .weak   global_int_weak_val_initialized[RW]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 1
+
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_int_weak_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:global_int_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 1
+@global_int_weak_val_initialized = weak global i32 1, align 4
+
+; CHECK-NEXT:      .csect  tls_global_int_weak_val_initialized[TL],2
+; CHECK-NEXT:      .weak   tls_global_int_weak_val_initialized[TL]
+; CHECK-NEXT:      .align  2
+; CHECK-NEXT:      .vbyte  4, 1
+
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_int_weak_val_initialized
+; NODATASEC-NEXT:  .align  2
+; NODATASEC-NEXT:tls_global_int_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 1
+@tls_global_int_weak_val_initialized = weak thread_local global i32 1, align 4
+
+; CHECK-NEXT:      .extern global_int_external_uninitialized[UA]
+; NODATASEC-NEXT:  .extern global_int_external_uninitialized[UA]
+@global_int_external_uninitialized = external global i32, align 4
+
+; CHECK-NEXT:      .extern tls_global_int_external_uninitialized[UL]
+; NODATASEC-NEXT:  .extern tls_global_int_external_uninitialized[UL]
+@tls_global_int_external_uninitialized = external thread_local global i32, align 4
+
+
+; double global variable, TLS/Non-TLS, common/external linkage
+
+; CHECK-NEXT:      .comm   global_double_common_zero_initialized[RW],8,3
+; NODATASEC-NEXT:  .comm   global_double_common_zero_initialized[RW],8,3
+@global_double_common_zero_initialized = common global double 0.000000e+00, align 8
+
+; CHECK-NEXT:      .comm   tls_global_double_common_zero_initialized[UL],8,3
+; NODATASEC-NEXT:  .comm   tls_global_double_common_zero_initialized[UL],8,3
+@tls_global_double_common_zero_initialized = common thread_local global double 0.000000e+00, align 8
+
+; CHECK-NEXT:      .extern global_double_external_uninitialized[UA]
+; NODATASEC-NEXT:  .extern global_double_external_uninitialized[UA]
+@global_double_external_uninitialized = external global i64, align 8
+
+; CHECK-NEXT:      .extern tls_global_double_external_uninitialized[UL]
+; NODATASEC-NEXT:  .extern tls_global_double_external_uninitialized[UL]
+@tls_global_double_external_uninitialized = external thread_local global i64, align 8
+
+
+; Long long global variable, TLS/Non-TLS, local/weak linkage
+
+; CHECK-NEXT:      .csect  global_long_long_internal_val_initialized[RW],3
+; CHECK-NEXT:      .lglobl global_long_long_internal_val_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 1
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .lglobl global_long_long_internal_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_internal_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 1
+@global_long_long_internal_val_initialized = internal global i64 1, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_internal_val_initialized[TL],3
+; CHECK-NEXT:      .lglobl tls_global_long_long_internal_val_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 1
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .lglobl tls_global_long_long_internal_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_internal_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 1
+@tls_global_long_long_internal_val_initialized = internal thread_local global i64 1, align 8
+
+; CHECK-NEXT:      .lcomm  global_long_long_internal_zero_initialized,8,global_long_long_internal_zero_initialized[BS],3
+; NODATASEC-NEXT:  .lcomm  global_long_long_internal_zero_initialized,8,global_long_long_internal_zero_initialized[BS],3
+@global_long_long_internal_zero_initialized = internal global i64 0, align 8
+
+; CHECK-NEXT:      .lcomm  tls_global_long_long_internal_zero_initialized,8,tls_global_long_long_internal_zero_initialized[UL],3
+; NODATASEC-NEXT:  .lcomm  tls_global_long_long_internal_zero_initialized,8,tls_global_long_long_internal_zero_initialized[UL],3
+@tls_global_long_long_internal_zero_initialized = internal thread_local global i64 0, align 8
+
+; CHECK-NEXT:      .csect  global_long_long_weak_val_initialized[RW],3
+; CHECK-NEXT:      .weak   global_long_long_weak_val_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 1
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_long_long_weak_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 1
+@global_long_long_weak_val_initialized = weak global i64 1, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_weak_val_initialized[TL],3
+; CHECK-NEXT:      .weak   tls_global_long_long_weak_val_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 1
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_long_long_weak_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 1
+@tls_global_long_long_weak_val_initialized = weak thread_local global i64 1, align 8
+
+; CHECK-NEXT:      .csect  global_long_long_weak_zero_initialized[RW],3
+; CHECK-NEXT:      .weak   global_long_long_weak_zero_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 0
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_long_long_weak_zero_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 0
+@global_long_long_weak_zero_initialized = weak global i64 0, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_weak_zero_initialized[TL],3
+; CHECK-NEXT:      .weak   tls_global_long_long_weak_zero_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  4, 0
+; CHECK-NEXT:      .vbyte  4, 0
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_long_long_weak_zero_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  4, 0
+; NODATASEC-NEXT:  .vbyte  4, 0
+@tls_global_long_long_weak_zero_initialized = weak thread_local global i64 0, align 8

llvm/test/CodeGen/PowerPC/aix-tls-variables-ppc64.ll

+; This file tests 64 bit TLS variable generation
+
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec -mtriple \
+; RUN:      powerpc64-ibm-aix-xcoff < %s | FileCheck %s
+; RUN: llc  -verify-machineinstrs -mcpu=pwr7 -mattr=-altivec -mtriple \
+; RUN:      powerpc64-ibm-aix-xcoff -data-sections=false < %s | FileCheck %s \
+; RUN:      --check-prefix=NODATASEC
+
+; When data-sections is true (default), we emit data into separate sections.
+; When data-sections is false, we emit data into the .data / .tdata sections.
+
+; Long long global variable, TLS/Non-TLS, local/weak linkage
+
+; CHECK:           .csect  global_long_long_internal_val_initialized[RW],3
+; CHECK-NEXT:      .lglobl global_long_long_internal_val_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 1
+; NODATASEC:       .csect  .data[RW],3
+; NODATASEC-NEXT:  .lglobl global_long_long_internal_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_internal_val_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 1
+@global_long_long_internal_val_initialized = internal global i64 1, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_internal_val_initialized[TL],3
+; CHECK-NEXT:      .lglobl tls_global_long_long_internal_val_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 1
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .lglobl tls_global_long_long_internal_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_internal_val_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 1
+@tls_global_long_long_internal_val_initialized = internal thread_local global i64 1, align 8
+
+; CHECK-NEXT:      .lcomm  global_long_long_internal_zero_initialized,8,global_long_long_internal_zero_initialized[BS],3
+; NODATASEC-NEXT:  .lcomm  global_long_long_internal_zero_initialized,8,global_long_long_internal_zero_initialized[BS],3
+@global_long_long_internal_zero_initialized = internal global i64 0, align 8
+
+; CHECK-NEXT:      .lcomm  tls_global_long_long_internal_zero_initialized,8,tls_global_long_long_internal_zero_initialized[UL],3
+; NODATASEC-NEXT:  .lcomm  tls_global_long_long_internal_zero_initialized,8,tls_global_long_long_internal_zero_initialized[UL],3
+@tls_global_long_long_internal_zero_initialized = internal thread_local global i64 0, align 8
+
+; CHECK-NEXT:      .csect  global_long_long_weak_val_initialized[RW],3
+; CHECK-NEXT:      .weak   global_long_long_weak_val_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 1
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_long_long_weak_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 1
+@global_long_long_weak_val_initialized = weak global i64 1, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_weak_val_initialized[TL],3
+; CHECK-NEXT:      .weak   tls_global_long_long_weak_val_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 1
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_long_long_weak_val_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_weak_val_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 1
+@tls_global_long_long_weak_val_initialized = weak thread_local global i64 1, align 8
+
+; CHECK-NEXT:      .csect  global_long_long_weak_zero_initialized[RW],3
+; CHECK-NEXT:      .weak   global_long_long_weak_zero_initialized[RW]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 0
+; NODATASEC-NEXT:  .csect  .data[RW],3
+; NODATASEC-NEXT:  .weak   global_long_long_weak_zero_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:global_long_long_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 0
+@global_long_long_weak_zero_initialized = weak global i64 0, align 8
+
+; CHECK-NEXT:      .csect  tls_global_long_long_weak_zero_initialized[TL],3
+; CHECK-NEXT:      .weak   tls_global_long_long_weak_zero_initialized[TL]
+; CHECK-NEXT:      .align  3
+; CHECK-NEXT:      .vbyte  8, 0
+; NODATASEC-NEXT:  .csect  .tdata[TL],3
+; NODATASEC-NEXT:  .weak   tls_global_long_long_weak_zero_initialized
+; NODATASEC-NEXT:  .align  3
+; NODATASEC-NEXT:tls_global_long_long_weak_zero_initialized:
+; NODATASEC-NEXT:  .vbyte  8, 0
+@tls_global_long_long_weak_zero_initialized = weak thread_local global i64 0, align 8