New lower emutls pass, fix linkage bugs.

[chh]

Summary

Previous implementation in http://reviews.llvm.org/D10522
created external references to __emutls_v.* variables.
Such references are inaccurate and cannot be handled by
all linkers, e.g. Android dynamic and gold linkers for aarch64.

Now a new LowerEmuTLS pass to go through all global variables,
and add emutls_v.* and emutls_t.* variables.
These __emutls* variables have the same linkage and
visibility as the associated user defined TLS variable.

Also removed old code that dump __emutls* variables in AsmPrinter.cpp,
and updated TLS unit tests.

[rnk]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

359–360

Remind me why we bother making an IR-level GlobalVariable if we never want to emit it?

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3057

I guess I didn't pay enough attention on the first review. :(

We really shouldn't be modifying the IR in TargetLowering. We have some instances of that, but we shouldn't be adding more.

Long term, if we want to stick with the IR lowering strategy, maybe we should do it as an IR pass. I seem to recall Chandler objected strongly to that, though.

Anyway, setting the initializer and linkage here for now seems fine.

3061

If we only need the global variable to make this GlobalAddress node, there are other ways to do that, like DAG.getExternalSymbol and DAG.getMCSymbol.

[chh]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

359–360

We do emit it, when the controlled variable xyz is visited by this function.
At that time, it is easier to emit both "emutls_v.xyz" and "emutls_t.xyz".
Originally, I didn't want to put either of them into symbol table,
but it turned out that it's easier to reuse codegen to access address of "__emutls_v.xyz"
if it is a global variable.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3057

Agree, I thought about a special pass for TLS too, until I found the target dependent LowerGlobalTLSAddres.
So, here LowerToTLSEmulateMode is only a target independent special case of LowerGlobalTLSAddres.

3061

I tried to do that, but maybe I did not make it all correct for an external symbol.
What I found out after my initial implementation is that get address of an external symbol would fail to link if the symbol is defined with internal linkage, for some linkers. That's what this patch to fix. So I would now make the symbol as real as necessary here to generate code to get the address.

[rnk]

I think the fact that you're excluding __emutls_v variables in AsmPrinter shows that this approach isn't a good one, and we need to go back and do it differently.

I think an IR preparation transform is going to be the cleanest, most portable way to do this.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

3061

True, you have to do lots of target specific things to get PIC access right.

[chh]

I think what you meant is to let AsmPrinter emit
emutls_v and emutls_t variables and those variables
should have complete definition in symbol table.
That should be doable.

How about the original TLS variable xyz?
It should be skipped when in emutls mode,
as it is done now in AsmPrinter, right?

[chh]

Now a new emutls lowering pass, plus fixing bugs.

[rnk]

The general approach seems reasonable to me, but IIRC a ModulePass creates a scheduling barrier in the PassManager pipeline. You probably want to schedule it first at the top of addPassesToGenerateCode in LLVMTargetMachine.cpp.

[chh]

Schedule LowerEmuTLS pass first at the top of addPassesToGenerateCode.

[chh]

ping.

[rnk]

lgtm

lib/CodeGen/LowerEmuTLS.cpp

43

What about COMDAT TLS variables? Consider this code:

template <typename T> struct A { static thread_local int x; };
int getX() { return A<int>::x++; }

Multiple TUs may instantiate A<int>::x, and we need __emutls_v._ZN1AIiE1xE to go in a comdat section.

Perhaps we can handle this in a subsequent patch, but it's good to think about.

106–107

Don't create an EVT object in IR transforms, they aren't needed.

A better way to write this would be:

IntegerType *WordType = DL.getIntPtrType(C, 0);

113–115

If you use cast<> instead of dyn_cast_or_null<>, this assertion will be performed for you implicitly.

[chh]

Handle comdat; simplify some code; add/update unit tests.

lib/CodeGen/LowerEmuTLS.cpp

43

Good catch. Please take a look of the new diff.
I added also some unit test to AArch64/emutls.ll.
Thanks.

106–107

Good suggestion. Thanks.

[rnk]

lgtm

include/llvm/CodeGen/AsmPrinter.h

public:
   ///
   virtual void EmitConstantPool();
 
   /// Print assembly representations of the jump tables used by the current
   /// function to the current output stream.
   ///
   virtual void EmitJumpTableInfo();
 
-  /// Emit the control variable for an emulated TLS variable.
-  virtual void EmitEmulatedTLSControlVariable(const GlobalVariable *GV,
-                                              MCSymbol *EmittedSym,
-                                              bool AllZeroInitValue);
-
   /// Emit the specified global variable to the .s file.
   virtual void EmitGlobalVariable(const GlobalVariable *GV);
 
   /// Check to see if the specified global is a special global used by LLVM. If
   /// so, emit it and return true, otherwise do nothing and return false.
   bool EmitSpecialLLVMGlobal(const GlobalVariable *GV);
 
   /// Emit an alignment directive to the specified power of two boundary. For

include/llvm/InitializePasses.h

 void initializeLoopUnswitchPass(PassRegistry&);
 void initializeLoopIdiomRecognizePass(PassRegistry&);
 void initializeLowerAtomicPass(PassRegistry&);
 void initializeLowerBitSetsPass(PassRegistry&);
 void initializeLowerExpectIntrinsicPass(PassRegistry&);
 void initializeLowerIntrinsicsPass(PassRegistry&);
 void initializeLowerInvokePass(PassRegistry&);
 void initializeLowerSwitchPass(PassRegistry&);
+void initializeLowerEmuTLSPass(PassRegistry&);
 void initializeMachineBlockFrequencyInfoPass(PassRegistry&);
 void initializeMachineBlockPlacementPass(PassRegistry&);
 void initializeMachineBlockPlacementStatsPass(PassRegistry&);
 void initializeMachineBranchProbabilityInfoPass(PassRegistry&);
 void initializeMachineCSEPass(PassRegistry&);
 void initializeImplicitNullChecksPass(PassRegistry&);
 void initializeMachineDominatorTreePass(PassRegistry&);
 void initializeMachineDominanceFrontierPass(PassRegistry&);

include/llvm/Transforms/Scalar.h

 //===----------------------------------------------------------------------===//
 //
 // LowerAtomic - Lower atomic intrinsics to non-atomic form
 //
 Pass *createLowerAtomicPass();
 
 //===----------------------------------------------------------------------===//
 //
+// LowerEmuTLS - This pass generates __emutls_[vt].xyz variables for all
+// TLS variables for the emulated TLS model.
+//
+ModulePass *createLowerEmuTLSPass(const TargetMachine *TM);
+
+//===----------------------------------------------------------------------===//
+//
 // ValuePropagation - Propagate CFG-derived value information
 //
 Pass *createCorrelatedValuePropagationPass();
 
 //===----------------------------------------------------------------------===//
 //
 // InstructionSimplifier - Remove redundant instructions.
 //

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

void AsmPrinter::getNameWithPrefix(SmallVectorImpl<char> &Name,
                                    const GlobalValue *GV) const {
   TM.getNameWithPrefix(Name, GV, *Mang);
 }
 
 MCSymbol *AsmPrinter::getSymbol(const GlobalValue *GV) const {
   return TM.getSymbol(GV, *Mang);
 }
 
-static MCSymbol *getOrCreateEmuTLSControlSym(MCSymbol *GVSym, MCContext &C) {
-  return C.getOrCreateSymbol(Twine("__emutls_v.") + GVSym->getName());
-}
-
-static MCSymbol *getOrCreateEmuTLSInitSym(MCSymbol *GVSym, MCContext &C) {
-  return C.getOrCreateSymbol(Twine("__emutls_t.") + GVSym->getName());
-}
-
-/// EmitEmulatedTLSControlVariable - Emit the control variable for an emulated TLS variable.
-void AsmPrinter::EmitEmulatedTLSControlVariable(const GlobalVariable *GV,
-                                                MCSymbol *EmittedSym,
-                                                bool AllZeroInitValue) {
-  MCSection *TLSVarSection = getObjFileLowering().getDataSection();
-  OutStreamer->SwitchSection(TLSVarSection);
-  MCSymbol *GVSym = getSymbol(GV);
-  EmitLinkage(GV, EmittedSym);  // same linkage as GV
-  const DataLayout &DL = GV->getParent()->getDataLayout();
-  uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
-  unsigned AlignLog = getGVAlignmentLog2(GV, DL);
-  unsigned WordSize = DL.getPointerSize();
-  unsigned Alignment = DL.getPointerABIAlignment();
-  EmitAlignment(Log2_32(Alignment));
-  OutStreamer->EmitLabel(EmittedSym);
-  OutStreamer->EmitIntValue(Size, WordSize);
-  OutStreamer->EmitIntValue((1 << AlignLog), WordSize);
-  OutStreamer->EmitIntValue(0, WordSize);
-  if (GV->hasInitializer() && !AllZeroInitValue) {
-    OutStreamer->EmitSymbolValue(
-        getOrCreateEmuTLSInitSym(GVSym, OutContext), WordSize);
-  } else
-    OutStreamer->EmitIntValue(0, WordSize);
-  if (MAI->hasDotTypeDotSizeDirective())
-    OutStreamer->emitELFSize(cast<MCSymbolELF>(EmittedSym),
-                             MCConstantExpr::create(4 * WordSize, OutContext));
-  OutStreamer->AddBlankLine();  // End of the __emutls_v.* variable.
-}
-
 /// EmitGlobalVariable - Emit the specified global variable to the .s file.
 void AsmPrinter::EmitGlobalVariable(const GlobalVariable *GV) {
-  bool IsEmuTLSVar =
-      GV->getThreadLocalMode() != llvm::GlobalVariable::NotThreadLocal &&
-      TM.Options.EmulatedTLS;
+  bool IsEmuTLSVar = TM.Options.EmulatedTLS && GV->isThreadLocal();
   assert(!(IsEmuTLSVar && GV->hasCommonLinkage()) &&
          "No emulated TLS variables in the common section");
 
+  // Never emit TLS variable xyz in emulated TLS model.
+  // The initialization value is in __emutls_t.xyz instead of xyz.
+  if (IsEmuTLSVar)
+    return;
+
   if (GV->hasInitializer()) {
     // Check to see if this is a special global used by LLVM, if so, emit it.
     if (EmitSpecialLLVMGlobal(GV))
       return;
 
     // Skip the emission of global equivalents. The symbol can be emitted later
     // on by emitGlobalGOTEquivs in case it turns out to be needed.
     if (GlobalGOTEquivs.count(getSymbol(GV)))
       return;
 
-    if (isVerbose() && !IsEmuTLSVar) {
+    if (isVerbose()) {
       // When printing the control variable __emutls_v.*,
       // we don't need to print the original TLS variable name.
       GV->printAsOperand(OutStreamer->GetCommentOS(),
                      /*PrintType=*/false, GV->getParent());
       OutStreamer->GetCommentOS() << '\n';
     }
   }
 
   MCSymbol *GVSym = getSymbol(GV);
-  MCSymbol *EmittedSym = IsEmuTLSVar ?
-      getOrCreateEmuTLSControlSym(GVSym, OutContext) : GVSym;
+  MCSymbol *EmittedSym = GVSym;
   // getOrCreateEmuTLSControlSym only creates the symbol with name and default attributes.
   // GV's or GVSym's attributes will be used for the EmittedSym.
 
   EmitVisibility(EmittedSym, GV->getVisibility(), !GV->isDeclaration());
 
   if (!GV->hasInitializer())   // External globals require no extra code.
     return;
 
void AsmPrinter::EmitGlobalVariable(const GlobalVariable *GV) {
   const DataLayout &DL = GV->getParent()->getDataLayout();
   uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
 
   // If the alignment is specified, we *must* obey it.  Overaligning a global
   // with a specified alignment is a prompt way to break globals emitted to
   // sections and expected to be contiguous (e.g. ObjC metadata).
   unsigned AlignLog = getGVAlignmentLog2(GV, DL);
 
-  bool AllZeroInitValue = false;
-  const Constant *InitValue = GV->getInitializer();
-  if (isa<ConstantAggregateZero>(InitValue))
-    AllZeroInitValue = true;
-  else {
-    const ConstantInt *InitIntValue = dyn_cast<ConstantInt>(InitValue);
-    if (InitIntValue && InitIntValue->isZero())
-      AllZeroInitValue = true;
-  }
-  if (IsEmuTLSVar)
-    EmitEmulatedTLSControlVariable(GV, EmittedSym, AllZeroInitValue);
-
   for (const HandlerInfo &HI : Handlers) {
     NamedRegionTimer T(HI.TimerName, HI.TimerGroupName, TimePassesIsEnabled);
     HI.Handler->setSymbolSize(GVSym, Size);
   }
 
   // Handle common and BSS local symbols (.lcomm).
   if (GVKind.isCommon() || GVKind.isBSSLocal()) {
-    assert(!(IsEmuTLSVar && GVKind.isCommon()) &&
-           "No emulated TLS variables in the common section");
     if (Size == 0) Size = 1;   // .comm Foo, 0 is undefined, avoid it.
     unsigned Align = 1 << AlignLog;
 
     // Handle common symbols.
     if (GVKind.isCommon()) {
       if (!getObjFileLowering().getCommDirectiveSupportsAlignment())
         Align = 0;
 
if (GVKind.isCommon() || GVKind.isBSSLocal()) {
 
     // .local _foo
     OutStreamer->EmitSymbolAttribute(GVSym, MCSA_Local);
     // .comm _foo, 42, 4
     OutStreamer->EmitCommonSymbol(GVSym, Size, Align);
     return;
   }
 
-  if (IsEmuTLSVar && AllZeroInitValue)
-    return;  // No need of initialization values.
+  MCSymbol *EmittedInitSym = GVSym;
 
-  MCSymbol *EmittedInitSym = IsEmuTLSVar ?
-      getOrCreateEmuTLSInitSym(GVSym, OutContext) : GVSym;
-  // getOrCreateEmuTLSInitSym only creates the symbol with name and default attributes.
-  // GV's or GVSym's attributes will be used for the EmittedInitSym.
-
-  MCSection *TheSection = IsEmuTLSVar ?
-      getObjFileLowering().getReadOnlySection() :
+  MCSection *TheSection =
       getObjFileLowering().SectionForGlobal(GV, GVKind, *Mang, TM);
 
   // Handle the zerofill directive on darwin, which is a special form of BSS
   // emission.
-  if (GVKind.isBSSExtern() && MAI->hasMachoZeroFillDirective() && !IsEmuTLSVar) {
+  if (GVKind.isBSSExtern() && MAI->hasMachoZeroFillDirective()) {
     if (Size == 0) Size = 1;  // zerofill of 0 bytes is undefined.
 
     // .globl _foo
     OutStreamer->EmitSymbolAttribute(GVSym, MCSA_Global);
     // .zerofill __DATA, __common, _foo, 400, 5
     OutStreamer->EmitZerofill(TheSection, GVSym, Size, 1 << AlignLog);
     return;
   }
 
   // Handle thread local data for mach-o which requires us to output an
   // additional structure of data and mangle the original symbol so that we
   // can reference it later.
   //
   // TODO: This should become an "emit thread local global" method on TLOF.
   // All of this macho specific stuff should be sunk down into TLOFMachO and
   // stuff like "TLSExtraDataSection" should no longer be part of the parent
   // TLOF class.  This will also make it more obvious that stuff like
   // MCStreamer::EmitTBSSSymbol is macho specific and only called from macho
   // specific code.
-  if (GVKind.isThreadLocal() && MAI->hasMachoTBSSDirective() && !IsEmuTLSVar) {
+  if (GVKind.isThreadLocal() && MAI->hasMachoTBSSDirective()) {
     // Emit the .tbss symbol
     MCSymbol *MangSym =
       OutContext.getOrCreateSymbol(GVSym->getName() + Twine("$tlv$init"));
 
     if (GVKind.isThreadBSS()) {
       TheSection = getObjFileLowering().getTLSBSSSection();
       OutStreamer->EmitTBSSSymbol(TheSection, MangSym, Size, 1 << AlignLog);
     } else if (GVKind.isThreadData()) {
if (GVKind.isThreadLocal() && MAI->hasMachoTBSSDirective()) {
     OutStreamer->EmitSymbolValue(MangSym, PtrSize);
 
     OutStreamer->AddBlankLine();
     return;
   }
 
   OutStreamer->SwitchSection(TheSection);
 
-  // emutls_t.* symbols are only used in the current compilation unit.
-  if (!IsEmuTLSVar)
-    EmitLinkage(GV, EmittedInitSym);
   EmitAlignment(AlignLog, GV);
 
   OutStreamer->EmitLabel(EmittedInitSym);
 
   EmitGlobalConstant(GV->getParent()->getDataLayout(), GV->getInitializer());
 
   if (MAI->hasDotTypeDotSizeDirective())
     // .size foo, 42

lib/CodeGen/CMakeLists.txt

add_llvm_library(LLVMCodeGen
   LiveIntervalUnion.cpp
   LiveRangeCalc.cpp
   LiveRangeEdit.cpp
   LiveRegMatrix.cpp
   LivePhysRegs.cpp
   LiveStackAnalysis.cpp
   LiveVariables.cpp
   LocalStackSlotAllocation.cpp
+  LowerEmuTLS.cpp
   MachineBasicBlock.cpp
   MachineBlockFrequencyInfo.cpp
   MachineBlockPlacement.cpp
   MachineBranchProbabilityInfo.cpp
   MachineCSE.cpp
   MachineCombiner.cpp
   MachineCopyPropagation.cpp
   MachineDominators.cpp

lib/CodeGen/LLVMTargetMachine.cpp

 
 /// addPassesToX helper drives creation and initialization of TargetPassConfig.
 static MCContext *
 addPassesToGenerateCode(LLVMTargetMachine *TM, PassManagerBase &PM,
                         bool DisableVerify, AnalysisID StartBefore,
                         AnalysisID StartAfter, AnalysisID StopAfter,
                         MachineFunctionInitializer *MFInitializer = nullptr) {
 
+  // When in emulated TLS mode, add the LowerEmuTLS pass.
+  if (TM->Options.EmulatedTLS)
+    PM.add(createLowerEmuTLSPass(TM));
+
   // Add internal analysis passes from the target machine.
   PM.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
 
   // Targets may override createPassConfig to provide a target-specific
   // subclass.
   TargetPassConfig *PassConfig = TM->createPassConfig(PM);
   PassConfig->setStartStopPasses(StartBefore, StartAfter, StopAfter);
 

lib/CodeGen/LowerEmuTLS.cpp

+//===- LowerEmuTLS.cpp - Add __emutls_[vt].* variables --------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This transformation is required for targets depending on libgcc style
+// emulated thread local storage variables. For every defined TLS variable xyz,
+// an __emutls_v.xyz is generated. If there is non-zero initialized value
+// an __emutls_t.xyz is also generated.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/Scalar.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Pass.h"
+#include "llvm/Target/TargetLowering.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "loweremutls"
+
+namespace {
+
+class LowerEmuTLS : public ModulePass {
+  const TargetMachine *TM;
+public:
+  static char ID; // Pass identification, replacement for typeid
+  explicit LowerEmuTLS() : ModulePass(ID), TM(nullptr) { }
+  explicit LowerEmuTLS(const TargetMachine *TM)
+      : ModulePass(ID), TM(TM) {
+    initializeLowerEmuTLSPass(*PassRegistry::getPassRegistry());
+  }
+  bool runOnModule(Module &M) override;
+private:
+  bool addEmuTlsVar(Module &M, const GlobalVariable *GV);
+  static void copyLinkageVisibility(Module &M,
+                                    const GlobalVariable *from,
+                                    GlobalVariable *to) {
+    to->setLinkage(from->getLinkage());
+    to->setVisibility(from->getVisibility());
+    if (from->hasComdat()) {
+      to->setComdat(M.getOrInsertComdat(to->getName()));
+      to->getComdat()->setSelectionKind(from->getComdat()->getSelectionKind());
+    }
+  }
+};
+}
+
+char LowerEmuTLS::ID = 0;
+
+INITIALIZE_PASS(LowerEmuTLS, "loweremutls",
+                "Add __emutls_[vt]. variables for emultated TLS model",
+                false, false)
+
+ModulePass *llvm::createLowerEmuTLSPass(const TargetMachine *TM) {
+  return new LowerEmuTLS(TM);
+}
+
+bool LowerEmuTLS::runOnModule(Module &M) {
+  if (!TM || !TM->Options.EmulatedTLS)
+    return false;
+
+  bool Changed = false;
+  SmallVector<const GlobalVariable*, 8> TlsVars;
+  for (const auto &G : M.globals()) {
+    if (G.isThreadLocal())
+      TlsVars.append({&G});
+  }
+  for (const auto G : TlsVars)
+    Changed |= addEmuTlsVar(M, G);
+  return Changed;
+}
+
+bool LowerEmuTLS::addEmuTlsVar(Module &M, const GlobalVariable *GV) {
+  LLVMContext &C = M.getContext();
+  PointerType *VoidPtrType = Type::getInt8PtrTy(C);
+
+  std::string EmuTlsVarName = ("__emutls_v." + GV->getName()).str();
+  GlobalVariable *EmuTlsVar = M.getNamedGlobal(EmuTlsVarName);
+  if (EmuTlsVar)
+    return false;  // It has been added before.
+
+  const DataLayout &DL = M.getDataLayout();
+  Constant *NullPtr = ConstantPointerNull::get(VoidPtrType);
+
+  // Get non-zero initializer from GV's initializer.
+  const Constant *InitValue = nullptr;
+  if (GV->hasInitializer()) {
+    InitValue = GV->getInitializer();
+    const ConstantInt *InitIntValue = dyn_cast<ConstantInt>(InitValue);
+    // When GV's init value is all 0, omit the EmuTlsTmplVar and let
+    // the emutls library function to reset newly allocated TLS variables.
+    if (isa<ConstantAggregateZero>(InitValue) ||
+        (InitIntValue && InitIntValue->isZero()))
+      InitValue = nullptr;
+  }
+
+  // Create the __emutls_v. symbol, whose type has 4 fields:
+  //     word size;   // size of GV in bytes
+  //     word align;  // alignment of GV
+  //     void *ptr;   // initialized to 0; set at run time per thread.
+  //     void *templ; // 0 or point to __emutls_t.*
+  // sizeof(word) should be the same as sizeof(void*) on target.
+  IntegerType *WordType = DL.getIntPtrType(C);
+  PointerType *InitPtrType = InitValue ?
+      PointerType::getUnqual(InitValue->getType()) : VoidPtrType;
+  Type *ElementTypes[4] = {WordType, WordType, VoidPtrType, InitPtrType};
+  ArrayRef<Type*> ElementTypeArray(ElementTypes, 4);
+  StructType *EmuTlsVarType = StructType::create(ElementTypeArray);
+  EmuTlsVar = cast<GlobalVariable>(
+      M.getOrInsertGlobal(EmuTlsVarName, EmuTlsVarType));
+  copyLinkageVisibility(M, GV, EmuTlsVar);
+
+  // Define "__emutls_t.*" and "__emutls_v.*" only if GV is defined.
+  if (!GV->hasInitializer())
+    return true;
+
+  Type *GVType = GV->getValueType();
+  unsigned GVAlignment = GV->getAlignment();
+  if (!GVAlignment) {
+    // When LLVM IL declares a variable without alignment, use
+    // the ABI default alignment for the type.
+    GVAlignment = DL.getABITypeAlignment(GVType);
+  }
+
+  // Define "__emutls_t.*" if there is InitValue
+  GlobalVariable *EmuTlsTmplVar = nullptr;
+  if (InitValue) {
+    std::string EmuTlsTmplName = ("__emutls_t." + GV->getName()).str();
+    EmuTlsTmplVar = dyn_cast_or_null<GlobalVariable>(
+        M.getOrInsertGlobal(EmuTlsTmplName, GVType));
+    assert(EmuTlsTmplVar && "Failed to create emualted TLS initializer");
+    EmuTlsTmplVar->setConstant(true);
+    EmuTlsTmplVar->setInitializer(const_cast<Constant*>(InitValue));
+    EmuTlsTmplVar->setAlignment(GVAlignment);
+    copyLinkageVisibility(M, GV, EmuTlsTmplVar);
+  }
+
+  // Define "__emutls_v.*" with initializer and alignment.
+  Constant *ElementValues[4] = {
+      ConstantInt::get(WordType, DL.getTypeStoreSize(GVType)),
+      ConstantInt::get(WordType, GVAlignment),
+      NullPtr, EmuTlsTmplVar ? EmuTlsTmplVar : NullPtr
+  };
+  ArrayRef<Constant*> ElementValueArray(ElementValues, 4);
+  EmuTlsVar->setInitializer(
+      ConstantStruct::get(EmuTlsVarType, ElementValueArray));
+  unsigned MaxAlignment = std::max(
+      DL.getABITypeAlignment(WordType),
+      DL.getABITypeAlignment(VoidPtrType));
+  EmuTlsVar->setAlignment(MaxAlignment);
+  return true;
+}

lib/CodeGen/SelectionDAG/TargetLowering.cpp

SDValue TargetLowering::LowerToTLSEmulatedModel(const GlobalAddressSDNode *GA,
   //   __emutls_get_address( address of global variable named "__emutls_v.xyz" )
   EVT PtrVT = getPointerTy(DAG.getDataLayout());
   PointerType *VoidPtrType = Type::getInt8PtrTy(*DAG.getContext());
   SDLoc dl(GA);
 
   ArgListTy Args;
   ArgListEntry Entry;
   std::string NameString = ("__emutls_v." + GA->getGlobal()->getName()).str();
   Module *VariableModule = const_cast<Module*>(GA->getGlobal()->getParent());
   StringRef EmuTlsVarName(NameString);
   GlobalVariable *EmuTlsVar = VariableModule->getNamedGlobal(EmuTlsVarName);
-  if (!EmuTlsVar)
-    EmuTlsVar = dyn_cast_or_null<GlobalVariable>(
-        VariableModule->getOrInsertGlobal(EmuTlsVarName, VoidPtrType));
+  assert(EmuTlsVar && "Cannot find EmuTlsVar ");
   Entry.Node = DAG.getGlobalAddress(EmuTlsVar, dl, PtrVT);
   Entry.Ty = VoidPtrType;
   Args.push_back(Entry);
 
   SDValue EmuTlsGetAddr = DAG.getExternalSymbol("__emutls_get_address", PtrVT);
 
   TargetLowering::CallLoweringInfo CLI(DAG);
   CLI.setDebugLoc(dl).setChain(DAG.getEntryNode());
   CLI.setCallee(CallingConv::C, VoidPtrType, EmuTlsGetAddr, std::move(Args), 0);

test/CodeGen/AArch64/emutls.ll

 ; ARM64-NEXT:   ldr x0, [x0, :got_lo12:__emutls_v.i1]
 ; ARM64-NEXT:   bl __emutls_get_address
 ; ARM64-NEXT:   ldp x29, x30, [sp]
 
 entry:
   ret i32* @i1
 }
 
+define i32 @f5() nounwind {
+; ARM64-LABEL: f5:
+; ARM64:        adrp x0, __emutls_v.i3
+; ARM64:        add x0, x0, :lo12:__emutls_v.i3
+; ARM64:        bl __emutls_get_address
+; ARM64-NEXT:   ldr w0, [x0]
+
+entry:
+  %tmp1 = load i32, i32* @i3
+  ret i32 %tmp1
+}
+
+define i32* @f6() {
+; ARM64-LABEL: f6:
+; ARM64:        adrp x0, __emutls_v.i3
+; ARM64:        add x0, x0, :lo12:__emutls_v.i3
+; ARM64-NEXT:   bl __emutls_get_address
+; ARM64-NEXT:   ldp x29, x30, [sp]
+
+entry:
+  ret i32* @i3
+}
+
+; Simple test of comdat __thread variables.
+; template <class T> struct A { static __thread T x; };
+; template <class T> T __thread A<T>::x;
+; int getIntX() { return A<int>::x++; }
+; float getFloatX() { return A<float>::x++; }
+
+$_ZN1AIiE1xE = comdat any
+$_ZN1AIfE1xE = comdat any
+@_ZN1AIiE1xE = linkonce_odr thread_local global i32 0, comdat, align 4
+@_ZN1AIfE1xE = linkonce_odr thread_local global float 0.000000e+00, comdat, align 4
+
+define i32 @_Z7getIntXv() {
+; ARM64-LABEL: _Z7getIntXv:
+; ARM64:        adrp x0, :got:__emutls_v._ZN1AIiE1xE
+; ARM64:        ldr x0, [x0, :got_lo12:__emutls_v._ZN1AIiE1xE]
+; ARM64-NEXT:   bl __emutls_get_address
+; ARM64-NEXT:   ldr {{.*}}, [x0]
+; ARM64:        add
+; ARM64:        str {{.*}}, [x0]
+
+entry:
+  %0 = load i32, i32* @_ZN1AIiE1xE, align 4
+  %inc = add nsw i32 %0, 1
+  store i32 %inc, i32* @_ZN1AIiE1xE, align 4
+  ret i32 %0
+}
+
+define float @_Z9getFloatXv() {
+; ARM64-LABEL: _Z9getFloatXv:
+; ARM64:        adrp x0, :got:__emutls_v._ZN1AIfE1xE
+; ARM64:        ldr x0, [x0, :got_lo12:__emutls_v._ZN1AIfE1xE]
+; ARM64-NEXT:   bl __emutls_get_address
+; ARM64-NEXT:   ldr {{.*}}, [x0]
+; ARM64:        fadd s{{.*}}, s
+; ARM64:        str s{{.*}}, [x0]
+
+entry:
+  %0 = load float, float* @_ZN1AIfE1xE, align 4
+  %inc = fadd float %0, 1.000000e+00
+  store float %inc, float* @_ZN1AIfE1xE, align 4
+  ret float %0
+}
+
+
 ;;;;;;;;;;;;;; 64-bit __emutls_v. and __emutls_t.
 
-; ARM64       .section .data.rel.local,
+; ARM64:      .data{{$}}
+; ARM64:      .globl __emutls_v.i1
 ; ARM64-LABEL: __emutls_v.i1:
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 0
 ; ARM64-NEXT: .xword __emutls_t.i1
 
-; ARM64       .section .rodata,
+; ARM64:      .section .rodata,
 ; ARM64-LABEL: __emutls_t.i1:
 ; ARM64-NEXT: .word 15
 
 ; ARM64-NOT:   __emutls_v.i2
 
-; ARM64       .section .data.rel.local,
+; ARM64:      .data{{$}}
+; ARM64-NOT:  .globl
 ; ARM64-LABEL: __emutls_v.i3:
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 0
 ; ARM64-NEXT: .xword __emutls_t.i3
 
-; ARM64       .section .rodata,
+; ARM64:      .section .rodata,
 ; ARM64-LABEL: __emutls_t.i3:
 ; ARM64-NEXT: .word 15
 
-; ARM64       .section .data.rel.local,
+; ARM64:      .hidden __emutls_v.i4
+; ARM64:      .data{{$}}
+; ARM64:      .globl __emutls_v.i4
 ; ARM64-LABEL: __emutls_v.i4:
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 4
 ; ARM64-NEXT: .xword 0
 ; ARM64-NEXT: .xword __emutls_t.i4
 
-; ARM64       .section .rodata,
+; ARM64:      .section .rodata,
 ; ARM64-LABEL: __emutls_t.i4:
 ; ARM64-NEXT: .word 15
 
 ; ARM64-NOT:   __emutls_v.i5:
-; ARM64       .hidden __emutls_v.i5
+; ARM64:      .hidden __emutls_v.i5
 ; ARM64-NOT:   __emutls_v.i5:
 
-; ARM64       .section .data.rel.local,
+; ARM64:      .data{{$}}
+; ARM64:      .globl __emutls_v.s1
 ; ARM64-LABEL: __emutls_v.s1:
 ; ARM64-NEXT: .xword 2
 ; ARM64-NEXT: .xword 2
 ; ARM64-NEXT: .xword 0
 ; ARM64-NEXT: .xword __emutls_t.s1
 
-; ARM64       .section .rodata,
+; ARM64:      .section .rodata,
 ; ARM64-LABEL: __emutls_t.s1:
 ; ARM64-NEXT: .hword 15
 
-; ARM64       .section .data.rel.local,
+; ARM64:      .data{{$}}
 ; ARM64-LABEL: __emutls_v.b1:
 ; ARM64-NEXT: .xword 1
 ; ARM64-NEXT: .xword 1
 ; ARM64-NEXT: .xword 0
 ; ARM64-NEXT: .xword 0
 
-; ARM64-NOT:  __emutls_t.b1
+; ARM64-NOT:   __emutls_t.b1
+
+; ARM64:      .section .data.__emutls_v._ZN1AIiE1xE,{{.*}},__emutls_v._ZN1AIiE1xE,comdat
+; ARM64:      .weak __emutls_v._ZN1AIiE1xE
+; ARM64:      .align 3
+; ARM64-LABEL: __emutls_v._ZN1AIiE1xE:
+; ARM64-NEXT: .xword 4
+; ARM64-NEXT: .xword 4
+; ARM64-NEXT: .xword 0
+; ARM64-NEXT: .xword 0
+
+; ARM64:      .section .data.__emutls_v._ZN1AIfE1xE,{{.*}},__emutls_v._ZN1AIfE1xE,comdat
+; ARM64:      .weak __emutls_v._ZN1AIfE1xE
+; ARM64:      .align 3
+; ARM64-LABEL: __emutls_v._ZN1AIfE1xE:
+; ARM64-NEXT: .xword 4
+; ARM64-NEXT: .xword 4
+; ARM64-NEXT: .xword 0
+; ARM64-NEXT: .xword __emutls_t._ZN1AIfE1xE
+
+; ARM64:      .section .rodata.__emutls_t._ZN1AIfE1xE,{{.*}},__emutls_t._ZN1AIfE1xE,comdat
+; ARM64:      .weak __emutls_t._ZN1AIfE1xE
+; ARM64:      .align 2
+; ARM64-LABEL: __emutls_t._ZN1AIfE1xE:
+; ARM64-NEXT: .word 0
+; ARM64-NEXT: .size

test/CodeGen/AArch64/emutls_generic.ll

 ; ARM_64-LABEL:  get_external_y:
 ; ARM_64:      __emutls_v.external_y
 ; ARM_64:      __emutls_get_address
 ; ARM_64-LABEL:  get_internal_y:
 ; ARM_64:      __emutls_v.internal_y
 ; ARM_64:      __emutls_get_address
 ; ARM_64-NOT:   __emutls_t.external_x
 ; ARM_64-NOT:   __emutls_v.external_x:
+; ARM_64:        .data{{$}}
+; ARM_64:        .globl __emutls_v.external_y
 ; ARM_64:        .align 3
 ; ARM_64-LABEL:  __emutls_v.external_y:
 ; ARM_64-NEXT:   .xword 1
 ; ARM_64-NEXT:   .xword 2
 ; ARM_64-NEXT:   .xword 0
 ; ARM_64-NEXT:   .xword __emutls_t.external_y
 ; ARM_64-NOT:    __emutls_v.external_x:
 ; ARM_64:        .section .rodata,
 ; ARM_64-LABEL:  __emutls_t.external_y:
 ; ARM_64-NEXT:   .byte 7
-; ARM_64:        .data
+; ARM_64:        .data{{$}}
+; ARM_64-NOT:    .globl __emutls_v
 ; ARM_64:        .align 3
 ; ARM_64-LABEL:  __emutls_v.internal_y:
 ; ARM_64-NEXT:   .xword 8
 ; ARM_64-NEXT:   .xword 16
 ; ARM_64-NEXT:   .xword 0
 ; ARM_64-NEXT:   .xword __emutls_t.internal_y
 ; ARM_64:        .section .rodata,
 ; ARM_64-LABEL:  __emutls_t.internal_y:
 ; ARM_64-NEXT:   .xword 9

test/CodeGen/ARM/emutls.ll

 ; RUN: llc -emulated-tls -mtriple=arm-linux-android \
 ; RUN:     -relocation-model=pic < %s | FileCheck -check-prefix=ARM32 %s
 
 ; Copied from X86/emutls.ll
 
 ; Use my_emutls_get_address like __emutls_get_address.
 @my_emutls_v_xyz = external global i8*, align 4
 declare i8* @my_emutls_get_address(i8*)
 
 define i32 @my_get_xyz() {
 ; ARM32-LABEL: my_get_xyz:
 ; ARM32:        ldr r0,
 ; ARM32:        ldr r0, [pc, r0]
 ; ARM32-NEXT:   bl my_emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
+; ARM32:        .long my_emutls_v_xyz(GOT_PREL)
 
 entry:
   %call = call i8* @my_emutls_get_address(i8* bitcast (i8** @my_emutls_v_xyz to i8*))
   %0 = bitcast i8* %call to i32*
   %1 = load i32, i32* %0, align 4
   ret i32 %1
 }
 
 @i1 = thread_local global i32 15
 @i2 = external thread_local global i32
 @i3 = internal thread_local global i32 15
 @i4 = hidden thread_local global i32 15
 @i5 = external hidden thread_local global i32
 @s1 = thread_local global i16 15
 @b1 = thread_local global i8 0
 
 define i32 @f1() {
 ; ARM32-LABEL: f1:
 ; ARM32:        ldr r0,
 ; ARM32:        ldr r0, [pc, r0]
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
+; ARM32:        .long __emutls_v.i1(GOT_PREL)
 
 entry:
   %tmp1 = load i32, i32* @i1
   ret i32 %tmp1
 }
 
 define i32* @f2() {
 ; ARM32-LABEL: f2:
 ; ARM32:        ldr r0,
 ; ARM32:        ldr r0, [pc, r0]
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   pop
+; ARM32:        .long __emutls_v.i1(GOT_PREL)
 
 entry:
   ret i32* @i1
 }
 
 define i32 @f3() nounwind {
 ; ARM32-LABEL: f3:
 ; ARM32:        ldr r0,
 ; ARM32:        ldr r0, [pc, r0]
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
+; ARM32:        .long __emutls_v.i2(GOT_PREL)
 
 entry:
   %tmp1 = load i32, i32* @i2
   ret i32 %tmp1
 }
 
 define i32* @f4() {
 ; ARM32-LABEL: f4:
 ; ARM32:        ldr r0,
 ; ARM32:        ldr r0, [pc, r0]
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   pop
+; ARM32:        .long __emutls_v.i2(GOT_PREL)
 
 entry:
   ret i32* @i2
 }
 
 define i32 @f5() nounwind {
 ; ARM32-LABEL: f5:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
+; ARM32:        .long __emutls_v.i3-
 
 entry:
   %tmp1 = load i32, i32* @i3
   ret i32 %tmp1
 }
 
 define i32* @f6() {
 ; ARM32-LABEL: f6:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   pop
+; ARM32:        .long __emutls_v.i3-
 
 entry:
   ret i32* @i3
 }
 
 define i32 @f7() {
 ; ARM32-LABEL: f7:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
+; ARM32:        .long __emutls_v.i4-(.LPC
 
 entry:
   %tmp1 = load i32, i32* @i4
   ret i32 %tmp1
 }
 
 define i32* @f8() {
 ; ARM32-LABEL: f8:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   pop
+; ARM32:        .long __emutls_v.i4-(.LPC
 
 entry:
   ret i32* @i4
 }
 
 define i32 @f9() {
 ; ARM32-LABEL: f9:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   ldr r0, [r0]
 
 entry:
   %tmp1 = load i32, i32* @i5
   ret i32 %tmp1
 }
 
 define i32* @f10() {
 ; ARM32-LABEL: f10:
 ; ARM32:        ldr r0,
-; ARM32:        ldr r0, [pc, r0]
+; ARM32:        add r0, pc, r0
 ; ARM32-NEXT:   bl __emutls_get_address(PLT)
 ; ARM32-NEXT:   pop
 
 entry:
   ret i32* @i5
 }
 
 define i16 @f11() {

 entry:
   %tmp1 = load i8, i8* @b1
   %tmp2 = sext i8 %tmp1 to i32
   ret i32 %tmp2
 }
 
 ;;;;;;;;;;;;;; 32-bit __emutls_v. and __emutls_t.
 
-; ARM32       .section .data.rel.local,
+; ARM32:      .data{{$}}
+; ARM32:      .globl __emutls_v.i1
 ; ARM32-LABEL: __emutls_v.i1:
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 0
 ; ARM32-NEXT: .long __emutls_t.i1
 
-; ARM32       .section .rodata,
+; ARM32:      .section .rodata,
 ; ARM32-LABEL: __emutls_t.i1:
 ; ARM32-NEXT: .long 15
 
 ; ARM32-NOT:   __emutls_v.i2
 
-; ARM32       .section .data.rel.local,
+; ARM32:      .data{{$}}
+; ARM32-NOT:  .globl
 ; ARM32-LABEL: __emutls_v.i3:
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 0
 ; ARM32-NEXT: .long __emutls_t.i3
 
-; ARM32       .section .rodata,
+; ARM32:      .section .rodata,
 ; ARM32-LABEL: __emutls_t.i3:
 ; ARM32-NEXT: .long 15
 
-; ARM32       .section .data.rel.local,
+; ARM32:      .data{{$}}
+; ARM32:      .globl __emutls_v.i4
 ; ARM32-LABEL: __emutls_v.i4:
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 4
 ; ARM32-NEXT: .long 0
 ; ARM32-NEXT: .long __emutls_t.i4
 
-; ARM32       .section .rodata,
+; ARM32:      .section .rodata,
 ; ARM32-LABEL: __emutls_t.i4:
 ; ARM32-NEXT: .long 15
 
 ; ARM32-NOT:   __emutls_v.i5:
-; ARM32       .hidden __emutls_v.i5
+; ARM32:      .hidden __emutls_v.i5
 ; ARM32-NOT:   __emutls_v.i5:
 
-; ARM32 .section .data.rel.local,
+; ARM32:      .data{{$}}
+; ARM32:      .globl __emutls_v.s1
 ; ARM32-LABEL: __emutls_v.s1:
 ; ARM32-NEXT: .long 2
 ; ARM32-NEXT: .long 2
 ; ARM32-NEXT: .long 0
 ; ARM32-NEXT: .long __emutls_t.s1
 
 ; ARM32 .section .rodata,
 ; ARM32-LABEL: __emutls_t.s1:
 ; ARM32-NEXT: .short 15
 
-; ARM32 .section .data.rel.local,
+; ARM32:      .data{{$}}
+; ARM32:      .globl __emutls_v.b1
 ; ARM32-LABEL: __emutls_v.b1:
 ; ARM32-NEXT: .long 1
 ; ARM32-NEXT: .long 1
 ; ARM32-NEXT: .long 0
 ; ARM32-NEXT: .long 0
 
 ; ARM32-NOT:   __emutls_t.b1

test/CodeGen/ARM/emutls_generic.ll

 
 ; ARM_32-LABEL:  get_external_x:
 ; ARM_32:        bl __emutls_get_address
 ; ARM_32:        .long __emutls_v.external_x
 ; ARM_32-LABEL:  get_external_y:
 ; ARM_32:        bl __emutls_get_address
 ; ARM_32:        .long __emutls_v.external_y
 ; ARM_32-LABEL:  get_internal_y:
-; ARM_32:      bl __emutls_get_address
-; ARM_32:      .long __emutls_v.internal_y
-; ARM_32-NOT:   __emutls_t.external_x
-; ARM_32-NOT:   __emutls_v.external_x:
-; ARM_32:        .data
+; ARM_32:        bl __emutls_get_address
+; ARM_32:        .long __emutls_v.internal_y
+; ARM_32-NOT:    __emutls_t.external_x
+; ARM_32-NOT:    __emutls_v.external_x:
+; ARM_32:        .data{{$}}
+; ARM_32:        .globl __emutls_v.external_y
 ; ARM_32:        .align 2
 ; ARM_32-LABEL:  __emutls_v.external_y:
 ; ARM_32-NEXT:   .long 1
 ; ARM_32-NEXT:   .long 2
 ; ARM_32-NEXT:   .long 0
 ; ARM_32-NEXT:   .long __emutls_t.external_y
 ; ARM_32:        .section .rodata,
 ; ARM_32-LABEL:  __emutls_t.external_y:
 ; ARM_32-NEXT:   .byte 7
-; ARM_32:        .data
+; ARM_32:        .data{{$}}
+; ARM_32-NOT:    .globl
 ; ARM_32:        .align 2
 ; ARM_32-LABEL:  __emutls_v.internal_y:
 ; ARM_32-NEXT:   .long 8
 ; ARM_32-NEXT:   .long 16
 ; ARM_32-NEXT:   .long 0
 ; ARM_32-NEXT:   .long __emutls_t.internal_y
 ; ARM_32-LABEL:  __emutls_t.internal_y:
 ; ARM_32-NEXT:   .long 9
 ; ARM_32-NEXT:   .long 0

test/CodeGen/X86/emutls-pic.ll

 entry:
   %0 = load i32, i32* @j, align 4
   %1 = load i32, i32* @k, align 4
   %add = add nsw i32 %0, %1
   ret i32 %add
 }
 
 ; X32-LABEL: f5:
-; X32:      movl __emutls_v.j@GOT(%ebx), %eax
+; X32:      leal __emutls_v.j@GOTOFF(%ebx), %eax
 ; X32-NEXT: movl %eax, (%esp)
 ; X32-NEXT: calll __emutls_get_address@PLT
 ; X32-NEXT: movl (%eax), %esi
-; X32-NEXT: movl __emutls_v.k@GOT(%ebx), %eax
+; X32-NEXT: leal __emutls_v.k@GOTOFF(%ebx), %eax
 ; X32-NEXT: movl %eax, (%esp)
 ; X32-NEXT: calll __emutls_get_address@PLT
 ; X32-NEXT: addl (%eax), %esi
 ; X32-NEXT: movl %esi, %eax
 
 ; X64-LABEL: f5:
-; X64:      movq __emutls_v.j@GOTPCREL(%rip), %rdi
+; X64:      leaq __emutls_v.j(%rip), %rdi
 ; X64-NEXT: callq __emutls_get_address@PLT
 ; X64-NEXT: movl (%rax), %ebx
-; X64-NEXT: movq __emutls_v.k@GOTPCREL(%rip), %rdi
+; X64-NEXT: leaq __emutls_v.k(%rip), %rdi
 ; X64-NEXT: callq __emutls_get_address@PLT
 ; X64-NEXT: addl (%rax), %ebx
 ; X64-NEXT: movl %ebx, %eax
 
 ;;;;; 32-bit targets
 
-; X32:      .data
+; X32:      .data{{$}}
+; X32:      .globl __emutls_v.i
 ; X32-LABEL: __emutls_v.i:
 ; X32-NEXT: .long 4
 ; X32-NEXT: .long 4
 ; X32-NEXT: .long 0
 ; X32-NEXT: .long __emutls_t.i
 
 ; X32:      .section .rodata,
 ; X32-LABEL: __emutls_t.i:
 ; X32-NEXT: .long 15
 
-; X32:      .data
+; X32:      .data{{$}}
+; X32-NOT:  .globl
 ; X32-LABEL: __emutls_v.j:
 ; X32-NEXT: .long 4
 ; X32-NEXT: .long 4
 ; X32-NEXT: .long 0
 ; X32-NEXT: .long __emutls_t.j
 
 ; X32:      .section .rodata,
 ; X32-LABEL: __emutls_t.j:
 ; X32-NEXT: .long 42
 
-; X32:      .data
+; X32:      .data{{$}}
+; X32-NOT:  .globl
 ; X32-LABEL: __emutls_v.k:
 ; X32-NEXT: .long 4
 ; X32-NEXT: .long 8
 ; X32-NEXT: .long 0
 ; X32-NEXT: .long 0
 
 ; X32-NOT:   __emutls_t.k:
 
 ;;;;; 64-bit targets
 
-; X64:      .data
+; X64:      .data{{$}}
+; X64:      .globl __emutls_v.i
 ; X64-LABEL: __emutls_v.i:
 ; X64-NEXT: .quad 4
 ; X64-NEXT: .quad 4
 ; X64-NEXT: .quad 0
 ; X64-NEXT: .quad __emutls_t.i
 
 ; X64:      .section .rodata,
 ; X64-LABEL: __emutls_t.i:
 ; X64-NEXT: .long 15
 
-; X64:      .data
+; X64:      .data{{$}}
+; X64-NOT:  .globl
 ; X64-LABEL: __emutls_v.j:
 ; X64-NEXT: .quad 4
 ; X64-NEXT: .quad 4
 ; X64-NEXT: .quad 0
 ; X64-NEXT: .quad __emutls_t.j
 
 ; X64:      .section .rodata,
 ; X64-LABEL: __emutls_t.j:
 ; X64-NEXT: .long 42
 
-; X64:      .data
+; X64:      .data{{$}}
+; X64-NOT:  .globl
 ; X64-LABEL: __emutls_v.k:
 ; X64-NEXT: .quad 4
 ; X64-NEXT: .quad 8
 ; X64-NEXT: .quad 0
 ; X64-NEXT: .quad 0
 
 ; X64-NOT:   __emutls_t.k:

test/CodeGen/X86/emutls_generic.ll

 ; CHECK-LABEL: __emutls_v.external_y:
 ; CHECK-LABEL: __emutls_t.external_y:
 ; CHECK:       __emutls_t.external_y
 ; CHECK-LABEL: __emutls_v.internal_y:
 ; CHECK-LABEL: __emutls_t.internal_y:
 ; CHECK:       __emutls_t.internal_y
 
 ; X86_32-LABEL:  get_external_x:
-; X86_32:        movl __emutls_v.external_x
+; X86_32:        movl __emutls_v.external_x@GOT(%ebx)
 ; X86_32:        calll __emutls_get_address
 ; X86_32-LABEL:  get_external_y:
-; X86_32:        movl __emutls_v.external_y
+; X86_32:        movl __emutls_v.external_y@GOT(%ebx)
 ; X86_32:        calll __emutls_get_address
 ; X86_32-LABEL:  get_internal_y:
-; X86_32:      movl __emutls_v.internal_y
-; X86_32:      calll __emutls_get_address
-; X86_32-NOT:   __emutls_t.external_x
-; X86_32-NOT:   __emutls_v.external_x:
-; X86_32:        .data
+; X86_32:        leal __emutls_v.internal_y@GOTOFF(%ebx)
+; X86_32:        calll __emutls_get_address
+; X86_32-NOT:    __emutls_t.external_x
+; X86_32-NOT:    __emutls_v.external_x:
+; X86_32:        .data{{$}}
+; X86_32:        .globl __emutls_v.external_y
 ; X86_32:        .align 4
 ; X86_32-LABEL:  __emutls_v.external_y:
 ; X86_32-NEXT:   .long 1
 ; X86_32-NEXT:   .long 2
 ; X86_32-NEXT:   .long 0
 ; X86_32-NEXT:   .long __emutls_t.external_y
 ; X86_32:        .section .rodata,
 ; X86_32-LABEL:  __emutls_t.external_y:
 ; X86_32-NEXT:   .byte 7
-; X86_32:        .data
+; X86_32:        .data{{$}}
+; X86_32-NOT:    .globl
 ; X86_32:        .align 4
 ; X86_32-LABEL:  __emutls_v.internal_y:
 ; X86_32-NEXT:   .long 8
 ; X86_32-NEXT:   .long 16
 ; X86_32-NEXT:   .long 0
 ; X86_32-NEXT:   .long __emutls_t.internal_y
 ; X86_32-LABEL:  __emutls_t.internal_y:
 ; X86_32-NEXT:   .quad 9
 ; X86_64-LABEL:  get_external_x:
-; X86_64:      __emutls_v.external_x
-; X86_64:      __emutls_get_address
+; X86_64:        __emutls_v.external_x@GOTPCREL(%rip)
+; X86_64:        __emutls_get_address
 ; X86_64-LABEL:  get_external_y:
-; X86_64:      __emutls_v.external_y
-; X86_64:      __emutls_get_address
+; X86_64:        __emutls_v.external_y@GOTPCREL(%rip)
+; X86_64:        __emutls_get_address
 ; X86_64-LABEL:  get_internal_y:
-; X86_64:      __emutls_v.internal_y
-; X86_64:      __emutls_get_address
-; X86_64-NOT:   __emutls_t.external_x
-; X86_64-NOT:   __emutls_v.external_x:
+; X86_64:        __emutls_v.internal_y(%rip)
+; X86_64:        __emutls_get_address
+; X86_64-NOT:    __emutls_t.external_x
+; X86_64-NOT:    __emutls_v.external_x:
+; X86_64:        .globl __emutls_v.external_y
 ; X86_64:        .align 8
 ; X86_64-LABEL:  __emutls_v.external_y:
 ; X86_64-NEXT:   .quad 1
 ; X86_64-NEXT:   .quad 2
 ; X86_64-NEXT:   .quad 0
 ; X86_64-NEXT:   .quad __emutls_t.external_y
 ; X86_64-NOT:    __emutls_v.external_x:
 ; X86_64:        .section .rodata,
 ; X86_64-LABEL:  __emutls_t.external_y:
 ; X86_64-NEXT:   .byte 7
-; X86_64:        .data
+; X86_64:        .data{{$}}
+; X86_64-NOT:    .globl
 ; X86_64:        .align 8
 ; X86_64-LABEL:  __emutls_v.internal_y:
 ; X86_64-NEXT:   .quad 8
 ; X86_64-NEXT:   .quad 16
 ; X86_64-NEXT:   .quad 0
 ; X86_64-NEXT:   .quad __emutls_t.internal_y
 ; X86_64:        .section .rodata,
 ; X86_64-LABEL:  __emutls_t.internal_y:
 ; X86_64-NEXT:   .quad 9