[OPENMP] Codegen for threadprivate variables

[ABataev]

Ping

[• fraggamuffin]

I kept getting an error in the build, which is directly traced back to this
change:
llvm[4]: Compiling CGDeclCXX.cpp for Debug+Asserts build
/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:
In member function 'void
clang::CodeGen::CodeGenModule::EmitCXXOMPThreadPrivateInitFunction(const
clang::VarDecl&, llvm::Constant*, bool, bool)':
/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:600:60:
error: no matching function for call to
'llvm::FunctionType::get(llvm::PointerType*&, <brace-enclosed initializer
list>, bool)'

/*isVarArg*/

false)->getPointerTo();

^

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:600:60:
note: candidates are:
In file included from
/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/Constants.h:28:0,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/ConstantFolder.h:20,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/IRBuilder.h:22,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGBuilder.h:13,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CodeGenFunction.h:17,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:14:
/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/DerivedTypes.h:105:24:
note: static llvm::FunctionType* llvm::FunctionType::get(llvm::Type*,
llvm::ArrayRef<llvm::Type*>, bool)

static FunctionType *get(Type *Result,
                     ^

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/DerivedTypes.h:105:24:
note: no known conversion for argument 2 from '<brace-enclosed
initializer list>' to 'llvm::ArrayRef<llvm::Type*>'

I am going to try again.

[ABataev]

Replaced initializer lists in function calls by arrays.

[ABataev]

Michael, try updated patch. I fixed possible problems.

Best regards,

Alexey Bataev

Software Engineer
Intel Compiler Team

18 Июнь 2014 г. 13:31:56, Michael Wong писал:

I kept getting an error in the build, which is directly traced back to this
change:
llvm[4]: Compiling CGDeclCXX.cpp for Debug+Asserts build
/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:
In member function 'void
clang::CodeGen::CodeGenModule::EmitCXXOMPThreadPrivateInitFunction(const
clang::VarDecl&, llvm::Constant*, bool, bool)':
/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:600:60:
error: no matching function for call to
'llvm::FunctionType::get(llvm::PointerType*&, <brace-enclosed initializer
list>, bool)'

/*isVarArg*/

false)->getPointerTo();

^

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:600:60:
note: candidates are:
In file included from
/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/Constants.h:28:0,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/ConstantFolder.h:20,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/IRBuilder.h:22,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGBuilder.h:13,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CodeGenFunction.h:17,

from

/nfs/terran/home/michaelw/clang-dev-new/llvm/tools/clang/lib/CodeGen/CGDeclCXX.cpp:14:
/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/DerivedTypes.h:105:24:
note: static llvm::FunctionType* llvm::FunctionType::get(llvm::Type*,
llvm::ArrayRef<llvm::Type*>, bool)

static FunctionType *get(Type *Result,
                     ^

/nfs/terran/home/michaelw/clang-dev-new/llvm/include/llvm/IR/DerivedTypes.h:105:24:
note: no known conversion for argument 2 from '<brace-enclosed
initializer list>' to 'llvm::ArrayRef<llvm::Type*>'

I am going to try again.

http://reviews.llvm.org/D4002

[ABataev]

Ping

[• ejstotzer]

These changes look good to me, but I am new to clang. I did not attempt a build with these changes applied.

[• fraggamuffin]

Builds fine. Change looks OK, but as this is codegen, will need more
established eyes like Hal.

[ABataev]

John (rjmccall), could you please review this patch?
Best regards,
Alexey Bataev.

[ABataev]

Ping!

[ABataev]

Ping!

[ABataev]

Ping!

[ABataev]

Ping

[• cbergstrom]

  Original Message  
From: Alexey Bataev
Sent: Monday, September 22, 2014 10:34
To: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Reply To: [e-mail]
Cc: [e-mail]
Subject: Re: [PATCH] [OPENMP] Codegen for threadprivate variables

Ping

http://reviews.llvm.org/D4002

It's a bit sad this community cares so little when there's good code to be merged. ‎

[hfinkel]

lib/CodeGen/CGDeclCXX.cpp

547

Like John said for EmitOMPSerialCall, this seems like it belongs in CGOpenMPRuntime.cpp.

620

Why not use the SourceLocation of the variable's initializer?

lib/Parse/Parser.cpp

627

Is this a separate change?

lib/Serialization/ASTReaderDecl.cpp

2186

This is a separate change.

[ABataev]

Ping

lib/CodeGen/CGDeclCXX.cpp

547

I'll rework this to remove direct calls to runtime functions.

620

Will be fixed.

lib/Parse/Parser.cpp

627

Nope, this is required for codegen, without it the declaration is not passed to codegen consumer.

lib/Serialization/ASTReaderDecl.cpp

2186

Nope, it's a part of codegen, without this change codegen for construct loaded from .pch does not work.

[hfinkel]

• Original Message -----

From: "Alexey Bataev" <[e-mail]>
To: "a bataev" <[e-mail]>, [e-mail], [e-mail], [e-mail],
[e-mail], [e-mail], [e-mail], [e-mail]
Cc: [e-mail]
Sent: Tuesday, September 23, 2014 5:47:21 AM
Subject: Re: [PATCH] [OPENMP] Codegen for threadprivate variables

[snip]

To be clear, regarding these changes below, is either of them separately testable? If so, please go ahead and commit them now with a test.

-Hal

Comment at: lib/Parse/Parser.cpp:627
@@ -626,4 +626,3 @@

case tok::annot_pragma_openmp:

• ParseOpenMPDeclarativeDirective();
• return DeclGroupPtrTy();

+ return ParseOpenMPDeclarativeDirective();

case tok::annot_pragma_ms_pointers_to_members:

---

hfinkel wrote:

Is this a separate change?

Nope, this is required for codegen, without it the declaration is not
passed to codegen consumer.

Comment at: lib/Serialization/ASTReaderDecl.cpp:2186
@@ -2185,3 +2185,3 @@

isa<ObjCImplDecl>(D) ||

• isa<ImportDecl>(D))

+ isa<ImportDecl>(D) || isa<OMPThreadPrivateDecl>(D))

return true;

---

hfinkel wrote:

This is a separate change.

Nope, it's a part of codegen, without this change codegen for
construct loaded from .pch does not work.

http://reviews.llvm.org/D4002

[ABataev]

Update after review

[hfinkel]

LGTM.

lib/CodeGen/CodeGenModule.cpp

1390

We should add a comment here explaining the separation of concerns between what EmitGlobalVarDefinition does, what EmitOMPThreadPrivateVarDecl does, and why they're both called here).

[rjmccall]

Okay, the big problem here is that the AST representation for threadprivate does not make any sense. This absolutely needs to be recorded on the target declaration itself somehow. I'm sorry this wasn't caught during the patch which added the Sema work.

I would suggest doing this by adding an implicit attribute to the decl. In order to work correctly with PCH/modules, you will potentially need to serialize what's called an "update record" to ensure that chained PCH files which add threadprivate to an existing var from a previous module include the added attribute.

This should also eliminate the need for these "is this a threadprivate variable" side tables.

lib/CodeGen/CGOpenMPRuntime.cpp

199

You have the comments backwards here.

201

Please don't make separate variables called kmpcCtorTy and kmpcCCtorTy. kmpcCopyCtorTy, maybe?

260

Please document the expectations about this cache variable. It's enough to say that it's a pointer's worth of storage that's reserved for use by the OpenMP runtime.

263

Why is this lookup by name, rather than by the canonical decl pointer?

270

Don't create a variable with non-internal linkage and a meaningless name. Two different translation units can use the same meaningless name for the caches for different variables.

If you're going to stick with a meaningless name, you need to use internal linkage for the cache variable.

Alternatively, if you use a meaningful name (perhaps based on the mangled name of the cached variable?), you can potentially share caches across translation units. The optimal sharing strategy depends on the linkage of the original global variable: if it's internal, you should use internal linkage with default visibility (because the cache must not be shared with a variable from a different file), and otherwise you should use linkonce_odr linkage and hidden visibility (so that the cache will be shared as broadly as possible without introducing unnecessary work for the dynamic linker).

292

Comment your parameters here, please. Especially Addr.

294

You should pass in the result of isDestructedType() instead of repeatedly recomputing it.

298

Great place for a comment explaining what each of these functions does. For example, this function re-emits the declaration's initializer into the thread-local copy of the variable.

313

Please just do a bitcast here.

323

Document the purpose of this function, please.

lib/CodeGen/CodeGenModule.cpp

1390

I would go so far as to say that EmitGlobalVarDefinition should implicitly handle the threadprivate stuff itself whenever possible.

3348

In C++, you can have non-trivial initializers on global variables of arbitrary type, like so:

void *buffer = malloc(1024);

So you may need to PerformInit regardless of whether RD is null.

3349

Use isDestructedType().

3353

It is really unfortunate that you need an eager global initializer to register a static local variable. Please change this so that it's only registered the first time that the variable declaration is "evaluated", within the initializer guards.

That should be straightforward: we'll never emit a static local variable before we've fully processed the function body (and hence seen any openmp pragmas lurking within).

lib/CodeGen/ModuleBuilder.cpp

108

Okay, at the very least, this should be guarded by the OpenMP language option. But I think it would be better to handle this directly in EmitGlobalVarDefinition, when a var is already known to be threadprivate during its emission, which should include all C++ static member cases. It's only true global-scoped variables where CodeGen might process the definition before knowing that it's supposed to be threadprivate.

lib/Serialization/ASTReaderDecl.cpp

2186

Please follow the pattern of putting each decl kind on its own line.

[rjmccall]

lib/CodeGen/CGOpenMPRuntime.cpp

346

As elsewhere, please rename CCtor to be more easily distinguishable from Ctor.

Please also document what the purpose of this function is and why it's okay to always pass null here.

[ABataev]

Hal, John, Thanks for the review!
I'll get back to you soon with the updated patch.

lib/CodeGen/CGOpenMPRuntime.cpp

199

I'll remove it.

201

Ok

260

Ok

263

VarDecl is the redeclarable construct and it may have several instances. MangledName is the same for all redeclarations. I'll try to improve it.

270

Agree, my fault. Earlier there was meaningful name, but it was removed somehow during previous updates.

292

Ok

294

Agree. I'll rework this

298

Agree, I'll add comments

313

Ok

323

Ok

346

Ok

lib/CodeGen/CodeGenModule.cpp

1390

Ok, I'll rework this. I'll add an attribute for thread private variables to handle it in GlobalVarDefinitions.

3348

I'll fix this

3349

Ok

3353

Agree, I'll try to fix this.

lib/CodeGen/ModuleBuilder.cpp

108

Ok

lib/Serialization/ASTReaderDecl.cpp

2186

Ok

[ABataev]

Reworked patch. Added attribute OMPThreadPrivateAttr to simplify codegen. Fixed problem with the codegen for static locals

[rjmccall]

Looks great! A few comments inline, and then I think it's ready to go.

lib/CodeGen/CGOpenMPRuntime.cpp

222

Please use

Twine(...) + ".cache."

instead of calling the ctor directly.

269

Can we avoid having to recompute isConstantInitializer? We should know this from context.

lib/CodeGen/CGOpenMPRuntime.h

168

Comment: it's a map from the unique names to the variables, not the other way around. Along those same lines, please name it something like InternalVars; the names are the keys, and OMP is implicit from context.

Using a StringMap here isn't the most efficient thing when you're hashing on a place where you have a declaration, but it's okay, and if you think it's worthwhile to reuse the same StringMap for different uses this way, fine. (But please document all the different things that get put in it!)

Instead of holding a naked llvm::GlobalVariable*, though, please use an llvm::AssertingVH<llvm::Constant>, which will correctly be updated if something needs to replaceAllUsesWith the original global variable you created.

lib/Sema/SemaOpenMP.cpp

852 ↗

(On Diff #15394)

If you want to get PCH / modules correct when the global decl has been separated from the threadprivate pragma (which might be an unnecessary corner case — up to you), you'll need to add a update record.

The way you do that is to:

1. check to see if the decl you're modifying came from an AST file, and if so:
2. add a new virtual method to ASTMutationListener,
3. implement it in ASTWriter by adding an update record to the AST, and then
4. handle that update record in ASTReader by adding the attribute retroactively (to all subsequent redeclarations).

[ABataev]

Updated patch after review.

[rjmccall]

You never actually call DeclarationMarkedOpenMPThreadPrivate in the first place. The right place to do that is at the point when you add the attribute.

You also need to test that this works. The basic idea is that the first PCH file should declare the variable, the second should declare it to be thread-private, and the third uses it (in some way that verifies that it was marked thread-private; preferably using a Sema check, but an IRGen test is also okay). You can look at test/PCH/chain-categories.m and test/PCH/chain-implicit-definition.cpp for examples of how to do that. (The test should go in test/PCH, not test/OpenMP.)

[ABataev]

Fix after review.

[rjmccall]

Awesome, thanks. Looks great.

[ABataev]

Closed by commit rL221663 (authored by @ABataev).

lib/AST/ASTContext.cpp

if (!VD->isFileVarDecl())
       return false;
     // Global named register variables (GNU extension) are never emitted.
     if (VD->getStorageClass() == SC_Register)
       return false;
   } else if (const FunctionDecl *FD = dyn_cast<FunctionDecl>(D)) {
     // We never need to emit an uninstantiated function template.
     if (FD->getTemplatedKind() == FunctionDecl::TK_FunctionTemplate)
       return false;
-  } else
+  } else if (isa<OMPThreadPrivateDecl>(D))
+    return true;
+  else
     return false;
 
   // If this is a member of a class template, we do not need to emit it.
   if (D->getDeclContext()->isDependentContext())
     return false;
 
   // Weak references don't produce any output by themselves.
   if (D->hasAttr<WeakRefAttr>())

lib/CodeGen/CGDecl.cpp

 #include "CodeGenFunction.h"
 #include "CGDebugInfo.h"
 #include "CGOpenCLRuntime.h"
 #include "CodeGenModule.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/CharUnits.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclObjC.h"
+#include "clang/AST/DeclOpenMP.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CGFunctionInfo.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/Type.h"
void CodeGenFunction::EmitDecl(const Decl &D) {
   case Decl::Function:  // void X();
   case Decl::Record:    // struct/union/class X;
   case Decl::Enum:      // enum X;
   case Decl::EnumConstant: // enum ? { X = ? }
   case Decl::CXXRecord: // struct/union/class X; [C++]
   case Decl::StaticAssert: // static_assert(X, ""); [C++0x]
   case Decl::Label:        // __label__ x;
   case Decl::Import:
-  case Decl::OMPThreadPrivate:
   case Decl::Empty:
     // None of these decls require codegen support.
     return;
-
+  case Decl::OMPThreadPrivate:
+    CGM.EmitOMPThreadPrivateDecl(cast<OMPThreadPrivateDecl>(&D));
+    return;
   case Decl::NamespaceAlias:
     if (CGDebugInfo *DI = getDebugInfo())
         DI->EmitNamespaceAlias(cast<NamespaceAliasDecl>(D));
     return;
   case Decl::Using:          // using X; [C++]
     if (CGDebugInfo *DI = getDebugInfo())
         DI->EmitUsingDecl(cast<UsingDecl>(D));
     return;

lib/CodeGen/CGDeclCXX.cpp

 //===--- CGDeclCXX.cpp - Emit LLVM Code for C++ declarations --------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This contains code dealing with code generation of C++ declarations
 //
 //===----------------------------------------------------------------------===//
 
 #include "CodeGenFunction.h"
 #include "CGCXXABI.h"
 #include "CGObjCRuntime.h"
+#include "CGOpenMPRuntime.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/IR/CallSite.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/Support/Path.h"
 
 using namespace clang;
 using namespace CodeGen;
 
 static void EmitDeclInit(CodeGenFunction &CGF, const VarDecl &D,
                          llvm::Constant *DeclPtr) {
llvm::Function *CodeGenFunction::generateDestroyHelper(
   StartFunction(VD, getContext().VoidTy, fn, FI, args);
 
   emitDestroy(addr, type, destroyer, useEHCleanupForArray);
   
   FinishFunction();
   
   return fn;
 }
+
+void CodeGenModule::EmitCXXOMPThreadPrivateInitFunction(const VarDecl &VD,
+                                                        llvm::Constant *Addr,
+                                                        bool PerformInit,
+                                                        bool PerformDestroy) {
+  QualType ASTTy = VD.getType();
+
+  llvm::Value *Ctor = nullptr, *CCtor = nullptr, *Dtor = nullptr;
+  if (PerformInit) {
+    CodeGenFunction CtorCGF(*this);
+    FunctionArgList Args;
+    ImplicitParamDecl Dst(Context, /*DC*/ nullptr, SourceLocation(),
+                          /*Id*/ nullptr, getContext().VoidPtrTy);
+    Args.push_back(&Dst);
+
+    const CGFunctionInfo &FI = getTypes().arrangeFreeFunctionDeclaration(
+        getContext().VoidPtrTy, Args, FunctionType::ExtInfo(),
+        /*isVariadic*/ false);
+    auto FTy = getTypes().GetFunctionType(FI);
+    auto Fn =
+        CreateGlobalInitOrDestructFunction(*this, FTy, ".__kmpc_global_ctor_.");
+    CtorCGF.StartFunction(GlobalDecl(), getContext().VoidPtrTy, Fn, FI, Args,
+                          SourceLocation());
+    auto Arg =
+        CtorCGF.EmitScalarConversion(Fn->arg_begin(), getContext().VoidPtrTy,
+                                     getContext().getPointerType(ASTTy));
+    CtorCGF.EmitAnyExprToMem(VD.getAnyInitializer(), Arg,
+                             VD.getAnyInitializer()->getType().getQualifiers(),
+                             /*IsInitializer*/ true);
+    CtorCGF.Builder.CreateStore(Fn->arg_begin(), CtorCGF.ReturnValue);
+    CtorCGF.FinishFunction();
+    Ctor = Fn;
+  }
+  if (PerformDestroy) {
+    CodeGenFunction DtorCGF(*this);
+    FunctionArgList Args;
+    ImplicitParamDecl Dst(Context, /*DC*/ nullptr, SourceLocation(),
+                          /*Id*/ nullptr, getContext().VoidPtrTy);
+    Args.push_back(&Dst);
+
+    const CGFunctionInfo &FI = getTypes().arrangeFreeFunctionDeclaration(
+        getContext().VoidTy, Args, FunctionType::ExtInfo(), false);
+    auto FTy = getTypes().GetFunctionType(FI);
+    auto Fn =
+        CreateGlobalInitOrDestructFunction(*this, FTy, ".__kmpc_global_dtor_.");
+    DtorCGF.StartFunction(GlobalDecl(), getContext().VoidTy, Fn, FI, Args,
+                          SourceLocation());
+    DtorCGF.emitDestroy(Fn->arg_begin(), ASTTy,
+                        DtorCGF.getDestroyer(ASTTy.isDestructedType()),
+                        DtorCGF.needsEHCleanup(ASTTy.isDestructedType()));
+    DtorCGF.FinishFunction();
+    Dtor = Fn;
+  }
+  llvm::Type *CCtorTyArgs[] = {VoidPtrTy, VoidPtrTy};
+  auto CCtorTy = llvm::FunctionType::get(VoidPtrTy, CCtorTyArgs,
+                                         /*isVarArg*/ false)->getPointerTo();
+  CCtor = llvm::Constant::getNullValue(CCtorTy);
+  if (Ctor == nullptr) {
+    auto CtorTy = llvm::FunctionType::get(VoidPtrTy, VoidPtrTy,
+                                          /*isVarArg*/ false)->getPointerTo();
+    Ctor = llvm::Constant::getNullValue(CtorTy);
+  }
+  if (Dtor == nullptr) {
+    auto DtorTy = llvm::FunctionType::get(VoidTy, VoidPtrTy,
+                                          /*isVarArg*/ false)->getPointerTo();
+    Dtor = llvm::Constant::getNullValue(DtorTy);
+  }
+  auto InitFunctionTy = llvm::FunctionType::get(VoidTy, /*isVarArg*/ false);
+  auto InitFunction = CreateGlobalInitOrDestructFunction(
+      *this, InitFunctionTy, ".__omp_threadprivate_init_.");
+  CodeGenFunction InitCGF(*this);
+  FunctionArgList ArgList;
+  InitCGF.StartFunction(GlobalDecl(), getContext().VoidTy, InitFunction,
+                        getTypes().arrangeNullaryFunction(), ArgList,
+                        SourceLocation());
+  // Call kmp_int32 __kmpc_global_thread_num(&loc) to init OpenMP runtime
+  // library.
+  InitCGF.EmitRuntimeCall(
+      OpenMPRuntime->CreateRuntimeFunction(
+          CGOpenMPRuntime::OMPRTL__kmpc_global_thread_num),
+      OpenMPRuntime->EmitOpenMPUpdateLocation(InitCGF, VD.getLocation()));
+  // Call __kmpc_threadprivate_register(&loc, &var, ctor, cctor/*NULL*/, dtor)
+  // to register constructor/destructor for variable.
+  llvm::Value *Args[] = {
+      OpenMPRuntime->EmitOpenMPUpdateLocation(InitCGF, VD.getLocation()),
+      InitCGF.Builder.CreatePointerCast(Addr, VoidPtrTy), Ctor, CCtor, Dtor};
+  InitCGF.EmitRuntimeCall(
+      OpenMPRuntime->CreateRuntimeFunction(
+          CGOpenMPRuntime::OMPRTL__kmpc_threadprivate_register),
+      Args);
+  InitCGF.FinishFunction();
+  CXXGlobalInits.push_back(InitFunction);
+}

lib/CodeGen/CGExpr.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "CodeGenFunction.h"
 #include "CGCXXABI.h"
 #include "CGCall.h"
 #include "CGDebugInfo.h"
 #include "CGObjCRuntime.h"
+#include "CGOpenMPRuntime.h"
 #include "CGRecordLayout.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/AST/Attr.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/ADT/Hashing.h"

 static llvm::Value *
 EmitBitCastOfLValueToProperType(CodeGenFunction &CGF,
                                 llvm::Value *V, llvm::Type *IRType,
                                 StringRef Name = StringRef()) {
   unsigned AS = cast<llvm::PointerType>(V->getType())->getAddressSpace();
   return CGF.Builder.CreateBitCast(V, IRType->getPointerTo(AS), Name);
 }
 
+static LValue EmitThreadPrivateVarDeclLValue(
+    CodeGenFunction &CGF, const VarDecl *VD, QualType T, llvm::Value *V,
+    llvm::Type *RealVarTy, CharUnits Alignment, SourceLocation Loc) {
+  V = CGF.CGM.getOpenMPRuntime().getAddrOfThreadPrivate(CGF, V, VD, Loc);
+  V = EmitBitCastOfLValueToProperType(CGF, V, RealVarTy);
+  return CGF.MakeAddrLValue(V, T, Alignment);
+}
+
 static LValue EmitGlobalVarDeclLValue(CodeGenFunction &CGF,
                                       const Expr *E, const VarDecl *VD) {
   QualType T = E->getType();
 
   // If it's thread_local, emit a call to its wrapper function instead.
   if (VD->getTLSKind() == VarDecl::TLS_Dynamic)
     return CGF.CGM.getCXXABI().EmitThreadLocalVarDeclLValue(CGF, VD, T);
 
   llvm::Value *V = CGF.CGM.GetAddrOfGlobalVar(VD);
   llvm::Type *RealVarTy = CGF.getTypes().ConvertTypeForMem(VD->getType());
   V = EmitBitCastOfLValueToProperType(CGF, V, RealVarTy);
   CharUnits Alignment = CGF.getContext().getDeclAlign(VD);
   LValue LV;
+  if (CGF.getLangOpts().OpenMP &&
+      CGF.CGM.getOpenMPRuntime().isThreadPrivateDecl(VD)) {
+    return EmitThreadPrivateVarDeclLValue(CGF, VD, T, V, RealVarTy, Alignment,
+                                          E->getExprLoc());
+  }
   if (VD->getType()->isReferenceType()) {
     llvm::LoadInst *LI = CGF.Builder.CreateLoad(V);
     LI->setAlignment(Alignment.getQuantity());
     V = LI;
     LV = CGF.MakeNaturalAlignAddrLValue(V, T);
   } else {
     LV = CGF.MakeAddrLValue(V, T, Alignment);
   }
if (VD->hasLinkage() || VD->isStaticDataMember())
       return EmitGlobalVarDeclLValue(*this, E, VD);
 
     bool isBlockVariable = VD->hasAttr<BlocksAttr>();
 
     llvm::Value *V = LocalDeclMap.lookup(VD);
     if (!V && VD->isStaticLocal())
       V = CGM.getStaticLocalDeclAddress(VD);
 
+    // Check if variable is threadprivate.
+    if (V && getLangOpts().OpenMP &&
+        CGM.getOpenMPRuntime().isThreadPrivateDecl(VD))
+      return EmitThreadPrivateVarDeclLValue(
+          *this, VD, T, V, getTypes().ConvertTypeForMem(VD->getType()),
+          Alignment, E->getExprLoc());
+
     // Use special handling for lambdas.
     if (!V) {
       if (FieldDecl *FD = LambdaCaptureFields.lookup(VD)) {
         return EmitCapturedFieldLValue(*this, FD, CXXABIThisValue);
       } else if (CapturedStmtInfo) {
         if (const FieldDecl *FD = CapturedStmtInfo->lookup(VD))
           return EmitCapturedFieldLValue(*this, FD,
                                          CapturedStmtInfo->getContextValue());

lib/CodeGen/CGOpenMPRuntime.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef CLANG_CODEGEN_OPENMPRUNTIME_H
 #define CLANG_CODEGEN_OPENMPRUNTIME_H
 
 #include "clang/AST/Type.h"
 #include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/ADT/StringMap.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 
 namespace llvm {
 class AllocaInst;
 class CallInst;
 class GlobalVariable;
 class Constant;
 class Function;
 class Module;
 class StructLayout;
 class FunctionType;
 class StructType;
 class Type;
 class Value;
 }
 
 namespace clang {
+class VarDecl;
 
 namespace CodeGen {
-
 class CodeGenFunction;
 class CodeGenModule;
 
 class CGOpenMPRuntime {
 public:
   /// \brief Values for bit flags used in the ident_t to describe the fields.
   /// All enumeric elements are named and described in accordance with the code
   /// from http://llvm.org/svn/llvm-project/openmp/trunk/runtime/src/kmp.h
enum OpenMPLocationFlags {
     /// \brief Implicit barrier in 'for' directive.
     OMP_IDENT_BARRIER_IMPL_FOR = 0x40,
     /// \brief Implicit barrier in 'sections' directive.
     OMP_IDENT_BARRIER_IMPL_SECTIONS = 0xC0,
     /// \brief Implicit barrier in 'single' directive.
     OMP_IDENT_BARRIER_IMPL_SINGLE = 0x140
   };
   enum OpenMPRTLFunction {
-    // Call to void __kmpc_fork_call(ident_t *loc, kmp_int32 argc, kmpc_micro
-    // microtask, ...);
+    /// \brief Call to void __kmpc_fork_call(ident_t *loc, kmp_int32 argc,
+    /// kmpc_micro microtask, ...);
     OMPRTL__kmpc_fork_call,
-    // Call to kmp_int32 kmpc_global_thread_num(ident_t *loc);
-    OMPRTL__kmpc_global_thread_num
+    /// \brief Call to kmp_int32 __kmpc_global_thread_num(ident_t *loc);
+    OMPRTL__kmpc_global_thread_num,
+    /// \brief Call to void *__kmpc_threadprivate_cached(ident_t *loc,
+    /// kmp_int32 global_tid, void *data, size_t size, void ***cache);
+    OMPRTL__kmpc_threadprivate_cached,
+    /// \brief Call to void __kmpc_threadprivate_register( ident_t *,
+    /// void *data, kmpc_ctor ctor, kmpc_cctor cctor, kmpc_dtor dtor);
+    OMPRTL__kmpc_threadprivate_register
   };
 
 private:
   CodeGenModule &CGM;
   /// \brief Default const ident_t object used for initialization of all other
   /// ident_t objects.
   llvm::Constant *DefaultOpenMPPSource;
   /// \brief Map of flags and corrsponding default locations.
private:
   /// typedef void (kmpc_micro)(kmp_int32 global_tid, kmp_int32 bound_tid,...);
   llvm::FunctionType *Kmpc_MicroTy;
   /// \brief Map of local debug location and functions.
   typedef llvm::DenseMap<llvm::Function *, llvm::Value *> OpenMPLocMapTy;
   OpenMPLocMapTy OpenMPLocMap;
   /// \brief Map of local gtid and functions.
   typedef llvm::DenseMap<llvm::Function *, llvm::Value *> OpenMPGtidMapTy;
   OpenMPGtidMapTy OpenMPGtidMap;
+  /// \brief Set of declarations marked as threadprivate.
+  typedef llvm::SmallPtrSet<const VarDecl *, 16> OpenMPThreadPrivateVarsTy;
+  OpenMPThreadPrivateVarsTy OpenMPThreadPrivateVars;
+  /// \brief Map of threadprivate vars and corresponding cache storages.
+  typedef llvm::StringMap<llvm::Constant *> OpenMPThreadPrivateMapTy;
+  OpenMPThreadPrivateMapTy OpenMPThreadPrivateMap;
+
+  /// \brief If the specified mangled name is not in the module, create and
+  /// return threadprivate cache object.
+  /// \param D Threadprivate variable.
+  /// \param Addr Original global variable.
+  /// \return Cache variable for the specified threadprivate.
+  llvm::Constant *getOrCreateThreadPrivateCache(const VarDecl *VD,
+                                                llvm::Value *Addr);
 
 public:
   CGOpenMPRuntime(CodeGenModule &CGM);
   ~CGOpenMPRuntime() {}
 
   /// \brief Cleans up references to the objects in finished function.
   /// \param CGF Reference to finished CodeGenFunction.
   ///
   void FunctionFinished(CodeGenFunction &CGF);
 
   /// \brief Emits object of ident_t type with info for source location.
   /// \param CGF Reference to current CodeGenFunction.
   /// \param Loc Clang source location.
   /// \param Flags Flags for OpenMP location.
   ///
   llvm::Value *
   EmitOpenMPUpdateLocation(CodeGenFunction &CGF, SourceLocation Loc,
                            OpenMPLocationFlags Flags = OMP_IDENT_KMPC);
public:
 
   /// \brief Returns pointer to kmpc_micro type;
   llvm::Type *getKmpc_MicroPointerTy();
 
   /// \brief Returns specified OpenMP runtime function.
   /// \param Function OpenMP runtime function.
   /// \return Specified function.
   llvm::Constant *CreateRuntimeFunction(OpenMPRTLFunction Function);
+
+  /// \brief Registers variable as threadprivate.
+  /// \param D ThreadpVariable declration to be checked.
+  void addThreadPrivateDecl(const VarDecl *VD);
+
+  /// \brief Checks if the specified declaration or any redeclarations are
+  /// marked as threadprivate.
+  /// \param D Variable declration to be checked.
+  /// \return true if any redeclrations of the specified declaration are marked
+  /// as threadprivate.
+  bool isThreadPrivateDecl(const VarDecl *VD) const;
+
+  /// \brief Returns address of the threadprivate variable for the current
+  /// thread.
+  /// \param CGF Reference to current CodeGenFunction.
+  /// \param Addr Address of the original variable.
+  /// \param D Threadprivate variable.
+  /// \param Loc Location of the reference to threadprivate var.
+  /// \return Address of the threadprivate variable for the current thread.
+  llvm::Value *getAddrOfThreadPrivate(CodeGenFunction &CGF, llvm::Value *Addr,
+                                      const VarDecl *VD, SourceLocation Loc);
 };
 }
 }
 
 #endif

lib/CodeGen/CGOpenMPRuntime.cpp

CGOpenMPRuntime::CreateRuntimeFunction(OpenMPRTLFunction Function) {
   llvm::Constant *RTLFn = nullptr;
   switch (Function) {
   case OMPRTL__kmpc_fork_call: {
     // Build void __kmpc_fork_call(ident_t *loc, kmp_int32 argc, kmpc_micro
     // microtask, ...);
     llvm::Type *TypeParams[] = { getIdentTyPointerTy(), CGM.Int32Ty,
                                  getKmpc_MicroPointerTy() };
     llvm::FunctionType *FnTy =
-        llvm::FunctionType::get(CGM.VoidTy, TypeParams, true);
+        llvm::FunctionType::get(CGM.VoidTy, TypeParams, /*isVarArg*/ true);
     RTLFn = CGM.CreateRuntimeFunction(FnTy, "__kmpc_fork_call");
     break;
   }
   case OMPRTL__kmpc_global_thread_num: {
     // Build kmp_int32 __kmpc_global_thread_num(ident_t *loc);
     llvm::Type *TypeParams[] = { getIdentTyPointerTy() };
     llvm::FunctionType *FnTy =
-        llvm::FunctionType::get(CGM.Int32Ty, TypeParams, false);
+        llvm::FunctionType::get(CGM.Int32Ty, TypeParams, /*isVarArg*/ false);
     RTLFn = CGM.CreateRuntimeFunction(FnTy, "__kmpc_global_thread_num");
     break;
   }
+  case OMPRTL__kmpc_threadprivate_cached: {
+    // Build void *__kmpc_threadprivate_cached(ident_t *loc,
+    // kmp_int32 global_tid, void *data, size_t size, void ***cache);
+    llvm::Type *TypeParams[] = {getIdentTyPointerTy(), CGM.Int32Ty,
+                                CGM.VoidPtrTy, CGM.SizeTy,
+                                CGM.VoidPtrTy->getPointerTo()->getPointerTo()};
+    llvm::FunctionType *FnTy =
+        llvm::FunctionType::get(CGM.VoidPtrTy, TypeParams, /*isVarArg*/ false);
+    RTLFn = CGM.CreateRuntimeFunction(FnTy, "__kmpc_threadprivate_cached");
+    break;
+  }
+  case OMPRTL__kmpc_threadprivate_register: {
+    // Build void __kmpc_threadprivate_register(ident_t *, void *data,
+    // kmpc_ctor ctor, kmpc_cctor cctor, kmpc_dtor dtor);
+    // typedef void *(*kmpc_ctor)(void *);
+    auto KmpcCtorTy =
+        llvm::FunctionType::get(CGM.VoidPtrTy, CGM.VoidPtrTy,
+                                /*isVarArg*/ false)->getPointerTo();
+    // typedef void (*kmpc_dtor)(void *);
+    llvm::Type *KmpcCCtorTyArgs[] = {CGM.VoidPtrTy, CGM.VoidPtrTy};
+    auto KmpcCCtorTy =
+        llvm::FunctionType::get(CGM.VoidPtrTy, KmpcCCtorTyArgs,
+                                /*isVarArg*/ false)->getPointerTo();
+    // typedef void *(*kmpc_cctor)(void *, void *);
+    auto KmpcDtorTy =
+        llvm::FunctionType::get(CGM.VoidTy, CGM.VoidPtrTy, /*isVarArg*/ false)
+            ->getPointerTo();
+    llvm::Type *FnTyArgs[] = {getIdentTyPointerTy(), CGM.VoidPtrTy, KmpcCtorTy,
+                              KmpcCCtorTy, KmpcDtorTy};
+    auto FnTy = llvm::FunctionType::get(CGM.VoidTy, FnTyArgs,
+                                        /*isVarArg*/ false);
+    RTLFn = CGM.CreateRuntimeFunction(FnTy, "__kmpc_threadprivate_register");
+    break;
+  }
   }
   return RTLFn;
 }
+
+void CGOpenMPRuntime::addThreadPrivateDecl(const VarDecl *VD) {
+  OpenMPThreadPrivateVars.insert(VD);
+}
+
+bool CGOpenMPRuntime::isThreadPrivateDecl(const VarDecl *VD) const {
+  auto CurDecl = VD->getMostRecentDecl();
+  while (CurDecl) {
+    if (OpenMPThreadPrivateVars.count(CurDecl) > 0)
+      return true;
+    CurDecl = CurDecl->getPreviousDecl();
+  }
+  return false;
+}
+
+llvm::Constant *
+CGOpenMPRuntime::getOrCreateThreadPrivateCache(const VarDecl *VD,
+                                               llvm::Value *Addr) {
+  StringRef MangledName = CGM.getMangledName(VD);
+  // Lookup the entry, lazily creating it if necessary.
+  auto &Entry = OpenMPThreadPrivateMap[MangledName];
+  if (Entry == nullptr) {
+    // Create cache memory for threadprivate variable void **Var.cache;
+    Entry = new llvm::GlobalVariable(
+        CGM.getModule(), CGM.Int8PtrPtrTy, /*IsConstant*/ false,
+        llvm::GlobalValue::CommonLinkage,
+        llvm::Constant::getNullValue(CGM.Int8PtrPtrTy), ".cache.",
+        dyn_cast<llvm::GlobalVariable>(Addr));
+  }
+
+  return Entry;
+}
+
+llvm::Value *CGOpenMPRuntime::getAddrOfThreadPrivate(CodeGenFunction &CGF,
+                                                     llvm::Value *Addr,
+                                                     const VarDecl *VD,
+                                                     SourceLocation Loc) {
+  auto VarTy = Addr->getType()->getPointerElementType();
+  llvm::Value *Args[] = {EmitOpenMPUpdateLocation(CGF, Loc),
+                         GetOpenMPGlobalThreadNum(CGF, Loc),
+                         CGF.Builder.CreatePointerCast(Addr, CGM.Int8PtrTy),
+                         CGM.getSize(CGM.GetTargetTypeStoreSize(VarTy)),
+                         getOrCreateThreadPrivateCache(VD, Addr)};
+  return CGF.EmitRuntimeCall(
+      CreateRuntimeFunction(OMPRTL__kmpc_threadprivate_cached), Args);
+}

lib/CodeGen/CodeGenModule.h

public:
   void addDeferredVTable(const CXXRecordDecl *RD) {
     DeferredVTables.push_back(RD);
   }
 
   /// Emit code for a singal global function or var decl. Forward declarations
   /// are emitted lazily.
   void EmitGlobal(GlobalDecl D);
 
+  /// \brief Emit a code for threadprivate directive.
+  /// \param D Threadprivate declaration.
+  void EmitOMPThreadPrivateDecl(const OMPThreadPrivateDecl *D);
+
+  /// \brief Emits the function which registers constructor/destructor for
+  /// the specified threadprivate variable.
+  /// \param VD A threadprivate variable.
+  /// \param Addr The address of the original variable.
+  /// \param PerformInit Need to create initialization function.
+  /// \param PerformDestroy Need to create destroy function.
+  void EmitCXXOMPThreadPrivateInitFunction(const VarDecl &VD,
+                                           llvm::Constant *Addr,
+                                           bool PerformInit,
+                                           bool PerformDestroy);
+
 private:
+  /// \brief Emit a code for threadprivate variable.
+  /// \brief VD Threadprivate variable.
+  void EmitOMPThreadPrivateVarDecl(const VarDecl *VD);
+
   llvm::GlobalValue *GetGlobalValue(StringRef Ref);
 
   llvm::Constant *
   GetOrCreateLLVMFunction(StringRef MangledName, llvm::Type *Ty, GlobalDecl D,
                           bool ForVTable, bool DontDefer = false,
                           llvm::AttributeSet ExtraAttrs = llvm::AttributeSet());
 
   llvm::Constant *GetOrCreateLLVMGlobal(StringRef MangledName,

lib/CodeGen/CodeGenModule.cpp

if (const auto *Method = dyn_cast<CXXMethodDecl>(D)) {
         getVTables().EmitThunks(GD);
 
       return;
     }
 
     return EmitGlobalFunctionDefinition(GD, GV);
   }
 
-  if (const auto *VD = dyn_cast<VarDecl>(D))
-    return EmitGlobalVarDefinition(VD);
+  if (const auto *VD = dyn_cast<VarDecl>(D)) {
+    EmitGlobalVarDefinition(VD);
+    if (getLangOpts().OpenMP && OpenMPRuntime->isThreadPrivateDecl(VD))
+      EmitOMPThreadPrivateVarDecl(VD);
+    return;
+  }
   
   llvm_unreachable("Invalid argument to EmitGlobalDefinition()");
 }
 
 /// GetOrCreateLLVMFunction - If the specified mangled name is not in the
 /// module, create and return an llvm Function with the specified type. If there
 /// is something in the module with the specified name, return it potentially
 /// bitcasted to the right type.
if (clang::Module *Owner = Import->getOwningModule()) {
           Owner->getTopLevelModule()->Name == getLangOpts().CurrentModule)
         break;
     }
 
     ImportedModules.insert(Import->getImportedModule());
     break;
   }
 
+  case Decl::OMPThreadPrivate:
+    EmitOMPThreadPrivateDecl(cast<OMPThreadPrivateDecl>(D));
+    break;
+
   case Decl::ClassTemplateSpecialization: {
     const auto *Spec = cast<ClassTemplateSpecializationDecl>(D);
     if (DebugInfo &&
         Spec->getSpecializationKind() == TSK_ExplicitInstantiationDefinition)
       DebugInfo->completeTemplateDefinition(*Spec);
   }
 
   default:
llvm::Constant *Fields[4] = {
     llvm::ConstantInt::get(Int32Ty, Uuid.substr(0,  8), 16),
     llvm::ConstantInt::get(Int16Ty, Uuid.substr(9,  4), 16),
     llvm::ConstantInt::get(Int16Ty, Uuid.substr(14, 4), 16),
     llvm::ConstantArray::get(llvm::ArrayType::get(Int8Ty, 8), Field3)
   };
 
   return llvm::ConstantStruct::getAnon(Fields);
 }
+
+void CodeGenModule::EmitOMPThreadPrivateVarDecl(const VarDecl *VD) {
+  if ((VD = VD->getDefinition(Context)) != nullptr) {
+    QualType ASTTy = VD->getType();
+    CXXRecordDecl *RD = ASTTy->getBaseElementTypeUnsafe()->getAsCXXRecordDecl();
+    bool PerformInit = RD && VD->getAnyInitializer() != nullptr;
+    bool PerformDestroy = RD && !RD->hasTrivialDestructor();
+    auto DeclPtr = VD->isStaticLocal() ? getStaticLocalDeclAddress(VD)
+                                       : GetAddrOfGlobalVar(VD);
+
+    if (DeclPtr && (PerformInit || PerformDestroy)) {
+      EmitCXXOMPThreadPrivateInitFunction(*VD, DeclPtr, PerformInit,
+                                          PerformDestroy);
+    }
+  }
+}
+
+void CodeGenModule::EmitOMPThreadPrivateDecl(const OMPThreadPrivateDecl *D) {
+  for (auto RefExpr : D->varlists()) {
+    const VarDecl *VD = cast<VarDecl>(cast<DeclRefExpr>(RefExpr)->getDecl());
+    EmitOMPThreadPrivateVarDecl(VD);
+    OpenMPRuntime->addThreadPrivateDecl(VD);
+  }
+}

lib/CodeGen/ModuleBuilder.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/CodeGen/ModuleBuilder.h"
 #include "CGDebugInfo.h"
 #include "CodeGenModule.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/DeclObjC.h"
+#include "clang/AST/DeclOpenMP.h"
 #include "clang/AST/Expr.h"
 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/Module.h"
public:
     /// to (e.g. struct, union, enum, class) is completed. This allows the
     /// client hack on the type, which can occur at any point in the file
     /// (because these can be defined in declspecs).
     void HandleTagDeclDefinition(TagDecl *D) override {
       if (Diags.hasErrorOccurred())
         return;
 
       Builder->UpdateCompletedType(D);
+      // In C++, we may have member functions that need to be emitted at this 
+      // point.
+      if (Ctx->getLangOpts().CPlusPlus && !D->isDependentContext()) {
+        for (auto *M : D->decls())
+          if (isa<OMPThreadPrivateDecl>(M))
+            Builder->EmitTopLevelDecl(M);
+      }
     }
 
     void HandleTagDeclRequiredDefinition(const TagDecl *D) override {
       if (Diags.hasErrorOccurred())
         return;
 
       if (CodeGen::CGDebugInfo *DI = Builder->getModuleDebugInfo())
         if (const RecordDecl *RD = dyn_cast<RecordDecl>(D))

lib/Parse/Parser.cpp

case tok::annot_pragma_redefine_extname:
     return DeclGroupPtrTy();
   case tok::annot_pragma_fp_contract:
     HandlePragmaFPContract();
     return DeclGroupPtrTy();
   case tok::annot_pragma_opencl_extension:
     HandlePragmaOpenCLExtension();
     return DeclGroupPtrTy();
   case tok::annot_pragma_openmp:
-    ParseOpenMPDeclarativeDirective();
-    return DeclGroupPtrTy();
+    return ParseOpenMPDeclarativeDirective();
   case tok::annot_pragma_ms_pointers_to_members:
     HandlePragmaMSPointersToMembers();
     return DeclGroupPtrTy();
   case tok::annot_pragma_ms_vtordisp:
     HandlePragmaMSVtorDisp();
     return DeclGroupPtrTy();
   case tok::annot_pragma_ms_pragma:
     HandlePragmaMSPragma();

lib/Serialization/ASTReaderDecl.cpp

 /// declarations with metadata, etc.
 static bool isConsumerInterestedIn(Decl *D, bool HasBody) {
   // An ObjCMethodDecl is never considered as "interesting" because its
   // implementation container always is.
 
   if (isa<FileScopeAsmDecl>(D) || 
       isa<ObjCProtocolDecl>(D) || 
       isa<ObjCImplDecl>(D) ||
-      isa<ImportDecl>(D))
+      isa<ImportDecl>(D) || isa<OMPThreadPrivateDecl>(D))
     return true;
   if (VarDecl *Var = dyn_cast<VarDecl>(D))
     return Var->isFileVarDecl() &&
            Var->isThisDeclarationADefinition() == VarDecl::Definition;
   if (FunctionDecl *Func = dyn_cast<FunctionDecl>(D))
     return Func->doesThisDeclarationHaveABody() || HasBody;
   
   return false;

test/OpenMP/threadprivate_codegen.cpp

+// RUN: %clang_cc1 -verify -DBODY -fopenmp=libiomp5 -triple x86_64-unknown-unknown -x c++ -emit-llvm %s -fexceptions -fcxx-exceptions -o - | FileCheck %s
+// RUN: %clang_cc1 -fopenmp=libiomp5 -x c++ -std=c++11 -triple x86_64-unknown-unknown -fexceptions -fcxx-exceptions -emit-pch -o %t %s
+// RUN: %clang_cc1 -fopenmp=libiomp5 -x c++ -DBODY -triple x86_64-unknown-unknown -fexceptions -fcxx-exceptions -g -std=c++11 -include-pch %t -verify %s -emit-llvm -o - | FileCheck --check-prefix=CHECK-DEBUG %s
+// expected-no-diagnostics
+#ifndef HEADER
+#define HEADER
+// CHECK-DAG: [[IDENT:%.+]] = type { i32, i32, i32, i32, i8* }
+// CHECK-DAG: [[S1:%.+]] = type { [[INT:i[0-9]+]] }
+// CHECK-DAG: [[S2:%.+]] = type { [[INT]], double }
+// CHECK-DAG: [[S3:%.+]] = type { [[INT]], float }
+// CHECK-DAG: [[S4:%.+]] = type { [[INT]], [[INT]] }
+// CHECK-DAG: [[S5:%.+]] = type { [[INT]], [[INT]], [[INT]] }
+// CHECK-DAG: [[SMAIN:%.+]] = type { [[INT]], double, double }
+// CHECK-DEBUG-DAG: [[IDENT:%.+]] = type { i32, i32, i32, i32, i8* }
+// CHECK-DEBUG-DAG: [[S1:%.+]] = type { [[INT:i[0-9]+]] }
+// CHECK-DEBUG-DAG: [[S2:%.+]] = type { [[INT]], double }
+// CHECK-DEBUG-DAG: [[S3:%.+]] = type { [[INT]], float }
+// CHECK-DEBUG-DAG: [[S4:%.+]] = type { [[INT]], [[INT]] }
+// CHECK-DEBUG-DAG: [[S5:%.+]] = type { [[INT]], [[INT]], [[INT]] }
+// CHECK-DEBUG-DAG: [[SMAIN:%.+]] = type { [[INT]], double, double }
+
+struct S1 {
+  int a;
+  S1()
+      : a(0) {
+  }
+  S1(int a)
+      : a(a) {
+  }
+  S1(const S1 &s) {
+    a = 12 + s.a;
+  }
+  ~S1() {
+    a = 0;
+  }
+};
+
+struct S2 {
+  int a;
+  double b;
+  S2()
+      : a(0) {
+  }
+  S2(int a)
+      : a(a) {
+  }
+  S2(const S2 &s) {
+    a = 12 + s.a;
+  }
+  ~S2() {
+    a = 0;
+  }
+};
+
+struct S3 {
+  int a;
+  float b;
+  S3()
+      : a(0) {
+  }
+  S3(int a)
+      : a(a) {
+  }
+  S3(const S3 &s) {
+    a = 12 + s.a;
+  }
+  ~S3() {
+    a = 0;
+  }
+};
+
+struct S4 {
+  int a, b;
+  S4()
+      : a(0) {
+  }
+  S4(int a)
+      : a(a) {
+  }
+  S4(const S4 &s) {
+    a = 12 + s.a;
+  }
+  ~S4() {
+    a = 0;
+  }
+};
+
+struct S5 {
+  int a, b, c;
+  S5()
+      : a(0) {
+  }
+  S5(int a)
+      : a(a) {
+  }
+  S5(const S5 &s) {
+    a = 12 + s.a;
+  }
+  ~S5() {
+    a = 0;
+  }
+};
+
+// CHECK:      [[CACHE1:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[GS1:@.+]] = internal global [[S1]] zeroinitializer
+// CHECK:      [[DEFAULT_LOC:@.+]] = private unnamed_addr constant [[IDENT]] { i32 0, i32 2, i32 0, i32 0, i8* getelementptr inbounds ([{{[0-9]+}} x i8]* {{@.+}}, i32 0, i32 0) }
+// CHECK-NOT:  {{@.+}} = common global i8** null
+// CHECK:      [[GS2:@.+]] = internal global [[S2]] zeroinitializer
+// CHECK:      [[CACHE2:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[ARR_X:@.+]] = global [2 x [3 x [[S1]]]] zeroinitializer
+// CHECK:      [[CACHE3:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[SM:@.+]] = internal global [[SMAIN]] zeroinitializer
+// CHECK:      [[CACHE4:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[STATIC_S:@.+]] = external global [[S3]]
+// CHECK:      [[CACHE5:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[GS3:@.+]] = external global [[S5]]
+// CHECK:      [[CACHE6:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[ST_INT_ST:@.+]] = linkonce_odr global i32 23
+// CHECK:      [[CACHE7:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[ST_FLOAT_ST:@.+]] = linkonce_odr global float 2.300000e+01
+// CHECK:      [[CACHE8:@[_.0-9a-zA-Z]+]] = common global i8** null
+// CHECK-NEXT: [[ST_S4_ST:@.+]] = linkonce_odr global %struct.S4 zeroinitializer
+// CHECK-NOT: {{@.+}} = common global i8** null
+// There is no cache for gs2 - it is not threadprivate. Check that there is only
+// 8 caches created (for Static::s, gs1, gs3, arr_x, main::sm, ST<int>::st,
+// ST<float>::st, ST<S4>::st)
+// CHECK-DEBUG-DAG: [[GS1:@.+]] = internal global [[S1]] zeroinitializer
+// CHECK-DEBUG-DAG: [[GS2:@.+]] = internal global [[S2]] zeroinitializer
+// CHECK-DEBUG-DAG: [[ARR_X:@.+]] = global [2 x [3 x [[S1]]]] zeroinitializer
+// CHECK-DEBUG-DAG: [[SM:@.+]] = internal global [[SMAIN]] zeroinitializer
+// CHECK-DEBUG-DAG: [[STATIC_S:@.+]] = external global [[S3]]
+// CHECK-DEBUG-DAG: [[GS3:@.+]] = external global [[S5]]
+// CHECK-DEBUG-DAG: [[ST_INT_ST:@.+]] = linkonce_odr global i32 23
+// CHECK-DEBUG-DAG: [[ST_FLOAT_ST:@.+]] = linkonce_odr global float 2.300000e+01
+// CHECK-DEBUG-DAG: [[ST_S4_ST:@.+]] = linkonce_odr global %struct.S4 zeroinitializer
+// CHECK-DEBUG-DAG: [[LOC1:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;;155;11;;\00"
+// CHECK-DEBUG-DAG: [[LOC2:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;;202;4;;\00"
+// CHECK-DEBUG-DAG: [[LOC3:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;292;19;;\00"
+// CHECK-DEBUG-DAG: [[LOC4:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;;292;16;;\00"
+// CHECK-DEBUG-DAG: [[LOC5:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;326;9;;\00"
+// CHECK-DEBUG-DAG: [[LOC6:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;343;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC7:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;360;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC8:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;386;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC9:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;407;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC10:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;422;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC11:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;439;27;;\00"
+// CHECK-DEBUG-DAG: [[LOC12:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;main;456;10;;\00"
+// CHECK-DEBUG-DAG: [[LOC13:@.*]] = private unnamed_addr constant [{{[0-9]+}} x i8] c";{{.*}}threadprivate_codegen.cpp;;264;10;;\00"
+
+struct Static {
+  static S3 s;
+#pragma omp threadprivate(s)
+};
+
+static S1 gs1(5);
+#pragma omp threadprivate(gs1)
+// CHECK:      define {{.*}} void [[S1_CTOR:@.*]]([[S1]]* %{{.*}},
+// CHECK:      define {{.*}} void [[S1_DTOR:@.*]]([[S1]]* %{{.*}})
+// CHECK:      define internal i8* [[GS1_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[S1]]*
+// CHECK-NEXT: call void [[S1_CTOR]]([[S1]]* [[RES]], {{.*}} 5)
+// CHECK-NEXT: ret i8* %0
+// CHECK-NEXT: }
+// CHECK:      define internal void [[GS1_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[S1]]*
+// CHECK-NEXT: call void [[S1_DTOR]]([[S1]]* [[RES]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK:      define internal void [[GS1_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK:      call void @__kmpc_threadprivate_register([[IDENT]]* [[DEFAULT_LOC]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i8* (i8*)* [[GS1_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[GS1_DTOR]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK-DEBUG:      define {{.*}} void [[S1_CTOR:@.*]]([[S1]]* %{{.*}},
+// CHECK-DEBUG:      define {{.*}} void [[S1_DTOR:@.*]]([[S1]]* %{{.*}})
+// CHECK-DEBUG:      define internal i8* [[GS1_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK-DEBUG:      [[RES:%.*]] = bitcast i8* %0 to [[S1]]*
+// CHECK-DEBUG-NEXT: call void [[S1_CTOR]]([[S1]]* [[RES]], {{.*}} 5)
+// CHECK-DEBUG-NEXT: ret i8* %0
+// CHECK-DEBUG-NEXT: }
+// CHECK-DEBUG:      define internal void [[GS1_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK-DEBUG:      [[RES:%.*]] = bitcast i8* %0 to [[S1]]*
+// CHECK-DEBUG-NEXT: call void [[S1_DTOR]]([[S1]]* [[RES]])
+// CHECK-DEBUG-NEXT: ret void
+// CHECK-DEBUG-NEXT: }
+// CHECK-DEBUG:      define internal void [[GS1_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+// CHECK-DEBUG:      @__kmpc_global_thread_num
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG:      store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC1]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      call void @__kmpc_threadprivate_register([[IDENT]]* [[KMPC_LOC_ADDR]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i8* (i8*)* [[GS1_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[GS1_DTOR]])
+// CHECK-DEBUG-NEXT: ret void
+// CHECK-DEBUG-NEXT: }
+static S2 gs2(27);
+// CHECK:      define {{.*}} void [[S2_CTOR:@.*]]([[S2]]* %{{.*}},
+// CHECK:      define {{.*}} void [[S2_DTOR:@.*]]([[S2]]* %{{.*}})
+// No another call for S2 constructor because it is not threadprivate
+// CHECK-NOT:  call void [[S2_CTOR]]([[S2]]*
+// CHECK-DEBUG:      define {{.*}} void [[S2_CTOR:@.*]]([[S2]]* %{{.*}},
+// CHECK-DEBUG:      define {{.*}} void [[S2_DTOR:@.*]]([[S2]]* %{{.*}})
+// No another call for S2 constructor because it is not threadprivate
+// CHECK-DEBUG-NOT:  call void [[S2_CTOR]]([[S2]]*
+S1 arr_x[2][3] = { { 1, 2, 3 }, { 4, 5, 6 } };
+#pragma omp threadprivate(arr_x)
+// CHECK:      define {{.*}} i8* [[ARR_X_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [2 x [3 x [[S1]]]]*
+// CHECK:      [[ARR1:%.*]] = getelementptr inbounds [2 x [3 x [[S1]]]]* [[RES]], i{{.*}} 0, i{{.*}} 0
+// CHECK:      [[ARR:%.*]] = getelementptr inbounds [3 x [[S1]]]* [[ARR1]], i{{.*}} 0, i{{.*}} 0
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR]], [[INT]] 1)
+// CHECK:      [[ARR_ELEMENT:%.*]] = getelementptr inbounds [[S1]]* [[ARR]], i{{.*}} 1
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR_ELEMENT]], [[INT]] 2)
+// CHECK:      [[ARR_ELEMENT2:%.*]] = getelementptr inbounds [[S1]]* [[ARR_ELEMENT]], i{{.*}} 1
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR_ELEMENT2]], [[INT]] 3)
+// CHECK:      [[ARR_ELEMENT3:%.*]] = getelementptr inbounds [3 x [[S1]]]* [[ARR1]], i{{.*}} 1
+// CHECK:      [[ARR_:%.*]] = getelementptr inbounds [3 x [[S1]]]* [[ARR_ELEMENT3]], i{{.*}} 0, i{{.*}} 0
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR_]], [[INT]] 4)
+// CHECK:      [[ARR_ELEMENT:%.*]] = getelementptr inbounds [[S1]]* [[ARR_]], i{{.*}} 1
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR_ELEMENT]], [[INT]] 5)
+// CHECK:      [[ARR_ELEMENT2:%.*]] = getelementptr inbounds [[S1]]* [[ARR_ELEMENT]], i{{.*}} 1
+// CHECK:      invoke void [[S1_CTOR]]([[S1]]* [[ARR_ELEMENT2]], [[INT]] 6)
+// CHECK:      ret i8* %0
+// CHECK:      }
+// CHECK:      define {{.*}} void [[ARR_X_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK:      [[ARR_BEGIN:%.*]] = bitcast i8* %0 to [[S1]]*
+// CHECK-NEXT: [[ARR_CUR:%.*]] = getelementptr inbounds [[S1]]* [[ARR_BEGIN]], i{{.*}} 6
+// CHECK-NEXT: br label %[[ARR_LOOP:.*]]
+// CHECK:      {{.*}}[[ARR_LOOP]]{{.*}}
+// CHECK-NEXT: [[ARR_ELEMENTPAST:%.*]] = phi [[S1]]* [ [[ARR_CUR]], {{.*}} ], [ [[ARR_ELEMENT:%.*]], {{.*}} ]
+// CHECK-NEXT: [[ARR_ELEMENT:%.*]] = getelementptr inbounds [[S1]]* [[ARR_ELEMENTPAST]], i{{.*}} -1
+// CHECK-NEXT: invoke void [[S1_DTOR]]([[S1]]* [[ARR_ELEMENT]])
+// CHECK:      [[ARR_DONE:%.*]] = icmp eq [[S1]]* [[ARR_ELEMENT]], [[ARR_BEGIN]]
+// CHECK-NEXT: br i1 [[ARR_DONE]], label %[[ARR_EXIT:.*]], label %[[ARR_LOOP]]
+// CHECK:      {{.*}}[[ARR_EXIT]]{{.*}}
+// CHECK-NEXT: ret void
+// CHECK:      }
+// CHECK:      define {{.*}} void [[ARR_X_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK:      call void @__kmpc_threadprivate_register([[IDENT]]* [[DEFAULT_LOC]], i8* bitcast ([2 x [3 x [[S1]]]]* [[ARR_X]] to i8*), i8* (i8*)* [[ARR_X_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[ARR_X_DTOR]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK-DEBUG:      define {{.*}} i8* [[ARR_X_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK-DEBUG:      }
+// CHECK-DEBUG:      define {{.*}} void [[ARR_X_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK-DEBUG:      }
+// CHECK-DEBUG:      define {{.*}} void [[ARR_X_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+// CHECK-DEBUG:      @__kmpc_global_thread_num
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG:      store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC2]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      call void @__kmpc_threadprivate_register([[IDENT]]* [[KMPC_LOC_ADDR]], i8* bitcast ([2 x [3 x [[S1]]]]* [[ARR_X]] to i8*), i8* (i8*)* [[ARR_X_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[ARR_X_DTOR]])
+// CHECK-DEBUG-NEXT: ret void
+// CHECK-DEBUG-NEXT: }
+extern S5 gs3;
+#pragma omp threadprivate(gs3)
+// No call for S5 constructor because gs3 has just declaration, not a definition.
+// CHECK-NOT:  call void {{.*}}([[S5]]*
+// CHECK-DEBUG-NOT:  call void {{.*}}([[S5]]*
+
+template <class T>
+struct ST {
+  static T st;
+#pragma omp threadprivate(st)
+};
+
+template <class T>
+T ST<T>::st(23);
+
+#endif
+
+#ifdef BODY
+
+// CHECK: define i32 @main()
+// CHECK-DEBUG: define i32 @main()
+int main() {
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+  int Res;
+  struct Smain {
+    int a;
+    double b, c;
+    Smain()
+        : a(0) {
+    }
+    Smain(int a)
+        : a(a) {
+    }
+    Smain(const Smain &s) {
+      a = 12 + s.a;
+    }
+    ~Smain() {
+      a = 0;
+    }
+  };
+
+  static Smain sm(gs1.a);
+// CHECK:      [[THREAD_NUM:%.+]] = call i32 @__kmpc_global_thread_num([[IDENT]]* [[DEFAULT_LOC]])
+// CHECK:      call i{{.*}} @__cxa_guard_acquire
+// CHECK:      [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE1]])
+// CHECK-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+// CHECK-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+// CHECK-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+// CHECK-NEXT: invoke void [[SMAIN_CTOR:.*]]([[SMAIN]]* [[SM]], [[INT]] [[GS1_A]])
+// CHECK:      call void @__cxa_guard_release
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC3]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG-NEXT: [[THREAD_NUM:%.+]] = call i32 @__kmpc_global_thread_num([[IDENT]]* [[KMPC_LOC_ADDR]])
+// CHECK-DEBUG:      call i{{.*}} @__cxa_guard_acquire
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC3]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+// CHECK-DEBUG-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+// CHECK-DEBUG-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+// CHECK-DEBUG-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+// CHECK-DEBUG-NEXT: invoke void [[SMAIN_CTOR:.*]]([[SMAIN]]* [[SM]], [[INT]] [[GS1_A]])
+// CHECK-DEBUG:      call void @__cxa_guard_release
+#pragma omp threadprivate(sm)
+  // CHECK:      [[STATIC_S_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S3]]* [[STATIC_S]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE4]])
+  // CHECK-NEXT: [[STATIC_S_ADDR:%.*]] = bitcast i8* [[STATIC_S_TEMP_ADDR]] to [[S3]]*
+  // CHECK-NEXT: [[STATIC_S_A_ADDR:%.*]] = getelementptr inbounds [[S3]]* [[STATIC_S_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[STATIC_S_A:%.*]] = load [[INT]]* [[STATIC_S_A_ADDR]]
+  // CHECK-NEXT: store [[INT]] [[STATIC_S_A]], [[INT]]* [[RES_ADDR:[^,]+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC5]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[STATIC_S_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S3]]* [[STATIC_S]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[STATIC_S_ADDR:%.*]] = bitcast i8* [[STATIC_S_TEMP_ADDR]] to [[S3]]*
+  // CHECK-DEBUG-NEXT: [[STATIC_S_A_ADDR:%.*]] = getelementptr inbounds [[S3]]* [[STATIC_S_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[STATIC_S_A:%.*]] = load [[INT]]* [[STATIC_S_A_ADDR]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[STATIC_S_A]], [[INT]]* [[RES_ADDR:[^,]+]]
+  Res = Static::s.a;
+  // CHECK:      [[SM_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[SMAIN]]* [[SM]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE3]])
+  // CHECK-NEXT: [[SM_ADDR:%.*]] = bitcast i8* [[SM_TEMP_ADDR]] to [[SMAIN]]*
+  // CHECK-NEXT: [[SM_A_ADDR:%.*]] = getelementptr inbounds [[SMAIN]]* [[SM_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[SM_A:%.*]] = load [[INT]]* [[SM_A_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[SM_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC6]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[SM_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[SMAIN]]* [[SM]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[SM_ADDR:%.*]] = bitcast i8* [[SM_TEMP_ADDR]] to [[SMAIN]]*
+  // CHECK-DEBUG-NEXT: [[SM_A_ADDR:%.*]] = getelementptr inbounds [[SMAIN]]* [[SM_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[SM_A:%.*]] = load [[INT]]* [[SM_A_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[SM_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += sm.a;
+  // CHECK:      [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE1]])
+  // CHECK-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+  // CHECK-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS1_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC7]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+  // CHECK-DEBUG-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS1_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += gs1.a;
+  // CHECK:      [[GS2_A:%.*]] = load [[INT]]* getelementptr inbounds ([[S2]]* [[GS2]], i{{.*}} 0, i{{.*}} 0)
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS2_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[GS2_A:%.*]] = load [[INT]]* getelementptr inbounds ([[S2]]* [[GS2]], i{{.*}} 0, i{{.*}} 0)
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS2_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += gs2.a;
+  // CHECK:      [[GS3_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S5]]* [[GS3]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE5]])
+  // CHECK-NEXT: [[GS3_ADDR:%.*]] = bitcast i8* [[GS3_TEMP_ADDR]] to [[S5]]*
+  // CHECK-NEXT: [[GS3_A_ADDR:%.*]] = getelementptr inbounds [[S5]]* [[GS3_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[GS3_A:%.*]] = load [[INT]]* [[GS3_A_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS3_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC8]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[GS3_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S5]]* [[GS3]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[GS3_ADDR:%.*]] = bitcast i8* [[GS3_TEMP_ADDR]] to [[S5]]*
+  // CHECK-DEBUG-NEXT: [[GS3_A_ADDR:%.*]] = getelementptr inbounds [[S5]]* [[GS3_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[GS3_A:%.*]] = load [[INT]]* [[GS3_A_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[GS3_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += gs3.a;
+  // CHECK:      [[ARR_X_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([2 x [3 x [[S1]]]]* [[ARR_X]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE2]])
+  // CHECK-NEXT: [[ARR_X_ADDR:%.*]] = bitcast i8* [[ARR_X_TEMP_ADDR]] to [2 x [3 x [[S1]]]]*
+  // CHECK-NEXT: [[ARR_X_1_ADDR:%.*]] = getelementptr inbounds [2 x [3 x [[S1]]]]* [[ARR_X_ADDR]], i{{.*}} 0, i{{.*}} 1
+  // CHECK-NEXT: [[ARR_X_1_1_ADDR:%.*]] = getelementptr inbounds [3 x [[S1]]]* [[ARR_X_1_ADDR]], i{{.*}} 0, i{{.*}} 1
+  // CHECK-NEXT: [[ARR_X_1_1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[ARR_X_1_1_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[ARR_X_1_1_A:%.*]] = load [[INT]]* [[ARR_X_1_1_A_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ARR_X_1_1_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC9]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT:      [[ARR_X_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([2 x [3 x [[S1]]]]* [[ARR_X]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[ARR_X_ADDR:%.*]] = bitcast i8* [[ARR_X_TEMP_ADDR]] to [2 x [3 x [[S1]]]]*
+  // CHECK-DEBUG-NEXT: [[ARR_X_1_ADDR:%.*]] = getelementptr inbounds [2 x [3 x [[S1]]]]* [[ARR_X_ADDR]], i{{.*}} 0, i{{.*}} 1
+  // CHECK-DEBUG-NEXT: [[ARR_X_1_1_ADDR:%.*]] = getelementptr inbounds [3 x [[S1]]]* [[ARR_X_1_ADDR]], i{{.*}} 0, i{{.*}} 1
+  // CHECK-DEBUG-NEXT: [[ARR_X_1_1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[ARR_X_1_1_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[ARR_X_1_1_A:%.*]] = load [[INT]]* [[ARR_X_1_1_A_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ARR_X_1_1_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += arr_x[1][1].a;
+  // CHECK:      [[ST_INT_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[INT]]* [[ST_INT_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE6]])
+  // CHECK-NEXT: [[ST_INT_ST_ADDR:%.*]] = bitcast i8* [[ST_INT_ST_TEMP_ADDR]] to [[INT]]*
+  // CHECK-NEXT: [[ST_INT_ST_VAL:%.*]] = load [[INT]]* [[ST_INT_ST_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ST_INT_ST_VAL]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC10]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[ST_INT_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[INT]]* [[ST_INT_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[ST_INT_ST_ADDR:%.*]] = bitcast i8* [[ST_INT_ST_TEMP_ADDR]] to [[INT]]*
+  // CHECK-DEBUG-NEXT: [[ST_INT_ST_VAL:%.*]] = load [[INT]]* [[ST_INT_ST_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ST_INT_ST_VAL]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += ST<int>::st;
+  // CHECK:      [[ST_FLOAT_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast (float* [[ST_FLOAT_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE7]])
+  // CHECK-NEXT: [[ST_FLOAT_ST_ADDR:%.*]] = bitcast i8* [[ST_FLOAT_ST_TEMP_ADDR]] to float*
+  // CHECK-NEXT: [[ST_FLOAT_ST_VAL:%.*]] = load float* [[ST_FLOAT_ST_ADDR]]
+  // CHECK-NEXT: [[FLOAT_TO_INT_CONV:%.*]] = fptosi float [[ST_FLOAT_ST_VAL]] to [[INT]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[FLOAT_TO_INT_CONV]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC11]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[ST_FLOAT_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast (float* [[ST_FLOAT_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[ST_FLOAT_ST_ADDR:%.*]] = bitcast i8* [[ST_FLOAT_ST_TEMP_ADDR]] to float*
+  // CHECK-DEBUG-NEXT: [[ST_FLOAT_ST_VAL:%.*]] = load float* [[ST_FLOAT_ST_ADDR]]
+  // CHECK-DEBUG-NEXT: [[FLOAT_TO_INT_CONV:%.*]] = fptosi float [[ST_FLOAT_ST_VAL]] to [[INT]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[FLOAT_TO_INT_CONV]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += static_cast<int>(ST<float>::st);
+  // CHECK:      [[ST_S4_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S4]]* [[ST_S4_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE8]])
+  // CHECK-NEXT: [[ST_S4_ST_ADDR:%.*]] = bitcast i8* [[ST_S4_ST_TEMP_ADDR]] to [[S4]]*
+  // CHECK-NEXT: [[ST_S4_ST_A_ADDR:%.*]] = getelementptr inbounds [[S4]]* [[ST_S4_ST_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-NEXT: [[ST_S4_ST_A:%.*]] = load [[INT]]* [[ST_S4_ST_A_ADDR]]
+  // CHECK-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ST_S4_ST_A]]
+  // CHECK-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  // CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+  // CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC12]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+  // CHECK-DEBUG-NEXT: [[ST_S4_ST_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S4]]* [[ST_S4_ST]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+  // CHECK-DEBUG-NEXT: [[ST_S4_ST_ADDR:%.*]] = bitcast i8* [[ST_S4_ST_TEMP_ADDR]] to [[S4]]*
+  // CHECK-DEBUG-NEXT: [[ST_S4_ST_A_ADDR:%.*]] = getelementptr inbounds [[S4]]* [[ST_S4_ST_ADDR]], i{{.*}} 0, i{{.*}} 0
+  // CHECK-DEBUG-NEXT: [[ST_S4_ST_A:%.*]] = load [[INT]]* [[ST_S4_ST_A_ADDR]]
+  // CHECK-DEBUG-NEXT: [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: [[ADD:%.*]] = add {{.*}} [[INT]] [[RES]], [[ST_S4_ST_A]]
+  // CHECK-DEBUG-NEXT: store [[INT]] [[ADD]], [[INT]]* [[RES:.+]]
+  Res += ST<S4>::st.a;
+  // CHECK:      [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-NEXT: ret [[INT]] [[RES]]
+  // CHECK-DEBUG:      [[RES:%.*]] = load [[INT]]* [[RES_ADDR]]
+  // CHECK-DEBUG-NEXT: ret [[INT]] [[RES]]
+  return Res;
+}
+// CHECK: }
+
+// CHECK:      define {{.*}} void [[SMAIN_CTOR]]([[SMAIN]]* %{{.*}},
+// CHECK:      define {{.*}} void [[SMAIN_DTOR:@.*]]([[SMAIN]]* %{{.*}})
+// CHECK:      define internal i8* [[SM_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK:      [[THREAD_NUM:%.+]] = call i32 @__kmpc_global_thread_num([[IDENT]]* [[DEFAULT_LOC]])
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[SMAIN]]*
+// CHECK:      [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[DEFAULT_LOC]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8*** [[CACHE1]])
+// CHECK-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+// CHECK-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+// CHECK-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+// CHECK-NEXT: call void [[SMAIN_CTOR]]([[SMAIN]]* [[RES]], [[INT]] [[GS1_A]])
+// CHECK-NEXT: ret i8* %0
+// CHECK-NEXT: }
+// CHECK:      define internal void [[SM_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[SMAIN]]*
+// CHECK-NEXT: call void [[SMAIN_DTOR]]([[SMAIN]]* [[RES]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK:      define internal void [[SM_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK:      call void @__kmpc_threadprivate_register([[IDENT]]* [[DEFAULT_LOC]], i8* bitcast ([[SMAIN]]* [[SM]] to i8*), i8* (i8*)* [[SM_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[SM_DTOR]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK-DEBUG:      define {{.*}} void [[SMAIN_CTOR]]([[SMAIN]]* %{{.*}},
+// CHECK-DEBUG:      define {{.*}} void [[SMAIN_DTOR:@.*]]([[SMAIN]]* %{{.*}})
+// CHECK-DEBUG:      define internal i8* [[SM_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC3]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG-NEXT: [[THREAD_NUM:%.+]] = call i32 @__kmpc_global_thread_num([[IDENT]]* [[KMPC_LOC_ADDR]])
+// CHECK-DEBUG:      [[RES:%.*]] = bitcast i8* %0 to [[SMAIN]]*
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC3]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      [[GS1_TEMP_ADDR:%.*]] = call i8* @__kmpc_threadprivate_cached([[IDENT]]* [[KMPC_LOC_ADDR]], i32 [[THREAD_NUM]], i8* bitcast ([[S1]]* [[GS1]] to i8*), i{{.*}} {{[0-9]+}}, i8***
+// CHECK-DEBUG-NEXT: [[GS1_ADDR:%.*]] = bitcast i8* [[GS1_TEMP_ADDR]] to [[S1]]*
+// CHECK-DEBUG-NEXT: [[GS1_A_ADDR:%.*]] = getelementptr inbounds [[S1]]* [[GS1_ADDR]], i{{.*}} 0, i{{.*}} 0
+// CHECK-DEBUG-NEXT: [[GS1_A:%.*]] = load [[INT]]* [[GS1_A_ADDR]]
+// CHECK-DEBUG-NEXT: call void [[SMAIN_CTOR]]([[SMAIN]]* [[RES]], [[INT]] [[GS1_A]])
+// CHECK-DEBUG-NEXT: ret i8* %0
+// CHECK-DEBUG-NEXT: }
+// CHECK-DEBUG:      define internal void [[SM_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK-DEBUG:      }
+// CHECK-DEBUG:      define internal void [[SM_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+// CHECK-DEBUG:      @__kmpc_global_thread_num
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC4]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      call void @__kmpc_threadprivate_register([[IDENT]]* [[KMPC_LOC_ADDR]], i8* bitcast ([[SMAIN]]* [[SM]] to i8*), i8* (i8*)* [[SM_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[SM_DTOR]])
+// CHECK-DEBUG-NEXT: ret void
+// CHECK-DEBUG-NEXT: }
+
+// CHECK:      define {{.*}} void [[S4_CTOR:@.*]]([[S4]]* %{{.*}},
+// CHECK:      define {{.*}} void [[S4_DTOR:@.*]]([[S4]]* %{{.*}})
+// CHECK:      define internal i8* [[ST_S4_ST_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[S4]]*
+// CHECK-NEXT: call void [[S4_CTOR]]([[S4]]* [[RES]], {{.*}} 23)
+// CHECK-NEXT: ret i8* %0
+// CHECK-NEXT: }
+// CHECK:      define internal void [[ST_S4_ST_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK:      [[RES:%.*]] = bitcast i8* %0 to [[S4]]*
+// CHECK-NEXT: call void [[S4_DTOR]]([[S4]]* [[RES]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK:      define internal void [[ST_S4_ST_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK:      call void @__kmpc_threadprivate_register([[IDENT]]* [[DEFAULT_LOC]], i8* bitcast ([[S4]]* [[ST_S4_ST]] to i8*), i8* (i8*)* [[ST_S4_ST_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[ST_S4_ST_DTOR]])
+// CHECK-NEXT: ret void
+// CHECK-NEXT: }
+// CHECK-DEBUG:      define {{.*}} void [[S4_CTOR:@.*]]([[S4]]* %{{.*}},
+// CHECK-DEBUG:      define {{.*}} void [[S4_DTOR:@.*]]([[S4]]* %{{.*}})
+// CHECK-DEBUG:      define internal i8* [[ST_S4_ST_CTOR:@\.__kmpc_global_ctor_\..*]](i8*)
+// CHECK-DEBUG:      }
+// CHECK-DEBUG:      define internal void [[ST_S4_ST_DTOR:@\.__kmpc_global_dtor_\..*]](i8*)
+// CHECK-DEBUG:      }
+// CHECK-DEBUG:      define internal void [[ST_S4_ST_INIT:@\.__omp_threadprivate_init_\..*]]()
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR:%.*]] = alloca [[IDENT]]
+// CHECK-DEBUG:      @__kmpc_global_thread_num
+// CHECK-DEBUG:      [[KMPC_LOC_ADDR_PSOURCE:%.*]] = getelementptr inbounds [[IDENT]]* [[KMPC_LOC_ADDR]], i{{.*}} 0, i{{.*}} 4
+// CHECK-DEBUG-NEXT: store i8* getelementptr inbounds ([{{.*}} x i8]* [[LOC13]], i{{.*}} 0, i{{.*}} 0), i8** [[KMPC_LOC_ADDR_PSOURCE]]
+// CHECK-DEBUG:      call void @__kmpc_threadprivate_register([[IDENT]]* [[KMPC_LOC_ADDR]], i8* bitcast ([[S4]]* [[ST_S4_ST]] to i8*), i8* (i8*)* [[ST_S4_ST_CTOR]], i8* (i8*, i8*)* null, void (i8*)* [[ST_S4_ST_DTOR]])
+// CHECK-DEBUG-NEXT: ret void
+// CHECK-DEBUG-NEXT: }
+
+// CHECK:      define internal void {{@.*}}()
+// CHECK-DAG:  call void [[GS1_INIT]]()
+// CHECK-DAG:  call void [[ARR_X_INIT]]()
+// CHECK-DAG:  call void [[SM_INIT]]()
+// CHECK-DAG:  call void [[ST_S4_ST_INIT]]()
+// CHECK:      ret void
+// CHECK-DEBUG:      define internal void {{@.*}}()
+// CHECK-DEBUG-DAG:  call void [[GS1_INIT]]()
+// CHECK-DEBUG-DAG:  call void [[ARR_X_INIT]]()
+// CHECK-DEBUG-DAG:  call void [[SM_INIT]]()
+// CHECK-DEBUG-DAG:  call void [[ST_S4_ST_INIT]]()
+// CHECK-DEBUG:      ret void
+#endif
+