[OpenMP][AMDGCN] Generate global variables and attributes for AMDGCN

[saiislam]

Summary

Provide support for amdgcn specific global variables and attributes.
Generalize allocation of various common global variables and provide
their specialized implementations for nvptx and amdgcn.

[ABataev]

clang/lib/CodeGen/CGOpenMPRuntime.h

502

Cab this type and corresponding functions be made AMDGCN-specific only?

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

119

Is this possible?

122

FlatAttrEmitted

132

CompileTimeThreadLimit

clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp

1314–1315

Restore original formatting.

clang/lib/CodeGen/CGOpenMPRuntimeGPU.h

249–277

Make them protected, not public if possible. Try the same for other new functions.

clang/lib/CodeGen/CGOpenMPRuntimeNVPTX.h

41–66

No need to add virtual, override is enough

[saiislam]

1. Moved amdgcn specific functions to CGOpenMPAMDGCN.cpp
2. Removed tautology condition
3. Corrected case of local variables
4. Restored original formatting
5. Changed back declaration of emit kernel methods as private
6. Added support of amdgcn specific PrePostActionTy implementation and its corresponding test cases
7. Changed static line numbers in new test cases with regex
8. Other small code corrections

[ABataev]

Reformat the code

clang/lib/CodeGen/CGOpenMPRuntime.h

501

Remove unnecessary formatting changes.

2491–2495

Better to make it a protected member function if you really require it. Plus, this function is very small and, I think, you simply create your own copy in CGOpenMPRuntimeAMDGCN

2499

Same here, make it protected or just create a copy, if it is small.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.h

29–31

Add comments for all new members

77

1. Do you really need to make this class public?
2. final

[saiislam]

1. Reformarting
2. Comments
3. Reduced scope of specialized PrePostActionTy

[saiislam]

clang/lib/CodeGen/CGOpenMPRuntime.h

2491–2495

Not making it protected because it is used by various static functions. And don't want to create an object pointer of subclass of CGOpenMPRuntime in CGOpenMPRuntime.

2499

It calls static functions which in turn call other static functions, so it won't make sense to create a copy of whole function chain in amdgcn.

[saiislam]

Ping.

[saiislam]

Ping.

[ABataev]

clang/lib/CodeGen/CGOpenMPRuntime.h

501

Still not removed

688

Restore original formatting

2494–2499

Better to encapsulate these functions into a new utility class and make them public static.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.h

30–63

Do you really need to expose all these new members as public?

31

Runtime does not support nested parallelism on GPU. Do you really need it?

93

It does not help to understand the functionality

clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp

1573

It leads to a mem leak.

clang/lib/CodeGen/CGOpenMPRuntimeGPU.h

37

1. Make it private or protected

2.Add default initializer

221–253

Are all these required to be public?

402–404

Make it private or protected

[saiislam]

1. Removed unnecessary formatting of untouched code.
2. Encapsulated addFieldToRecordDecl and createGlobalStruct methods in a class and made them static (triggered change at all calling sites).
3. Marked most of the member methods of CGOpenMPRuntimeAMDGCN as private (forgot to do same change in nvptx)
4. Fixed the memory leak
5. Marked appropriate member variables as protected in CGOpenMPRuntimeGPU

[JonChesterfield]

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

178

The nvptx emitSPMDKernelWrapper does nothing and the amdgcn one appends some metadata.

How about 'nvptx::generateMetadata(...)' that does nothing and 'amdgcn::generateMetadata(...)` that does this stuff, called from the end of emitSPMDKernel?

200

This metadata generation could be split out from the other changes.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.h

43

I'm not convinced by this abstraction. It looks like amdgcn and nvptx want almost exactly the same variable in each case. The difference appears to be that nvptx uses internal linkage and amdgcn uses weak + externally initialized, in which case we're better off with

bool nvptx::needsExternalInitialization() {return false;}
bool amdgpu::needsExternalInitialization() {return true;}

Or, if the inline ternary is unappealing, amdgcn::NewGlobalVariable(...) that passes the arguments to llvm::GlobalVariable while setting the two fields that differ between the two.

clang/lib/CodeGen/CGOpenMPRuntimeGPU.h

204

Please put this back to the previous location so we can see whether it changed in the diff

[saiislam]

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

178

It will be then difficult to track what all things are being done differently in the two. So, the common code has been generalized and (no change in nvptx + some changes in amdgcn) has been used as specialization.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.h

43

I understand what you are suggesting. But, there are multiple such variables where linkage between nvptx and amdgcn are different. Also current style gives flexibility to a future implementation to define these variables in their own way.
What do you think?

clang/lib/CodeGen/CGOpenMPRuntimeGPU.h

204

This movement changes them from private to protected.
I could have just added access specifiers and not move the definitions. It would have simplified the review, but it would have decreased the readability for future.

221–253

Yes, they are being called from outside class.

[JonChesterfield]

clang/lib/CodeGen/CGOpenMPRuntimeNVPTX.cpp

59

Perhaps (typed into browser):

llvm::GlobalVariable *CGOpenMPRuntimeNVPTX::createGlobal( CodeGenModule &CGM, llvm::ArrayType *Ty, StringRef Name) {
  return new llvm::GlobalVariable(
      CGM.getModule(), Ty, /*isConstant=*/false,
      llvm::GlobalVariable::CommonLinkage, llvm::Constant::getNullValue(Ty),
      Name,
      /*InsertBefore=*/nullptr, llvm::GlobalVariable::NotThreadLocal,
      CGM.getContext().getTargetAddressSpace(LangAS::cuda_shared),
/*isExternallyInitialized*/ true);
}

llvm::GlobalVariable *CGOpenMPRuntimeAMDGCN::createGlobal( CodeGenModule &CGM, llvm::ArrayType *Ty, StringRef Name) {
  return new llvm::GlobalVariable(
      CGM.getModule(), Ty, /*isConstant=*/false,
      llvm::GlobalVariable::WeakAnyLinkage, llvm::Constant::getNullValue(Ty),
      Name,
      /*InsertBefore=*/nullptr, llvm::GlobalVariable::NotThreadLocal,
      CGM.getContext().getTargetAddressSpace(LangAS::cuda_shared),
/*isExternallyInitialized*/ false);
}

[saiislam]

1. Simplifies overall patch after D90248.
2. Removes MaxParallelLevel and thus target specific PrePostActionTy.
3. Removes ExternallyInitialized qualifier from shared variables for AMDGCN.

[JonChesterfield]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

1344

This appears to be the same as the free function we had before, except now all the call sites are prefixed CodegenUtil. Is there a functional change I'm missing?

The rest of this patch would be easier to read with this part split off.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

77

This is a very verbose way to say that amdgcn calls emitmetatdata at the end of emitkernel and nvptx doesn't. Suggest unconditionally calling emitmetatdata, and having emitmetatdata be a no-op for nvptx.

89

I think there's a credible chance this is useful to nvptx, so doesn't have to be amdgcn specific

108

I think this is about computing a maximum workgroup size which the runtime uses to limit the number of threads it launches. If so, this is probably useful for nvptx and amdgcn. I'm having trouble working out what the conditions are though. Maybe it's based on an openmp clause?

150

I think I remember seeing a diff that makes this attribute unconditionally emitted by some other part of the toolchain. If so, it may no longer be required

169

HostServices is unused. Mode is redundant with exec_mode. wg_size is redundant with the other wg_size symbol added above. This kern_desc object should be deleted, not upstreamed.

[saiislam]

clang/lib/CodeGen/CGOpenMPRuntime.cpp

1344

addFieldToRecordDecl and createGlobalStruct methods had file static scope. To make them callable from other files, from amdgcn specific file in this case, they were put in this utility class.

D92167 puts this change into a separate patch. Will update this patch once D92167 gets accepted.

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

77

Won't the no-op approach be less extensible? Current way, though verbose, leaves scope for attaching prefix/suffix code as and when required around emitkernel. While in case of no-op, every implementing arch might have to use the exact same pattern of methods with and without code.

89

You are right, it can be useful for nvptx as well. May be we can club its generalization with the nvptx's use-case when it arrives in the future?

108

Yes, the if block in 111-147 corresponds to "number of threads" for thread_limit and num_threads clauses in teams and parallel directives.

169

Ok, thanks. Will update in next revision.

[JonChesterfield]

I don't believe the contents of this patch is necessary for codegen on amdgpu. One of the internal/weak distinctions works around a bug in the gfx800 toolchain, but we should root cause and fix that bug instead. The kern_desc object is redundant. I think amdgpu-flat-work-group-size is already emitted, but if not, we might want that.

The wg_size code is interesting but architecture independent, and it's probably more user friendly for nvptx and amdgcn to have the same handling of wg_size constraints.

clang/lib/CodeGen/CGOpenMPRuntime.h

private:
   ///   char      *name;       // Name of the function or global.
   ///   size_t     size;       // Size of the entry info (0 if it a function).
   ///   int32_t flags;
   ///   int32_t reserved;
   /// };
   QualType TgtOffloadEntryQTy;
   /// Entity that registers the offloading constants that were emitted so
   /// far.
   class OffloadEntriesInfoManagerTy {
     CodeGenModule &CGM;
 
     /// Number of entries registered so far.
     unsigned OffloadingEntriesNum = 0;
 
   public:
     /// Base class of the entries info.
     class OffloadEntryInfo {
     public:
bool hasDeviceGlobalVarEntryInfo(StringRef VarName) const {
       return OffloadEntriesDeviceGlobalVar.count(VarName) > 0;
     }
     /// Applies action \a Action on all registered entries.
     typedef llvm::function_ref<void(StringRef,
                                     const OffloadEntryInfoDeviceGlobalVar &)>
         OffloadDeviceGlobalVarEntryInfoActTy;
     void actOnDeviceGlobalVarEntriesInfo(
         const OffloadDeviceGlobalVarEntryInfoActTy &Action);
 
   private:
     // Storage for target region entries kind. The storage is to be indexed by
     // file ID, device ID, parent function name and line number.
     typedef llvm::DenseMap<unsigned, OffloadEntryInfoTargetRegion>
         OffloadEntriesTargetRegionPerLine;
     typedef llvm::StringMap<OffloadEntriesTargetRegionPerLine>
         OffloadEntriesTargetRegionPerParentName;
     typedef llvm::DenseMap<unsigned, OffloadEntriesTargetRegionPerParentName>
public:
 
   /// Gets the OpenMP-specific address of the local variable.
   Address getAddressOfLocalVariable(CodeGenFunction &CGF,
                                     const VarDecl *VD) override {
     return Address::invalid();
   }
 };
 
+/// To encapsulate helper methods to be used by target specific specializations
+/// of CGOpenMPRuntimeGPU.
+class CodeGenUtil {
+public:
+  static FieldDecl *addFieldToRecordDecl(ASTContext &C, DeclContext *DC,
+                                         QualType FieldTy);
+
+  template <class... As>
+  static llvm::GlobalVariable *
+  createGlobalStruct(CodeGenModule &CGM, QualType Ty, bool IsConstant,
+                     ArrayRef<llvm::Constant *> Data, const Twine &Name,
+                     As &&... Args);
+};
+
 } // namespace CodeGen
 } // namespace clang
 
 #endif

clang/lib/CodeGen/CGOpenMPRuntime.cpp

 
 LValue CGOpenMPTaskOutlinedRegionInfo::getThreadIDVariableLValue(
     CodeGenFunction &CGF) {
   return CGF.MakeAddrLValue(CGF.GetAddrOfLocalVar(getThreadIDVariable()),
                             getThreadIDVariable()->getType(),
                             AlignmentSource::Decl);
 }
 
-static FieldDecl *addFieldToRecordDecl(ASTContext &C, DeclContext *DC,
-                                       QualType FieldTy) {
-  auto *Field = FieldDecl::Create(
-      C, DC, SourceLocation(), SourceLocation(), /*Id=*/nullptr, FieldTy,
-      C.getTrivialTypeSourceInfo(FieldTy, SourceLocation()),
-      /*BW=*/nullptr, /*Mutable=*/false, /*InitStyle=*/ICIS_NoInit);
-  Field->setAccess(AS_public);
-  DC->addDecl(Field);
-  return Field;
-}
-
 CGOpenMPRuntime::CGOpenMPRuntime(CodeGenModule &CGM, StringRef FirstSeparator,
                                  StringRef Separator)
     : CGM(CGM), FirstSeparator(FirstSeparator), Separator(Separator),
       OMPBuilder(CGM.getModule()), OffloadEntriesInfoManager(CGM) {
   KmpCriticalNameTy = llvm::ArrayType::get(CGM.Int32Ty, /*NumElements*/ 8);
 
   // Initialize Types used in OpenMPIRBuilder from OMPKinds.def
   OMPBuilder.initialize();
for (const FieldDecl *FD : RD->fields()) {
     for (unsigned I = PrevIdx; I < Idx; ++I)
       Fields.add(llvm::Constant::getNullValue(StructTy->getElementType(I)));
     PrevIdx = Idx + 1;
     Fields.add(*DI);
     ++DI;
   }
 }
 
+FieldDecl *clang::CodeGen::CodeGenUtil::CodeGenUtil::addFieldToRecordDecl(
+    ASTContext &C, DeclContext *DC, QualType FieldTy) {
+  auto *Field = FieldDecl::Create(
+      C, DC, SourceLocation(), SourceLocation(), /*Id=*/nullptr, FieldTy,
+      C.getTrivialTypeSourceInfo(FieldTy, SourceLocation()),
+      /*BW=*/nullptr, /*Mutable=*/false, /*InitStyle=*/ICIS_NoInit);
+  Field->setAccess(AS_public);
+  DC->addDecl(Field);
+  return Field;
+}
+
 template <class... As>
-static llvm::GlobalVariable *
-createGlobalStruct(CodeGenModule &CGM, QualType Ty, bool IsConstant,
-                   ArrayRef<llvm::Constant *> Data, const Twine &Name,
-                   As &&... Args) {
+llvm::GlobalVariable *clang::CodeGen::CodeGenUtil::createGlobalStruct(
+    CodeGenModule &CGM, QualType Ty, bool IsConstant,
+    ArrayRef<llvm::Constant *> Data, const Twine &Name, As &&... Args) {
   const auto *RD = cast<RecordDecl>(Ty->getAsTagDecl());
   const CGRecordLayout &RL = CGM.getTypes().getCGRecordLayout(RD);
   ConstantInitBuilder CIBuilder(CGM);
   ConstantStructBuilder Fields = CIBuilder.beginStruct(RL.getLLVMType());
   buildStructValue(Fields, CGM, RD, RL, Data);
   return Fields.finishAndCreateGlobal(
       Name, CGM.getContext().getAlignOfGlobalVarInChars(Ty), IsConstant,
       std::forward<As>(Args)...);
void CGOpenMPRuntime::createOffloadEntry(
 
   llvm::Constant *Data[] = {
       llvm::ConstantExpr::getPointerBitCastOrAddrSpaceCast(ID, CGM.VoidPtrTy),
       llvm::ConstantExpr::getPointerBitCastOrAddrSpaceCast(Str, CGM.Int8PtrTy),
       llvm::ConstantInt::get(CGM.SizeTy, Size),
       llvm::ConstantInt::get(CGM.Int32Ty, Flags),
       llvm::ConstantInt::get(CGM.Int32Ty, 0)};
   std::string EntryName = getName({"omp_offloading", "entry", ""});
-  llvm::GlobalVariable *Entry = createGlobalStruct(
+  llvm::GlobalVariable *Entry = CodeGenUtil::createGlobalStruct(
       CGM, getTgtOffloadEntryQTy(), /*IsConstant=*/true, Data,
       Twine(EntryName).concat(Name), llvm::GlobalValue::WeakAnyLinkage);
 
   // The entry has to be created in the section the linker expects it to be.
   Entry->setSection("omp_offloading_entries");
 }
 
 void CGOpenMPRuntime::createOffloadEntriesAndInfoMetadata() {
QualType CGOpenMPRuntime::getTgtOffloadEntryQTy() {
   //   size_t     size;       // Size of the entry info (0 if it a function).
   //   int32_t    flags;      // Flags associated with the entry, e.g. 'link'.
   //   int32_t    reserved;   // Reserved, to use by the runtime library.
   // };
   if (TgtOffloadEntryQTy.isNull()) {
     ASTContext &C = CGM.getContext();
     RecordDecl *RD = C.buildImplicitRecord("__tgt_offload_entry");
     RD->startDefinition();
-    addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-    addFieldToRecordDecl(C, RD, C.getPointerType(C.CharTy));
-    addFieldToRecordDecl(C, RD, C.getSizeType());
-    addFieldToRecordDecl(
+    CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, C.getPointerType(C.CharTy));
+    CodeGenUtil::addFieldToRecordDecl(C, RD, C.getSizeType());
+    CodeGenUtil::addFieldToRecordDecl(
         C, RD, C.getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/true));
-    addFieldToRecordDecl(
+    CodeGenUtil::addFieldToRecordDecl(
         C, RD, C.getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/true));
     RD->completeDefinition();
     RD->addAttr(PackedAttr::CreateImplicit(C));
     TgtOffloadEntryQTy = C.getRecordType(RD);
   }
   return TgtOffloadEntryQTy;
 }
 
for (const auto &Pair : Privates) {
       // If the private variable is a local variable with lvalue ref type,
       // allocate the pointer instead of the pointee type.
       if (Pair.second.isLocalPrivate()) {
         if (VD->getType()->isLValueReferenceType())
           Type = C.getPointerType(Type);
         if (isAllocatableDecl(VD))
           Type = C.getPointerType(Type);
       }
-      FieldDecl *FD = addFieldToRecordDecl(C, RD, Type);
+      FieldDecl *FD = CodeGenUtil::addFieldToRecordDecl(C, RD, Type);
       if (VD->hasAttrs()) {
         for (specific_attr_iterator<AlignedAttr> I(VD->getAttrs().begin()),
              E(VD->getAttrs().end());
              I != E; ++I)
           FD->addAttr(*I);
       }
     }
     RD->completeDefinition();
createKmpTaskTRecordDecl(CodeGenModule &CGM, OpenMPDirectiveKind Kind,
   //         kmp_uint64          lb;
   //         kmp_uint64          ub;
   //         kmp_int64           st;
   //         kmp_int32           liter;
   //         void *              reductions;
   //       };
   RecordDecl *UD = C.buildImplicitRecord("kmp_cmplrdata_t", TTK_Union);
   UD->startDefinition();
-  addFieldToRecordDecl(C, UD, KmpInt32Ty);
-  addFieldToRecordDecl(C, UD, KmpRoutineEntryPointerQTy);
+  CodeGenUtil::addFieldToRecordDecl(C, UD, KmpInt32Ty);
+  CodeGenUtil::addFieldToRecordDecl(C, UD, KmpRoutineEntryPointerQTy);
   UD->completeDefinition();
   QualType KmpCmplrdataTy = C.getRecordType(UD);
   RecordDecl *RD = C.buildImplicitRecord("kmp_task_t");
   RD->startDefinition();
-  addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  addFieldToRecordDecl(C, RD, KmpRoutineEntryPointerQTy);
-  addFieldToRecordDecl(C, RD, KmpInt32Ty);
-  addFieldToRecordDecl(C, RD, KmpCmplrdataTy);
-  addFieldToRecordDecl(C, RD, KmpCmplrdataTy);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, KmpRoutineEntryPointerQTy);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt32Ty);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, KmpCmplrdataTy);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, KmpCmplrdataTy);
   if (isOpenMPTaskLoopDirective(Kind)) {
     QualType KmpUInt64Ty =
         CGM.getContext().getIntTypeForBitwidth(/*DestWidth=*/64, /*Signed=*/0);
     QualType KmpInt64Ty =
         CGM.getContext().getIntTypeForBitwidth(/*DestWidth=*/64, /*Signed=*/1);
-    addFieldToRecordDecl(C, RD, KmpUInt64Ty);
-    addFieldToRecordDecl(C, RD, KmpUInt64Ty);
-    addFieldToRecordDecl(C, RD, KmpInt64Ty);
-    addFieldToRecordDecl(C, RD, KmpInt32Ty);
-    addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, KmpUInt64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, KmpUInt64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt32Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
   }
   RD->completeDefinition();
   return RD;
 }
 
 static RecordDecl *
 createKmpTaskTWithPrivatesRecordDecl(CodeGenModule &CGM, QualType KmpTaskTQTy,
                                      ArrayRef<PrivateDataTy> Privates) {
   ASTContext &C = CGM.getContext();
   // Build struct kmp_task_t_with_privates {
   //         kmp_task_t task_data;
   //         .kmp_privates_t. privates;
   //       };
   RecordDecl *RD = C.buildImplicitRecord("kmp_task_t_with_privates");
   RD->startDefinition();
-  addFieldToRecordDecl(C, RD, KmpTaskTQTy);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, KmpTaskTQTy);
   if (const RecordDecl *PrivateRD = createPrivatesRecordDecl(CGM, Privates))
-    addFieldToRecordDecl(C, RD, C.getRecordType(PrivateRD));
+    CodeGenUtil::addFieldToRecordDecl(C, RD, C.getRecordType(PrivateRD));
   RD->completeDefinition();
   return RD;
 }
 
 /// Emit a proxy function which accepts kmp_task_t as the second
 /// argument.
 /// \code
 /// kmp_int32 .omp_task_entry.(kmp_int32 gtid, kmp_task_t *tt) {

 
 /// Builds kmp_depend_info, if it is not built yet, and builds flags type.
 static void getKmpAffinityType(ASTContext &C, QualType &KmpTaskAffinityInfoTy) {
   QualType FlagsTy = C.getIntTypeForBitwidth(32, /*Signed=*/false);
   if (KmpTaskAffinityInfoTy.isNull()) {
     RecordDecl *KmpAffinityInfoRD =
         C.buildImplicitRecord("kmp_task_affinity_info_t");
     KmpAffinityInfoRD->startDefinition();
-    addFieldToRecordDecl(C, KmpAffinityInfoRD, C.getIntPtrType());
-    addFieldToRecordDecl(C, KmpAffinityInfoRD, C.getSizeType());
-    addFieldToRecordDecl(C, KmpAffinityInfoRD, FlagsTy);
+    CodeGenUtil::addFieldToRecordDecl(C, KmpAffinityInfoRD, C.getIntPtrType());
+    CodeGenUtil::addFieldToRecordDecl(C, KmpAffinityInfoRD, C.getSizeType());
+    CodeGenUtil::addFieldToRecordDecl(C, KmpAffinityInfoRD, FlagsTy);
     KmpAffinityInfoRD->completeDefinition();
     KmpTaskAffinityInfoTy = C.getRecordType(KmpAffinityInfoRD);
   }
 }
 
 CGOpenMPRuntime::TaskResultTy
 CGOpenMPRuntime::emitTaskInit(CodeGenFunction &CGF, SourceLocation Loc,
                               const OMPExecutableDirective &D,

 
 /// Builds kmp_depend_info, if it is not built yet, and builds flags type.
 static void getDependTypes(ASTContext &C, QualType &KmpDependInfoTy,
                            QualType &FlagsTy) {
   FlagsTy = C.getIntTypeForBitwidth(C.getTypeSize(C.BoolTy), /*Signed=*/false);
   if (KmpDependInfoTy.isNull()) {
     RecordDecl *KmpDependInfoRD = C.buildImplicitRecord("kmp_depend_info");
     KmpDependInfoRD->startDefinition();
-    addFieldToRecordDecl(C, KmpDependInfoRD, C.getIntPtrType());
-    addFieldToRecordDecl(C, KmpDependInfoRD, C.getSizeType());
-    addFieldToRecordDecl(C, KmpDependInfoRD, FlagsTy);
+    CodeGenUtil::addFieldToRecordDecl(C, KmpDependInfoRD, C.getIntPtrType());
+    CodeGenUtil::addFieldToRecordDecl(C, KmpDependInfoRD, C.getSizeType());
+    CodeGenUtil::addFieldToRecordDecl(C, KmpDependInfoRD, FlagsTy);
     KmpDependInfoRD->completeDefinition();
     KmpDependInfoTy = C.getRecordType(KmpDependInfoRD);
   }
 }
 
 std::pair<llvm::Value *, LValue>
 CGOpenMPRuntime::getDepobjElements(CodeGenFunction &CGF, LValue DepobjLVal,
                                    SourceLocation Loc) {
llvm::Value *CGOpenMPRuntime::emitTaskReductionInit(
   //   void *reduce_init; // data initialization routine
   //   void *reduce_fini; // data finalization routine
   //   void *reduce_comb; // data combiner routine
   //   kmp_task_red_flags_t flags; // flags for additional info from compiler
   // } kmp_taskred_input_t;
   ASTContext &C = CGM.getContext();
   RecordDecl *RD = C.buildImplicitRecord("kmp_taskred_input_t");
   RD->startDefinition();
-  const FieldDecl *SharedFD = addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  const FieldDecl *OrigFD = addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  const FieldDecl *SizeFD = addFieldToRecordDecl(C, RD, C.getSizeType());
-  const FieldDecl *InitFD  = addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  const FieldDecl *FiniFD = addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  const FieldDecl *CombFD = addFieldToRecordDecl(C, RD, C.VoidPtrTy);
-  const FieldDecl *FlagsFD = addFieldToRecordDecl(
+  const FieldDecl *SharedFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  const FieldDecl *OrigFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  const FieldDecl *SizeFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.getSizeType());
+  const FieldDecl *InitFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  const FieldDecl *FiniFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  const FieldDecl *CombFD =
+      CodeGenUtil::addFieldToRecordDecl(C, RD, C.VoidPtrTy);
+  const FieldDecl *FlagsFD = CodeGenUtil::addFieldToRecordDecl(
       C, RD, C.getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/false));
   RD->completeDefinition();
   QualType RDType = C.getRecordType(RD);
   unsigned Size = Data.ReductionVars.size();
   llvm::APInt ArraySize(/*numBits=*/64, Size);
   QualType ArrayRDType = C.getConstantArrayType(
       RDType, ArraySize, nullptr, ArrayType::Normal, /*IndexTypeQuals=*/0);
   // kmp_task_red_input_t .rd_input.[Size];
static void emitNonContiguousDescriptor(
   //  uint64_t count;
   //  uint64_t stride
   // };
   ASTContext &C = CGF.getContext();
   QualType Int64Ty = C.getIntTypeForBitwidth(/*DestWidth=*/64, /*Signed=*/0);
   RecordDecl *RD;
   RD = C.buildImplicitRecord("descriptor_dim");
   RD->startDefinition();
-  addFieldToRecordDecl(C, RD, Int64Ty);
-  addFieldToRecordDecl(C, RD, Int64Ty);
-  addFieldToRecordDecl(C, RD, Int64Ty);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
+  CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
   RD->completeDefinition();
   QualType DimTy = C.getRecordType(RD);
 
   enum { OffsetFD = 0, CountFD, StrideFD };
   // We need two index variable here since the size of "Dims" is the same as the
   // size of Components, however, the size of offset, count, and stride is equal
   // to the size of base declaration that is non-contiguous.
   for (unsigned I = 0, L = 0, E = NonContigInfo.Dims.size(); I < E; ++I) {
void CGOpenMPRuntime::emitDoacrossInit(CodeGenFunction &CGF,
   if (KmpDimTy.isNull()) {
     // Build struct kmp_dim {  // loop bounds info casted to kmp_int64
     //  kmp_int64 lo; // lower
     //  kmp_int64 up; // upper
     //  kmp_int64 st; // stride
     // };
     RD = C.buildImplicitRecord("kmp_dim");
     RD->startDefinition();
-    addFieldToRecordDecl(C, RD, Int64Ty);
-    addFieldToRecordDecl(C, RD, Int64Ty);
-    addFieldToRecordDecl(C, RD, Int64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
+    CodeGenUtil::addFieldToRecordDecl(C, RD, Int64Ty);
     RD->completeDefinition();
     KmpDimTy = C.getRecordType(RD);
   } else {
     RD = cast<RecordDecl>(KmpDimTy->getAsTagDecl());
   }
   llvm::APInt Size(/*numBits=*/32, NumIterations.size());
   QualType ArrayTy =
       C.getConstantArrayType(KmpDimTy, Size, nullptr, ArrayType::Normal, 0);
Address CGOpenMPRuntime::emitLastprivateConditionalInit(CodeGenFunction &CGF,
   QualType NewType;
   const FieldDecl *VDField;
   const FieldDecl *FiredField;
   LValue BaseLVal;
   auto VI = I->getSecond().find(VD);
   if (VI == I->getSecond().end()) {
     RecordDecl *RD = C.buildImplicitRecord("lasprivate.conditional");
     RD->startDefinition();
-    VDField = addFieldToRecordDecl(C, RD, VD->getType().getNonReferenceType());
-    FiredField = addFieldToRecordDecl(C, RD, C.CharTy);
+    VDField = CodeGenUtil::addFieldToRecordDecl(
+        C, RD, VD->getType().getNonReferenceType());
+    FiredField = CodeGenUtil::addFieldToRecordDecl(C, RD, C.CharTy);
     RD->completeDefinition();
     NewType = C.getRecordType(RD);
     Address Addr = CGF.CreateMemTemp(NewType, C.getDeclAlign(VD), VD->getName());
     BaseLVal = CGF.MakeAddrLValue(Addr, NewType, AlignmentSource::Decl);
     I->getSecond().try_emplace(VD, NewType, VDField, FiredField, BaseLVal);
   } else {
     NewType = std::get<0>(VI->getSecond());
     VDField = std::get<1>(VI->getSecond());

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.h

 
 namespace clang {
 namespace CodeGen {
 
 class CGOpenMPRuntimeAMDGCN final : public CGOpenMPRuntimeGPU {
 
 public:
   explicit CGOpenMPRuntimeAMDGCN(CodeGenModule &CGM);
 
+private:
+  /// Struct to store kernel descriptors
+  QualType TgtAttributeStructQTy;
+
   /// Get the GPU warp size.
   llvm::Value *getGPUWarpSize(CodeGenFunction &CGF) override;
 
   /// Get the id of the current thread on the GPU.
   llvm::Value *getGPUThreadID(CodeGenFunction &CGF) override;
 
   /// Get the maximum number of threads in a block of the GPU.
   llvm::Value *getGPUNumThreads(CodeGenFunction &CGF) override;
+
+  /// Target independent wrapper over target specific emitSPMDKernel()
+  void emitSPMDKernelWrapper(const OMPExecutableDirective &D,
+                             StringRef ParentName, llvm::Function *&OutlinedFn,
+                             llvm::Constant *&OutlinedFnID, bool IsOffloadEntry,
+                             const RegionCodeGenTy &CodeGen) override;
+
+  /// Target independent wrapper over target specific emitNonSPMDKernel()
+  void emitNonSPMDKernelWrapper(const OMPExecutableDirective &D,
+                                StringRef ParentName,
+                                llvm::Function *&OutlinedFn,
+                                llvm::Constant *&OutlinedFnID,
+                                bool IsOffloadEntry,
+                                const RegionCodeGenTy &CodeGen) override;
+
+  /// Create a unique global variable to indicate the flat-work-group-size
+  /// for this region. Values are [256..1024].
+  static void setPropertyWorkGroupSize(CodeGenModule &CGM, StringRef Name,
+                                       unsigned WGSize);
+
+  /// Generate global variables _wg_size, kern_desc, __tgt_attribute_struct.
+  /// Also generate appropriate value of attribute amdgpu-flat-work-group-size
+  void generateMetaData(CodeGenModule &CGM, const OMPExecutableDirective &D,
+                        llvm::Function *&OutlinedFn, bool IsGeneric);
+
+  /// Returns __tgt_attribute_struct type.
+  QualType getTgtAttributeStructQTy();
+
+  /// Emit structure descriptor for a kernel
+  void emitStructureKernelDesc(CodeGenModule &CGM, StringRef Name,
+                               int16_t WG_Size, int8_t Mode,
+                               int8_t HostServices);
 };
 
 } // namespace CodeGen
 } // namespace clang
 
 #endif // LLVM_CLANG_LIB_CODEGEN_CGOPENMPRUNTIMEAMDGCN_H

clang/lib/CodeGen/CGOpenMPRuntimeAMDGCN.cpp

 //===-- CGOpenMPRuntimeAMDGCN.cpp - Interface to OpenMP AMDGCN Runtimes --===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This provides a class for OpenMP runtime code generation specialized to
 // AMDGCN targets from generalized CGOpenMPRuntimeGPU class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CGOpenMPRuntimeAMDGCN.h"
+#include "CGOpenMPRuntime.h"
 #include "CGOpenMPRuntimeGPU.h"
 #include "CodeGenFunction.h"
 #include "clang/AST/Attr.h"
 #include "clang/AST/DeclOpenMP.h"
 #include "clang/AST/StmtOpenMP.h"
 #include "clang/AST/StmtVisitor.h"
 #include "clang/Basic/Cuda.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/IR/IntrinsicsAMDGPU.h"
 
 using namespace clang;
 using namespace CodeGen;
 using namespace llvm::omp;
 
+//
+// Definitions of all virtual methods defined in CGOpenMPRuntimeGPU
+//
 CGOpenMPRuntimeAMDGCN::CGOpenMPRuntimeAMDGCN(CodeGenModule &CGM)
     : CGOpenMPRuntimeGPU(CGM) {
   if (!CGM.getLangOpts().OpenMPIsDevice)
     llvm_unreachable("OpenMP AMDGCN can only handle device code.");
+  KernelStaticGlobalizedLinkage = llvm::GlobalValue::WeakAnyLinkage;
 }
 
 llvm::Value *CGOpenMPRuntimeAMDGCN::getGPUWarpSize(CodeGenFunction &CGF) {
   CGBuilderTy &Bld = CGF.Builder;
   // return constant compile-time target-specific warp size
   unsigned WarpSize = CGF.getTarget().getGridValue(llvm::omp::GV_Warp_Size);
   return Bld.getInt32(WarpSize);
 }
llvm::Value *CGOpenMPRuntimeAMDGCN::getGPUNumThreads(CodeGenFunction &CGF) {
   if (!F) {
     F = llvm::Function::Create(
         llvm::FunctionType::get(CGF.Int64Ty, {CGF.Int32Ty}, false),
         llvm::GlobalVariable::ExternalLinkage, LocSize, &CGF.CGM.getModule());
   }
   return Bld.CreateTrunc(
       Bld.CreateCall(F, {Bld.getInt32(0)}, "nvptx_num_threads"), CGF.Int32Ty);
 }
+
+void CGOpenMPRuntimeAMDGCN::emitSPMDKernelWrapper(
+    const OMPExecutableDirective &D, StringRef ParentName,
+    llvm::Function *&OutlinedFn, llvm::Constant *&OutlinedFnID,
+    bool IsOffloadEntry, const RegionCodeGenTy &CodeGen) {
+  emitSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
+                 CodeGen);
+  generateMetaData(CGM, D, OutlinedFn, /*SPMD*/ false);
+}
+
+void CGOpenMPRuntimeAMDGCN::emitNonSPMDKernelWrapper(
+    const OMPExecutableDirective &D, StringRef ParentName,
+    llvm::Function *&OutlinedFn, llvm::Constant *&OutlinedFnID,
+    bool IsOffloadEntry, const RegionCodeGenTy &CodeGen) {
+  emitNonSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
+                    CodeGen);
+  generateMetaData(CGM, D, OutlinedFn, /*Generic*/ true);
+}
+
+//
+// Definitions of AMDGCN specific methods
+//
+void CGOpenMPRuntimeAMDGCN::setPropertyWorkGroupSize(CodeGenModule &CGM,
+                                                     StringRef Name,
+                                                     unsigned WGSize) {
+  auto *GVMode = new llvm::GlobalVariable(
+      CGM.getModule(), CGM.Int16Ty, /*isConstant=*/true,
+      llvm::GlobalValue::WeakAnyLinkage,
+      llvm::ConstantInt::get(CGM.Int16Ty, WGSize), Twine(Name, "_wg_size"),
+      /*InsertBefore=*/nullptr, llvm::GlobalVariable::NotThreadLocal,
+      CGM.getContext().getTargetAddressSpace(LangAS::cuda_device));
+
+  CGM.addCompilerUsedGlobal(GVMode);
+}
+
+void CGOpenMPRuntimeAMDGCN::generateMetaData(CodeGenModule &CGM,
+                                             const OMPExecutableDirective &D,
+                                             llvm::Function *&OutlinedFn,
+                                             bool IsGeneric) {
+  int FlatAttr = 0;
+  bool FlatAttrEmitted = false;
+  unsigned DefaultWorkGroupSz =
+      CGM.getTarget().getGridValue(llvm::omp::GVIDX::GV_Default_WG_Size);
+
+  if (isOpenMPTeamsDirective(D.getDirectiveKind()) ||
+      isOpenMPParallelDirective(D.getDirectiveKind())) {
+    const auto *ThreadLimitClause = D.getSingleClause<OMPThreadLimitClause>();
+    const auto *NumThreadsClause = D.getSingleClause<OMPNumThreadsClause>();
+    unsigned MaxWorkGroupSz =
+        CGM.getTarget().getGridValue(llvm::omp::GVIDX::GV_Max_WG_Size);
+    unsigned CompileTimeThreadLimit = 0;
+    // Only one of thread_limit or num_threads is used, cant do it for both
+    if (ThreadLimitClause && !NumThreadsClause) {
+      Expr *ThreadLimitExpr = ThreadLimitClause->getThreadLimit();
+      clang::Expr::EvalResult Result;
+      if (ThreadLimitExpr->EvaluateAsInt(Result, CGM.getContext()))
+        CompileTimeThreadLimit = Result.Val.getInt().getExtValue();
+    } else if (!ThreadLimitClause && NumThreadsClause) {
+      Expr *NumThreadsExpr = NumThreadsClause->getNumThreads();
+      clang::Expr::EvalResult Result;
+      if (NumThreadsExpr->EvaluateAsInt(Result, CGM.getContext()))
+        CompileTimeThreadLimit = Result.Val.getInt().getExtValue();
+    }
+
+    // Add kernel metadata if ThreadLimit Clause is compile time constant > 0
+    if (CompileTimeThreadLimit > 0) {
+      // Add the WarpSize to generic, to reflect what runtime dispatch does.
+      if (IsGeneric)
+        CompileTimeThreadLimit +=
+            CGM.getTarget().getGridValue(llvm::omp::GVIDX::GV_Warp_Size);
+      if (CompileTimeThreadLimit > MaxWorkGroupSz)
+        CompileTimeThreadLimit = MaxWorkGroupSz;
+      std::string AttrVal = llvm::utostr(CompileTimeThreadLimit);
+      FlatAttr = CompileTimeThreadLimit;
+      OutlinedFn->addFnAttr("amdgpu-flat-work-group-size",
+                            AttrVal + "," + AttrVal);
+      setPropertyWorkGroupSize(CGM, OutlinedFn->getName(),
+                               CompileTimeThreadLimit);
+    }
+    FlatAttrEmitted = true;
+  } // end of amdgcn teams or parallel directive
+
+  // emit amdgpu-flat-work-group-size if not emitted already.
+  if (!FlatAttrEmitted) {
+    std::string FlatAttrVal = llvm::utostr(DefaultWorkGroupSz);
+    OutlinedFn->addFnAttr("amdgpu-flat-work-group-size",
+                          FlatAttrVal + "," + FlatAttrVal);
+  }
+  // Emit a kernel descriptor for runtime.
+  StringRef KernDescName = OutlinedFn->getName();
+  CGOpenMPRuntimeAMDGCN::emitStructureKernelDesc(
+      CGM, KernDescName, FlatAttr, IsGeneric, 1 /* Uses HostServices */);
+}
+
+/// Emit structure descriptor for a kernel
+void CGOpenMPRuntimeAMDGCN::emitStructureKernelDesc(CodeGenModule &CGM,
+                                                    StringRef Name,
+                                                    int16_t WG_Size,
+                                                    int8_t Mode,
+                                                    int8_t HostServices) {
+
+  // Create all device images
+  llvm::Constant *AttrData[] = {
+      llvm::ConstantInt::get(CGM.Int16Ty, 2), // Version
+      llvm::ConstantInt::get(CGM.Int16Ty, 9), // Size in bytes
+      llvm::ConstantInt::get(CGM.Int16Ty, WG_Size),
+      llvm::ConstantInt::get(CGM.Int8Ty, Mode), // 0 => SPMD, 1 => GENERIC
+      llvm::ConstantInt::get(CGM.Int8Ty, HostServices) // 1 => use HostServices
+  };
+
+  llvm::GlobalVariable *AttrImages =
+      clang::CodeGen::CodeGenUtil::createGlobalStruct(
+          CGM, getTgtAttributeStructQTy(), isDefaultLocationConstant(),
+          AttrData, Twine(Name, "_kern_desc"),
+          llvm::GlobalValue::WeakAnyLinkage);
+
+  CGM.addCompilerUsedGlobal(AttrImages);
+}
+
+// Create Tgt Attribute Struct type.
+QualType CGOpenMPRuntimeAMDGCN::getTgtAttributeStructQTy() {
+  ASTContext &C = CGM.getContext();
+  QualType KmpInt8Ty = C.getIntTypeForBitwidth(/*Width=*/8, /*Signed=*/1);
+  QualType KmpInt16Ty = C.getIntTypeForBitwidth(/*Width=*/16, /*Signed=*/1);
+  if (TgtAttributeStructQTy.isNull()) {
+    RecordDecl *RD = C.buildImplicitRecord("__tgt_attribute_struct");
+    RD->startDefinition();
+    // Version
+    clang::CodeGen::CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt16Ty);
+    // Struct Size in bytes.
+    clang::CodeGen::CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt16Ty);
+    // WG_size
+    clang::CodeGen::CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt16Ty);
+    // Mode
+    clang::CodeGen::CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt8Ty);
+    // HostServices
+    clang::CodeGen::CodeGenUtil::addFieldToRecordDecl(C, RD, KmpInt8Ty);
+    RD->completeDefinition();
+    TgtAttributeStructQTy = C.getRecordType(RD);
+  }
+  return TgtAttributeStructQTy;
+}

clang/lib/CodeGen/CGOpenMPRuntimeGPU.h

public:
   enum ExecutionMode {
     /// SPMD execution mode (all threads are worker threads).
     EM_SPMD,
     /// Non-SPMD execution mode (1 master thread, others are workers).
     EM_NonSPMD,
     /// Unknown execution mode (orphaned directive).
     EM_Unknown,
   };
+
+protected:
+  /// Linkage type of KernelStaticGlobalized variable
+  llvm::GlobalValue::LinkageTypes KernelStaticGlobalizedLinkage;
+
 private:
   /// Parallel outlined function work for workers to execute.
   llvm::SmallVector<llvm::Function *, 16> Work;
 
   struct EntryFunctionState {
     llvm::BasicBlock *ExitBB = nullptr;
   };
 
private:
   //
 
   /// Creates offloading entry for the provided entry ID \a ID,
   /// address \a Addr, size \a Size, and flags \a Flags.
   void createOffloadEntry(llvm::Constant *ID, llvm::Constant *Addr,
                           uint64_t Size, int32_t Flags,
                           llvm::GlobalValue::LinkageTypes Linkage) override;
 
+protected:
   /// Emit outlined function specialized for the Fork-Join
   /// programming model for applicable target directives on the NVPTX device.
   /// \param D Directive to emit.
   /// \param ParentName Name of the function that encloses the target region.
   /// \param OutlinedFn Outlined function value to be defined by this call.
   /// \param OutlinedFnID Outlined function ID value to be defined by this call.
   /// \param IsOffloadEntry True if the outlined function is an offload entry.
   /// An outlined function may not be an entry if, e.g. the if clause always
protected:
   /// \param CodeGen Object containing the target statements.
   /// An outlined function may not be an entry if, e.g. the if clause always
   /// evaluates to false.
   void emitSPMDKernel(const OMPExecutableDirective &D, StringRef ParentName,
                       llvm::Function *&OutlinedFn,
                       llvm::Constant *&OutlinedFnID, bool IsOffloadEntry,
                       const RegionCodeGenTy &CodeGen);
 
+private:
   /// Emit outlined function for 'target' directive on the NVPTX
   /// device.
   /// \param D Directive to emit.
   /// \param ParentName Name of the function that encloses the target region.
   /// \param OutlinedFn Outlined function value to be defined by this call.
   /// \param OutlinedFnID Outlined function ID value to be defined by this call.
   /// \param IsOffloadEntry True if the outlined function is an offload entry.
   /// An outlined function may not be an entry if, e.g. the if clause always
protected:
   bool isDefaultLocationConstant() const override { return true; }
 
   /// Returns additional flags that can be stored in reserved_2 field of the
   /// default location.
   /// For NVPTX target contains data about SPMD/Non-SPMD execution mode +
   /// Full/Lightweight runtime mode. Used for better optimization.
   unsigned getDefaultLocationReserved2Flags() const override;
 
 public:
   explicit CGOpenMPRuntimeGPU(CodeGenModule &CGM);
   void clear() override;
 
   /// Declare generalized virtual functions which need to be defined
   /// by all specializations of OpenMPGPURuntime Targets like AMDGCN
   /// and NVPTX.
 
   /// Get the GPU warp size.
   virtual llvm::Value *getGPUWarpSize(CodeGenFunction &CGF) = 0;
 
   /// Get the id of the current thread on the GPU.
   virtual llvm::Value *getGPUThreadID(CodeGenFunction &CGF) = 0;
 
   /// Get the maximum number of threads in a block of the GPU.
   virtual llvm::Value *getGPUNumThreads(CodeGenFunction &CGF) = 0;
 
+  /// Target independent wrapper over target specific emitSPMDKernel()
+  virtual void emitSPMDKernelWrapper(const OMPExecutableDirective &D,
+                                     StringRef ParentName,
+                                     llvm::Function *&OutlinedFn,
+                                     llvm::Constant *&OutlinedFnID,
+                                     bool IsOffloadEntry,
+                                     const RegionCodeGenTy &CodeGen) = 0;
+
+  /// Target independent wrapper over target specific emitNonSPMDKernel()
+  virtual void emitNonSPMDKernelWrapper(const OMPExecutableDirective &D,
+                                        StringRef ParentName,
+                                        llvm::Function *&OutlinedFn,
+                                        llvm::Constant *&OutlinedFnID,
+                                        bool IsOffloadEntry,
+                                        const RegionCodeGenTy &CodeGen) = 0;
+
   /// Emit call to void __kmpc_push_proc_bind(ident_t *loc, kmp_int32
   /// global_tid, int proc_bind) to generate code for 'proc_bind' clause.
   virtual void emitProcBindClause(CodeGenFunction &CGF,
                                   llvm::omp::ProcBindKind ProcBind,
                                   SourceLocation Loc) override;
 
   /// Emits call to void __kmpc_push_num_threads(ident_t *loc, kmp_int32
   /// global_tid, kmp_int32 num_threads) to generate code for 'num_threads'
   /// clause.
   /// \param NumThreads An integer value of threads.
   virtual void emitNumThreadsClause(CodeGenFunction &CGF,
                                     llvm::Value *NumThreads,
                                     SourceLocation Loc) override;
 
   /// This function ought to emit, in the general case, a call to
   // the openmp runtime kmpc_push_num_teams. In NVPTX backend it is not needed
   // as these numbers are obtained through the PTX grid and block configuration.
   /// \param NumTeams An integer expression of teams.
   /// \param ThreadLimit An integer expression of threads.
   void emitNumTeamsClause(CodeGenFunction &CGF, const Expr *NumTeams,
                           const Expr *ThreadLimit, SourceLocation Loc) override;
 
   /// Emits inlined function for the specified OpenMP parallel
   //  directive.
   /// \a D. This outlined function has type void(*)(kmp_int32 *ThreadID,
   /// kmp_int32 BoundID, struct context_vars*).
   /// \param D OpenMP directive.
   /// \param ThreadIDVar Variable for thread id in the current OpenMP region.
   /// \param InnermostKind Kind of innermost directive (for simple directives it
   /// is a directive itself, for combined - its innermost directive).
   /// \param CodeGen Code generation sequence for the \a D directive.
   llvm::Function *
   emitParallelOutlinedFunction(const OMPExecutableDirective &D,
                                const VarDecl *ThreadIDVar,
                                OpenMPDirectiveKind InnermostKind,
                                const RegionCodeGenTy &CodeGen) override;
 
   /// Emits inlined function for the specified OpenMP teams
   //  directive.
   /// \a D. This outlined function has type void(*)(kmp_int32 *ThreadID,
   /// kmp_int32 BoundID, struct context_vars*).
   /// \param D OpenMP directive.
   /// \param ThreadIDVar Variable for thread id in the current OpenMP region.
   /// \param InnermostKind Kind of innermost directive (for simple directives it
   /// is a directive itself, for combined - its innermost directive).
   /// \param CodeGen Code generation sequence for the \a D directive.
   llvm::Function *
   emitTeamsOutlinedFunction(const OMPExecutableDirective &D,
                             const VarDecl *ThreadIDVar,
public:
   /// their declaration context.
   enum DataSharingMode {
     /// CUDA data sharing mode.
     CUDA,
     /// Generic data-sharing mode.
     Generic,
   };
 
   /// Cleans up references to the objects in finished function.
   ///
   void functionFinished(CodeGenFunction &CGF) override;
 
   /// Choose a default value for the dist_schedule clause.
   void getDefaultDistScheduleAndChunk(CodeGenFunction &CGF,
       const OMPLoopDirective &S, OpenMPDistScheduleClauseKind &ScheduleKind,
       llvm::Value *&Chunk) const override;
 
   /// Choose a default value for the schedule clause.
   void getDefaultScheduleAndChunk(CodeGenFunction &CGF,

clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp

 //===---- CGOpenMPRuntimeGPU.cpp - Interface to OpenMP GPU Runtimes ----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This provides a generalized class for OpenMP runtime code generation
 // specialized by GPU targets NVPTX and AMDGCN.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CGOpenMPRuntimeGPU.h"
+#include "CGOpenMPRuntimeAMDGCN.h"
 #include "CGOpenMPRuntimeNVPTX.h"
 #include "CodeGenFunction.h"
 #include "clang/AST/Attr.h"
 #include "clang/AST/DeclOpenMP.h"
 #include "clang/AST/StmtOpenMP.h"
 #include "clang/AST/StmtVisitor.h"
 #include "clang/Basic/Cuda.h"
 #include "llvm/ADT/SmallPtrSet.h"
void Exit(CodeGenFunction &CGF) override {
       RT.emitNonSPMDEntryFooter(CGF, EST);
     }
   } Action(EST, WST);
   CodeGen.setAction(Action);
   IsInTTDRegion = true;
   // Reserve place for the globalized memory.
   GlobalizedRecords.emplace_back();
   if (!KernelStaticGlobalized) {
+    auto &RT = static_cast<CGOpenMPRuntimeGPU &>(CGM.getOpenMPRuntime());
     KernelStaticGlobalized = new llvm::GlobalVariable(
         CGM.getModule(), CGM.VoidPtrTy, /*isConstant=*/false,
-        llvm::GlobalValue::InternalLinkage,
-        llvm::UndefValue::get(CGM.VoidPtrTy),
+        RT.KernelStaticGlobalizedLinkage, llvm::UndefValue::get(CGM.VoidPtrTy),
         "_openmp_kernel_static_glob_rd$ptr", /*InsertBefore=*/nullptr,
         llvm::GlobalValue::NotThreadLocal,
         CGM.getContext().getTargetAddressSpace(LangAS::cuda_shared));
   }
   emitTargetOutlinedFunctionHelper(D, ParentName, OutlinedFn, OutlinedFnID,
                                    IsOffloadEntry, CodeGen);
   IsInTTDRegion = false;
 
void Exit(CodeGenFunction &CGF) override {
       RT.emitSPMDEntryFooter(CGF, EST);
     }
   } Action(*this, EST, D);
   CodeGen.setAction(Action);
   IsInTTDRegion = true;
   // Reserve place for the globalized memory.
   GlobalizedRecords.emplace_back();
   if (!KernelStaticGlobalized) {
+    auto &RT = static_cast<CGOpenMPRuntimeGPU &>(CGM.getOpenMPRuntime());
     KernelStaticGlobalized = new llvm::GlobalVariable(
         CGM.getModule(), CGM.VoidPtrTy, /*isConstant=*/false,
-        llvm::GlobalValue::InternalLinkage,
-        llvm::UndefValue::get(CGM.VoidPtrTy),
+        RT.KernelStaticGlobalizedLinkage, llvm::UndefValue::get(CGM.VoidPtrTy),
         "_openmp_kernel_static_glob_rd$ptr", /*InsertBefore=*/nullptr,
         llvm::GlobalValue::NotThreadLocal,
         CGM.getContext().getTargetAddressSpace(LangAS::cuda_shared));
   }
   emitTargetOutlinedFunctionHelper(D, ParentName, OutlinedFn, OutlinedFnID,
                                    IsOffloadEntry, CodeGen);
   IsInTTDRegion = false;
 }

 // 'generic', the runtime reserves one warp for the master, otherwise, all
 // warps participate in parallel work.
 static void setPropertyExecutionMode(CodeGenModule &CGM, StringRef Name,
                                      bool Mode) {
   auto *GVMode =
       new llvm::GlobalVariable(CGM.getModule(), CGM.Int8Ty, /*isConstant=*/true,
                                llvm::GlobalValue::WeakAnyLinkage,
                                llvm::ConstantInt::get(CGM.Int8Ty, Mode ? 0 : 1),
                                Twine(Name, "_exec_mode"));
   CGM.addCompilerUsedGlobal(GVMode);
 }
 
 void CGOpenMPRuntimeGPU::emitWorkerFunction(WorkerFunctionState &WST) {
   ASTContext &Ctx = CGM.getContext();
 
   CodeGenFunction CGF(CGM, /*suppressNewContext=*/true);
   CGF.StartFunction(GlobalDecl(), Ctx.VoidTy, WST.WorkerFn, WST.CGFI, {},
                     WST.Loc, WST.Loc);
void CGOpenMPRuntimeGPU::emitTargetOutlinedFunction(
     bool IsOffloadEntry, const RegionCodeGenTy &CodeGen) {
   if (!IsOffloadEntry) // Nothing to do.
     return;
 
   assert(!ParentName.empty() && "Invalid target region parent name!");
 
   bool Mode = supportsSPMDExecutionMode(CGM.getContext(), D);
   if (Mode)
-    emitSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
-                   CodeGen);
+    emitSPMDKernelWrapper(D, ParentName, OutlinedFn, OutlinedFnID,
+                          IsOffloadEntry, CodeGen);
   else
-    emitNonSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
-                      CodeGen);
+    emitNonSPMDKernelWrapper(D, ParentName, OutlinedFn, OutlinedFnID,
+                             IsOffloadEntry, CodeGen);
 
   setPropertyExecutionMode(CGM, OutlinedFn->getName(), Mode);
 }
 
 namespace {
 LLVM_ENABLE_BITMASK_ENUMS_IN_NAMESPACE();
 /// Enum for accesseing the reserved_2 field of the ident_t struct.
 enum ModeFlagsTy : unsigned {
void Enter(CodeGenFunction &CGF) override {
       PrevIsInParallelRegion = IsInParallelRegion;
       IsInParallelRegion = true;
     }
     void Exit(CodeGenFunction &CGF) override {
       IsInParallelRegion = PrevIsInParallelRegion;
     }
   } Action(IsInParallelRegion);
   CodeGen.setAction(Action);
   bool PrevIsInTTDRegion = IsInTTDRegion;
   IsInTTDRegion = false;
   bool PrevIsInTargetMasterThreadRegion = IsInTargetMasterThreadRegion;
   IsInTargetMasterThreadRegion = false;
   auto *OutlinedFun =
       cast<llvm::Function>(CGOpenMPRuntime::emitParallelOutlinedFunction(
           D, ThreadIDVar, InnermostKind, CodeGen));
   if (CGM.getLangOpts().Optimize) {
     OutlinedFun->removeFnAttr(llvm::Attribute::NoInline);

clang/lib/CodeGen/CGOpenMPRuntimeNVPTX.h

 namespace clang {
 namespace CodeGen {
 
 class CGOpenMPRuntimeNVPTX final : public CGOpenMPRuntimeGPU {
 
 public:
   explicit CGOpenMPRuntimeNVPTX(CodeGenModule &CGM);
 
+private:
   /// Get the GPU warp size.
   llvm::Value *getGPUWarpSize(CodeGenFunction &CGF) override;
 
   /// Get the id of the current thread on the GPU.
   llvm::Value *getGPUThreadID(CodeGenFunction &CGF) override;
 
   /// Get the maximum number of threads in a block of the GPU.
   llvm::Value *getGPUNumThreads(CodeGenFunction &CGF) override;
+
+  /// Target independent wrapper over target specific emitSPMDKernel()
+  void emitSPMDKernelWrapper(const OMPExecutableDirective &D,
+                             StringRef ParentName, llvm::Function *&OutlinedFn,
+                             llvm::Constant *&OutlinedFnID, bool IsOffloadEntry,
+                             const RegionCodeGenTy &CodeGen) override;
+
+  /// Target independent wrapper over target specific emitNonSPMDKernel()
+  void emitNonSPMDKernelWrapper(const OMPExecutableDirective &D,
+                                StringRef ParentName,
+                                llvm::Function *&OutlinedFn,
+                                llvm::Constant *&OutlinedFnID,
+                                bool IsOffloadEntry,
+                                const RegionCodeGenTy &CodeGen) override;
 };
 
 } // CodeGen namespace.
 } // clang namespace.
 
 #endif // LLVM_CLANG_LIB_CODEGEN_CGOPENMPRUNTIMENVPTX_H

clang/lib/CodeGen/CGOpenMPRuntimeNVPTX.cpp

 using namespace clang;
 using namespace CodeGen;
 using namespace llvm::omp;
 
 CGOpenMPRuntimeNVPTX::CGOpenMPRuntimeNVPTX(CodeGenModule &CGM)
     : CGOpenMPRuntimeGPU(CGM) {
   if (!CGM.getLangOpts().OpenMPIsDevice)
     llvm_unreachable("OpenMP NVPTX can only handle device code.");
+  KernelStaticGlobalizedLinkage = llvm::GlobalValue::InternalLinkage;
 }
 
 llvm::Value *CGOpenMPRuntimeNVPTX::getGPUWarpSize(CodeGenFunction &CGF) {
   return CGF.EmitRuntimeCall(
       llvm::Intrinsic::getDeclaration(
           &CGF.CGM.getModule(), llvm::Intrinsic::nvvm_read_ptx_sreg_warpsize),
       "nvptx_warp_size");
 }
 
 llvm::Value *CGOpenMPRuntimeNVPTX::getGPUThreadID(CodeGenFunction &CGF) {
   CGBuilderTy &Bld = CGF.Builder;
   llvm::Function *F;
   F = llvm::Intrinsic::getDeclaration(
       &CGF.CGM.getModule(), llvm::Intrinsic::nvvm_read_ptx_sreg_tid_x);
   return Bld.CreateCall(F, llvm::None, "nvptx_tid");
 }
 
 llvm::Value *CGOpenMPRuntimeNVPTX::getGPUNumThreads(CodeGenFunction &CGF) {
   CGBuilderTy &Bld = CGF.Builder;
   llvm::Function *F;
   F = llvm::Intrinsic::getDeclaration(
       &CGF.CGM.getModule(), llvm::Intrinsic::nvvm_read_ptx_sreg_ntid_x);
   return Bld.CreateCall(F, llvm::None, "nvptx_num_threads");
 }
+
+void CGOpenMPRuntimeNVPTX::emitSPMDKernelWrapper(
+    const OMPExecutableDirective &D, StringRef ParentName,
+    llvm::Function *&OutlinedFn, llvm::Constant *&OutlinedFnID,
+    bool IsOffloadEntry, const RegionCodeGenTy &CodeGen) {
+  emitSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
+                 CodeGen);
+}
+
+void CGOpenMPRuntimeNVPTX::emitNonSPMDKernelWrapper(
+    const OMPExecutableDirective &D, StringRef ParentName,
+    llvm::Function *&OutlinedFn, llvm::Constant *&OutlinedFnID,
+    bool IsOffloadEntry, const RegionCodeGenTy &CodeGen) {
+  emitNonSPMDKernel(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry,
+                    CodeGen);
+}