[OpenCL] Add global_device and global_host address spaces

[sidorovd]

Summary

This patch introduces 2 new address spaces in OpenCL: global_device and global_host
which are a subset of a global address space, so the address space scheme will be
looking like:

generic->global->host
                          ->device
             ->private
             ->local
constant

Justification: USM allocations may be associated with both host and device memory. We
want to give users a way to tell the compiler the allocation type of a USM pointer for
optimization purposes. (Link to the Unified Shared Memory extension:
https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/USM/cl_intel_unified_shared_memory.asciidoc)

Before this patch USM pointer could be only in opencl_global
address space, hence a device backend can't tell if a particular pointer
points to host or device memory. On FPGAs at least we can generate more
efficient hardware code if the user tells us where the pointer can point -
being able to distinguish between these types of pointers at compile time
allows us to instantiate simpler load-store units to perform memory
transactions.

[Anastasia]

I think you should also test the conversions of address spaces and it would be good to add a cl file i.e. OpenCL C level test.

clang/include/clang/Basic/AttrDocs.td

3132

USM abbreviation is not clear here. I suggest to writenit in full.

Is there a language extension spec for this or any other form of documentation already?

If not I would suggest explaining the conversions rules/relation to other OpenCL address spaces here. Alternatively you can also consider describing this in Claqng extensions: https://clang.llvm.org/docs/LanguageExtensions.html

clang/lib/CodeGen/CodeGenModule.cpp

1340

This is numbers are governed by SPIR that doesn't have this address spaces. Any reasons not to use continuous numbering?

[sidorovd]

clang/include/clang/Basic/AttrDocs.td

3132

Is there a language extension spec for this or any other form of documentation already?

Only for SYCL at this point: https://github.com/intel/llvm/commit/66bdf4a6c753448376aec1d7411dc626c2c5cf1c

Okay, I'll expand attribute documentation here, thanks!

clang/lib/CodeGen/CodeGenModule.cpp

1340

I didn't use continuous numbering because rest number are occupied by other target in clang, for example numbers from 5 to 7 are assigned to cuda_device, cuda_constant and cuda_shared address spaces. Do you mean that within SPIR target we can overlap with address spaces from different targets?

[Anastasia]

Btw could you upload a full diff, please?
https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface
It will simplify the review....

clang/lib/CodeGen/CodeGenModule.cpp

1340

I think here we are using SPIR address spaces - not the same as Clang internal address spaces?

[Anastasia]

clang/lib/CodeGen/CodeGenModule.cpp

1340

I.e. this is just used in the metadata so technically this should just be the same as corresponding address spaces in the address space map of SPIR target which is what you have. However, you could also just use continuous numbering I guess i.e. 5,6?

[Anastasia]

clang/include/clang/Basic/AttrDocs.td

3132

Btw do I understand it correctly that global_host means it is visible to the host and the device as opposed to global_device that is only visible on the device?

[sidorovd]

Uploaded the full diff. I have already forgot have to Phabricator properly :)

In the update I addressed:

1. Updated info in AttrDocs;
2. Added test in address space conversion.

TODO: update address space numerical values for SPIR target
Regarding the TODO. Still I like an idea of detached address spaces a little bit more, but I see your point. I would love to check how it works with SPIR-V address space mapping in SPIR-V to LLVM IR translator. I know, it shall have a little impact on a clang work, but still what to see the work I have to do there first (5 and 6 are already occupied there).

[sidorovd]

clang/include/clang/Basic/AttrDocs.td

3132

Yes!

[aaron.ballman]

clang/include/clang/Basic/Attr.td

1183

I'd appreciate a slightly different name here. OpenCLGlobalAddressSpacesDocs and OpenCLAddressSpaceGlobalDocs are pretty similar names. In fact, given how this relates to opencl_global, perhaps that documentation should be expanded instead of introducing a different doc category.

clang/include/clang/Basic/AttrDocs.td

3130

You should use backticks instead of parens for the attribute names (and fully spell them out), as in other parts of the docs.

3131

distinguishing -> distinguish

3132

"pointers that access device memory that access global memory" -- I'm not certain what you're trying to say with this, but I also don't know the domain very well. Do you mean "pointers that access global device memory from those that access global host memory"?

3134

of the opencl_global (add the the), also put backticks around opencl_global.

3135

is looking like -> looks like

3142–3143

Backticks around syntax identifiers.

[Anastasia]

TODO: update address space numerical values for SPIR target
Regarding the TODO. Still I like an idea of detached address spaces a little bit more, but I see your point. I would love to check how it works with SPIR-V address space mapping in SPIR-V to LLVM IR translator. I know, it shall have a little impact on a clang work, but still what to see the work I have to do there first (5 and 6 are already occupied there).

Makes sense. I think conversion to SPIR-V is one of the primary use cases. However, if checking doesn't need a lot of time it would be better to commit with the final numbering scheme. Otherwise things like this are pretty hard to change once they start to be used.

[sidorovd]

clang/include/clang/Basic/Attr.td

1183

I would rather leave it in a separate doc as it's a vendor extension now. If/when we collect a positive feedback I don't mind to merge it with OpenCLAddressSpaceGlobalDocs. What about OpenCLGlobalDeviceHostAddressSpacesDocs?

[sidorovd]

In D82174#2109491, @Anastasia wrote:

TODO: update address space numerical values for SPIR target
Regarding the TODO. Still I like an idea of detached address spaces a little bit more, but I see your point. I would love to check how it works with SPIR-V address space mapping in SPIR-V to LLVM IR translator. I know, it shall have a little impact on a clang work, but still what to see the work I have to do there first (5 and 6 are already occupied there).

Makes sense. I think conversion to SPIR-V is one of the primary use cases. However, if checking doesn't need a lot of time it would be better to commit with the final numbering scheme. Otherwise things like this are pretty hard to change once they start to be used.

I have checked it. Values 5 and 6 are occupied for mapping Input and Output storage classes and I have a feeling that they are actually mapped on some real address spaces from some internal frontend. I asked the author of this change in the translator https://github.com/KhronosGroup/SPIRV-LLVM-Translator/pull/579#discussion_r444377669 if I can move SPIRAS_Input/Output or not, waiting for a reply.

[aaron.ballman]

clang/include/clang/Basic/Attr.td

1183

I feel like the combined docs could be written to clarify it's a vendor extension, but I don't insist on consolidating the docs if that's your preference. How about OpenCLAddressSpaceGlobalExtDocs to make it clear it's an extension related to the other docs?

[sidorovd]

clang/include/clang/Basic/Attr.td

1183

I like it, thanks!

[aaron.ballman]

The attribute bits LGTM, but I have no comment on the OpenCL bits.

[sidorovd]

Made SPIR AS map continuous.

[sidorovd]

In D82174#2109576, @sidorovd wrote:

In D82174#2109491, @Anastasia wrote:

TODO: update address space numerical values for SPIR target
Regarding the TODO. Still I like an idea of detached address spaces a little bit more, but I see your point. I would love to check how it works with SPIR-V address space mapping in SPIR-V to LLVM IR translator. I know, it shall have a little impact on a clang work, but still what to see the work I have to do there first (5 and 6 are already occupied there).

Makes sense. I think conversion to SPIR-V is one of the primary use cases. However, if checking doesn't need a lot of time it would be better to commit with the final numbering scheme. Otherwise things like this are pretty hard to change once they start to be used.

It appeared to be, that SPIRVAS_Input/Output has no mapping on real LLVM address spaces, so I'll just move them in the translator.

I have checked it. Values 5 and 6 are occupied for mapping Input and Output storage classes and I have a feeling that they are actually mapped on some real address spaces from some internal frontend. I asked the author of this change in the translator https://github.com/KhronosGroup/SPIRV-LLVM-Translator/pull/579#discussion_r444377669 if I can move SPIRAS_Input/Output or not, waiting for a reply.

[Anastasia]

clang/include/clang/Basic/AttrDocs.td

3134

Let's add the official spelling too:

opencl_global -> __global/opencl_global

The same applies below.

3134

so address space scheme for OpenCL 2.0 with the extension looks like -> the full address space set model for OpenCL 2.0 with the extension looks as follows:

3142

Do you refer to implicit conversion here? What about converting the other direction i.e. global_host to global?

I would also encourage you to add a reference to embedded C (ISO/IEC TR 18037) section 5 which OpenCL adopts too that explains subsets orverlapping and conversions in details.

clang/lib/AST/ASTContext.cpp

922–935

Perhaps not critical but I believe we could make numeration here continuous too as it is used only for testing purposes I believe. However it might require some tests to be changed.

clang/lib/AST/ItaniumMangle.cpp

2400

Why not to follow the same convention and use lower case CLhost. I would also suggest the full spelling i.e. CLglobalhost

clang/lib/AST/MicrosoftMangle.cpp

1827

The same here, why not _ASCLdevice or even _ASCLglobaldevice

clang/lib/AST/TypePrinter.cpp

1872

This should technically print as a source spelling. Have you considered adding this spelling to clang? Alternatively you could print attribute spelling.

Also you should test this either via diagnostic output or an AST dump.

[sidorovd]

clang/include/clang/Basic/AttrDocs.td

3142

What about converting the other direction i.e. global_host to global?

This possibility is yet being discussed. I'm not ready to disallow this type of conversion of explicitly specify that it's allowed in this patch (or at least right now).

clang/lib/AST/ItaniumMangle.cpp

2400

I think it's a bad idea, since demanglers may consider it to be opencl_global address space + unknown token.

[Anastasia]

clang/include/clang/Basic/AttrDocs.td

3142

Ok, I was asking to document the current behavior of your patch. You can continue the discussion and amend it later on if needed.

But actually is global_host/global_device a subset of global? Then you should not be able to convert the other direction implicitly but only explicitly. Perhaps your testing should be improved in this area.

clang/lib/AST/ItaniumMangle.cpp

2400

I would imagine demanglers would read until the next tag to complete the decoding of the current element.

Also your comment above is inconsistent because it explains the mangling scheme as follows:

"CL" [ "global" | "local" | "constant" |
      //                                "private"| "generic" | "global_device" |
      //                                "global_host" ]

[sidorovd]

Sorry for a late response and thank you very much for the feedback!

clang/include/clang/Basic/AttrDocs.td

3142

Thanks for pointing it out, added the test.

clang/lib/AST/ItaniumMangle.cpp

2400

Made the mangling be CLdevice and CLhost.
I've made some experiments with an online demangler (not specialized on clang and OpenCL).
I took void foo(attribute((opencl_global)) int * Val); function, which as mangled as _Z3fooPU8CLglobali , and with changing it's mangling processed it through this mangler. Here is the result:
_Z3fooPU8CLglobali -> foo(int CLglobal*) wasn't parsed properly, but still OK
_Z3fooPU8CLdevicei -> foo(int CLdevice*) same
_Z3fooPU8CLglobal_devicei -> _Z3fooPU8CLglobal_devicei yeap, it wasn't demangled
_Z3fooPU8CLglobaldevicei -> foo(double CLglobal*, long double, void, int, char, long double, int) wow
_Z3fooPU8CLglobalDevicei ->foo(decimal128 CLglobal*, void, int, char, long double, int) // wow

So I think keeping mangling as easy as possible without overlapping with something known for demanglers is a safer approach.

clang/lib/AST/TypePrinter.cpp

1872

I would prefer to leave AST spelling here to be consistent with other address spaces. The appropriate test is in place.

[Anastasia]

clang/test/SemaOpenCL/usm-address-spaces-conversions.cl

23

Looking good! Can you extend for conversions with other address spaces please? It should result in an error apart from generic should work?

[sidorovd]

clang/test/SemaOpenCL/usm-address-spaces-conversions.cl

23

Expanded the test. I left it in its own source file to distinguish extension testing from the core, but if you have objections - I can merge it with address-spaces-conversions-cl2.0.cl .

[Anastasia]

LGTM, Thanks! Although I left a comment about possible test simplification that can be addressed before committing.

clang/test/SemaOpenCL/usm-address-spaces-conversions.cl

36

I think you can simplify this test by removing this macro. It doesn't seem to change the test coverage and you can minimize the number of run lines...

[sidorovd]

Simplified the test

clang/include/clang/AST/Type.h

public:
   /// CL2.0 adds:
   ///   __generic is a superset of any address space except for __constant.
   static bool isAddressSpaceSupersetOf(LangAS A, LangAS B) {
     // Address spaces must match exactly.
     return A == B ||
            // Otherwise in OpenCLC v2.0 s6.5.5: every address space except
            // for __constant can be used as __generic.
            (A == LangAS::opencl_generic && B != LangAS::opencl_constant) ||
+           // We also define global_device and global_host address spaces,
+           // to distinguish global pointers allocated on host from pointers
+           // allocated on device, which are a subset of __global.
+           (A == LangAS::opencl_global && (B == LangAS::opencl_global_device ||
+                                           B == LangAS::opencl_global_host)) ||
            // Consider pointer size address spaces to be equivalent to default.
            ((isPtrSizeAddressSpace(A) || A == LangAS::Default) &&
             (isPtrSizeAddressSpace(B) || B == LangAS::Default));
   }
 
   /// Returns true if the address space in these qualifiers is equal to or
   /// a superset of the address space in the argument qualifiers.
   bool isAddressSpaceSupersetOf(Qualifiers other) const {

clang/include/clang/Basic/AddressSpaces.h

enum class LangAS : unsigned {
   // In OpenCL each l-value must have certain non-default address space, each
   // r-value must have no address space (i.e. the default address space). The
   // pointee of a pointer must have non-default address space.
   opencl_global,
   opencl_local,
   opencl_constant,
   opencl_private,
   opencl_generic,
+  opencl_global_device,
+  opencl_global_host,
 
   // CUDA specific address spaces.
   cuda_device,
   cuda_constant,
   cuda_shared,
 
   // Pointer size and extension address spaces.
   ptr32_sptr,

clang/include/clang/Basic/Attr.td

def OpenCLPrivateAddressSpace : TypeAttr {
   let Documentation = [OpenCLAddressSpacePrivateDocs];
 }
 
 def OpenCLGlobalAddressSpace : TypeAttr {
   let Spellings = [Keyword<"__global">, Keyword<"global">, Clang<"opencl_global">];
   let Documentation = [OpenCLAddressSpaceGlobalDocs];
 }
 
+def OpenCLGlobalDeviceAddressSpace : TypeAttr {
+  let Spellings = [Clang<"opencl_global_device">];
+  let Documentation = [OpenCLAddressSpaceGlobalExtDocs];
+}
+
+def OpenCLGlobalHostAddressSpace : TypeAttr {
+  let Spellings = [Clang<"opencl_global_host">];
+  let Documentation = [OpenCLAddressSpaceGlobalExtDocs];
+}
+
 def OpenCLLocalAddressSpace : TypeAttr {
   let Spellings = [Keyword<"__local">, Keyword<"local">, Clang<"opencl_local">];
   let Documentation = [OpenCLAddressSpaceLocalDocs];
 }
 
 def OpenCLConstantAddressSpace : TypeAttr {
   let Spellings = [Keyword<"__constant">, Keyword<"constant">, Clang<"opencl_constant">];
   let Documentation = [OpenCLAddressSpaceConstantDocs];

clang/include/clang/Basic/AttrDocs.td

 global memory, which is accessible by all work items. The content stored in this
 memory area persists between kernel executions. Pointer types to the global
 address space are allowed as function parameters or local variables. Starting
 with OpenCL v2.0, the global address space can be used with global (program
 scope) variables and static local variable as well.
   }];
 }
 
+def OpenCLAddressSpaceGlobalExtDocs : Documentation {
+  let Category = DocOpenCLAddressSpaces;
+  let Heading = "[[clang::opencl_global_device]], [[clang::opencl_global_host]]";
+  let Content = [{
+The ``global_device`` and ``global_host`` address space attributes specify that
+an object is allocated in global memory on the device/host. It helps to
+distinguish USM (Unified Shared Memory) pointers that access global device
+memory from those that access global host memory. These new address spaces are
+a subset of the ``__global/opencl_global`` address space, the full address space
+set model for OpenCL 2.0 with the extension looks as follows:
+  generic->global->host
+                 ->device
+         ->private
+         ->local
+  constant
+
+As ``global_device`` and ``global_host`` are a subset of
+``__global/opencl_global`` address spaces it is allowed to convert
+``global_device`` and ``global_host`` address spaces to
+``__global/opencl_global`` address spaces (following ISO/IEC TR 18037 5.1.3
+"Address space nesting and rules for pointers).
+  }];
+}
+
 def OpenCLAddressSpaceLocalDocs : Documentation {
   let Category = DocOpenCLAddressSpaces;
   let Heading = "__local, local, [[clang::opencl_local]]";
   let Content = [{
 The local address space specifies that an object is allocated in the local (work
 group) memory area, which is accessible to all work items in the same work
 group. The content stored in this memory region is not accessible after
 the kernel execution ends. In a kernel function scope, any variable can be in

clang/include/clang/Sema/ParsedAttr.h

public:
   /// If this is an OpenCL addr space attribute returns its representation
   /// in LangAS, otherwise returns default addr space.
   LangAS asOpenCLLangAS() const {
     switch (getParsedKind()) {
     case ParsedAttr::AT_OpenCLConstantAddressSpace:
       return LangAS::opencl_constant;
     case ParsedAttr::AT_OpenCLGlobalAddressSpace:
       return LangAS::opencl_global;
+    case ParsedAttr::AT_OpenCLGlobalDeviceAddressSpace:
+      return LangAS::opencl_global_device;
+    case ParsedAttr::AT_OpenCLGlobalHostAddressSpace:
+      return LangAS::opencl_global_host;
     case ParsedAttr::AT_OpenCLLocalAddressSpace:
       return LangAS::opencl_local;
     case ParsedAttr::AT_OpenCLPrivateAddressSpace:
       return LangAS::opencl_private;
     case ParsedAttr::AT_OpenCLGenericAddressSpace:
       return LangAS::opencl_generic;
     default:
       return LangAS::Default;

clang/lib/AST/ASTContext.cpp

 }
 
 static const LangASMap *getAddressSpaceMap(const TargetInfo &T,
                                            const LangOptions &LOpts) {
   if (LOpts.FakeAddressSpaceMap) {
     // The fake address space map must have a distinct entry for each
     // language-specific address space.
     static const unsigned FakeAddrSpaceMap[] = {
-        0, // Default
-        1, // opencl_global
-        3, // opencl_local
-        2, // opencl_constant
-        0, // opencl_private
-        4, // opencl_generic
-        5, // cuda_device
-        6, // cuda_constant
-        7, // cuda_shared
-        8, // ptr32_sptr
-        9, // ptr32_uptr
-        10 // ptr64
+        0,  // Default
+        1,  // opencl_global
+        3,  // opencl_local
+        2,  // opencl_constant
+        0,  // opencl_private
+        4,  // opencl_generic
+        5,  // opencl_global_device
+        6,  // opencl_global_host
+        7,  // cuda_device
+        8,  // cuda_constant
+        9,  // cuda_shared
+        10, // ptr32_sptr
+        11, // ptr32_uptr
+        12  // ptr64
     };
     return &FakeAddrSpaceMap;
   } else {
     return &T.getAddressSpaceMap();
   }
 }
 
 static bool isAddrSpaceMapManglingEnabled(const TargetInfo &TI,

clang/lib/AST/ItaniumMangle.cpp

if (Context.getASTContext().addressSpaceMapManglingFor(AS)) {
       //  <target-addrspace> ::= "AS" <address-space-number>
       unsigned TargetAS = Context.getASTContext().getTargetAddressSpace(AS);
       if (TargetAS != 0)
         ASString = "AS" + llvm::utostr(TargetAS);
     } else {
       switch (AS) {
       default: llvm_unreachable("Not a language specific address space");
       //  <OpenCL-addrspace> ::= "CL" [ "global" | "local" | "constant" |
-      //                                "private"| "generic" ]
-      case LangAS::opencl_global:   ASString = "CLglobal";   break;
-      case LangAS::opencl_local:    ASString = "CLlocal";    break;
-      case LangAS::opencl_constant: ASString = "CLconstant"; break;
-      case LangAS::opencl_private:  ASString = "CLprivate";  break;
-      case LangAS::opencl_generic:  ASString = "CLgeneric";  break;
+      //                                "private"| "generic" | "device" |
+      //                                "host" ]
+      case LangAS::opencl_global:
+        ASString = "CLglobal";
+        break;
+      case LangAS::opencl_global_device:
+        ASString = "CLdevice";
+        break;
+      case LangAS::opencl_global_host:
+        ASString = "CLhost";
+        break;
+      case LangAS::opencl_local:
+        ASString = "CLlocal";
+        break;
+      case LangAS::opencl_constant:
+        ASString = "CLconstant";
+        break;
+      case LangAS::opencl_private:
+        ASString = "CLprivate";
+        break;
+      case LangAS::opencl_generic:
+        ASString = "CLgeneric";
+        break;
       //  <CUDA-addrspace> ::= "CU" [ "device" | "constant" | "shared" ]
-      case LangAS::cuda_device:     ASString = "CUdevice";   break;
-      case LangAS::cuda_constant:   ASString = "CUconstant"; break;
-      case LangAS::cuda_shared:     ASString = "CUshared";   break;
+      case LangAS::cuda_device:
+        ASString = "CUdevice";
+        break;
+      case LangAS::cuda_constant:
+        ASString = "CUconstant";
+        break;
+      case LangAS::cuda_shared:
+        ASString = "CUshared";
+        break;
       //  <ptrsize-addrspace> ::= [ "ptr32_sptr" | "ptr32_uptr" | "ptr64" ]
       case LangAS::ptr32_sptr:
         ASString = "ptr32_sptr";
         break;
       case LangAS::ptr32_uptr:
         ASString = "ptr32_uptr";
         break;
       case LangAS::ptr64:

clang/lib/AST/MicrosoftMangle.cpp

void MicrosoftCXXNameMangler::mangleAddressSpaceType(QualType T,
                                                      SourceRange Range) {
   // Address space is mangled as an unqualified templated type in the __clang
   // namespace. The demangled version of this is:
   // In the case of a language specific address space:
   // __clang::struct _AS[language_addr_space]<Type>
   // where:
   //  <language_addr_space> ::= <OpenCL-addrspace> | <CUDA-addrspace>
   //    <OpenCL-addrspace> ::= "CL" [ "global" | "local" | "constant" |
-  //                                "private"| "generic" ]
+  //                                "private"| "generic" | "device" | "host" ]
   //    <CUDA-addrspace> ::= "CU" [ "device" | "constant" | "shared" ]
   //    Note that the above were chosen to match the Itanium mangling for this.
   //
   // In the case of a non-language specific address space:
   //  __clang::struct _AS<TargetAS, Type>
   assert(Quals.hasAddressSpace() && "Not valid without address space");
   llvm::SmallString<32> ASMangling;
   llvm::raw_svector_ostream Stream(ASMangling);
   MicrosoftCXXNameMangler Extra(Context, Stream);
   Stream << "?$";
 
   LangAS AS = Quals.getAddressSpace();
   if (Context.getASTContext().addressSpaceMapManglingFor(AS)) {
     unsigned TargetAS = Context.getASTContext().getTargetAddressSpace(AS);
     Extra.mangleSourceName("_AS");
     Extra.mangleIntegerLiteral(llvm::APSInt::getUnsigned(TargetAS),
                                /*IsBoolean*/ false);
   } else {
     switch (AS) {
     default:
       llvm_unreachable("Not a language specific address space");
     case LangAS::opencl_global:
       Extra.mangleSourceName("_ASCLglobal");
       break;
+    case LangAS::opencl_global_device:
+      Extra.mangleSourceName("_ASCLdevice");
+      break;
+    case LangAS::opencl_global_host:
+      Extra.mangleSourceName("_ASCLhost");
+      break;
     case LangAS::opencl_local:
       Extra.mangleSourceName("_ASCLlocal");
       break;
     case LangAS::opencl_constant:
       Extra.mangleSourceName("_ASCLconstant");
       break;
     case LangAS::opencl_private:
       Extra.mangleSourceName("_ASCLprivate");

clang/lib/AST/TypePrinter.cpp

 #define TYPE_ATTR(NAME)
 #define DECL_OR_TYPE_ATTR(NAME)
 #define ATTR(NAME) case attr::NAME:
 #include "clang/Basic/AttrList.inc"
     llvm_unreachable("non-type attribute attached to type");
 
   case attr::OpenCLPrivateAddressSpace:
   case attr::OpenCLGlobalAddressSpace:
+  case attr::OpenCLGlobalDeviceAddressSpace:
+  case attr::OpenCLGlobalHostAddressSpace:
   case attr::OpenCLLocalAddressSpace:
   case attr::OpenCLConstantAddressSpace:
   case attr::OpenCLGenericAddressSpace:
     // FIXME: Update printAttributedBefore to print these once we generate
     // AttributedType nodes for them.
     break;
 
   case attr::LifetimeBound:
std::string Qualifiers::getAddrSpaceAsString(LangAS AS) {
   case LangAS::opencl_local:
     return "__local";
   case LangAS::opencl_private:
     return "__private";
   case LangAS::opencl_constant:
     return "__constant";
   case LangAS::opencl_generic:
     return "__generic";
+  case LangAS::opencl_global_device:
+    return "__global_device";
+  case LangAS::opencl_global_host:
+    return "__global_host";
   case LangAS::cuda_device:
     return "__device__";
   case LangAS::cuda_constant:
     return "__constant__";
   case LangAS::cuda_shared:
     return "__shared__";
   case LangAS::ptr32_sptr:
     return "__sptr __ptr32";

clang/lib/Basic/Targets/AMDGPU.cpp

 
 const LangASMap AMDGPUTargetInfo::AMDGPUDefIsGenMap = {
     Generic,  // Default
     Global,   // opencl_global
     Local,    // opencl_local
     Constant, // opencl_constant
     Private,  // opencl_private
     Generic,  // opencl_generic
+    Global,   // opencl_global_device
+    Global,   // opencl_global_host
     Global,   // cuda_device
     Constant, // cuda_constant
     Local,    // cuda_shared
     Generic,  // ptr32_sptr
     Generic,  // ptr32_uptr
     Generic   // ptr64
 };
 
 const LangASMap AMDGPUTargetInfo::AMDGPUDefIsPrivMap = {
     Private,  // Default
     Global,   // opencl_global
     Local,    // opencl_local
     Constant, // opencl_constant
     Private,  // opencl_private
     Generic,  // opencl_generic
+    Global,   // opencl_global_device
+    Global,   // opencl_global_host
     Global,   // cuda_device
     Constant, // cuda_constant
     Local,    // cuda_shared
     Generic,  // ptr32_sptr
     Generic,  // ptr32_uptr
     Generic   // ptr64
 
 };

clang/lib/Basic/Targets/NVPTX.h

 static const unsigned NVPTXAddrSpaceMap[] = {
     0, // Default
     1, // opencl_global
     3, // opencl_local
     4, // opencl_constant
     0, // opencl_private
     // FIXME: generic has to be added to the target
     0, // opencl_generic
+    1, // opencl_global_device
+    1, // opencl_global_host
     1, // cuda_device
     4, // cuda_constant
     3, // cuda_shared
     0, // ptr32_sptr
     0, // ptr32_uptr
     0  // ptr64
 };
 

clang/lib/Basic/Targets/SPIR.h

 
 static const unsigned SPIRAddrSpaceMap[] = {
     0, // Default
     1, // opencl_global
     3, // opencl_local
     2, // opencl_constant
     0, // opencl_private
     4, // opencl_generic
+    5, // opencl_global_device
+    6, // opencl_global_host
     0, // cuda_device
     0, // cuda_constant
     0, // cuda_shared
     0, // ptr32_sptr
     0, // ptr32_uptr
     0  // ptr64
 };
 

clang/lib/Basic/Targets/TCE.h

 // publicly available in http://tce.cs.tut.fi
 
 static const unsigned TCEOpenCLAddrSpaceMap[] = {
     0, // Default
     3, // opencl_global
     4, // opencl_local
     5, // opencl_constant
     0, // opencl_private
+    1, // opencl_global_device
+    1, // opencl_global_host
     // FIXME: generic has to be added to the target
     0, // opencl_generic
     0, // cuda_device
     0, // cuda_constant
     0, // cuda_shared
     0, // ptr32_sptr
     0, // ptr32_uptr
     0, // ptr64

clang/lib/Basic/Targets/X86.h

 
 static const unsigned X86AddrSpaceMap[] = {
     0,   // Default
     0,   // opencl_global
     0,   // opencl_local
     0,   // opencl_constant
     0,   // opencl_private
     0,   // opencl_generic
+    0,   // opencl_global_device
+    0,   // opencl_global_host
     0,   // cuda_device
     0,   // cuda_constant
     0,   // cuda_shared
     270, // ptr32_sptr
     271, // ptr32_uptr
     272  // ptr64
 };
 

clang/lib/CodeGen/CodeGenModule.cpp

 // Returns the address space id that should be produced to the
 // kernel_arg_addr_space metadata. This is always fixed to the ids
 // as specified in the SPIR 2.0 specification in order to differentiate
 // for example in clGetKernelArgInfo() implementation between the address
 // spaces with targets without unique mapping to the OpenCL address spaces
 // (basically all single AS CPUs).
 static unsigned ArgInfoAddressSpace(LangAS AS) {
   switch (AS) {
-  case LangAS::opencl_global:   return 1;
-  case LangAS::opencl_constant: return 2;
-  case LangAS::opencl_local:    return 3;
-  case LangAS::opencl_generic:  return 4; // Not in SPIR 2.0 specs.
+  case LangAS::opencl_global:
+    return 1;
+  case LangAS::opencl_constant:
+    return 2;
+  case LangAS::opencl_local:
+    return 3;
+  case LangAS::opencl_generic:
+    return 4; // Not in SPIR 2.0 specs.
+  case LangAS::opencl_global_device:
+    return 5;
+  case LangAS::opencl_global_host:
+    return 6;
   default:
     return 0; // Assume private.
   }
 }
 
 void CodeGenModule::GenOpenCLArgMetadata(llvm::Function *Fn,
                                          const FunctionDecl *FD,
                                          CodeGenFunction *CGF) {
   assert(((FD && CGF) || (!FD && !CGF)) &&
          "Incorrect use - FD and CGF should either be both null or not!");
return Context.toCharUnitsFromBits(
       getDataLayout().getTypeStoreSizeInBits(Ty));
 }
 
 LangAS CodeGenModule::GetGlobalVarAddressSpace(const VarDecl *D) {
   LangAS AddrSpace = LangAS::Default;
   if (LangOpts.OpenCL) {
     AddrSpace = D ? D->getType().getAddressSpace() : LangAS::opencl_global;
     assert(AddrSpace == LangAS::opencl_global ||
+           AddrSpace == LangAS::opencl_global_device ||
+           AddrSpace == LangAS::opencl_global_host ||
            AddrSpace == LangAS::opencl_constant ||
            AddrSpace == LangAS::opencl_local ||
            AddrSpace >= LangAS::FirstTargetAddressSpace);
     return AddrSpace;
   }
 
   if (LangOpts.CUDA && LangOpts.CUDAIsDevice) {
     if (D && D->hasAttr<CUDAConstantAttr>())

clang/lib/Sema/SemaType.cpp

 }
 
 static bool isAddressSpaceKind(const ParsedAttr &attr) {
   auto attrKind = attr.getKind();
 
   return attrKind == ParsedAttr::AT_AddressSpace ||
          attrKind == ParsedAttr::AT_OpenCLPrivateAddressSpace ||
          attrKind == ParsedAttr::AT_OpenCLGlobalAddressSpace ||
+         attrKind == ParsedAttr::AT_OpenCLGlobalDeviceAddressSpace ||
+         attrKind == ParsedAttr::AT_OpenCLGlobalHostAddressSpace ||
          attrKind == ParsedAttr::AT_OpenCLLocalAddressSpace ||
          attrKind == ParsedAttr::AT_OpenCLConstantAddressSpace ||
          attrKind == ParsedAttr::AT_OpenCLGenericAddressSpace;
 }
 
 static void processTypeAttrs(TypeProcessingState &state, QualType &type,
                              TypeAttrLocation TAL,
                              ParsedAttributesView &attrs) {
for (ParsedAttr &attr : AttrsCopy) {
 
     case ParsedAttr::AT_MayAlias:
       // FIXME: This attribute needs to actually be handled, but if we ignore
       // it it breaks large amounts of Linux software.
       attr.setUsedAsTypeAttr();
       break;
     case ParsedAttr::AT_OpenCLPrivateAddressSpace:
     case ParsedAttr::AT_OpenCLGlobalAddressSpace:
+    case ParsedAttr::AT_OpenCLGlobalDeviceAddressSpace:
+    case ParsedAttr::AT_OpenCLGlobalHostAddressSpace:
     case ParsedAttr::AT_OpenCLLocalAddressSpace:
     case ParsedAttr::AT_OpenCLConstantAddressSpace:
     case ParsedAttr::AT_OpenCLGenericAddressSpace:
     case ParsedAttr::AT_AddressSpace:
       HandleAddressSpaceTypeAttribute(type, attr, state);
       attr.setUsedAsTypeAttr();
       break;
     OBJC_POINTER_TYPE_ATTRS_CASELIST:

clang/test/AST/language_address_space_attribute.cpp

 
 void langas() {
   // CHECK: VarDecl {{.*}} x_global '__global int *'
   __attribute__((opencl_global)) int *x_global;
 
   // CHECK: VarDecl {{.*}} z_global '__global int *'
   [[clang::opencl_global]] int *z_global;
 
+  // CHECK: VarDecl {{.*}} x_global_device '__global_device int *'
+  __attribute__((opencl_global_device)) int *x_global_device;
+
+  // CHECK: VarDecl {{.*}} z_global_device '__global_device int *'
+  [[clang::opencl_global_device]] int *z_global_device;
+
+  // CHECK: VarDecl {{.*}} x_global_host '__global_host int *'
+  __attribute__((opencl_global_host)) int *x_global_host;
+
+  // CHECK: VarDecl {{.*}} z_global_host '__global_host int *'
+  [[clang::opencl_global_host]] int *z_global_host;
+
   // CHECK: VarDecl {{.*}} x_local '__local int *'
   __attribute__((opencl_local)) int *x_local;
 
   // CHECK: VarDecl {{.*}} z_local '__local int *'
   [[clang::opencl_local]] int *z_local;
 
   // CHECK: VarDecl {{.*}} x_constant '__constant int *'
   __attribute__((opencl_constant)) int *x_constant;

clang/test/CodeGenCXX/mangle-address-space.cpp

 
 #ifdef __OPENCL_CPP_VERSION__
 // CHECKOCL-LABEL: define {{.*}}void @_Z6ocl_f0PU9CLprivatec
 // WINOCL-LABEL: define {{.*}}void @"?ocl_f0@@YAXPEAU?$_ASCLprivate@$$CAD@__clang@@@Z"
 void ocl_f0(char __private *p) { }
 
 struct ocl_OpaqueType;
 typedef ocl_OpaqueType __global * ocl_OpaqueTypePtr;
+typedef ocl_OpaqueType __attribute__((opencl_global_host)) * ocl_OpaqueTypePtrH;
+typedef ocl_OpaqueType
+    __attribute__((opencl_global_device)) *
+    ocl_OpaqueTypePtrD;
 
 // CHECKOCL-LABEL: define {{.*}}void @_Z6ocl_f0PU8CLglobal14ocl_OpaqueType
 // WINOCL-LABEL: define {{.*}}void @"?ocl_f0@@YAXPEAU?$_ASCLglobal@$$CAUocl_OpaqueType@@@__clang@@@Z"
 void ocl_f0(ocl_OpaqueTypePtr) { }
 
 // CHECKOCL-LABEL: define {{.*}}void @_Z6ocl_f1PU10CLconstantKc
 // WINOCL-LABEL: define {{.*}}void @"?ocl_f1@@YAXPEAU?$_ASCLconstant@$$CBD@__clang@@@Z"
 void ocl_f1(char __constant const *p) {}
 
 // Ensure we can do return values, which change in MS mode.
 // CHECKOCL-LABEL: define {{.*}}float* @_Z6ocl_f1PU9CLgenericKc
 // WINOCL-LABEL: define {{.*}}float* @"?ocl_f1@@YAPEAU?$_ASCLconstant@$$CAM@__clang@@PEAU?$_ASCLgeneric@$$CBD@2@@Z"
 __constant float *ocl_f1(char __generic const *p) { return 0;}
 
 // Ensure we skip return values, since non-pointers aren't supposed to have an AS.
 // CHECKOCL-LABEL: define {{.*}}float* @_Z6ocl_f2PU9CLgenericKc
 // WINOCL-LABEL: define {{.*}}float* @"?ocl_f2@@YAPEAU?$_ASCLgeneric@$$CAM@__clang@@QEAU?$_ASCLgeneric@$$CBD@2@@Z"
 __generic float *ocl_f2(__generic char const * const p) { return 0;}
+
+// CHECKOCL-LABEL: define {{.*}}void @_Z6ocl_f3PU6CLhost14ocl_OpaqueType
+// WINOCL-LABEL: define {{.*}}void @"?ocl_f3@@YAXPEAU?$_ASCLhost@$$CAUocl_OpaqueType@@@__clang@@@Z"
+void ocl_f3(ocl_OpaqueTypePtrH) {}
+
+// CHECKOCL-LABEL: define {{.*}}void @_Z6ocl_f4PU8CLdevice14ocl_OpaqueType
+// WINOCL-LABEL: define {{.*}}void @"?ocl_f4@@YAXPEAU?$_ASCLdevice@$$CAUocl_OpaqueType@@@__clang@@@Z"
+void ocl_f4(ocl_OpaqueTypePtrD) {}
 #endif

clang/test/CodeGenOpenCL/address-spaces-conversions.cl

 // RUN: %clang_cc1 %s -triple x86_64-unknown-linux-gnu -O0 -ffake-address-space-map -cl-std=CL2.0 -emit-llvm -o - | FileCheck %s
 // RUN: %clang_cc1 %s -triple x86_64-unknown-linux-gnu -O0 -cl-std=CL2.0 -emit-llvm -o - | FileCheck --check-prefix=CHECK-NOFAKE %s
 // When -ffake-address-space-map is not used, all addr space mapped to 0 for x86_64.
 
 // test that we generate address space casts everywhere we need conversions of
 // pointers to different address spaces
 
 // CHECK: define void @test
-void test(global int *arg_glob, generic int *arg_gen) {
+void test(global int *arg_glob, generic int *arg_gen,
+          __attribute__((opencl_global_device)) int *arg_device,
+          __attribute__((opencl_global_host)) int *arg_host) {
   int var_priv;
   arg_gen = arg_glob; // implicit cast global -> generic
   // CHECK: %{{[0-9]+}} = addrspacecast i32 addrspace(1)* %{{[0-9]+}} to i32 addrspace(4)*
   // CHECK-NOFAKE-NOT: addrspacecast
 
   arg_gen = &var_priv; // implicit cast with obtaining adr, private -> generic
   // CHECK: %{{[._a-z0-9]+}} = addrspacecast i32* %{{[._a-z0-9]+}} to i32 addrspace(4)*
   // CHECK-NOFAKE-NOT: addrspacecast
void test(global int *arg_glob, generic int *arg_gen,
   var_priv = arg_gen > arg_glob; // comparison
   // CHECK: %{{[0-9]+}} = addrspacecast i32 addrspace(1)* %{{[0-9]+}} to i32 addrspace(4)*
 
   generic void *var_gen_v = arg_glob;
   // CHECK: addrspacecast
   // CHECK-NOT: bitcast
   // CHECK-NOFAKE: bitcast
   // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_glob = arg_device; // implicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_glob = arg_host; // implicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_glob = (global int *)arg_device; // explicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_glob = (global int *)arg_host; // explicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_device = (__attribute((opencl_global_device)) int *)arg_glob; // explicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
+
+  arg_host = (__attribute((opencl_global_host)) int *)arg_glob; // explicit cast
+  // CHECK: addrspacecast
+  // CHECK-NOFAKE-NOT: addrspacecast
 }
 
 // Test ternary operator.
 // CHECK: define void @test_ternary
 void test_ternary(void) {
   global int *var_glob;
   generic int *var_gen;
   generic int *var_gen2;

clang/test/CodeGenOpenCL/address-spaces.cl

 
 // CHECK: i32 addrspace(3)* %arg
 void fl(local int *arg) {}
 
 // SPIR: i32 addrspace(2)* %arg
 // AMDGCN: i32 addrspace(4)* %arg
 void fc(constant int *arg) {}
 
+// SPIR: i32 addrspace(5)* %arg
+// AMDGCN: i32 addrspace(1)* %arg
+void fd(__attribute__((opencl_global_device)) int *arg) {}
+
+// SPIR: i32 addrspace(6)* %arg
+// AMDGCN: i32 addrspace(1)* %arg
+void fh(__attribute__((opencl_global_host)) int *arg) {}
+
 #ifdef CL20
 int i;
 // CL20-DAG: @i = {{(dso_local )?}}addrspace(1) global i32 0
 int *ptr;
 // CL20SPIR-DAG: @ptr = {{(common )?}}{{(dso_local )?}}addrspace(1) global i32 addrspace(4)* null
 // CL20AMDGCN-DAG: @ptr = {{(dso_local )?}}addrspace(1) global i32* null
 #endif
 

clang/test/SemaOpenCL/usm-address-spaces-conversions.cl

+// RUN: %clang_cc1 %s -ffake-address-space-map -verify -pedantic -fsyntax-only -cl-std=CL2.0
+// RUN: %clang_cc1 %s -ffake-address-space-map -verify -pedantic -fsyntax-only -cl-std=CL2.0 -DGENERIC
+// RUN: %clang_cc1 %s -ffake-address-space-map -verify -pedantic -fsyntax-only -cl-std=CL2.0 -DCONSTANT
+// RUN: %clang_cc1 %s -ffake-address-space-map -verify -pedantic -fsyntax-only -cl-std=CL2.0 -DLOCAL
+
+/* USM (unified shared memory) extension for OpenCLC 2.0 adds two new address
+ * spaces: global_device and global_host that are a subset of __global address
+ * space. As ISO/IEC TR 18037 5.1.3 declares - it's possible to implicitly
+ * convert a subset address space to a superset address space, while conversion
+ * in a reversed direction could be achived only with an explicit cast */
+
+#ifdef GENERIC
+#define AS_COMP __generic
+#else
+#define AS_COMP __global
+#endif // GENERIC
+
+#ifdef CONSTANT
+#define AS_INCOMP __constant
+#elif LOCAL
+#define AS_INCOMP __local
+#else // PRIVATE
+#define AS_INCOMP __private
+#endif // CONSTANT
+
+void test(AS_COMP int *arg_comp,
+          __attribute__((opencl_global_device)) int *arg_device,
+          __attribute__((opencl_global_host)) int *arg_host) {
+  AS_COMP int *var_glob1 = arg_device;
+  AS_COMP int *var_glob2 = arg_host;
+  AS_COMP int *var_glob3 = (AS_COMP int *)arg_device;
+  AS_COMP int *var_glob4 = (AS_COMP int *)arg_host;
+  arg_device = (__attribute__((opencl_global_device)) int *)arg_comp;
+  arg_host = (__attribute__((opencl_global_host)) int *)arg_comp;
+#ifdef GENERIC
+  // expected-error@+6{{assigning '__generic int *__private' to '__global_device int *__private' changes address space of pointer}}
+  // expected-error@+6{{assigning '__generic int *__private' to '__global_host int *__private' changes address space of pointer}}
+#else
+  // expected-error@+3{{assigning '__global int *__private' to '__global_device int *__private' changes address space of pointer}}
+  // expected-error@+3{{assigning '__global int *__private' to '__global_host int *__private' changes address space of pointer}}
+#endif // GENERIC
+  arg_device = arg_comp;
+  arg_host = arg_comp;
+
+#ifdef CONSTANT
+  // expected-error@+15{{initializing '__constant int *__private' with an expression of type '__global_device int *__private' changes address space of pointer}}
+  // expected-error@+15{{initializing '__constant int *__private' with an expression of type '__global_host int *__private' changes address space of pointer}}
+  // expected-error@+15{{initializing '__constant int *__private' with an expression of type '__global_device int *' changes address space of pointer}}
+  // expected-error@+16{{initializing '__constant int *__private' with an expression of type '__global_host int *' changes address space of pointer}}
+#elif LOCAL
+  // expected-error@+10{{initializing '__local int *__private' with an expression of type '__global_device int *__private' changes address space of pointer}}
+  // expected-error@+10{{initializing '__local int *__private' with an expression of type '__global_host int *__private' changes address space of pointer}}
+  // expected-error@+10{{initializing '__local int *__private' with an expression of type '__global_device int *' changes address space of pointer}}
+  // expected-error@+11{{initializing '__local int *__private' with an expression of type '__global_host int *' changes address space of pointer}}
+#else // PRIVATE
+  // expected-error@+5{{initializing '__private int *__private' with an expression of type '__global_device int *__private' changes address space of pointer}}
+  // expected-error@+5{{initializing '__private int *__private' with an expression of type '__global_host int *__private' changes address space of pointer}}
+  // expected-error@+5{{initializing '__private int *__private' with an expression of type '__global_device int *' changes address space of pointer}}
+  // expected-error@+6{{initializing '__private int *__private' with an expression of type '__global_host int *' changes address space of pointer}}
+#endif // CONSTANT
+  AS_INCOMP int *var_incomp1 = arg_device;
+  AS_INCOMP int *var_incomp2 = arg_host;
+  AS_INCOMP int *var_incomp3 =
+      (__attribute__((opencl_global_device)) int *)arg_device;
+  AS_INCOMP int *var_incomp4 =
+      (__attribute__((opencl_global_host)) int *)arg_host;
+}

clang/test/SemaTemplate/address_space-dependent.cpp

 
 template <int I>
 void neg() {
   __attribute__((address_space(I))) int *bounds; // expected-error {{address space is negative}}
 }
 
 template <long int I>
 void tooBig() {
-  __attribute__((address_space(I))) int *bounds; // expected-error {{address space is larger than the maximum supported (8388595)}}
+  __attribute__((address_space(I))) int *bounds; // expected-error {{address space is larger than the maximum supported (8388593)}}
 }
 
 template <long int I>
 void correct() {
   __attribute__((address_space(I))) int *bounds;
 }
 
 template <int I, int J>

 
 int main() {  
   int __attribute__((address_space(1))) * p1;
   int p = GetAddressSpaceValue(p1);
 
   car<1, 2, 3>(); // expected-note {{in instantiation of function template specialization 'car<1, 2, 3>' requested here}}
   HasASTemplateFields<1> HASTF;
   neg<-1>(); // expected-note {{in instantiation of function template specialization 'neg<-1>' requested here}}
-  correct<0x7FFFF3>();
+  correct<0x7FFFF1>();
   tooBig<8388650>(); // expected-note {{in instantiation of function template specialization 'tooBig<8388650>' requested here}}
 
   __attribute__((address_space(1))) char *x;
   __attribute__((address_space(2))) char *y;
   cmp<1, 2>(x, y); // expected-note {{in instantiation of function template specialization 'cmp<1, 2>' requested here}}
 
   fooFunction<1> ff;
   ff.get_0(); // expected-note {{in instantiation of member function 'fooFunction<1>::get_0' requested here}}
   ff.qf();
   ff.test3(); // expected-note {{in instantiation of member function 'fooFunction<1>::test3' requested here}}
   
   static_assert(partial_spec_deduce_as<int __attribute__((address_space(3))) *>::value == 3, "address space value has been incorrectly deduced"); 
 
   return 0;
 }