[HLSL] Add groupshare address space.

[python3kgae]

Summary

Added keyword, LangAS and TypeAttrbute for groupshared.

Tanslate it to LangAS with asHLSLLangAS.

Make sure it translated into address space 3 for DirectX target.

[python3kgae]

Rebase

[python3kgae]

Fix test fail caused by max address space change.

[python3kgae]

Change Category to DocCatVariable.

[aaron.ballman]

At a high-level: can you make sure that functional changes all have corresponding test coverage?

clang/include/clang/Basic/Attr.td

4069

So this is being exposed as both a keyword and an attribute? Why? (No tests are using it as an attribute.)

clang/include/clang/Basic/DiagnosticSemaKinds.td

11718–11719

It looks like err_opencl_return_value_with_address_space could be modified to make this work. (The existing phrasing of 'return value' is strange and I'd be fine changing that to 'return type' instead.)

clang/lib/Basic/Targets/AMDGPU.cpp

59

83

clang/lib/Basic/Targets/DirectX.h

44

clang/lib/Basic/Targets/NVPTX.h

46

clang/lib/Basic/Targets/SPIR.h

46

76

clang/lib/Basic/Targets/TCE.h

53

clang/lib/Basic/Targets/X86.h

47

clang/lib/Parse/ParseExprCXX.cpp

1408

Do you expect to add more address spaces for HLSL? Do we want a helper function here for isHLSLQualifier() instead?

Also, there's no test coverage for this change.

clang/lib/Sema/SemaDecl.cpp

15002–15006

Why is this being done from ActOnStartOfFunctionDef() instead of CheckFunctionReturnType()?

We're missing test coverage for specifying this on a function declaration, a friend function declaration, or a member function (declaration or definition).

What about non-function situations like conversion operators? Can you write operator groupshared int() const; ?

[python3kgae]

Code cleanup to match comments.
Reuse opencl error.
Fix typo.

clang/include/clang/Basic/Attr.td

4069

It should be only keyword. I misunderstood what the Clang<> means here :(

clang/lib/Parse/ParseExprCXX.cpp

1408

Added isHLSLQualifier.
Maybe not other address space in a short term but will need other qualifiers for sure. :)

HLSL doesn't support lambda at this moment.
Should I just remove this and add it back when lambda is supported?

clang/lib/Sema/SemaDecl.cpp

15002–15006

Moved to ActOnFunctionDeclarator following err_opencl_return_value_with_address_space.
Also added tests.
operator overload is not supported in current version of hlsl though.

[aaron.ballman]

clang/include/clang/Basic/DiagnosticSemaKinds.td

10304–10305

I think we can drop the opencl and hlsl from the name at this point.

11718

Looks like this change can be reverted out.

clang/lib/Parse/ParseDecl.cpp

4344

It took me a while to convince myself that this was correct -- the token is consumed at the end of the while loop.

5844

Same thing here -- the end of the while loop is what consumes the token, so this is also correct.

clang/lib/Parse/ParseExprCXX.cpp

1410

And this consume is correct because the parse function doesn't consume any tokens.

clang/test/SemaHLSL/group_shared.hlsl

58

Some other Sema test cases that would be useful:

// Do we reject it on a return type in a function pointer?
groupshared void (*fp)();
// What about on parameters in a function pointer?
void (*fp2)(groupshared float);
// Trailing return types?
auto func() -> groupshared void;
// What about in a novel qualifier order?
void groupshared f();

struct S {
  // Do we reject it as a function qualifier on a member function?
  void f() groupshared;
};

// Does it impact size or alignment?
static_assert(sizeof(float) == sizeof(groupshared float));
static_assert(alignof(double) == alignof(groupshared double));

// Does it impact type identity for templates?
template <typename Ty>
struct S {
  static constexpr bool value = false;
};

template <>
struct S<groupshared float> {
  static constexpr bool value = true;
};
static_assert(!S<float>::value);
static_assert(S<groupshared float>::value);

// Can you overload based on the qualifier?
void func(float f) {}
void func(groupshared float f) {} // Error on redefinition?

(Note, I'm taking guesses on some of these test cases; if I've guessed wrong, feel free to correct me!)

I'd also expect tests to show up in ParserHLSL testing the parsing behavior:

// Do we accept novel qualifier orders?
extern groupshared float f;
extern float groupshared f; // Ok, redeclaration?

// Can we parse it on a lambda? Note, be sure to add more lambda tests with 
// more complex parsing situations to make sure you're happy with the parsing
// behavior.
[]() groupshared {};

// Can we still parse attributes written on the type?
float groupshared [[]] i = 12;

[beanz]

clang/include/clang/Basic/Attr.td

4070

Shouldn't this have let Subjects = SubjectList<[Var]>;?

I don't think we support groupshared on anything except variable declarations.

[python3kgae]

Add SubjectList<[Var]> and add tests.

[python3kgae]

clang/test/SemaHLSL/group_shared.hlsl

58

Unfortunately, things like pointer and lambda are not supported in HLSL yet.
So some tests will not work.

[beanz]

clang/test/ParserHLSL/group_shared.hlsl

12

What happens to this if you set the language standard to hlsl 202x? I setup 202x in Clang to be C++11-based since I think that is the direction 202x will take (still a lot unsure about the exact features).

[python3kgae]

clang/test/ParserHLSL/group_shared.hlsl

12

hlsl 202x works!
I'll need to add more tests for hlsl 202x since lambda compiles.

[python3kgae]

Add hlsl202x test for C++ 11 features like lambda.

[aaron.ballman]

clang/lib/Sema/SemaDecl.cpp

10131

clang/lib/Sema/SemaLambda.cpp

977

clang/test/ParserHLSL/group_shared_202x.hlsl

6–7

One test I think we need in SemaCXX for this is what happens when the lambda return type (or function return type) is deduced. e.g.,

extern groupshared float f;

auto func() {
  return f;
}

void other() {
  [&]() { return f; };
}

Does use of return f cause us to deduce a return type that's annotated with groupshared or does that get stripped off thanks to lvalue to rvalue conversions and so we deduce float?

[python3kgae]

Update comments and add more test cases.

[python3kgae]

clang/test/ParserHLSL/group_shared_202x.hlsl

6–7

groupshared is stripped because of lvalue to rvalue conversions.

And even hlsl202x not enable CXX14 :( .

[aaron.ballman]

LGTM

clang/test/ParserHLSL/group_shared_202x.hlsl

6–7

Excellent, thank you!

clang/include/clang/Basic/AddressSpaces.h

enum class LangAS : unsigned {
   sycl_local,
   sycl_private,
 
   // Pointer size and extension address spaces.
   ptr32_sptr,
   ptr32_uptr,
   ptr64,
 
+  // HLSL specific address spaces.
+  hlsl_groupshared,
+
   // This denotes the count of language-specific address spaces and also
   // the offset added to the target-specific address spaces, which are usually
   // specified by address space attributes __attribute__(address_space(n))).
   FirstTargetAddressSpace
 };
 
 /// The type of a lookup table which maps from language-specific address spaces
 /// to target-specific ones.

clang/include/clang/Basic/Attr.td

let Args = [EnumArgument<"ResourceType", "ResourceClass",
                             "StructuredBuffer", "CBufferKind", "SamplerKind",
                             "TBuffer", "RTAccelerationStructure", "FeedbackTexture2D",
                             "FeedbackTexture2DArray"]
                             >
               ];
   let Documentation = [InternalOnly];
 }
 
+def HLSLGroupSharedAddressSpace : TypeAttr {
+  let Spellings = [Keyword<"groupshared">];
+  let Subjects = SubjectList<[Var]>;
+  let Documentation = [HLSLGroupSharedAddressSpaceDocs];
+}
+
 def RandomizeLayout : InheritableAttr {
   let Spellings = [GCC<"randomize_layout">];
   let Subjects = SubjectList<[Record]>;
   let Documentation = [ClangRandomizeLayoutDocs];
   let LangOpts = [COnly];
 }
 
 def NoRandomizeLayout : InheritableAttr {

clang/include/clang/Basic/AttrDocs.td

 when the struct is used as a parameter type.
 The semantic on the field is ignored when not used as a parameter.
 This attribute is only supported in compute shaders.
 
 The full documentation is available here: https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/sv-dispatchthreadid
   }];
 }
 
+def HLSLGroupSharedAddressSpaceDocs : Documentation {
+  let Category = DocCatVariable;
+  let Content = [{
+HLSL enables threads of a compute shader to exchange values via shared memory.
+HLSL provides barrier primitives such as GroupMemoryBarrierWithGroupSync,
+and so on to ensure the correct ordering of reads and writes to shared memory
+in the shader and to avoid data races.
+Here's an example to declare a groupshared variable.
+.. code-block:: c++
+
+  groupshared GSData data[5*5*1];
+
+The full documentation is available here: https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-variable-syntax#group-shared
+  }];
+}
+
 def AnnotateTypeDocs : Documentation {
   let Category = DocCatType;
   let Heading = "annotate_type";
   let Content = [{
 This attribute is used to add annotations to types, typically for use by static
 analysis tools that are not integrated into the core Clang compiler (e.g.,
 Clang-Tidy checks or out-of-tree Clang-based tools). It is a counterpart to the
 `annotate` attribute, which serves the same purpose, but for declarations.

clang/include/clang/Basic/DiagnosticSemaKinds.td

def err_opencl_global_invalid_addr_space : Error<
   "%select{program scope|static local|extern}0 variable must reside in %1 address space">;
 def err_missing_actual_pipe_type : Error<
   "missing actual type specifier for pipe">;
 def err_reference_pipe_type : Error <
   "pipes packet types cannot be of reference type">;
 def err_opencl_no_main : Error<"%select{function|kernel}0 cannot be called 'main'">;
 def err_opencl_kernel_attr :
   Error<"attribute %0 can only be applied to an OpenCL kernel function">;
-def err_opencl_return_value_with_address_space : Error<
-  "return value cannot be qualified with address space">;
+def err_return_value_with_address_space : Error<
+  "return type cannot be qualified with address space">;
 def err_opencl_constant_no_init : Error<
   "variable in constant address space must be initialized">;
 def err_opencl_atomic_init: Error<
   "atomic variable can be %select{assigned|initialized}0 to a variable only "
   "in global address space">;
 def err_opencl_implicit_vector_conversion : Error<
   "implicit conversions between vector types (%0 and %1) are not permitted">;
 def err_opencl_invalid_type_array : Error<

 def err_hlsl_unsupported_register_type : Error<"invalid resource class specifier '%0' used; expected 'b', 's', 't', or 'u'">;
 def err_hlsl_unsupported_register_number : Error<"register number should be an integer">;
 def err_hlsl_expected_space : Error<"invalid space specifier '%0' used; expected 'space' followed by an integer, like space1">;
 def err_hlsl_pointers_unsupported : Error<
   "%select{pointers|references}0 are unsupported in HLSL">;
 
 def err_hlsl_operator_unsupported : Error<
   "the '%select{&|*|->}0' operator is unsupported in HLSL">;
 
 // Layout randomization diagnostics.
 def err_non_designated_init_used : Error<
   "a randomized struct can only be initialized with a designated initializer">;
 def err_cast_from_randomized_struct : Error<
   "casting from randomized structure pointer type %0 to %1">;
 } // end of sema component.

clang/include/clang/Basic/TokenKinds.def

 KEYWORD(addrspace_cast              , KEYOPENCLCXX)
 
 // CUDA/HIP function attributes
 KEYWORD(__noinline__                , KEYCUDA)
 
 // HLSL keywords.
 KEYWORD(cbuffer                     , KEYHLSL)
 KEYWORD(tbuffer                     , KEYHLSL)
+KEYWORD(groupshared                 , KEYHLSL)
 
 // OpenMP Type Traits
 UNARY_EXPR_OR_TYPE_TRAIT(__builtin_omp_required_simd_align, OpenMPRequiredSimdAlign, KEYALL)
 
 // Borland Extensions.
 KEYWORD(__pascal                    , KEYALL)
 
 // Altivec Extension.

clang/include/clang/Parse/Parser.h

private:
   void DiagnoseAndSkipExtendedMicrosoftTypeAttributes();
   SourceLocation SkipExtendedMicrosoftTypeAttributes();
   void ParseMicrosoftInheritanceClassAttributes(ParsedAttributes &attrs);
   void ParseBorlandTypeAttributes(ParsedAttributes &attrs);
   void ParseOpenCLKernelAttributes(ParsedAttributes &attrs);
   void ParseOpenCLQualifiers(ParsedAttributes &Attrs);
   void ParseNullabilityTypeSpecifiers(ParsedAttributes &attrs);
   void ParseCUDAFunctionAttributes(ParsedAttributes &attrs);
+  bool isHLSLQualifier(const Token &Tok) const;
+  void ParseHLSLQualifiers(ParsedAttributes &Attrs);
 
   VersionTuple ParseVersionTuple(SourceRange &Range);
   void ParseAvailabilityAttribute(IdentifierInfo &Availability,
                                   SourceLocation AvailabilityLoc,
                                   ParsedAttributes &attrs,
                                   SourceLocation *endLoc,
                                   IdentifierInfo *ScopeName,
                                   SourceLocation ScopeLoc,

clang/include/clang/Sema/ParsedAttr.h

LangAS asSYCLLangAS() const {
     case ParsedAttr::AT_OpenCLPrivateAddressSpace:
       return LangAS::sycl_private;
     case ParsedAttr::AT_OpenCLGenericAddressSpace:
     default:
       return LangAS::Default;
     }
   }
 
+  /// If this is an HLSL address space attribute, returns its representation
+  /// in LangAS, otherwise returns default address space.
+  LangAS asHLSLLangAS() const {
+    switch (getParsedKind()) {
+    case ParsedAttr::AT_HLSLGroupSharedAddressSpace:
+      return LangAS::hlsl_groupshared;
+    default:
+      return LangAS::Default;
+    }
+  }
+
   AttributeCommonInfo::Kind getKind() const {
     return AttributeCommonInfo::Kind(Info.AttrKind);
   }
   const ParsedAttrInfo &getInfo() const { return Info; }
 };
 
 class AttributePool;
 /// A factory, from which one makes pools, from which one creates

clang/lib/AST/ASTContext.cpp

static const unsigned FakeAddrSpaceMap[] = {
         9,  // cuda_shared
         1,  // sycl_global
         5,  // sycl_global_device
         6,  // sycl_global_host
         3,  // sycl_local
         0,  // sycl_private
         10, // ptr32_sptr
         11, // ptr32_uptr
-        12  // ptr64
+        12, // ptr64
+        13, // hlsl_groupshared
     };
     return &FakeAddrSpaceMap;
   } else {
     return &T.getAddressSpaceMap();
   }
 }
 
 static bool isAddrSpaceMapManglingEnabled(const TargetInfo &TI,

clang/lib/AST/TypePrinter.cpp

#include "clang/Basic/AttrList.inc"
 
   case attr::OpenCLPrivateAddressSpace:
   case attr::OpenCLGlobalAddressSpace:
   case attr::OpenCLGlobalDeviceAddressSpace:
   case attr::OpenCLGlobalHostAddressSpace:
   case attr::OpenCLLocalAddressSpace:
   case attr::OpenCLConstantAddressSpace:
   case attr::OpenCLGenericAddressSpace:
+  case attr::HLSLGroupSharedAddressSpace:
     // FIXME: Update printAttributedBefore to print these once we generate
     // AttributedType nodes for them.
     break;
 
   case attr::LifetimeBound:
   case attr::TypeNonNull:
   case attr::TypeNullable:
   case attr::TypeNullableResult:
std::string Qualifiers::getAddrSpaceAsString(LangAS AS) {
   case LangAS::cuda_shared:
     return "__shared__";
   case LangAS::ptr32_sptr:
     return "__sptr __ptr32";
   case LangAS::ptr32_uptr:
     return "__uptr __ptr32";
   case LangAS::ptr64:
     return "__ptr64";
+  case LangAS::hlsl_groupshared:
+    return "groupshared";
   default:
     return std::to_string(toTargetAddressSpace(AS));
   }
 }
 
 // Appends qualifiers to the given string, separated by spaces.  Will
 // prefix a space if the string is non-empty.  Will not append a final
 // space.

clang/lib/Basic/Targets/AMDGPU.cpp

const LangASMap AMDGPUTargetInfo::AMDGPUDefIsGenMap = {
     Local,    // cuda_shared
     Global,   // sycl_global
     Global,   // sycl_global_device
     Global,   // sycl_global_host
     Local,    // sycl_local
     Private,  // sycl_private
     Generic,  // ptr32_sptr
     Generic,  // ptr32_uptr
-    Generic   // ptr64
+    Generic,  // ptr64
+    Generic,  // hlsl_groupshared
 };
 
 const LangASMap AMDGPUTargetInfo::AMDGPUDefIsPrivMap = {
     Private,  // Default
     Global,   // opencl_global
     Local,    // opencl_local
     Constant, // opencl_constant
     Private,  // opencl_private
     Generic,  // opencl_generic
     Global,   // opencl_global_device
     Global,   // opencl_global_host
     Global,   // cuda_device
     Constant, // cuda_constant
     Local,    // cuda_shared
     // SYCL address space values for this map are dummy
-    Generic,  // sycl_global
-    Generic,  // sycl_global_device
-    Generic,  // sycl_global_host
-    Generic,  // sycl_local
-    Generic,  // sycl_private
-    Generic,  // ptr32_sptr
-    Generic,  // ptr32_uptr
-    Generic   // ptr64
+    Generic, // sycl_global
+    Generic, // sycl_global_device
+    Generic, // sycl_global_host
+    Generic, // sycl_local
+    Generic, // sycl_private
+    Generic, // ptr32_sptr
+    Generic, // ptr32_uptr
+    Generic, // ptr64
+    Generic, // hlsl_groupshared
 
 };
 } // namespace targets
 } // namespace clang
 
 const Builtin::Info AMDGPUTargetInfo::BuiltinInfo[] = {
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
   {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},

clang/lib/Basic/Targets/DirectX.h

static const unsigned DirectXAddrSpaceMap[] = {
     // SYCL address space values for this map are dummy
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    3, // hlsl_groupshared
 };
 
 class LLVM_LIBRARY_VISIBILITY DirectXTargetInfo : public TargetInfo {
 public:
   DirectXTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     TLSSupported = false;
     VLASupported = false;

clang/lib/Basic/Targets/NVPTX.h

static const unsigned NVPTXAddrSpaceMap[] = {
     3, // cuda_shared
     1, // sycl_global
     1, // sycl_global_device
     1, // sycl_global_host
     3, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_groupshared
 };
 
 /// The DWARF address class. Taken from
 /// https://docs.nvidia.com/cuda/archive/10.0/ptx-writers-guide-to-interoperability/index.html#cuda-specific-dwarf
 static const int NVPTXDWARFAddrSpaceMap[] = {
     -1, // Default, opencl_private or opencl_generic - not defined
     5,  // opencl_global
     -1,

clang/lib/Basic/Targets/SPIR.h

static const unsigned SPIRDefIsPrivMap[] = {
     // SYCL address space values for this map are dummy
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_groupshared
 };
 
 // Used by both the SPIR and SPIR-V targets.
 static const unsigned SPIRDefIsGenMap[] = {
     4, // Default
     // OpenCL address space values for this map are dummy and they can't be used
     0, // opencl_global
     0, // opencl_local
static const unsigned SPIRDefIsGenMap[] = {
     3, // cuda_shared
     1, // sycl_global
     5, // sycl_global_device
     6, // sycl_global_host
     3, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_groupshared
 };
 
 // Base class for SPIR and SPIR-V target info.
 class LLVM_LIBRARY_VISIBILITY BaseSPIRTargetInfo : public TargetInfo {
 protected:
   BaseSPIRTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     assert((Triple.isSPIR() || Triple.isSPIRV()) &&

clang/lib/Basic/Targets/TCE.h

static const unsigned TCEOpenCLAddrSpaceMap[] = {
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
     0, // ptr64
+    0, // hlsl_groupshared
 };
 
 class LLVM_LIBRARY_VISIBILITY TCETargetInfo : public TargetInfo {
 public:
   TCETargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     TLSSupported = false;
     IntWidth = 32;

clang/lib/Basic/Targets/X86.h

static const unsigned X86AddrSpaceMap[] = {
     0,   // cuda_shared
     0,   // sycl_global
     0,   // sycl_global_device
     0,   // sycl_global_host
     0,   // sycl_local
     0,   // sycl_private
     270, // ptr32_sptr
     271, // ptr32_uptr
-    272  // ptr64
+    272, // ptr64
+    0,   // hlsl_groupshared
 };
 
 // X86 target abstract base class; x86-32 and x86-64 are very close, so
 // most of the implementation can be shared.
 class LLVM_LIBRARY_VISIBILITY X86TargetInfo : public TargetInfo {
 
   enum X86SSEEnum {
     NoSSE,

clang/lib/Parse/ParseDecl.cpp

 
 void Parser::ParseOpenCLQualifiers(ParsedAttributes &Attrs) {
   IdentifierInfo *AttrName = Tok.getIdentifierInfo();
   SourceLocation AttrNameLoc = Tok.getLocation();
   Attrs.addNew(AttrName, AttrNameLoc, nullptr, AttrNameLoc, nullptr, 0,
                ParsedAttr::AS_Keyword);
 }
 
+bool Parser::isHLSLQualifier(const Token &Tok) const {
+  return Tok.is(tok::kw_groupshared);
+}
+
+void Parser::ParseHLSLQualifiers(ParsedAttributes &Attrs) {
+  IdentifierInfo *AttrName = Tok.getIdentifierInfo();
+  SourceLocation AttrNameLoc = ConsumeToken();
+  Attrs.addNew(AttrName, AttrNameLoc, nullptr, AttrNameLoc, nullptr, 0,
+               ParsedAttr::AS_Keyword);
+}
+
 void Parser::ParseNullabilityTypeSpecifiers(ParsedAttributes &attrs) {
   // Treat these like attributes, even though they're type specifiers.
   while (true) {
     switch (Tok.getKind()) {
     case tok::kw__Nonnull:
     case tok::kw__Nullable:
     case tok::kw__Nullable_result:
     case tok::kw__Null_unspecified: {
#include "clang/Basic/TransformTypeTraits.def"
     case tok::kw___constant:
     // OpenCL access qualifiers:
     case tok::kw___read_only:
     case tok::kw___write_only:
     case tok::kw___read_write:
       ParseOpenCLQualifiers(DS.getAttributes());
       break;
 
+    case tok::kw_groupshared:
+      // NOTE: ParseHLSLQualifiers will consume the qualifier token.
+      ParseHLSLQualifiers(DS.getAttributes());
+      continue;
+
     case tok::less:
       // GCC ObjC supports types like "<SomeProtocol>" as a synonym for
       // "id<SomeProtocol>".  This is hopelessly old fashioned and dangerous,
       // but we support it.
       if (DS.hasTypeSpecifier() || !getLangOpts().ObjC)
         goto DoneWithDeclSpec;
 
       SourceLocation StartLoc = Tok.getLocation();
#include "clang/Basic/OpenCLImageTypes.def"
   case tok::kw___private:
   case tok::kw___local:
   case tok::kw___global:
   case tok::kw___constant:
   case tok::kw___generic:
   case tok::kw___read_only:
   case tok::kw___read_write:
   case tok::kw___write_only:
+
+  case tok::kw_groupshared:
     return true;
 
   case tok::kw_private:
     return getLangOpts().OpenCL;
 
   // C11 _Atomic
   case tok::kw__Atomic:
     return true;
bool Parser::isDeclarationSpecifier(
   case tok::kw___constant:
   case tok::kw___generic:
   case tok::kw___read_only:
   case tok::kw___read_write:
   case tok::kw___write_only:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
 
+  case tok::kw_groupshared:
     return true;
 
   case tok::kw_private:
     return getLangOpts().OpenCL;
   }
 }
 
 bool Parser::isConstructorDeclarator(bool IsUnqualified, bool DeductionGuide,
while (true) {
     case tok::kw___constant:
     case tok::kw___generic:
     case tok::kw___read_only:
     case tok::kw___write_only:
     case tok::kw___read_write:
       ParseOpenCLQualifiers(DS.getAttributes());
       break;
 
+    case tok::kw_groupshared:
+      // NOTE: ParseHLSLQualifiers will consume the qualifier token.
+      ParseHLSLQualifiers(DS.getAttributes());
+      continue;
+
     case tok::kw___unaligned:
       isInvalid = DS.SetTypeQual(DeclSpec::TQ_unaligned, Loc, PrevSpec, DiagID,
                                  getLangOpts());
       break;
     case tok::kw___uptr:
       // GNU libc headers in C mode use '__uptr' as an identifier which conflicts
       // with the MS modifier keyword.
       if ((AttrReqs & AR_DeclspecAttributesParsed) && !getLangOpts().CPlusPlus &&

clang/lib/Parse/ParseExprCXX.cpp

auto ParseLambdaSpecifiers =
         // Parse OpenCL addr space attribute.
         if (Tok.isOneOf(tok::kw___private, tok::kw___global, tok::kw___local,
                         tok::kw___constant, tok::kw___generic)) {
           ParseOpenCLQualifiers(DS.getAttributes());
           ConsumeToken();
         }
 
         SourceLocation FunLocalRangeEnd = DeclEndLoc;
 
         // Parse trailing-return-type[opt].
         if (Tok.is(tok::arrow)) {
           FunLocalRangeEnd = Tok.getLocation();
           SourceRange Range;
           TrailingReturnType = ParseTrailingReturnType(
               Range, /*MayBeFollowedByDirectInit*/ false);
           TrailingReturnTypeLoc = Range.getBegin();
           if (Range.getEnd().isValid())
             DeclEndLoc = Range.getEnd();
         }
if (Tok.is(tok::l_paren)) {
 
     // Parse requires-clause[opt].
     if (Tok.is(tok::kw_requires))
       ParseTrailingRequiresClause(D);
   } else if (Tok.isOneOf(tok::kw_mutable, tok::arrow, tok::kw___attribute,
                          tok::kw_constexpr, tok::kw_consteval, tok::kw_static,
                          tok::kw___private, tok::kw___global, tok::kw___local,
                          tok::kw___constant, tok::kw___generic,
-                         tok::kw_requires, tok::kw_noexcept) ||
+                         tok::kw_groupshared, tok::kw_requires,
+                         tok::kw_noexcept) ||
              (Tok.is(tok::l_square) && NextToken().is(tok::l_square))) {
     if (!getLangOpts().CPlusPlus2b)
       // It's common to forget that one needs '()' before 'mutable', an
       // attribute specifier, the result type, or the requires clause. Deal with
       // this.
       Diag(Tok, diag::ext_lambda_missing_parens)
           << FixItHint::CreateInsertion(Tok.getLocation(), "() ");
 

clang/lib/Parse/ParseTentative.cpp

Parser::isCXXDeclarationSpecifier(ImplicitTypenameContext AllowImplicitTypename,
   case tok::kw___generic:
     // OpenCL access qualifiers
   case tok::kw___read_only:
   case tok::kw___write_only:
   case tok::kw___read_write:
     // OpenCL pipe
   case tok::kw_pipe:
 
+    // HLSL address space qualifiers
+  case tok::kw_groupshared:
+
     // GNU
   case tok::kw_restrict:
   case tok::kw__Complex:
   case tok::kw___attribute:
   case tok::kw___auto_type:
     return TPResult::True;
 
     // Microsoft

clang/lib/Sema/SemaDecl.cpp

Sema::ActOnFunctionDeclarator(Scope *S, Declarator &D, DeclContext *DC,
   // Handle attributes.
   ProcessDeclAttributes(S, NewFD, D);
 
   if (getLangOpts().OpenCL) {
     // OpenCL v1.1 s6.5: Using an address space qualifier in a function return
     // type declaration will generate a compilation error.
     LangAS AddressSpace = NewFD->getReturnType().getAddressSpace();
     if (AddressSpace != LangAS::Default) {
-      Diag(NewFD->getLocation(),
-           diag::err_opencl_return_value_with_address_space);
+      Diag(NewFD->getLocation(), diag::err_return_value_with_address_space);
       NewFD->setInvalidDecl();
     }
   }
 
   if (getLangOpts().HLSL) {
     auto &TargetInfo = getASTContext().getTargetInfo();
     // Skip operator overload which not identifier.
     // Also make sure NewFD is in translation-unit scope.
     if (!NewFD->isInvalidDecl() && Name.isIdentifier() &&
         NewFD->getName() == TargetInfo.getTargetOpts().HLSLEntry &&
         S->getDepth() == 0) {
       CheckHLSLEntryPoint(NewFD);
       if (!NewFD->isInvalidDecl()) {
         auto Env = TargetInfo.getTriple().getEnvironment();
         AttributeCommonInfo AL(NewFD->getBeginLoc());
         HLSLShaderAttr::ShaderType ShaderType =
             static_cast<HLSLShaderAttr::ShaderType>(
                 hlsl::getStageFromEnvironment(Env));
         // To share code with HLSLShaderAttr, add HLSLShaderAttr to entry
         // function.
         if (HLSLShaderAttr *Attr = mergeHLSLShaderAttr(NewFD, AL, ShaderType))
           NewFD->addAttr(Attr);
       }
     }
+    // HLSL does not support specifying an address space on a function return
+    // type.
+    LangAS AddressSpace = NewFD->getReturnType().getAddressSpace();
+    if (AddressSpace != LangAS::Default) {
+      Diag(NewFD->getLocation(), diag::err_return_value_with_address_space);
+      NewFD->setInvalidDecl();
+    }
   }
 
   if (!getLangOpts().CPlusPlus) {
     // Perform semantic checking on the function declaration.
     if (!NewFD->isInvalidDecl() && NewFD->isMain())
       CheckMain(NewFD, D.getDeclSpec());
 
     if (!NewFD->isInvalidDecl() && NewFD->isMSVCRTEntryPoint())
Decl *Sema::ActOnStartOfFunctionDef(Scope *FnBodyScope, Decl *D,
   // unless the function is deleted (C++ specifc, C++ [dcl.fct.def.general]p2)
   QualType ResultType = FD->getReturnType();
   if (!ResultType->isDependentType() && !ResultType->isVoidType() &&
       !FD->isInvalidDecl() && BodyKind != FnBodyKind::Delete &&
       RequireCompleteType(FD->getLocation(), ResultType,
                           diag::err_func_def_incomplete_result))
     FD->setInvalidDecl();
 
   if (FnBodyScope)
     PushDeclContext(FnBodyScope, FD);
 
   // Check the validity of our function parameters
   if (BodyKind != FnBodyKind::Delete)
     CheckParmsForFunctionDef(FD->parameters(),
                              /*CheckParameterNames=*/true);
 
   // Add non-parameter declarations already in the function to the current
   // scope.
   if (FnBodyScope) {
     for (Decl *NPD : FD->decls()) {
       auto *NonParmDecl = dyn_cast<NamedDecl>(NPD);

clang/lib/Sema/SemaLambda.cpp

if (ParamInfo.getNumTypeObjects() == 0) {
     }
 
     MethodTyInfo = GetTypeForDeclarator(ParamInfo, CurScope);
     assert(MethodTyInfo && "no type from lambda-declarator");
     EndLoc = ParamInfo.getSourceRange().getEnd();
 
     ExplicitResultType = FTI.hasTrailingReturnType();
 
+    if (ExplicitResultType && getLangOpts().HLSL) {
+      QualType RetTy = FTI.getTrailingReturnType().get();
+      if (!RetTy.isNull()) {
+        // HLSL does not support specifying an address space on a lambda return
+        // type.
+        LangAS AddressSpace = RetTy.getAddressSpace();
+        if (AddressSpace != LangAS::Default)
+          Diag(FTI.getTrailingReturnTypeLoc(),
+               diag::err_return_value_with_address_space);
+      }
+    }
+
     if (FTIHasNonVoidParameters(FTI)) {
       Params.reserve(FTI.NumParams);
       for (unsigned i = 0, e = FTI.NumParams; i != e; ++i)
         Params.push_back(cast<ParmVarDecl>(FTI.Params[i].Param));
     }
 
     // Check for unexpanded parameter packs in the method type.
     if (MethodTyInfo->getType()->containsUnexpandedParameterPack())

clang/lib/Sema/SemaType.cpp

if (T.hasNonTrivialToPrimitiveDestructCUnion() ||
     checkNonTrivialCUnion(T, Loc, NTCUC_FunctionReturn,
                           NTCUK_Destruct|NTCUK_Copy);
 
   // C++2a [dcl.fct]p12:
   //   A volatile-qualified return type is deprecated
   if (T.isVolatileQualified() && getLangOpts().CPlusPlus20)
     Diag(Loc, diag::warn_deprecated_volatile_return) << T;
 
+  if (T.getAddressSpace() != LangAS::Default && getLangOpts().HLSL)
+    return true;
   return false;
 }
 
 /// Check the extended parameter information.  Most of the necessary
 /// checking should occur when applying the parameter attribute; the
 /// only other checks required are positional restrictions.
 static void checkExtParameterInfos(Sema &S, ArrayRef<QualType> paramTypes,
                     const FunctionProtoType::ExtProtoInfo &EPI,
if (Attr.getKind() == ParsedAttr::AT_AddressSpace) {
     if (!T.isNull())
       Type = T;
     else
       Attr.setInvalid();
   } else {
     // The keyword-based type attributes imply which address space to use.
     ASIdx = S.getLangOpts().SYCLIsDevice ? Attr.asSYCLLangAS()
                                          : Attr.asOpenCLLangAS();
+    if (S.getLangOpts().HLSL)
+      ASIdx = Attr.asHLSLLangAS();
 
     if (ASIdx == LangAS::Default)
       llvm_unreachable("Invalid address space");
 
     if (DiagnoseMultipleAddrSpaceAttributes(S, Type.getAddressSpace(), ASIdx,
                                             Attr.getLoc())) {
       Attr.setInvalid();
       return;
case ParsedAttr::AT_MayAlias:
       break;
     case ParsedAttr::AT_OpenCLPrivateAddressSpace:
     case ParsedAttr::AT_OpenCLGlobalAddressSpace:
     case ParsedAttr::AT_OpenCLGlobalDeviceAddressSpace:
     case ParsedAttr::AT_OpenCLGlobalHostAddressSpace:
     case ParsedAttr::AT_OpenCLLocalAddressSpace:
     case ParsedAttr::AT_OpenCLConstantAddressSpace:
     case ParsedAttr::AT_OpenCLGenericAddressSpace:
+    case ParsedAttr::AT_HLSLGroupSharedAddressSpace:
     case ParsedAttr::AT_AddressSpace:
       HandleAddressSpaceTypeAttribute(type, attr, state);
       attr.setUsedAsTypeAttr();
       break;
     OBJC_POINTER_TYPE_ATTRS_CASELIST:
       if (!handleObjCPointerTypeAttr(state, attr, type))
         distributeObjCPointerTypeAttr(state, attr, type);
       attr.setUsedAsTypeAttr();

clang/test/AST/HLSL/group_shared.hlsl

+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -ast-dump -o - %s | FileCheck %s
+
+//CHECK:VarDecl 0x[[A:[0-9a-f]+]] <{{.*}} col:24> col:20 used a 'groupshared float[10]'
+ groupshared float a[10];
+
+// CHECK:FunctionDecl 0x{{[0-9a-f]+}} <{{.*}}> line:[[@LINE+2]]:7 main 'void ()'
+ [numthreads(8,8,1)]
+ void main() {
+// CHECK:BinaryOperator 0x{{[0-9a-f]+}} <{{.*}}> 'groupshared float' lvalue '='
+// CHECK:ArraySubscriptExpr 0x{{[0-9a-f]+}} <col:4, col:7> 'groupshared float' lvalue
+// CHECK:ImplicitCastExpr 0x{{[0-9a-f]+}} <col:4> 'groupshared float *' <ArrayToPointerDecay>
+// CHECK:DeclRefExpr 0x{{[0-9a-f]+}} <col:4> 'groupshared float[10]' lvalue Var 0x[[A]] 'a' 'groupshared float[10]'
+// CHECK:IntegerLiteral 0x{{[0-9a-f]+}} <col:6> 'int' 0
+// CHECK:ImplicitCastExpr 0x{{[0-9a-f]+}} <col:11> 'float' <IntegralToFloating>
+// CHECK:IntegerLiteral 0x{{[0-9a-f]+}} <col:11> 'int' 1
+   a[0] = 1;
+ }
+
+
+// CHECK:HLSLNumThreadsAttr 0x{{[0-9a-f]+}} <{{.*}}> 8 8 1

clang/test/CodeGenHLSL/group_shared.hlsl

+
+// RUN: %clang_cc1 -std=hlsl2021 -finclude-default-header -x hlsl -triple \
+// RUN:   dxil-pc-shadermodel6.3-library %s \
+// RUN:   -emit-llvm -disable-llvm-passes -o - | FileCheck %s
+
+// Make sure groupshared translated into address space 3.
+// CHECK:@"?a@@3PAMA" = addrspace(3) global [10 x float]
+
+ groupshared float a[10];
+
+ [numthreads(8,8,1)]
+ void main() {
+   a[0] = 1;
+ }
+

clang/test/Parser/opencl-cxx-keywords.cl

class A{
   private:
   private int i; //expected-error{{field may not be qualified with an address space}}
 };
 
 private ::A i; //expected-error{{program scope variable must reside in global or constant address space}}
 
 void foo(private int i);
 
-private int bar(); //expected-error{{return value cannot be qualified with address space}}
+private int bar(); //expected-error{{return type cannot be qualified with address space}}

clang/test/ParserHLSL/group_shared.hlsl

+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -o - -fsyntax-only %s -verify
+extern groupshared float f;
+extern float groupshared f; // Ok, redeclaration?
+
+
+// NOTE:lambda is not enabled except for hlsl202x.
+// expected-error@+2 {{expected expression}}
+// expected-warning@+1 {{'auto' type specifier is a C++11 extension}}
+auto l = []() groupshared  {};
+
+
+// NOTE: remove this error once [[]] attribute is supported except for hlsl202x.
+// expected-error@+1 {{expected expression}}
+float groupshared [[]] i = 12;
+
+float groupshared const i2 = 12;

clang/test/ParserHLSL/group_shared_202x.hlsl

+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -std=hlsl202x  -o - -fsyntax-only %s -verify
+extern groupshared float f;
+extern float groupshared f; // Ok, redeclaration?
+
+// expected-error@+1 {{return type cannot be qualified with address space}}
+auto l = []() -> groupshared void {};
+// expected-error@+1 {{expected a type}}
+auto l2 = []() -> groupshared {};
+
+float groupshared [[]] i = 12;
+
+float groupshared const i2 = 12;
+
+void foo() {
+    l();
+}
+
+extern groupshared float f;
+const float cf = f;
+// expected-error@+1 {{'auto' return without trailing return type; deduced return types are a C++14 extension}}
+auto func() {
+  return f;
+}
+
+void other() {
+  // NOTE: groupshared and const are stripped off thanks to lvalue to rvalue conversions and we deduce float for the return type.
+  auto l = [&]() { return f; };
+  auto l2 = [&]() { return cf; };
+}

clang/test/SemaHLSL/group_shared.hlsl

+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -o - -fsyntax-only %s -verify
+
+ groupshared float a[10];
+
+ [numthreads(8,8,1)]
+ void main() {
+   a[0] = 1;
+   // expected-error@+1 {{automatic variable qualified with an address space}}
+   groupshared float b;
+ }
+
+// expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+// expected-error@+1 {{return type cannot be qualified with address space}}
+ groupshared float foo() {
+  static groupshared float foo0;
+    return 1;
+ }
+// expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+// expected-error@+1 {{return type cannot be qualified with address space}}
+  groupshared void bar() {
+    extern groupshared float bar0;
+  }
+// expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+// expected-error@+1 {{return type cannot be qualified with address space}}
+  groupshared float decl() {
+      return 1;
+  }
+
+  class C {
+      // expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+      // expected-error@+1 {{return type cannot be qualified with address space}}
+      groupshared void foo() {}
+      // expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+      // expected-error@+1 {{return type cannot be qualified with address space}}
+      groupshared void bar();
+      // expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+      // expected-error@+1 {{return type cannot be qualified with address space}}
+      friend groupshared void friend_def() {}
+      // expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+      // expected-error@+1 {{return type cannot be qualified with address space}}
+      friend groupshared void friend_decl();
+  };
+
+  struct S {
+    // expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+    // expected-error@+1 {{field may not be qualified with an address space}}
+    groupshared float f;
+    static groupshared float g;
+  };
+
+  // expected-error@+1 {{parameter may not be qualified with an address space}}
+  float foo2(groupshared float a) {
+    return a;
+  }
+
+// expected-note@+2 {{parameter may not be qualified with an address space}}
+template<typename T>
+  T tfoo(T t) {
+     return t;
+  }
+  // expected-warning@+1 {{alias declarations are a C++11 extension}}
+ using GSF = groupshared float;
+ GSF gs;
+ // expected-error@+1 {{no matching function for call to 'tfoo'}}
+ GSF gs2 = tfoo<GSF>(gs);
+
+// NOTE:This one didn't report error on the groupshared return type,
+// it is caused by return type check is after pointer check which is acceptable.
+// expected-error@+1 {{pointers are unsupported in HLSL}}
+groupshared void (*fp)();
+// expected-error@+2 {{pointers are unsupported in HLSL}}
+// expected-error@+1 {{parameter may not be qualified with an address space}}
+void (*fp2)(groupshared float);
+// NOTE: HLSL not support trailing return types.
+// expected-warning@+2 {{'auto' type specifier is a C++11 extension}}
+// expected-error@+1 {{expected function body after function declarator}}
+auto func() -> groupshared void;
+// expected-warning@+2 {{'groupshared' attribute only applies to variables}}
+// expected-error@+1 {{return type cannot be qualified with address space}}
+void groupshared f();
+
+struct S2 {
+  // Do we reject it as a function qualifier on a member function?
+  void f() groupshared;
+};
+
+// Does it impact size or alignment?
+_Static_assert(sizeof(float) == sizeof(groupshared float), "");
+_Static_assert(_Alignof(double) == _Alignof(groupshared double),"");
+
+// Does it impact type identity for templates?
+template <typename Ty>
+struct S3 {
+  static const bool value = false;
+};
+
+template <>
+struct S3<groupshared float> {
+  static const bool value = true;
+};
+_Static_assert(!S3<float>::value, "");
+_Static_assert(S3<groupshared float>::value, "");
+
+// Can you overload based on the qualifier?
+void func(float f) {}
+// expected-error@+1 {{parameter may not be qualified with an address space}}
+void func(groupshared float f) {}

clang/test/SemaHLSL/group_shared_202x.hlsl

+// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -std=hlsl202x  -o - -fsyntax-only %s -verify
+
+// expected-error@+1 {{return type cannot be qualified with address space}}
+auto func() -> groupshared void;
+
+// expected-error@+1 {{parameter may not be qualified with an address space}}
+auto func(float groupshared) -> void;
+
+// expected-error@+1 {{parameter may not be qualified with an address space}}
+auto l = [](groupshared float ) {};
+
+// expected-error@+1 {{return type cannot be qualified with address space}}
+auto l2 = []() -> groupshared void {};
+
+struct S {
+// expected-error@+1 {{return type cannot be qualified with address space}}
+operator groupshared int() const;
+
+};

clang/test/SemaOpenCL/address-spaces.cl

void nested(__global int *g, __global int * __private *gg, __local int *l, __local int * __private *ll, __global float * __private *gg_f) {
   pg = pl;  // expected-error {{assigning '__private l_t *__private' (aka '__local int *__private *__private') to '__private g_t *' (aka '__global int *__private *') changes address space of nested pointer}}
   pl = pg;  // expected-error {{assigning '__private g_t *__private' (aka '__global int *__private *__private') to '__private l_t *' (aka '__local int *__private *') changes address space of nested pointer}}
 
   ll = (__local int * __private *)(void *)gg;
   void *vp = ll;
 }
 #endif
 
-__private int func_return_priv(void);       //expected-error {{return value cannot be qualified with address space}}
-__global int func_return_global(void);      //expected-error {{return value cannot be qualified with address space}}
-__local int func_return_local(void);        //expected-error {{return value cannot be qualified with address space}}
-__constant int func_return_constant(void);  //expected-error {{return value cannot be qualified with address space}}
+__private int func_return_priv(void);       //expected-error {{return type cannot be qualified with address space}}
+__global int func_return_global(void);      //expected-error {{return type cannot be qualified with address space}}
+__local int func_return_local(void);        //expected-error {{return type cannot be qualified with address space}}
+__constant int func_return_constant(void);  //expected-error {{return type cannot be qualified with address space}}
 #if __OPENCL_C_VERSION__ >= 200
-__generic int func_return_generic(void);    //expected-error {{return value cannot be qualified with address space}}
+__generic int func_return_generic(void);    //expected-error {{return type cannot be qualified with address space}}
 #endif
 
 void func_multiple_addr(void) {
   typedef __private int private_int_t;
   __private __local int var1;   // expected-error {{multiple address spaces specified for type}}
   __private __local int *var2;  // expected-error {{multiple address spaces specified for type}}
   __local private_int_t var3;   // expected-error {{multiple address spaces specified for type}}
   __local private_int_t *var4;  // expected-error {{multiple address spaces specified for type}}

clang/test/SemaOpenCL/invalid-kernel.cl

 // RUN: %clang_cc1 -verify %s
 // RUN: %clang_cc1 -cl-std=CL2.0 -verify %s
 
 kernel int bar()  { // expected-error {{kernel must have void return type}}
   return 6;
 }
 
 kernel void main() { // expected-error {{kernel cannot be called 'main'}}
 
 }
 
 int main() { // expected-error {{function cannot be called 'main'}}
   return 0;
 }
 
-int* global x(int* x) { // expected-error {{return value cannot be qualified with address space}}
+int* global x(int* x) { // expected-error {{return type cannot be qualified with address space}}
   return x + 1;
 }
 
-int* local x(int* x) { // expected-error {{return value cannot be qualified with address space}}
+int* local x(int* x) { // expected-error {{return type cannot be qualified with address space}}
   return x + 1;
 }
 
-int* constant x(int* x) { // expected-error {{return value cannot be qualified with address space}}
+int* constant x(int* x) { // expected-error {{return type cannot be qualified with address space}}
   return x + 1;
 }

clang/test/SemaTemplate/address_space-dependent.cpp

 
 template <int I>
 void neg() {
   __attribute__((address_space(I))) int *bounds; // expected-error {{address space is negative}}
 }
 
 template <long int I>
 void tooBig() {
-  __attribute__((address_space(I))) int *bounds; // expected-error {{address space is larger than the maximum supported (8388588)}}
+  __attribute__((address_space(I))) int *bounds; // expected-error {{address space is larger than the maximum supported (8388587)}}
 }
 
 template <long int I>
 void correct() {
   __attribute__((address_space(I))) int *bounds;
 }
 
 template <int I, int J>