[Clang][WIP]Experimental implementation of data member packs in dependent context.

[SlaterLatiao]

Summary

• Experimental implementation of discourse#71333.
• Supports access data member packs in dependent context.
• Expands access to member packs on transforming member access expression.
• Renames expanded field names to avoid duplication.

Depends on D156546

[cjdb]

clang/lib/Sema/SemaTemplateInstantiate.cpp

3292

Does LLVM have some form of absl::StrCat that we can use instead of operator+?

clang/lib/Sema/TreeTransform.h

4191–4218

It may be worth putting this into its own named function to help with readability.

clang/test/CodeGenCXX/data_member_packs.cpp

73

This needs to be passed to one of the sum functions and checked that it's generating the correct code.

[denik]

clang/lib/Sema/SemaExprMember.cpp

528–529

if (isa<PackExpansionType>(Field->getType())) { ...

1018

expanded

clang/lib/Sema/SemaTemplateInstantiate.cpp

3289

nit: Maybe inaccessible by?

clang/lib/Sema/TreeTransform.h

4177

Expanded

4198–4199

Why is this in a loop?

4208

Can we add a check that Arg never exceeds the number of arguments in the full expansion (is it Expansion->getNumExpansions()?)

clang/test/CodeGenCXX/data_member_packs.cpp

76

This is a good test case!
I would also add a test for the partial expansion of the member pack. And also checking the right order of the fields. Something like:

template<typename T> constexpr auto foo(T t) { return t; }
template<typename T, typename ... Ts> constexpr auto foo(T t, Ts ... ts) { return t + 2 * foo(ts...); }

S1<int, int, int> s = {1,2,3};
static_assert(foo(s.ts...) == 17);

[MaskRay]

clang/lib/Sema/SemaTemplateInstantiate.cpp

3292

(Twine(...) + ...).str()

Many times Twine can be omitted.

clang/lib/Sema/TreeTransform.h

4218

https://llvm.org/docs/CodingStandards.html#prefer-preincrement

[dblaikie]

clang/lib/Sema/SemaExprMember.cpp

519–521

I think you can skip the const in the first dyn_cast (think it works implicitly?) & could use const auto * on the LHS of both these ifs, since the RHS makes it clear what the type is

523–524

No need for the assert if you're immediately going to cast anyway, it'll assert. Though perhaps the custom assert here gives you a chance to make it more explicit that this is intentional.

525–527

This could be llvm::find_if, maybe? Not sure if it'd be tidier, but maybe more legible

1023–1025

Usually don't put {} on single-line scopes in the LLVM codebase.

clang/lib/Sema/SemaTemplateInstantiate.cpp

3292

there's at least llvm::Twine which can avoid manifesting the intermediate strings

clang/lib/Sema/TreeTransform.h

4171

Might be worth pulling out this new code as a separate function - the continue at the end is many lines away from the start or end of the loop, making control flow a bit hard to follow (probably the existing code could do with some of this massaging even before/regardless of the new code you're adding here)

[SlaterLatiao]

Address review comments.

[SlaterLatiao]

clang/lib/Sema/SemaExprMember.cpp

523–524

Removed the assert.

525–527

Rewrote with llvm::find_if.

[SlaterLatiao]

Remove changes in D156546 which doesn't belong to this patch.

[SlaterLatiao]

• Minor fix on comment.
• Replace string '+' with Twine.

[SlaterLatiao]

• Make transforming expressions for member packs a separate function.
• Minor edits based on review comments.

[SlaterLatiao]

clang/lib/Sema/TreeTransform.h

4171

Made the logic a separate function and added a comment to explain the continue.

4198–4199

Each field need to have its own MemberAccessExpression. This loop iterates through the expanded fields.

4208

Expansion->getNumExpansions() returns the number of expansions only when we know it. In this case it returns std::nullopt.
A more legit way to check Arg is to search for the specialization of the struct based on the template args of the current context, but I didn't find a proper way to achieve that.

[SlaterLatiao]

• Added test case for partial expansion.

[SlaterLatiao]

clang/test/CodeGenCXX/data_member_packs.cpp

73

It's passed to sum and generation of sum(int, int) is checked.

76

We can't check the runtime state in LLVM/Clang tests. I added a test case for partial expansion and checked the IR. The order of fields can be checked by looking at the order of types in the instantiation of sum.

clang/include/clang/AST/ExprCXX.h

class CXXDependentScopeMemberExpr final
   NestedNameSpecifierLoc QualifierLoc;
 
   /// The member to which this member expression refers, which
   /// can be name, overloaded operator, or destructor.
   ///
   /// FIXME: could also be a template-id
   DeclarationNameInfo MemberNameInfo;
 
+  /// Whether the member is a member pack.
+  bool isMemberPack;
+
   // CXXDependentScopeMemberExpr is followed by several trailing objects,
   // some of which optional. They are in order:
   //
   // * An optional ASTTemplateKWAndArgsInfo for the explicitly specified
   //   template keyword and arguments. Present if and only if
   //   hasTemplateKWAndArgsInfo().
   //
   // * An array of getNumTemplateArgs() TemplateArgumentLoc containing location
class CXXDependentScopeMemberExpr final
   unsigned numTrailingObjects(OverloadToken<TemplateArgumentLoc>) const {
     return getNumTemplateArgs();
   }
 
   unsigned numTrailingObjects(OverloadToken<NamedDecl *>) const {
     return hasFirstQualifierFoundInScope();
   }
 
-  CXXDependentScopeMemberExpr(const ASTContext &Ctx, Expr *Base,
-                              QualType BaseType, bool IsArrow,
-                              SourceLocation OperatorLoc,
-                              NestedNameSpecifierLoc QualifierLoc,
-                              SourceLocation TemplateKWLoc,
-                              NamedDecl *FirstQualifierFoundInScope,
-                              DeclarationNameInfo MemberNameInfo,
-                              const TemplateArgumentListInfo *TemplateArgs);
+  CXXDependentScopeMemberExpr(
+      const ASTContext &Ctx, Expr *Base, QualType BaseType, bool IsArrow,
+      SourceLocation OperatorLoc, NestedNameSpecifierLoc QualifierLoc,
+      SourceLocation TemplateKWLoc, NamedDecl *FirstQualifierFoundInScope,
+      const TemplateArgumentListInfo *TemplateArgs, bool isMemberPack);
 
   CXXDependentScopeMemberExpr(EmptyShell Empty, bool HasTemplateKWAndArgsInfo,
                               bool HasFirstQualifierFoundInScope);
 
 public:
   static CXXDependentScopeMemberExpr *
   Create(const ASTContext &Ctx, Expr *Base, QualType BaseType, bool IsArrow,
          SourceLocation OperatorLoc, NestedNameSpecifierLoc QualifierLoc,
          SourceLocation TemplateKWLoc, NamedDecl *FirstQualifierFoundInScope,
          DeclarationNameInfo MemberNameInfo,
-         const TemplateArgumentListInfo *TemplateArgs);
+         const TemplateArgumentListInfo *TemplateArgs,
+         bool isMemberPack = false);
 
   static CXXDependentScopeMemberExpr *
   CreateEmpty(const ASTContext &Ctx, bool HasTemplateKWAndArgsInfo,
               unsigned NumTemplateArgs, bool HasFirstQualifierFoundInScope);
 
   /// True if this is an implicit access, i.e. one in which the
   /// member being accessed was not written in the source.  The source
   /// location of the operator is invalid in this case.
   bool isImplicitAccess() const {
     if (!Base)
       return true;
     return cast<Expr>(Base)->isImplicitCXXThis();
   }
 
   /// Retrieve the base object of this member expressions,
   /// e.g., the \c x in \c x.m.
   Expr *getBase() const {
     assert(!isImplicitAccess());
     return cast<Expr>(Base);
   }
 
   QualType getBaseType() const { return BaseType; }
 
+  bool isMemberPackAccess() const { return isMemberPack; }
+
   /// Determine whether this member expression used the '->'
   /// operator; otherwise, it used the '.' operator.
   bool isArrow() const { return CXXDependentScopeMemberExprBits.IsArrow; }
 
   /// Retrieve the location of the '->' or '.' operator.
   SourceLocation getOperatorLoc() const {
     return CXXDependentScopeMemberExprBits.OperatorLoc;
   }

clang/lib/AST/ComputeDependence.cpp

ExprDependence clang::computeDependence(CXXDependentScopeMemberExpr *E) {
   auto D = ExprDependence::TypeValueInstantiation;
   if (!E->isImplicitAccess())
     D |= E->getBase()->getDependence();
   if (auto *Q = E->getQualifier())
     D |= toExprDependence(Q->getDependence());
   D |= getDependenceInExpr(E->getMemberNameInfo());
   for (const auto &A : E->template_arguments())
     D |= toExprDependence(A.getArgument().getDependence());
+  if (E->isMemberPackAccess())
+    D |= ExprDependence::UnexpandedPack;
   return D;
 }
 
 ExprDependence clang::computeDependence(MaterializeTemporaryExpr *E) {
   return E->getSubExpr()->getDependence();
 }
 
 ExprDependence clang::computeDependence(CXXFoldExpr *E) {

clang/lib/AST/ExprCXX.cpp

SourceLocation CXXUnresolvedConstructExpr::getBeginLoc() const {
   return TypeAndInitForm.getPointer()->getTypeLoc().getBeginLoc();
 }
 
 CXXDependentScopeMemberExpr::CXXDependentScopeMemberExpr(
     const ASTContext &Ctx, Expr *Base, QualType BaseType, bool IsArrow,
     SourceLocation OperatorLoc, NestedNameSpecifierLoc QualifierLoc,
     SourceLocation TemplateKWLoc, NamedDecl *FirstQualifierFoundInScope,
     DeclarationNameInfo MemberNameInfo,
-    const TemplateArgumentListInfo *TemplateArgs)
+    const TemplateArgumentListInfo *TemplateArgs, bool isMemberPack)
     : Expr(CXXDependentScopeMemberExprClass, Ctx.DependentTy, VK_LValue,
            OK_Ordinary),
       Base(Base), BaseType(BaseType), QualifierLoc(QualifierLoc),
-      MemberNameInfo(MemberNameInfo) {
+      MemberNameInfo(MemberNameInfo), isMemberPack(isMemberPack) {
   CXXDependentScopeMemberExprBits.IsArrow = IsArrow;
   CXXDependentScopeMemberExprBits.HasTemplateKWAndArgsInfo =
       (TemplateArgs != nullptr) || TemplateKWLoc.isValid();
   CXXDependentScopeMemberExprBits.HasFirstQualifierFoundInScope =
       FirstQualifierFoundInScope != nullptr;
   CXXDependentScopeMemberExprBits.OperatorLoc = OperatorLoc;
 
   if (TemplateArgs) {

 CXXDependentScopeMemberExpr::CXXDependentScopeMemberExpr(
     EmptyShell Empty, bool HasTemplateKWAndArgsInfo,
     bool HasFirstQualifierFoundInScope)
     : Expr(CXXDependentScopeMemberExprClass, Empty) {
   CXXDependentScopeMemberExprBits.HasTemplateKWAndArgsInfo =
       HasTemplateKWAndArgsInfo;
   CXXDependentScopeMemberExprBits.HasFirstQualifierFoundInScope =
       HasFirstQualifierFoundInScope;
+  isMemberPack = false;
 }
 
 CXXDependentScopeMemberExpr *CXXDependentScopeMemberExpr::Create(
     const ASTContext &Ctx, Expr *Base, QualType BaseType, bool IsArrow,
     SourceLocation OperatorLoc, NestedNameSpecifierLoc QualifierLoc,
     SourceLocation TemplateKWLoc, NamedDecl *FirstQualifierFoundInScope,
     DeclarationNameInfo MemberNameInfo,
-    const TemplateArgumentListInfo *TemplateArgs) {
+    const TemplateArgumentListInfo *TemplateArgs,
+    bool isMemberPack) {
   bool HasTemplateKWAndArgsInfo =
       (TemplateArgs != nullptr) || TemplateKWLoc.isValid();
   unsigned NumTemplateArgs = TemplateArgs ? TemplateArgs->size() : 0;
   bool HasFirstQualifierFoundInScope = FirstQualifierFoundInScope != nullptr;
 
   unsigned Size = totalSizeToAlloc<ASTTemplateKWAndArgsInfo,
                                    TemplateArgumentLoc, NamedDecl *>(
       HasTemplateKWAndArgsInfo, NumTemplateArgs, HasFirstQualifierFoundInScope);
 
   void *Mem = Ctx.Allocate(Size, alignof(CXXDependentScopeMemberExpr));
   return new (Mem) CXXDependentScopeMemberExpr(
       Ctx, Base, BaseType, IsArrow, OperatorLoc, QualifierLoc, TemplateKWLoc,
-      FirstQualifierFoundInScope, MemberNameInfo, TemplateArgs);
+      FirstQualifierFoundInScope, MemberNameInfo, TemplateArgs, isMemberPack);
 }
 
 CXXDependentScopeMemberExpr *CXXDependentScopeMemberExpr::CreateEmpty(
     const ASTContext &Ctx, bool HasTemplateKWAndArgsInfo,
     unsigned NumTemplateArgs, bool HasFirstQualifierFoundInScope) {
   assert(NumTemplateArgs == 0 || HasTemplateKWAndArgsInfo);
 
   unsigned Size = totalSizeToAlloc<ASTTemplateKWAndArgsInfo,

clang/lib/Sema/SemaExprMember.cpp

Sema::ActOnDependentMemberExpr(Expr *BaseExpr, QualType BaseType,
 
   assert(BaseType->isDependentType() || NameInfo.getName().isDependentName() ||
          isDependentScopeSpecifier(SS) ||
          (TemplateArgs && llvm::any_of(TemplateArgs->arguments(),
                                        [](const TemplateArgumentLoc &Arg) {
                                          return Arg.getArgument().isDependent();
                                        })));
 
-  // Get the type being accessed in BaseType.  If this is an arrow, the BaseExpr
+  bool isMemberPack = false;
+  // Check for member packs by looking up base's template decl.
+  if (const ElaboratedType *ET = dyn_cast<const ElaboratedType>(BaseType)) {
+    if (const TemplateSpecializationType *TST =
+            dyn_cast<TemplateSpecializationType>(ET->getNamedType())) {
+      auto *TD = TST->getTemplateName().getAsTemplateDecl();
+      assert(isa<ClassTemplateDecl>(TD) &&
+             "template decl in member access is not ClassTemplateDecl");
+      for (FieldDecl *Field :
+           cast<ClassTemplateDecl>(TD)->getTemplatedDecl()->fields()) {
+        if (Field->getDeclName() == NameInfo.getName()) {
+          if (const PackExpansionType *PET =
+                  dyn_cast<PackExpansionType>(Field->getType())) {
+            isMemberPack = true;
+          }
+          break;
+        }
+      }
+    }
+  }
+  // Get the type being accessed in BaseType. If this is an arrow, the BaseExpr
   // must have pointer type, and the accessed type is the pointee.
   return CXXDependentScopeMemberExpr::Create(
       Context, BaseExpr, BaseType, IsArrow, OpLoc,
       SS.getWithLocInContext(Context), TemplateKWLoc, FirstQualifierInScope,
-      NameInfo, TemplateArgs);
+      NameInfo, TemplateArgs, isMemberPack);
 }
 
 /// We know that the given qualified member reference points only to
 /// declarations which do not belong to the static type of the base
 /// expression.  Diagnose the problem.
 static void DiagnoseQualifiedMemberReference(Sema &SemaRef,
                                              Expr *BaseExpr,
                                              QualType BaseType,
Sema::BuildMemberReferenceExpr(Expr *BaseExpr, QualType BaseExprType,
   if (S && BaseExpr && FD &&
       (isa<CXXDestructorDecl>(FD) || isa<CXXConstructorDecl>(FD)) &&
       isa<CXXThisExpr>(BaseExpr->IgnoreImpCasts()) &&
       IsInFnTryBlockHandler(S))
     Diag(MemberLoc, diag::warn_cdtor_function_try_handler_mem_expr)
         << isa<CXXDestructorDecl>(FD);
 
   if (R.empty()) {
+    // The member is expaned from member pack if its name has `@` in it. In this
+    // case a failed name lookup is not an error, but represents it's at the end
+    // of the expanded members. Return an empty expression to inform its
+    // caller.
+    auto MemberNameStr = MemberNameInfo.getName().getAsString();
+    if (MemberNameStr.find('@') != std::string::npos) {
+      return ExprEmpty();
+    }
     // Rederive where we looked up.
-    DeclContext *DC = (SS.isSet()
-                       ? computeDeclContext(SS, false)
+    DeclContext *DC = (SS.isSet() ? computeDeclContext(SS, false)
                                   : BaseType->castAs<RecordType>()->getDecl());
 
     if (ExtraArgs) {
       ExprResult RetryExpr;
       if (!IsArrow && BaseExpr) {
         SFINAETrap Trap(*this, true);
         ParsedType ObjectType;
         bool MayBePseudoDestructor = false;
         RetryExpr = ActOnStartCXXMemberReference(getCurScope(), BaseExpr,

clang/lib/Sema/SemaTemplateInstantiate.cpp

if (FieldDecl *Field = dyn_cast<FieldDecl>(Member);
         // Generate a new field from PackExpansion field.
         if (Decl *NewMember = Instantiator.Visit(Member)) {
           FieldDecl *PackedField = cast<FieldDecl>(NewMember);
           Sema::ArgumentPackSubstitutionIndexRAII SubstIndex(*this, Arg);
           QualType T =
               SubstType(PatternType, TemplateArgs, PackedField->getLocation(),
                         PackedField->getDeclName());
           PackedField->setType(T);
+          // Rename the expanded fields to avoid ambiguous names in
+          // name lookup. The renamed fields need to be invisible to users so an
+          // `@` is added to the name.
+          std::string NewFieldName =
+              PackedField->getName().str() + "@" + std::to_string(Arg);
+          PackedField->setDeclName(&Context.Idents.get(NewFieldName));
           Fields.push_back(PackedField);
           if (NewMember->isInvalidDecl()) {
             // When `NewMember` has type of `PackExpansionType`, it escapes
             // validation checks in `Visit`. Handling of such cases
             // will be implemented in a future commit.
             // Currently this branch should never be reached.
             assert(false && "not implemented");
             Instantiation->setInvalidDecl();

clang/lib/Sema/TreeTransform.h

if (IsCall && getDerived().DropCallArgument(Inputs[I])) {
         *ArgChanged = true;
 
       break;
     }
 
     if (PackExpansionExpr *Expansion = dyn_cast<PackExpansionExpr>(Inputs[I])) {
       Expr *Pattern = Expansion->getPattern();
 
+      if (CXXDependentScopeMemberExpr *MemberExpr =
+              dyn_cast<CXXDependentScopeMemberExpr>(Pattern)) {
+        if (ArgChanged)
+          *ArgChanged = true;
+        assert(MemberExpr->isMemberPackAccess() &&
+               "trying to expand non-pack member access");
+        std::string UnExpanedNameStr =
+            MemberExpr->getMemberNameInfo().getName().getAsString();
+
+        ExprResult Base = getDerived().TransformExpr(MemberExpr->getBase());
+        if (Base.isInvalid())
+          return true;
+        QualType BaseType = ((Expr *)Base.get())->getType();
+        if (MemberExpr->isArrow()) {
+          assert(BaseType->isPointerType());
+          BaseType = BaseType->castAs<PointerType>()->getPointeeType();
+        }
+
+        unsigned Arg = 0;
+        while (true) {
+          // Transform unexpanded field name and create a new member expression.
+          DeclarationName ExpandedName = &SemaRef.Context.Idents.get(
+              UnExpanedNameStr + "@" + std::to_string(Arg));
+          // Construct name info with new name and keep other members the same.
+          DeclarationNameInfo ExpandedNameInfo = DeclarationNameInfo(
+              ExpandedName, MemberExpr->getMemberNameInfo().getLoc(),
+              MemberExpr->getMemberNameInfo().getInfo());
+          TemplateArgumentListInfo TemplateArgs = TemplateArgumentListInfo();
+          MemberExpr->copyTemplateArgumentsInto(TemplateArgs);
+          auto *ExpandedMemberExpr = CXXDependentScopeMemberExpr::Create(
+              SemaRef.Context, MemberExpr->getBase(), MemberExpr->getBaseType(),
+              MemberExpr->isArrow(), MemberExpr->getOperatorLoc(),
+              MemberExpr->getQualifierLoc(),
+              MemberExpr->getTemplateKeywordLoc(),
+              MemberExpr->getFirstQualifierFoundInScope(), ExpandedNameInfo,
+              &TemplateArgs, MemberExpr->isMemberPackAccess());
+
+          Sema::ArgumentPackSubstitutionIndexRAII SubstIndex(getSema(), Arg);
+          ExprResult Out = getDerived().TransformExpr(ExpandedMemberExpr);
+          if (Out.isInvalid())
+            return true;
+          // An empty expression is returned when name lookup fails in accessing
+          // member packs. This means the last field in member pack has been
+          // processd and time to exit the loop.
+          if (Out.isUnset())
+            break;
+          Outputs.push_back(Out.get());
+          Arg++;
+        }
+        continue;
+      }
+
       SmallVector<UnexpandedParameterPack, 2> Unexpanded;
       getSema().collectUnexpandedParameterPacks(Pattern, Unexpanded);
       assert(!Unexpanded.empty() && "Pack expansion without parameter packs?");
 
       // Determine whether the set of unexpanded parameter packs can and should
       // be expanded.
       bool Expand = true;
       bool RetainExpansion = false;

clang/test/CodeGenCXX/data_member_packs.cpp

 // RUN: %clang_cc1 --std=c++20 %s -emit-llvm -o - -triple x86_64-linux | FileCheck %s --check-prefixes=CHECK
 
 // Tests declaration data member packs.
 template<typename... Ts> struct S1 {
     Ts... ts;
 };
 
 template<typename T, typename... Ts> struct S2 {
     T t[2];
     Ts... ts;
 };
 
+template<typename T, typename... Ts> struct S3 {
+    T t;
+    Ts... ts;
+};
+
 // CHECK: %struct.S1 = type { i32 }
 S1<int> s1;
 // CHECK-NEXT: %struct.S1.0 = type { i32, float, double }
 S1<int, float, double> s2;
 // Test template args as the last arg.
 // CHECK-NEXT: %struct.S2 = type { [2 x i32], float, double }
 S2<int, float, double> s3;
 // Test nested template args.
 // CHECK-NEXT: %struct.S1.1 = type { i32, float, %struct.S1.2 }
 // CHECK-NEXT: %struct.S1.2 = type { double, double }
 S1<int, float, S1<double, double>> s4;
 // Test empty template arg.
 // CHECK-NEXT: %struct.S1.3 = type { i8 }
 S1<> s5;
 // Test duplicate types in template args.
 // CHECK-NEXT: %struct.S1.4 = type { i32, i32 }
 S1<int, int> s6;
 
+S1<int, long, float, double> s7;
+
+template<typename... Ts> auto sum(Ts... ts) {
+  return (ts + ...);
+}
+
+auto take_empty() {
+  return 0;
+}
+
+template<typename... Ts> auto sum_pack(S1<Ts...> s) {
+  return sum(s.ts...);
+}
+// Test template arg + expansion.
+template<typename T, typename... Ts> auto sum_pack2(S1<T, Ts...> s) {
+  return sum(s.ts...);
+}
+// Test empty expansion.
+template<typename... Ts> auto take_empty(S3<Ts...> s) {
+  return take_empty(s.ts...);
+}
+// Test nested template args and multiple expansions.
+template<typename... Ts> struct S4 {
+  template<typename... Ts2> auto sum_pack(S1<Ts..., Ts2...> s) {
+    return sum(s.ts...);
+  }
+};
+
+int main() {
+  // Check calling take_empty()
+  // CHECK: %call = call noundef i32 @_Z10take_emptyv()
+  take_empty(S3<int>{});
+  // Check instantiation of sum(int, float, double)
+  // CHECK: double @_Z3sumIJifdEEDaDpT_(i32 noundef %ts, float noundef %ts1, double noundef %ts3)
+  sum_pack(s2);
+  // Check instantiation of sum(int, int)
+  // CHECK: i32 @_Z3sumIJiiEEDaDpT_(i32 noundef %ts, i32 noundef %ts1)
+  sum_pack2<int, int>(s6);
+  // Check instantiation of sum(int, long, float, double)
+  // CHECK: double @_Z3sumIJilfdEEDaDpT_(i32 noundef %ts, i64 noundef %ts1, float noundef %ts3, double noundef %ts5)
+  S4<int, long>{}.sum_pack<float, double>(s7);
+  return 0;
+}
+