[clang] Implement sugared substitution changes to infrastructure

[mizvekov]

Summary

Implements the changes required to perform substitution with
non-canonical template arguments, and to 'finalize' them
by not placing 'Subst' nodes.

A finalized substitution means we won't resugar them later,
because these templates themselves were eagerly substituted
with the intended arguments at the point of use. We may still
resugar other templates used within those, though.

This patch does not actually implement any uses of this
functionality, those will be added in subsequent patches,
so expect no changes to existing tests.

Signed-off-by: Matheus Izvekov <[e-mail]>

[ldionne]

As explained in more details here (https://reviews.llvm.org/D133249#inline-1289205), it would be nice to stop using the DELETE.ME idiom to kick off libc++ CI, unless you really think there's a good chance to cause failures in libc++ specifically.

TLDR of the link: the libc++ CI has 50+ jobs, exactly one of which will exercise your Clang changes. All the other jobs are using pre-compiled versions of Clang. So in reality, this DELETE.ME file trick only results in wasting libc++ CI resources, in addition to adding the libc++ review group on reviews that we probably don't need to look at. If you do suspect that the change might break libc++ specifically (such as changing diagnostics or whatever), then you're welcome to do so, although in the mid-term we'll try to simply run the libc++ tests as part of Clang's own CI setup.

[mizvekov]

In D134604#3814986, @ldionne wrote:

As explained in more details here (https://reviews.llvm.org/D133249#inline-1289205), it would be nice to stop using the DELETE.ME idiom to kick off libc++ CI, unless you really think there's a good chance to cause failures in libc++ specifically.

TLDR of the link: the libc++ CI has 50+ jobs, exactly one of which will exercise your Clang changes. All the other jobs are using pre-compiled versions of Clang. So in reality, this DELETE.ME file trick only results in wasting libc++ CI resources, in addition to adding the libc++ review group on reviews that we probably don't need to look at. If you do suspect that the change might break libc++ specifically (such as changing diagnostics or whatever), then you're welcome to do so, although in the mid-term we'll try to simply run the libc++ tests as part of Clang's own CI setup.

Hello!

I was aware of those discussions, and I even made a workaround to avoid this extra cost of running unrelated pipelines.
The patch on the very bottom of this stack (D134079) disables all the other pipelines except the bootstrapping one.

For this patch, I only needed a pipeline run for the initial submission, but I don't plan to keep running it, I will remove the DELETE.ME on the next submission.

I have two other patches in the stack which I will need to keep running the libcxx-ci as I update them: D131858 and D127695. These have had non-trivial libcxx breakages, and I have even seen new problems introduced as you keep working on the ranges implementation.

Thanks by the way for the awesome work on the libcxx-CI, and I hope in the near future these workarounds will not be needed anymore!

[ldionne]

In D134604#3815634, @mizvekov wrote:

I was aware of those discussions, and I even made a workaround to avoid this extra cost of running unrelated pipelines.
The patch on the very bottom of this stack (D134079) disables all the other pipelines except the bootstrapping one.

Oh, awesome, I wasn't aware of that. Okay, this mitigates a lot of my concern then. The only remaining annoyance is the additional traffic on the libc++ review group, but that's manageable if it gets you the testing you need until we've improved the CI situation w/ Clang.

I have two other patches in the stack which I will need to keep running the libcxx-ci as I update them: D131858 and D127695. These have had non-trivial libcxx breakages, and I have even seen new problems introduced as you keep working on the ranges implementation.

Understood, in that case I think your usage of the CI is definitely justified. I mostly wanted to get the word out that this is not a pattern we want to encourage more generally, for the reasons explained, but I think it doesn't apply here.

Thanks by the way for the awesome work on the libcxx-CI, and I hope in the near future these workarounds will not be needed anymore!

Hey, thanks to you for wanting to ensure your patches don't break libc++ :-)!

[erichkeane]

A couple of nits, otherwise I don't have to see this again.

clang/include/clang/AST/TemplateName.h

158

Based on this comment, should this be 'Finalize' instead of 'Final'? Or something like "ShouldFinalize"?

clang/include/clang/Sema/Template.h

104–105

Hrmph, this additional flag likely requires rebasing/rebuilding on a patch I just committed, which uses this MLTAL again. Hopefully not too bad to propagate.

clang/lib/AST/ASTContext.cpp

3049

typically we name these sorts of things based on their return type, so this would be more hasCanonicalTemplateArguments. BUT the modifying nature of it makes this an awkward function all the same.

I think I'd suggest putting both of the 'output's here on the same side of the function, either both a return, or both an out-param, so the name isn't ambiguous/confusing like this.

clang/include/clang/AST/ASTContext.h

public:
 
   QualType getBTFTagAttributedType(const BTFTypeTagAttr *BTFAttr,
                                    QualType Wrapped);
 
   QualType getSubstTemplateTypeParmType(QualType Replacement,
                                         Decl *AssociatedDecl, unsigned Index,
                                         Optional<unsigned> PackIndex) const;
   QualType getSubstTemplateTypeParmPackType(Decl *AssociatedDecl,
-                                            unsigned Index,
+                                            unsigned Index, bool Final,
                                             const TemplateArgument &ArgPack);
 
   QualType
   getTemplateTypeParmType(unsigned Depth, unsigned Index,
                           bool ParameterPack,
                           TemplateTypeParmDecl *ParmDecl = nullptr) const;
 
   QualType getTemplateSpecializationType(TemplateName T,
public:
   TemplateName getDependentTemplateName(NestedNameSpecifier *NNS,
                                         OverloadedOperatorKind Operator) const;
   TemplateName getSubstTemplateTemplateParm(TemplateName replacement,
                                             Decl *AssociatedDecl,
                                             unsigned Index,
                                             Optional<unsigned> PackIndex) const;
   TemplateName getSubstTemplateTemplateParmPack(const TemplateArgument &ArgPack,
                                                 Decl *AssociatedDecl,
-                                                unsigned Index) const;
+                                                unsigned Index,
+                                                bool Final) const;
 
   enum GetBuiltinTypeError {
     /// No error
     GE_None,
 
     /// Missing a type
     GE_Missing_type,
 

clang/include/clang/AST/PropertiesBase.td

def : Property<"argumentPack", TemplateArgument> {
     let Read = [{ parm->getArgumentPack() }];
   }
   def : Property<"associatedDecl", DeclRef> {
     let Read = [{ parm->getAssociatedDecl() }];
   }
   def : Property<"index", UInt32> {
     let Read = [{ parm->getIndex() }];
   }
+  def : Property<"sugared", Bool> {
+    let Read = [{ parm->getFinal() }];
+  }
   def : Creator<[{
-    return ctx.getSubstTemplateTemplateParmPack(argumentPack, associatedDecl, index);
+    return ctx.getSubstTemplateTemplateParmPack(argumentPack, associatedDecl, index, sugared);
   }]>;
 }
 
 // Type cases for TemplateArgument.
 def : PropertyTypeKind<TemplateArgument, TemplateArgumentKind,
                        "node.getKind()">;
 let Class = PropertyTypeCase<TemplateArgument, "Null"> in {
   def : Creator<[{

clang/include/clang/AST/TemplateName.h

 /// parameter pack.
 ///
 /// This kind of template names occurs when the parameter pack has been
 /// provided with a template template argument pack in a context where its
 /// enclosing pack expansion could not be fully expanded.
 class SubstTemplateTemplateParmPackStorage : public UncommonTemplateNameStorage,
                                              public llvm::FoldingSetNode {
   const TemplateArgument *Arguments;
-  Decl *AssociatedDecl;
+  llvm::PointerIntPair<Decl *, 1, bool> AssociatedDeclAndFinal;
 
 public:
   SubstTemplateTemplateParmPackStorage(ArrayRef<TemplateArgument> ArgPack,
-                                       Decl *AssociatedDecl, unsigned Index)
-      : UncommonTemplateNameStorage(SubstTemplateTemplateParmPack, Index,
-                                    ArgPack.size()),
-        Arguments(ArgPack.data()), AssociatedDecl(AssociatedDecl) {
-    assert(AssociatedDecl != nullptr);
-  }
+                                       Decl *AssociatedDecl, unsigned Index,
+                                       bool Final);
 
   /// A template-like entity which owns the whole pattern being substituted.
   /// This will own a set of template parameters.
-  Decl *getAssociatedDecl() const { return AssociatedDecl; }
+  Decl *getAssociatedDecl() const;
 
   /// Returns the index of the replaced parameter in the associated declaration.
   /// This should match the result of `getParameterPack()->getIndex()`.
   unsigned getIndex() const { return Bits.Index; }
 
+  // When true the substitution will be 'Final' (subst node won't be placed).
+  bool getFinal() const;
+
   /// Retrieve the template template parameter pack being substituted.
   TemplateTemplateParmDecl *getParameterPack() const;
 
   /// Retrieve the template template argument pack with which this
   /// parameter was substituted.
   TemplateArgument getArgumentPack() const;
 
   void Profile(llvm::FoldingSetNodeID &ID, ASTContext &Context);
 
   static void Profile(llvm::FoldingSetNodeID &ID, ASTContext &Context,
                       const TemplateArgument &ArgPack, Decl *AssociatedDecl,
-                      unsigned Index);
+                      unsigned Index, bool Final);
 };
 
 /// Represents a C++ template name within the type system.
 ///
 /// A C++ template name refers to a template within the C++ type
 /// system. In most cases, a template name is simply a reference to a
 /// class template, e.g.
 ///

clang/include/clang/AST/Type.h

 /// at the current pack substitution index.
 class SubstTemplateTypeParmPackType : public Type, public llvm::FoldingSetNode {
   friend class ASTContext;
 
   /// A pointer to the set of template arguments that this
   /// parameter pack is instantiated with.
   const TemplateArgument *Arguments;
 
-  Decl *AssociatedDecl;
+  llvm::PointerIntPair<Decl *, 1, bool> AssociatedDeclAndFinal;
 
   SubstTemplateTypeParmPackType(QualType Canon, Decl *AssociatedDecl,
-                                unsigned Index,
+                                unsigned Index, bool Final,
                                 const TemplateArgument &ArgPack);
 
 public:
   IdentifierInfo *getIdentifier() const;
 
   /// A template-like entity which owns the whole pattern being substituted.
   /// This will usually own a set of template parameters, or in some
   /// cases might even be a template parameter itself.
-  Decl *getAssociatedDecl() const { return AssociatedDecl; }
+  Decl *getAssociatedDecl() const;
 
   /// Gets the template parameter declaration that was substituted for.
   const TemplateTypeParmDecl *getReplacedParameter() const;
 
   /// Returns the index of the replaced parameter in the associated declaration.
   /// This should match the result of `getReplacedParameter()->getIndex()`.
   unsigned getIndex() const { return SubstTemplateTypeParmPackTypeBits.Index; }
 
+  // When true the substitution will be 'Final' (subst node won't be placed).
+  bool getFinal() const;
+
   unsigned getNumArgs() const {
     return SubstTemplateTypeParmPackTypeBits.NumArgs;
   }
 
   bool isSugared() const { return false; }
   QualType desugar() const { return QualType(this, 0); }
 
   TemplateArgument getArgumentPack() const;
 
   void Profile(llvm::FoldingSetNodeID &ID);
   static void Profile(llvm::FoldingSetNodeID &ID, const Decl *AssociatedDecl,
-                      unsigned Index, const TemplateArgument &ArgPack);
+                      unsigned Index, bool Final,
+                      const TemplateArgument &ArgPack);
 
   static bool classof(const Type *T) {
     return T->getTypeClass() == SubstTemplateTypeParmPack;
   }
 };
 
 /// Common base class for placeholders for types that get replaced by
 /// placeholder type deduction: C++11 auto, C++14 decltype(auto), C++17 deduced

clang/include/clang/AST/TypeProperties.td

 
 let Class = SubstTemplateTypeParmPackType in {
   def : Property<"associatedDecl", DeclRef> {
     let Read = [{ node->getAssociatedDecl() }];
   }
   def : Property<"Index", UInt32> {
     let Read = [{ node->getIndex() }];
   }
+  def : Property<"Final", Bool> {
+    let Read = [{ node->getFinal() }];
+  }
   def : Property<"replacementPack", TemplateArgument> {
     let Read = [{ node->getArgumentPack() }];
   }
 
   def : Creator<[{
     return ctx.getSubstTemplateTypeParmPackType(
-                        associatedDecl, Index, replacementPack);
+                        associatedDecl, Index, Final, replacementPack);
   }]>;
 }
 
 let Class = BuiltinType in {
   def : Property<"kind", BuiltinTypeKind> {
     let Read = [{ node->getKind() }];
   }
 

clang/include/clang/Sema/Sema.h

   static void MarkDeducedTemplateParameters(ASTContext &Ctx,
                                   const FunctionTemplateDecl *FunctionTemplate,
                                   llvm::SmallBitVector &Deduced);
 
   //===--------------------------------------------------------------------===//
   // C++ Template Instantiation
   //
 
-  MultiLevelTemplateArgumentList getTemplateInstantiationArgs(
-      const NamedDecl *D, const TemplateArgumentList *Innermost = nullptr,
-      bool RelativeToPrimary = false, const FunctionDecl *Pattern = nullptr,
+  MultiLevelTemplateArgumentList
+  getTemplateInstantiationArgs(const NamedDecl *D, bool Final = false,
+                               const TemplateArgumentList *Innermost = nullptr,
+                               bool RelativeToPrimary = false,
+                               const FunctionDecl *Pattern = nullptr,
                                bool ForConstraintInstantiation = false);
 
   /// A context in which code is being synthesized (where a source location
   /// alone is not sufficient to identify the context). This covers template
   /// instantiation and various forms of implicitly-generated functions.
   struct CodeSynthesisContext {
     /// The kind of template instantiation we are performing
     enum SynthesisKind {
       /// We are instantiating a template declaration. The entity is

   QualType BuildObjCTypeParamType(const ObjCTypeParamDecl *Decl,
                                   SourceLocation ProtocolLAngleLoc,
                                   ArrayRef<ObjCProtocolDecl *> Protocols,
                                   ArrayRef<SourceLocation> ProtocolLocs,
                                   SourceLocation ProtocolRAngleLoc,
                                   bool FailOnError = false);
 
   /// Build an Objective-C object pointer type.
-  QualType BuildObjCObjectType(QualType BaseType,
-                               SourceLocation Loc,
-                               SourceLocation TypeArgsLAngleLoc,
-                               ArrayRef<TypeSourceInfo *> TypeArgs,
-                               SourceLocation TypeArgsRAngleLoc,
-                               SourceLocation ProtocolLAngleLoc,
-                               ArrayRef<ObjCProtocolDecl *> Protocols,
-                               ArrayRef<SourceLocation> ProtocolLocs,
-                               SourceLocation ProtocolRAngleLoc,
-                               bool FailOnError = false);
+  QualType BuildObjCObjectType(
+      QualType BaseType, SourceLocation Loc, SourceLocation TypeArgsLAngleLoc,
+      ArrayRef<TypeSourceInfo *> TypeArgs, SourceLocation TypeArgsRAngleLoc,
+      SourceLocation ProtocolLAngleLoc, ArrayRef<ObjCProtocolDecl *> Protocols,
+      ArrayRef<SourceLocation> ProtocolLocs, SourceLocation ProtocolRAngleLoc,
+      bool FailOnError, bool Rebuilding);
 
   /// Ensure attributes are consistent with type.
   /// \param [in, out] Attributes The attributes to check; they will
   /// be modified to be consistent with \p PropertyTy.
   void CheckObjCPropertyAttributes(Decl *PropertyPtrTy,
                                    SourceLocation Loc,
                                    unsigned &Attributes,
                                    bool propertyInPrimaryClass);

clang/include/clang/Sema/Template.h

enum class TemplateSubstitutionKind : char {
   /// When instantiating X<int>::Y<17>::f, the multi-level template argument
   /// list will contain a template argument list (int) at depth 0 and a
   /// template argument list (17) at depth 1.
   class MultiLevelTemplateArgumentList {
     /// The template argument list at a certain template depth
 
     using ArgList = ArrayRef<TemplateArgument>;
     struct ArgumentListLevel {
-      Decl *AssociatedDecl;
+      llvm::PointerIntPair<Decl *, 1, bool> AssociatedDeclAsFinal;
       ArgList Args;
     };
     using ContainerType = SmallVector<ArgumentListLevel, 4>;
 
     using ArgListsIterator = ContainerType::iterator;
     using ConstArgListsIterator = ContainerType::const_iterator;
 
     /// The template argument lists, stored from the innermost template
     /// argument list (first) to the outermost template argument list (last).
     ContainerType TemplateArgumentLists;
 
     /// The number of outer levels of template arguments that are not
     /// being substituted.
     unsigned NumRetainedOuterLevels = 0;
 
     /// The kind of substitution described by this argument list.
     TemplateSubstitutionKind Kind = TemplateSubstitutionKind::Specialization;
 
   public:
     /// Construct an empty set of template argument lists.
     MultiLevelTemplateArgumentList() = default;
 
     /// Construct a single-level template argument list.
-    MultiLevelTemplateArgumentList(Decl *D, ArgList Args) {
-      addOuterTemplateArguments(D, Args);
+    MultiLevelTemplateArgumentList(Decl *D, ArgList Args, bool Final) {
+      addOuterTemplateArguments(D, Args, Final);
     }
 
     void setKind(TemplateSubstitutionKind K) { Kind = K; }
 
     /// Determine the kind of template substitution being performed.
     TemplateSubstitutionKind getKind() const { return Kind; }
 
     /// Determine whether we are rewriting template parameters rather than
const TemplateArgument &operator()(unsigned Depth, unsigned Index) const {
       assert(Index <
              TemplateArgumentLists[getNumLevels() - Depth - 1].Args.size());
       return TemplateArgumentLists[getNumLevels() - Depth - 1].Args[Index];
     }
 
     /// A template-like entity which owns the whole pattern being substituted.
     /// This will usually own a set of template parameters, or in some
     /// cases might even be a template parameter itself.
-    Decl *getAssociatedDecl(unsigned Depth) const {
+    /// Returns nullptr for fully-sugared substitutions.
+    std::pair<Decl *, bool> getAssociatedDecl(unsigned Depth) const {
       assert(NumRetainedOuterLevels <= Depth && Depth < getNumLevels());
-      return TemplateArgumentLists[getNumLevels() - Depth - 1].AssociatedDecl;
+      auto AD = TemplateArgumentLists[getNumLevels() - Depth - 1]
+                    .AssociatedDeclAsFinal;
+      return {AD.getPointer(), AD.getInt()};
     }
 
     /// Determine whether there is a non-NULL template argument at the
     /// given depth and index.
     ///
     /// There must exist a template argument list at the given depth.
     bool hasTemplateArgument(unsigned Depth, unsigned Index) const {
       assert(Depth < getNumLevels());
void setArgument(unsigned Depth, unsigned Index,
       assert(Index <
              TemplateArgumentLists[getNumLevels() - Depth - 1].Args.size());
       const_cast<TemplateArgument &>(
           TemplateArgumentLists[getNumLevels() - Depth - 1].Args[Index]) = Arg;
     }
 
     /// Add a new outmost level to the multi-level template argument
     /// list.
-    void addOuterTemplateArguments(Decl *AssociatedDecl, ArgList Args) {
+    /// A 'Final' substitution means that Subst* nodes won't be built
+    /// for the replacements.
+    void addOuterTemplateArguments(Decl *AssociatedDecl, ArgList Args,
+                                   bool Final) {
       assert(!NumRetainedOuterLevels &&
              "substituted args outside retained args?");
       assert(getKind() == TemplateSubstitutionKind::Specialization);
-      assert(AssociatedDecl != nullptr || Args.size() == 0);
       TemplateArgumentLists.push_back(
-          {AssociatedDecl ? AssociatedDecl->getCanonicalDecl() : nullptr,
-           Args});
+          {{AssociatedDecl->getCanonicalDecl(), Final}, Args});
     }
 
     void addOuterTemplateArguments(ArgList Args) {
       assert(!NumRetainedOuterLevels &&
              "substituted args outside retained args?");
       assert(getKind() == TemplateSubstitutionKind::Rewrite);
-      TemplateArgumentLists.push_back({nullptr, Args});
+      TemplateArgumentLists.push_back({{}, Args});
     }
 
     void addOuterTemplateArguments(llvm::NoneType) {
       assert(!NumRetainedOuterLevels &&
              "substituted args outside retained args?");
       TemplateArgumentLists.push_back({});
     }
 

clang/lib/AST/ASTContext.cpp

 }
 
 const ASTRecordLayout &
 ASTContext::getASTObjCImplementationLayout(
                                         const ObjCImplementationDecl *D) const {
   return getObjCLayout(D->getClassInterface(), D);
 }
 
+static auto getCanonicalTemplateArguments(const ASTContext &C,
+                                          ArrayRef<TemplateArgument> Args,
+                                          bool &AnyNonCanonArgs) {
+  SmallVector<TemplateArgument, 16> CanonArgs(Args);
+  for (auto &Arg : CanonArgs) {
+    TemplateArgument OrigArg = Arg;
+    Arg = C.getCanonicalTemplateArgument(Arg);
+    AnyNonCanonArgs |= !Arg.structurallyEquals(OrigArg);
+  }
+  return CanonArgs;
+}
+
 //===----------------------------------------------------------------------===//
 //                   Type creation/memoization methods
 //===----------------------------------------------------------------------===//
 
 QualType
 ASTContext::getExtQualType(const Type *baseType, Qualifiers quals) const {
   unsigned fastQuals = quals.getFastQualifiers();
   quals.removeFastQualifiers();
if (!SubstParm) {
     Types.push_back(SubstParm);
     SubstTemplateTypeParmTypes.InsertNode(SubstParm, InsertPos);
   }
 
   return QualType(SubstParm, 0);
 }
 
 /// Retrieve a
-QualType ASTContext::getSubstTemplateTypeParmPackType(
-    Decl *AssociatedDecl, unsigned Index, const TemplateArgument &ArgPack) {
+QualType
+ASTContext::getSubstTemplateTypeParmPackType(Decl *AssociatedDecl,
+                                             unsigned Index, bool Final,
+                                             const TemplateArgument &ArgPack) {
 #ifndef NDEBUG
-  for (const auto &P : ArgPack.pack_elements()) {
+  for (const auto &P : ArgPack.pack_elements())
     assert(P.getKind() == TemplateArgument::Type && "Pack contains a non-type");
-    assert(P.getAsType().isCanonical() && "Pack contains non-canonical type");
-  }
 #endif
 
   llvm::FoldingSetNodeID ID;
-  SubstTemplateTypeParmPackType::Profile(ID, AssociatedDecl, Index, ArgPack);
+  SubstTemplateTypeParmPackType::Profile(ID, AssociatedDecl, Index, Final,
+                                         ArgPack);
   void *InsertPos = nullptr;
   if (SubstTemplateTypeParmPackType *SubstParm =
           SubstTemplateTypeParmPackTypes.FindNodeOrInsertPos(ID, InsertPos))
     return QualType(SubstParm, 0);
 
   QualType Canon;
-  if (!AssociatedDecl->isCanonicalDecl()) {
-    Canon = getSubstTemplateTypeParmPackType(AssociatedDecl->getCanonicalDecl(),
-                                             Index, ArgPack);
+  {
+    TemplateArgument CanonArgPack = getCanonicalTemplateArgument(ArgPack);
+    if (!AssociatedDecl->isCanonicalDecl() ||
+        !CanonArgPack.structurallyEquals(ArgPack)) {
+      Canon = getSubstTemplateTypeParmPackType(
+          AssociatedDecl->getCanonicalDecl(), Index, Final, CanonArgPack);
+      [[maybe_unused]] const auto *Nothing =
           SubstTemplateTypeParmPackTypes.FindNodeOrInsertPos(ID, InsertPos);
+      assert(!Nothing);
+    }
   }
 
-  auto *SubstParm = new (*this, TypeAlignment)
-      SubstTemplateTypeParmPackType(Canon, AssociatedDecl, Index, ArgPack);
+  auto *SubstParm = new (*this, TypeAlignment) SubstTemplateTypeParmPackType(
+      Canon, AssociatedDecl, Index, Final, ArgPack);
   Types.push_back(SubstParm);
   SubstTemplateTypeParmPackTypes.InsertNode(SubstParm, InsertPos);
   return QualType(SubstParm, 0);
 }
 
 /// Retrieve the template type parameter type for a template
 /// parameter or parameter pack with the given depth, index, and (optionally)
 /// name.
ASTContext::getTemplateSpecializationType(TemplateName Template,
   auto *Spec
     = new (Mem) TemplateSpecializationType(Template, Args, CanonType,
                                          IsTypeAlias ? Underlying : QualType());
 
   Types.push_back(Spec);
   return QualType(Spec, 0);
 }
 
-static bool
-getCanonicalTemplateArguments(const ASTContext &C,
-                              ArrayRef<TemplateArgument> OrigArgs,
-                              SmallVectorImpl<TemplateArgument> &CanonArgs) {
-  bool AnyNonCanonArgs = false;
-  unsigned NumArgs = OrigArgs.size();
-  CanonArgs.resize(NumArgs);
-  for (unsigned I = 0; I != NumArgs; ++I) {
-    const TemplateArgument &OrigArg = OrigArgs[I];
-    TemplateArgument &CanonArg = CanonArgs[I];
-    CanonArg = C.getCanonicalTemplateArgument(OrigArg);
-    if (!CanonArg.structurallyEquals(OrigArg))
-      AnyNonCanonArgs = true;
-  }
-  return AnyNonCanonArgs;
-}
-
 QualType ASTContext::getCanonicalTemplateSpecializationType(
     TemplateName Template, ArrayRef<TemplateArgument> Args) const {
   assert(!Template.getAsDependentTemplateName() &&
          "No dependent template names here!");
 
   // Look through qualified template names.
   if (QualifiedTemplateName *QTN = Template.getAsQualifiedTemplateName())
     Template = TemplateName(QTN->getUnderlyingTemplate());
 
   // Build the canonical template specialization type.
   TemplateName CanonTemplate = getCanonicalTemplateName(Template);
-  SmallVector<TemplateArgument, 4> CanonArgs;
-  ::getCanonicalTemplateArguments(*this, Args, CanonArgs);
+  bool AnyNonCanonArgs = false;
+  auto CanonArgs =
+      ::getCanonicalTemplateArguments(*this, Args, AnyNonCanonArgs);
 
   // Determine whether this canonical template specialization type already
   // exists.
   llvm::FoldingSetNodeID ID;
   TemplateSpecializationType::Profile(ID, CanonTemplate,
                                       CanonArgs, *this);
 
   void *InsertPos = nullptr;
ASTContext::getDependentTemplateSpecializationType(
   if (T)
     return QualType(T, 0);
 
   NestedNameSpecifier *CanonNNS = getCanonicalNestedNameSpecifier(NNS);
 
   ElaboratedTypeKeyword CanonKeyword = Keyword;
   if (Keyword == ETK_None) CanonKeyword = ETK_Typename;
 
-  SmallVector<TemplateArgument, 16> CanonArgs;
-  bool AnyNonCanonArgs =
-      ::getCanonicalTemplateArguments(*this, Args, CanonArgs);
+  bool AnyNonCanonArgs = false;
+  auto CanonArgs =
+      ::getCanonicalTemplateArguments(*this, Args, AnyNonCanonArgs);
 
   QualType Canon;
   if (AnyNonCanonArgs || CanonNNS != NNS || CanonKeyword != Keyword) {
     Canon = getDependentTemplateSpecializationType(CanonKeyword, CanonNNS,
                                                    Name,
                                                    CanonArgs);
 
     // Find the insert position again.
ASTContext::getCanonicalTemplateName(const TemplateName &Name) const {
 
   case TemplateName::SubstTemplateTemplateParmPack: {
     SubstTemplateTemplateParmPackStorage *subst =
         Name.getAsSubstTemplateTemplateParmPack();
     TemplateArgument canonArgPack =
         getCanonicalTemplateArgument(subst->getArgumentPack());
     return getSubstTemplateTemplateParmPack(
         canonArgPack, subst->getAssociatedDecl()->getCanonicalDecl(),
-        subst->getIndex());
+        subst->getFinal(), subst->getIndex());
   }
   }
 
   llvm_unreachable("bad template name!");
 }
 
 bool ASTContext::hasSameTemplateName(const TemplateName &X,
                                      const TemplateName &Y) const {
switch (Arg.getKind()) {
 
     case TemplateArgument::Integral:
       return TemplateArgument(Arg, getCanonicalType(Arg.getIntegralType()));
 
     case TemplateArgument::Type:
       return TemplateArgument(getCanonicalType(Arg.getAsType()));
 
     case TemplateArgument::Pack: {
-      if (Arg.pack_size() == 0)
+      bool AnyNonCanonArgs = false;
+      auto CanonArgs = ::getCanonicalTemplateArguments(
+          *this, Arg.pack_elements(), AnyNonCanonArgs);
+      if (!AnyNonCanonArgs)
         return Arg;
-
-      auto *CanonArgs = new (*this) TemplateArgument[Arg.pack_size()];
-      unsigned Idx = 0;
-      for (TemplateArgument::pack_iterator A = Arg.pack_begin(),
-                                        AEnd = Arg.pack_end();
-           A != AEnd; (void)++A, ++Idx)
-        CanonArgs[Idx] = getCanonicalTemplateArgument(*A);
-
-      return TemplateArgument(llvm::makeArrayRef(CanonArgs, Arg.pack_size()));
+      return TemplateArgument::CreatePackCopy(const_cast<ASTContext &>(*this),
+                                              CanonArgs);
     }
   }
 
   // Silence GCC warning
   llvm_unreachable("Unhandled template argument kind");
 }
 
 NestedNameSpecifier *
ASTContext::getSubstTemplateTemplateParm(TemplateName Replacement,
   }
 
   return TemplateName(subst);
 }
 
 TemplateName
 ASTContext::getSubstTemplateTemplateParmPack(const TemplateArgument &ArgPack,
                                              Decl *AssociatedDecl,
-                                             unsigned Index) const {
+                                             unsigned Index, bool Final) const {
   auto &Self = const_cast<ASTContext &>(*this);
   llvm::FoldingSetNodeID ID;
   SubstTemplateTemplateParmPackStorage::Profile(ID, Self, ArgPack,
-                                                AssociatedDecl, Index);
+                                                AssociatedDecl, Index, Final);
 
   void *InsertPos = nullptr;
   SubstTemplateTemplateParmPackStorage *Subst
     = SubstTemplateTemplateParmPacks.FindNodeOrInsertPos(ID, InsertPos);
 
   if (!Subst) {
     Subst = new (*this) SubstTemplateTemplateParmPackStorage(
-        ArgPack.pack_elements(), AssociatedDecl, Index);
+        ArgPack.pack_elements(), AssociatedDecl, Index, Final);
     SubstTemplateTemplateParmPacks.InsertNode(Subst, InsertPos);
   }
 
   return TemplateName(Subst);
 }
 
 /// getFromTargetType - Given one of the integer types provided by
 /// TargetInfo, produce the corresponding type. The unsigned @p Type

clang/lib/AST/ASTImporter.cpp

ExpectedType ASTNodeImporter::VisitSubstTemplateTypeParmPackType(
   if (!ReplacedOrErr)
     return ReplacedOrErr.takeError();
 
   Expected<TemplateArgument> ToArgumentPack = import(T->getArgumentPack());
   if (!ToArgumentPack)
     return ToArgumentPack.takeError();
 
   return Importer.getToContext().getSubstTemplateTypeParmPackType(
-      *ReplacedOrErr, T->getIndex(), *ToArgumentPack);
+      *ReplacedOrErr, T->getIndex(), T->getFinal(), *ToArgumentPack);
 }
 
 ExpectedType ASTNodeImporter::VisitTemplateSpecializationType(
                                        const TemplateSpecializationType *T) {
   auto ToTemplateOrErr = import(T->getTemplateName());
   if (!ToTemplateOrErr)
     return ToTemplateOrErr.takeError();
 
case TemplateName::SubstTemplateTemplateParmPack: {
     if (!ArgPackOrErr)
       return ArgPackOrErr.takeError();
 
     auto AssociatedDeclOrErr = Import(SubstPack->getAssociatedDecl());
     if (!AssociatedDeclOrErr)
       return AssociatedDeclOrErr.takeError();
 
     return ToContext.getSubstTemplateTemplateParmPack(
-        *ArgPackOrErr, *AssociatedDeclOrErr, SubstPack->getIndex());
+        *ArgPackOrErr, *AssociatedDeclOrErr, SubstPack->getIndex(),
+        SubstPack->getFinal());
   }
   case TemplateName::UsingTemplate: {
     auto UsingOrError = Import(From.getAsUsingShadowDecl());
     if (!UsingOrError)
       return UsingOrError.takeError();
     return TemplateName(cast<UsingShadowDecl>(*UsingOrError));
   }
   }

clang/lib/AST/TemplateName.cpp

void SubstTemplateTemplateParmStorage::Profile(llvm::FoldingSetNodeID &ID,
                                                unsigned Index,
                                                Optional<unsigned> PackIndex) {
   Replacement.Profile(ID);
   ID.AddPointer(AssociatedDecl);
   ID.AddInteger(Index);
   ID.AddInteger(PackIndex ? *PackIndex + 1 : 0);
 }
 
+SubstTemplateTemplateParmPackStorage::SubstTemplateTemplateParmPackStorage(
+    ArrayRef<TemplateArgument> ArgPack, Decl *AssociatedDecl, unsigned Index,
+    bool Final)
+    : UncommonTemplateNameStorage(SubstTemplateTemplateParmPack, Index,
+                                  ArgPack.size()),
+      Arguments(ArgPack.data()), AssociatedDeclAndFinal(AssociatedDecl, Final) {
+  assert(AssociatedDecl != nullptr);
+}
+
 void SubstTemplateTemplateParmPackStorage::Profile(llvm::FoldingSetNodeID &ID,
                                                    ASTContext &Context) {
-  Profile(ID, Context, getArgumentPack(), getAssociatedDecl(), getIndex());
+  Profile(ID, Context, getArgumentPack(), getAssociatedDecl(), getIndex(),
+          getFinal());
+}
+
+Decl *SubstTemplateTemplateParmPackStorage::getAssociatedDecl() const {
+  return AssociatedDeclAndFinal.getPointer();
+}
+
+bool SubstTemplateTemplateParmPackStorage::getFinal() const {
+  return AssociatedDeclAndFinal.getInt();
 }
 
 void SubstTemplateTemplateParmPackStorage::Profile(
     llvm::FoldingSetNodeID &ID, ASTContext &Context,
-    const TemplateArgument &ArgPack, Decl *AssociatedDecl, unsigned Index) {
+    const TemplateArgument &ArgPack, Decl *AssociatedDecl, unsigned Index,
+    bool Final) {
   ArgPack.Profile(ID, Context);
   ID.AddPointer(AssociatedDecl);
   ID.AddInteger(Index);
+  ID.AddBoolean(Final);
 }
 
 TemplateName::TemplateName(void *Ptr) {
   Storage = StorageType::getFromOpaqueValue(Ptr);
 }
 
 TemplateName::TemplateName(TemplateDecl *Template) : Storage(Template) {}
 TemplateName::TemplateName(OverloadedTemplateStorage *Storage)

clang/lib/AST/Type.cpp

 }
 
 const TemplateTypeParmDecl *
 SubstTemplateTypeParmType::getReplacedParameter() const {
   return ::getReplacedParameter(getAssociatedDecl(), getIndex());
 }
 
 SubstTemplateTypeParmPackType::SubstTemplateTypeParmPackType(
-    QualType Canon, Decl *AssociatedDecl, unsigned Index,
+    QualType Canon, Decl *AssociatedDecl, unsigned Index, bool Final,
     const TemplateArgument &ArgPack)
     : Type(SubstTemplateTypeParmPack, Canon,
            TypeDependence::DependentInstantiation |
                TypeDependence::UnexpandedPack),
-      Arguments(ArgPack.pack_begin()), AssociatedDecl(AssociatedDecl) {
+      Arguments(ArgPack.pack_begin()),
+      AssociatedDeclAndFinal(AssociatedDecl, Final) {
   SubstTemplateTypeParmPackTypeBits.Index = Index;
   SubstTemplateTypeParmPackTypeBits.NumArgs = ArgPack.pack_size();
   assert(AssociatedDecl != nullptr);
 }
 
+Decl *SubstTemplateTypeParmPackType::getAssociatedDecl() const {
+  return AssociatedDeclAndFinal.getPointer();
+}
+
+bool SubstTemplateTypeParmPackType::getFinal() const {
+  return AssociatedDeclAndFinal.getInt();
+}
+
 const TemplateTypeParmDecl *
 SubstTemplateTypeParmPackType::getReplacedParameter() const {
   return ::getReplacedParameter(getAssociatedDecl(), getIndex());
 }
 
 IdentifierInfo *SubstTemplateTypeParmPackType::getIdentifier() const {
   return getReplacedParameter()->getIdentifier();
 }
 
 TemplateArgument SubstTemplateTypeParmPackType::getArgumentPack() const {
   return TemplateArgument(llvm::makeArrayRef(Arguments, getNumArgs()));
 }
 
 void SubstTemplateTypeParmPackType::Profile(llvm::FoldingSetNodeID &ID) {
-  Profile(ID, getAssociatedDecl(), getIndex(), getArgumentPack());
+  Profile(ID, getAssociatedDecl(), getIndex(), getFinal(), getArgumentPack());
 }
 
 void SubstTemplateTypeParmPackType::Profile(llvm::FoldingSetNodeID &ID,
                                             const Decl *AssociatedDecl,
-                                            unsigned Index,
+                                            unsigned Index, bool Final,
                                             const TemplateArgument &ArgPack) {
   ID.AddPointer(AssociatedDecl);
   ID.AddInteger(Index);
+  ID.AddBoolean(Final);
   ID.AddInteger(ArgPack.pack_size());
   for (const auto &P : ArgPack.pack_elements())
     ID.AddPointer(P.getAsType().getAsOpaquePtr());
 }
 
 bool TemplateSpecializationType::anyDependentTemplateArguments(
     const TemplateArgumentListInfo &Args, ArrayRef<TemplateArgument> Converted) {
   return anyDependentTemplateArguments(Args.arguments(), Converted);

clang/lib/Sema/SemaConcept.cpp

if (Inst.isInvalid())
       return true;
 
     // addInstantiatedParametersToScope creates a map of 'uninstantiated' to
     // 'instantiated' parameters and adds it to the context. For the case where
     // this function is a template being instantiated NOW, we also need to add
     // the list of current template arguments to the list so that they also can
     // be picked out of the map.
     if (auto *SpecArgs = FD->getTemplateSpecializationArgs()) {
-      MultiLevelTemplateArgumentList JustTemplArgs(FD, SpecArgs->asArray());
+      MultiLevelTemplateArgumentList JustTemplArgs(FD, SpecArgs->asArray(),
+                                                   /*Final=*/false);
       if (addInstantiatedParametersToScope(
               FD, PrimaryTemplate->getTemplatedDecl(), Scope, JustTemplArgs))
         return true;
     }
 
     // If this is a member function, make sure we get the parameters that
     // reference the original primary template.
     if (const auto *FromMemTempl =

 Sema::SetupConstraintCheckingTemplateArgumentsAndScope(
     FunctionDecl *FD, llvm::Optional<ArrayRef<TemplateArgument>> TemplateArgs,
     LocalInstantiationScope &Scope) {
   MultiLevelTemplateArgumentList MLTAL;
 
   // Collect the list of template arguments relative to the 'primary' template.
   // We need the entire list, since the constraint is completely uninstantiated
   // at this point.
-  MLTAL = getTemplateInstantiationArgs(FD, /*Innermost=*/nullptr,
-                                       /*RelativeToPrimary=*/true,
-                                       /*Pattern=*/nullptr,
-                                       /*ForConstraintInstantiation=*/true);
+  MLTAL =
+      getTemplateInstantiationArgs(FD, /*Final=*/false, /*Innermost=*/nullptr,
   if (SetupConstraintScope(FD, TemplateArgs, MLTAL, Scope))
     return llvm::None;
 
   return MLTAL;
 }
 
 bool Sema::CheckFunctionConstraints(const FunctionDecl *FD,
                                     ConstraintSatisfaction &Satisfaction,

 
 
 // Figure out the to-translation-unit depth for this function declaration for
 // the purpose of seeing if they differ by constraints. This isn't the same as
 // getTemplateDepth, because it includes already instantiated parents.
 static unsigned CalculateTemplateDepthForConstraints(Sema &S,
                                                      const NamedDecl *ND) {
   MultiLevelTemplateArgumentList MLTAL = S.getTemplateInstantiationArgs(
-      ND, /*Innermost=*/nullptr, /*RelativeToPrimary=*/true,
+      ND, /*Final=*/false, /*Innermost=*/nullptr, /*RelativeToPrimary=*/true,
       /*Pattern=*/nullptr,
       /*ForConstraintInstantiation=*/true);
   return MLTAL.getNumSubstitutedLevels();
 }
 
 namespace {
   class AdjustConstraintDepth : public TreeTransform<AdjustConstraintDepth> {
   unsigned TemplateDepth = 0;
   public:
   using inherited = TreeTransform<AdjustConstraintDepth>;
   AdjustConstraintDepth(Sema &SemaRef, unsigned TemplateDepth)
       : inherited(SemaRef), TemplateDepth(TemplateDepth) {}
+
+  using inherited::TransformTemplateTypeParmType;
   QualType TransformTemplateTypeParmType(TypeLocBuilder &TLB,
-                                         TemplateTypeParmTypeLoc TL) {
+                                         TemplateTypeParmTypeLoc TL, bool) {
     const TemplateTypeParmType *T = TL.getTypePtr();
 
     TemplateTypeParmDecl *NewTTPDecl = nullptr;
     if (TemplateTypeParmDecl *OldTTPDecl = T->getDecl())
       NewTTPDecl = cast_or_null<TemplateTypeParmDecl>(
           TransformDecl(TL.getNameLoc(), OldTTPDecl));
 
     QualType Result = getSema().Context.getTemplateTypeParmType(
substituteParameterMappings(Sema &S, NormalizedConstraint &N,
   Atomic.ParameterMapping.emplace(TempArgs, SubstArgs.size());
   return false;
 }
 
 static bool substituteParameterMappings(Sema &S, NormalizedConstraint &N,
                                         const ConceptSpecializationExpr *CSE) {
   TemplateArgumentList TAL{TemplateArgumentList::OnStack,
                            CSE->getTemplateArguments()};
-  MultiLevelTemplateArgumentList MLTAL =
-      S.getTemplateInstantiationArgs(CSE->getNamedConcept(), &TAL,
-                                     /*RelativeToPrimary=*/true,
-                                     /*Pattern=*/nullptr,
-                                     /*ForConstraintInstantiation=*/true);
+  MultiLevelTemplateArgumentList MLTAL = S.getTemplateInstantiationArgs(
+      CSE->getNamedConcept(), /*Final=*/false, &TAL,
 
   return substituteParameterMappings(S, N, CSE->getNamedConcept(), MLTAL,
                                      CSE->getTemplateArgsAsWritten());
 }
 
 Optional<NormalizedConstraint>
 NormalizedConstraint::fromConstraintExprs(Sema &S, NamedDecl *D,
                                           ArrayRef<const Expr *> E) {

clang/lib/Sema/SemaExprCXX.cpp

else if (ReturnTypeRequirement.isTypeConstraint()) {
     QualType MatchedType =
         Context.getReferenceQualifiedType(E).getCanonicalType();
     llvm::SmallVector<TemplateArgument, 1> Args;
     Args.push_back(TemplateArgument(MatchedType));
 
     auto *Param = cast<TemplateTypeParmDecl>(TPL->getParam(0));
 
     TemplateArgumentList TAL(TemplateArgumentList::OnStack, Args);
-    MultiLevelTemplateArgumentList MLTAL(Param, TAL.asArray());
+    MultiLevelTemplateArgumentList MLTAL(Param, TAL.asArray(),
+                                         /*Final=*/false);
     MLTAL.addOuterRetainedLevels(TPL->getDepth());
     Expr *IDC = Param->getTypeConstraint()->getImmediatelyDeclaredConstraint();
     ExprResult Constraint = SubstExpr(IDC, MLTAL);
     if (Constraint.isInvalid()) {
       Status = concepts::ExprRequirement::SS_ExprSubstitutionFailure;
     } else {
       SubstitutedConstraintExpr =
           cast<ConceptSpecializationExpr>(Constraint.get());

clang/lib/Sema/SemaTemplate.cpp

ConstraintRefersToContainingTemplateChecker(Sema &SemaRef,
                                               unsigned TemplateDepth)
       : inherited(SemaRef), Friend(Friend), TemplateDepth(TemplateDepth) {}
   bool getResult() const { return Result; }
 
   // This should be the only template parm type that we have to deal with.
   // SubstTempalteTypeParmPack, SubstNonTypeTemplateParmPack, and
   // FunctionParmPackExpr are all partially substituted, which cannot happen
   // with concepts at this point in translation.
+  using inherited::TransformTemplateTypeParmType;
   QualType TransformTemplateTypeParmType(TypeLocBuilder &TLB,
-                                         TemplateTypeParmTypeLoc TL) {
+                                         TemplateTypeParmTypeLoc TL, bool) {
     assert(TL.getDecl()->getDepth() <= TemplateDepth &&
            "Nothing should reference a value below the actual template depth, "
            "depth is likely wrong");
     if (TL.getDecl()->getDepth() != TemplateDepth)
       Result = true;
-    return inherited::TransformTemplateTypeParmType(TLB, TL);
+    return inherited::TransformTemplateTypeParmType(
+        TLB, TL,
+        /*SuppressObjCLifetime=*/false);
   }
 
   Decl *TransformDecl(SourceLocation Loc, Decl *D) {
     if (!D)
       return D;
     // FIXME : This is possibly an incomplete list, but it is unclear what other
     // Decl kinds could be used to refer to the template parameters.  This is a
     // best guess so far based on examples currently available, but the
QualType Sema::CheckTemplateIdType(TemplateName Name,
   if (TypeAliasTemplateDecl *AliasTemplate =
           dyn_cast<TypeAliasTemplateDecl>(Template)) {
 
     // Find the canonical type for this type alias template specialization.
     TypeAliasDecl *Pattern = AliasTemplate->getTemplatedDecl();
     if (Pattern->isInvalidDecl())
       return QualType();
 
-    TemplateArgumentList StackTemplateArgs(TemplateArgumentList::OnStack,
-                                           CanonicalConverted);
-
     // Only substitute for the innermost template argument list.
     MultiLevelTemplateArgumentList TemplateArgLists;
-    TemplateArgLists.addOuterTemplateArguments(Template,
-                                               StackTemplateArgs.asArray());
+    TemplateArgLists.addOuterTemplateArguments(Template, CanonicalConverted,
+                                               /*Final=*/false);
     TemplateArgLists.addOuterRetainedLevels(
         AliasTemplate->getTemplateParameters()->getDepth());
 
     LocalInstantiationScope Scope(*this);
     InstantiatingTemplate Inst(*this, TemplateLoc, Template);
     if (Inst.isInvalid())
       return QualType();
 
if (!Decl) {
       if (ClassTemplate->isOutOfLine())
         Decl->setLexicalDeclContext(ClassTemplate->getLexicalDeclContext());
     }
 
     if (Decl->getSpecializationKind() == TSK_Undeclared &&
         ClassTemplate->getTemplatedDecl()->hasAttrs()) {
       InstantiatingTemplate Inst(*this, TemplateLoc, Decl);
       if (!Inst.isInvalid()) {
-        MultiLevelTemplateArgumentList TemplateArgLists;
-        TemplateArgLists.addOuterTemplateArguments(Template,
-                                                   CanonicalConverted);
+        MultiLevelTemplateArgumentList TemplateArgLists(Template,
+                                                        CanonicalConverted,
+                                                        /*Final=*/false);
         InstantiateAttrsForDecl(TemplateArgLists,
                                 ClassTemplate->getTemplatedDecl(), Decl);
       }
     }
 
     // Diagnose uses of this specialization.
     (void)DiagnoseUseOfDecl(Decl, TemplateLoc);
 
Sema::CheckConceptTemplateId(const CXXScopeSpec &SS,
 
   auto *CSD = ImplicitConceptSpecializationDecl::Create(
       Context, NamedConcept->getDeclContext(), NamedConcept->getLocation(),
       CanonicalConverted);
   ConstraintSatisfaction Satisfaction;
   bool AreArgsDependent =
       TemplateSpecializationType::anyDependentTemplateArguments(
           *TemplateArgs, CanonicalConverted);
-  MultiLevelTemplateArgumentList MLTAL(NamedConcept, CanonicalConverted);
+  MultiLevelTemplateArgumentList MLTAL(NamedConcept, CanonicalConverted,
+                                       /*Final=*/false);
   LocalInstantiationScope Scope(*this);
 
   EnterExpressionEvaluationContext EECtx{
       *this, ExpressionEvaluationContext::ConstantEvaluated, CSD};
 
   if (!AreArgsDependent &&
       CheckConstraintSatisfaction(
           NamedConcept, {NamedConcept->getConstraintExpr()}, MLTAL,
SubstDefaultTemplateArgument(Sema &SemaRef,
   // on the previously-computed template arguments.
   if (ArgType->getType()->isInstantiationDependentType()) {
     Sema::InstantiatingTemplate Inst(SemaRef, TemplateLoc,
                                      Param, Template, Converted,
                                      SourceRange(TemplateLoc, RAngleLoc));
     if (Inst.isInvalid())
       return nullptr;
 
-    TemplateArgumentList TemplateArgs(TemplateArgumentList::OnStack, Converted);
-
     // Only substitute for the innermost template argument list.
-    MultiLevelTemplateArgumentList TemplateArgLists(Template,
-                                                    TemplateArgs.asArray());
+    MultiLevelTemplateArgumentList TemplateArgLists(Template, Converted,
+                                                    /*Final=*/false);
     for (unsigned i = 0, e = Param->getDepth(); i != e; ++i)
       TemplateArgLists.addOuterTemplateArguments(None);
 
     bool ForLambdaCallOperator = false;
     if (const auto *Rec = dyn_cast<CXXRecordDecl>(Template->getDeclContext()))
       ForLambdaCallOperator = Rec->isLambda();
     Sema::ContextRAII SavedContext(SemaRef, Template->getDeclContext(),
                                    !ForLambdaCallOperator);
SubstDefaultTemplateArgument(Sema &SemaRef,
                              NonTypeTemplateParmDecl *Param,
                         SmallVectorImpl<TemplateArgument> &Converted) {
   Sema::InstantiatingTemplate Inst(SemaRef, TemplateLoc,
                                    Param, Template, Converted,
                                    SourceRange(TemplateLoc, RAngleLoc));
   if (Inst.isInvalid())
     return ExprError();
 
-  TemplateArgumentList TemplateArgs(TemplateArgumentList::OnStack, Converted);
-
   // Only substitute for the innermost template argument list.
-  MultiLevelTemplateArgumentList TemplateArgLists(Template,
-                                                  TemplateArgs.asArray());
+  MultiLevelTemplateArgumentList TemplateArgLists(Template, Converted,
+                                                  /*Final=*/false);
   for (unsigned i = 0, e = Param->getDepth(); i != e; ++i)
     TemplateArgLists.addOuterTemplateArguments(None);
 
   Sema::ContextRAII SavedContext(SemaRef, Template->getDeclContext());
   EnterExpressionEvaluationContext ConstantEvaluated(
       SemaRef, Sema::ExpressionEvaluationContext::ConstantEvaluated);
   return SemaRef.SubstExpr(Param->getDefaultArgument(), TemplateArgLists);
 }
SubstDefaultTemplateArgument(Sema &SemaRef,
                        SmallVectorImpl<TemplateArgument> &Converted,
                              NestedNameSpecifierLoc &QualifierLoc) {
   Sema::InstantiatingTemplate Inst(
       SemaRef, TemplateLoc, TemplateParameter(Param), Template, Converted,
       SourceRange(TemplateLoc, RAngleLoc));
   if (Inst.isInvalid())
     return TemplateName();
 
-  TemplateArgumentList TemplateArgs(TemplateArgumentList::OnStack, Converted);
-
   // Only substitute for the innermost template argument list.
-  MultiLevelTemplateArgumentList TemplateArgLists(Template,
-                                                  TemplateArgs.asArray());
+  MultiLevelTemplateArgumentList TemplateArgLists(Template, Converted,
+                                                  /*Final=*/false);
   for (unsigned i = 0, e = Param->getDepth(); i != e; ++i)
     TemplateArgLists.addOuterTemplateArguments(None);
 
   Sema::ContextRAII SavedContext(SemaRef, Template->getDeclContext());
   // Substitute into the nested-name-specifier first,
   QualifierLoc = Param->getDefaultArgument().getTemplateQualifierLoc();
   if (QualifierLoc) {
     QualifierLoc =
if (NTTPType->isInstantiationDependentType() &&
         !Template->getDeclContext()->isDependentContext()) {
       // Do substitution on the type of the non-type template parameter.
       InstantiatingTemplate Inst(*this, TemplateLoc, Template, NTTP,
                                  CanonicalConverted,
                                  SourceRange(TemplateLoc, RAngleLoc));
       if (Inst.isInvalid())
         return true;
 
-      TemplateArgumentList TemplateArgs(TemplateArgumentList::OnStack,
-                                        CanonicalConverted);
-
-      MultiLevelTemplateArgumentList MLTAL(Template, TemplateArgs.asArray());
+      MultiLevelTemplateArgumentList MLTAL(Template, CanonicalConverted,
+                                           /*Final=*/false);
       // If the parameter is a pack expansion, expand this slice of the pack.
       if (auto *PET = NTTPType->getAs<PackExpansionType>()) {
         Sema::ArgumentPackSubstitutionIndexRAII SubstIndex(*this,
                                                            ArgumentPackIndex);
         NTTPType = SubstType(PET->getPattern(), MLTAL, NTTP->getLocation(),
                              NTTP->getDeclName());
       } else {
         NTTPType = SubstType(NTTPType, MLTAL, NTTP->getLocation(),
// FIXME: Skip this if the parameters aren't instantiation-dependent.
     // Set up a template instantiation context.
     LocalInstantiationScope Scope(*this);
     InstantiatingTemplate Inst(*this, TemplateLoc, Template, TempParm,
                                SugaredConverted,
                                SourceRange(TemplateLoc, RAngleLoc));
     if (Inst.isInvalid())
       return true;
 
-    Params = SubstTemplateParams(
-        Params, CurContext,
-        MultiLevelTemplateArgumentList(Template, CanonicalConverted));
+    Params =
+        SubstTemplateParams(Params, CurContext,
+                            MultiLevelTemplateArgumentList(
+                                Template, CanonicalConverted, /*Final=*/false));
     if (!Params)
       return true;
   }
 
   // C++1z [temp.local]p1: (DR1004)
   //   When [the injected-class-name] is used [...] as a template-argument for
   //   a template template-parameter [...] it refers to the class template
   //   itself.
if (!PartialTemplateArgs) {
     if (const auto *Method =
             dyn_cast_or_null<CXXMethodDecl>(Template->getTemplatedDecl()))
       ThisQuals = Method->getMethodQualifiers();
 
     ContextRAII Context(*this, NewContext);
     CXXThisScopeRAII(*this, RD, ThisQuals, RD != nullptr);
 
     MultiLevelTemplateArgumentList MLTAL = getTemplateInstantiationArgs(
-        Template, &StackTemplateArgs, /*RelativeToPrimary=*/true,
+        Template, /*Final=*/false, &StackTemplateArgs,
+        /*RelativeToPrimary=*/true,
         /*Pattern=*/nullptr,
         /*ForConceptInstantiation=*/true);
     if (EnsureTemplateArgumentListConstraints(
             Template, MLTAL,
             SourceRange(TemplateLoc, TemplateArgs.getRAngleLoc()))) {
       if (ConstraintsNotSatisfied)
         *ConstraintsNotSatisfied = true;
       return true;

clang/lib/Sema/SemaTemplateDeduction.cpp

for (const auto &P : Arg.pack_elements()) {
       SugaredPackedArgsBuilder.push_back(SugaredOutput.pop_back_val());
       CanonicalPackedArgsBuilder.push_back(CanonicalOutput.pop_back_val());
     }
 
     // If the pack is empty, we still need to substitute into the parameter
     // itself, in case that substitution fails.
     if (SugaredPackedArgsBuilder.empty()) {
       LocalInstantiationScope Scope(S);
-      TemplateArgumentList TemplateArgs(TemplateArgumentList::OnStack,
-                                        CanonicalOutput);
-      MultiLevelTemplateArgumentList Args(Template, TemplateArgs.asArray());
+      MultiLevelTemplateArgumentList Args(Template, CanonicalOutput,
+                                          /*Final=*/false);
 
       if (auto *NTTP = dyn_cast<NonTypeTemplateParmDecl>(Param)) {
         Sema::InstantiatingTemplate Inst(S, Template->getLocation(), Template,
                                          NTTP, CanonicalOutput,
                                          Template->getSourceRange());
         if (Inst.isInvalid() ||
             S.SubstType(NTTP->getType(), Args, NTTP->getLocation(),
                         NTTP->getDeclName()).isNull())
CheckDeducedArgumentConstraints(Sema &S, TemplateDeclT *Template,
   llvm::SmallVector<const Expr *, 3> AssociatedConstraints;
   Template->getAssociatedConstraints(AssociatedConstraints);
 
   bool NeedsReplacement = DeducedArgsNeedReplacement(Template);
   TemplateArgumentList DeducedTAL{TemplateArgumentList::OnStack,
                                   CanonicalDeducedArgs};
 
   MultiLevelTemplateArgumentList MLTAL = S.getTemplateInstantiationArgs(
-      Template, /*InnerMost=*/NeedsReplacement ? nullptr : &DeducedTAL,
+      Template, /*Final=*/false,
+      /*InnerMost=*/NeedsReplacement ? nullptr : &DeducedTAL,
       /*RelativeToPrimary=*/true, /*Pattern=*/
       nullptr, /*ForConstraintInstantiation=*/true);
 
   // getTemplateInstantiationArgs picks up the non-deduced version of the
   // template args when this is a variable template partial specialization and
   // not class-scope explicit specialization, so replace with Deduced Args
   // instead of adding to inner-most.
   if (NeedsReplacement)
FinishTemplateArgumentDeduction(
   LocalInstantiationScope InstScope(S);
   auto *Template = Partial->getSpecializedTemplate();
   const ASTTemplateArgumentListInfo *PartialTemplArgInfo =
       Partial->getTemplateArgsAsWritten();
 
   TemplateArgumentListInfo InstArgs(PartialTemplArgInfo->LAngleLoc,
                                     PartialTemplArgInfo->RAngleLoc);
 
-  if (S.SubstTemplateArguments(
-          PartialTemplArgInfo->arguments(),
+  if (S.SubstTemplateArguments(PartialTemplArgInfo->arguments(),
                                MultiLevelTemplateArgumentList(
-              Partial, CanonicalDeducedArgumentList->asArray()),
+                                   Partial,
+                                   CanonicalDeducedArgumentList->asArray(),
+                                   /*Final=*/false),
                                InstArgs)) {
     unsigned ArgIdx = InstArgs.size(), ParamIdx = ArgIdx;
     if (ParamIdx >= Partial->getTemplateParameters()->size())
       ParamIdx = Partial->getTemplateParameters()->size() - 1;
 
     Decl *Param = const_cast<NamedDecl *>(
         Partial->getTemplateParameters()->getParam(ParamIdx));
     Info.Param = makeTemplateParameter(Param);
     Info.FirstArg = (*PartialTemplArgInfo)[ArgIdx].getArgument();
const FunctionProtoType *Proto
     = Function->getType()->getAs<FunctionProtoType>();
   assert(Proto && "Function template does not have a prototype?");
 
   // Isolate our substituted parameters from our caller.
   LocalInstantiationScope InstScope(*this, /*MergeWithOuterScope*/true);
 
   ExtParameterInfoBuilder ExtParamInfos;
 
+  MultiLevelTemplateArgumentList MLTAL(FunctionTemplate,
+                                       CanonicalExplicitArgumentList->asArray(),
+                                       /*Final=*/false);
+
   // Instantiate the types of each of the function parameters given the
   // explicitly-specified template arguments. If the function has a trailing
   // return type, substitute it after the arguments to ensure we substitute
   // in lexical order.
   if (Proto->hasTrailingReturn()) {
-    if (SubstParmTypes(
-            Function->getLocation(), Function->parameters(),
-            Proto->getExtParameterInfosOrNull(),
-            MultiLevelTemplateArgumentList(
-                FunctionTemplate, CanonicalExplicitArgumentList->asArray()),
-            ParamTypes, /*params=*/nullptr, ExtParamInfos))
+    if (SubstParmTypes(Function->getLocation(), Function->parameters(),
+                       Proto->getExtParameterInfosOrNull(), MLTAL, ParamTypes,
+                       /*params=*/nullptr, ExtParamInfos))
       return TDK_SubstitutionFailure;
   }
 
   // Instantiate the return type.
   QualType ResultType;
   {
     // C++11 [expr.prim.general]p3:
     //   If a declaration declares a member function or member function
     //   template of a class X, the expression this is a prvalue of type
     //   "pointer to cv-qualifier-seq X" between the optional cv-qualifer-seq
     //   and the end of the function-definition, member-declarator, or
     //   declarator.
     Qualifiers ThisTypeQuals;
     CXXRecordDecl *ThisContext = nullptr;
     if (CXXMethodDecl *Method = dyn_cast<CXXMethodDecl>(Function)) {
       ThisContext = Method->getParent();
       ThisTypeQuals = Method->getMethodQualifiers();
     }
 
     CXXThisScopeRAII ThisScope(*this, ThisContext, ThisTypeQuals,
                                getLangOpts().CPlusPlus11);
 
-    ResultType = SubstType(
-        Proto->getReturnType(),
-        MultiLevelTemplateArgumentList(
-            FunctionTemplate, CanonicalExplicitArgumentList->asArray()),
+    ResultType =
+        SubstType(Proto->getReturnType(), MLTAL,
                   Function->getTypeSpecStartLoc(), Function->getDeclName());
     if (ResultType.isNull() || Trap.hasErrorOccurred())
       return TDK_SubstitutionFailure;
     // CUDA: Kernel function must have 'void' return type.
     if (getLangOpts().CUDA)
       if (Function->hasAttr<CUDAGlobalAttr>() && !ResultType->isVoidType()) {
         Diag(Function->getLocation(), diag::err_kern_type_not_void_return)
             << Function->getType() << Function->getSourceRange();
         return TDK_SubstitutionFailure;
       }
   }
 
   // Instantiate the types of each of the function parameters given the
   // explicitly-specified template arguments if we didn't do so earlier.
   if (!Proto->hasTrailingReturn() &&
-      SubstParmTypes(
-          Function->getLocation(), Function->parameters(),
-          Proto->getExtParameterInfosOrNull(),
-          MultiLevelTemplateArgumentList(
-              FunctionTemplate, CanonicalExplicitArgumentList->asArray()),
-          ParamTypes, /*params*/ nullptr, ExtParamInfos))
+      SubstParmTypes(Function->getLocation(), Function->parameters(),
+                     Proto->getExtParameterInfosOrNull(), MLTAL, ParamTypes,
+                     /*params*/ nullptr, ExtParamInfos))
     return TDK_SubstitutionFailure;
 
   if (FunctionType) {
     auto EPI = Proto->getExtProtoInfo();
     EPI.ExtParameterInfos = ExtParamInfos.getPointerOrNull(ParamTypes.size());
 
     // In C++1z onwards, exception specifications are part of the function type,
     // so substitution into the type must also substitute into the exception
     // specification.
     SmallVector<QualType, 4> ExceptionStorage;
     if (getLangOpts().CPlusPlus17 &&
-        SubstExceptionSpec(
-            Function->getLocation(), EPI.ExceptionSpec, ExceptionStorage,
-            MultiLevelTemplateArgumentList(
-                FunctionTemplate, CanonicalExplicitArgumentList->asArray())))
+        SubstExceptionSpec(Function->getLocation(), EPI.ExceptionSpec,
+                           ExceptionStorage, MLTAL))
       return TDK_SubstitutionFailure;
 
     *FunctionType = BuildFunctionType(ResultType, ParamTypes,
                                       Function->getLocation(),
                                       Function->getDeclName(),
                                       EPI);
     if (FunctionType->isNull() || Trap.hasErrorOccurred())
       return TDK_SubstitutionFailure;
Sema::TemplateDeductionResult Sema::FinishTemplateArgumentDeduction(
   Info.reset(SugaredDeducedArgumentList, CanonicalDeducedArgumentList);
 
   // Substitute the deduced template arguments into the function template
   // declaration to produce the function template specialization.
   DeclContext *Owner = FunctionTemplate->getDeclContext();
   if (FunctionTemplate->getFriendObjectKind())
     Owner = FunctionTemplate->getLexicalDeclContext();
   MultiLevelTemplateArgumentList SubstArgs(
-      FunctionTemplate, CanonicalDeducedArgumentList->asArray());
+      FunctionTemplate, CanonicalDeducedArgumentList->asArray(),
+      /*Final=*/false);
   Specialization = cast_or_null<FunctionDecl>(
       SubstDecl(FunctionTemplate->getTemplatedDecl(), Owner, SubstArgs));
   if (!Specialization || Specialization->isInvalidDecl())
     return TDK_SubstitutionFailure;
 
   assert(Specialization->getPrimaryTemplate()->getCanonicalDecl() ==
          FunctionTemplate->getCanonicalDecl());
 
static bool CheckDeducedPlaceholderConstraints(Sema &S, const AutoType &Type,
   for (unsigned I = 0, C = TypeLoc.getNumArgs(); I != C; ++I)
     TemplateArgs.addArgument(TypeLoc.getArgLoc(I));
 
   llvm::SmallVector<TemplateArgument, 4> SugaredConverted, CanonicalConverted;
   if (S.CheckTemplateArgumentList(Concept, SourceLocation(), TemplateArgs,
                                   /*PartialTemplateArgs=*/false,
                                   SugaredConverted, CanonicalConverted))
     return true;
-  MultiLevelTemplateArgumentList MLTAL(Concept, CanonicalConverted);
+  MultiLevelTemplateArgumentList MLTAL(Concept, CanonicalConverted,
+                                       /*Final=*/false);
   if (S.CheckConstraintSatisfaction(Concept, {Concept->getConstraintExpr()},
                                     MLTAL, TypeLoc.getLocalSourceRange(),
                                     Satisfaction))
     return true;
   if (!Satisfaction.IsSatisfied) {
     std::string Buf;
     llvm::raw_string_ostream OS(Buf);
     OS << "'" << Concept->getName();

clang/lib/Sema/SemaTemplateInstantiate.cpp

HandleVarTemplateSpec(const VarTemplateSpecializationDecl *VarTemplSpec,
   // If this variable template specialization was instantiated from a
   // specialized member that is a variable template, we're done.
   assert(VarTemplSpec->getSpecializedTemplate() && "No variable template?");
   llvm::PointerUnion<VarTemplateDecl *, VarTemplatePartialSpecializationDecl *>
       Specialized = VarTemplSpec->getSpecializedTemplateOrPartial();
   if (VarTemplatePartialSpecializationDecl *Partial =
           Specialized.dyn_cast<VarTemplatePartialSpecializationDecl *>()) {
     Result.addOuterTemplateArguments(
-        Partial, VarTemplSpec->getTemplateInstantiationArgs().asArray());
+        Partial, VarTemplSpec->getTemplateInstantiationArgs().asArray(),
+        /*Final=*/false);
     if (Partial->isMemberSpecialization())
       return Response::Done();
   } else {
     VarTemplateDecl *Tmpl = Specialized.get<VarTemplateDecl *>();
     Result.addOuterTemplateArguments(
-        Tmpl, VarTemplSpec->getTemplateInstantiationArgs().asArray());
+        Tmpl, VarTemplSpec->getTemplateInstantiationArgs().asArray(),
+        /*Final=*/false);
     if (Tmpl->isMemberSpecialization())
       return Response::Done();
   }
   return Response::DontClearRelativeToPrimaryNextDecl(VarTemplSpec);
 }
 
 // If we have a template template parameter with translation unit context,
 // then we're performing substitution into a default template argument of
HandleClassTemplateSpec(const ClassTemplateSpecializationDecl *ClassTemplSpec,
   if (!ClassTemplSpec->isClassScopeExplicitSpecialization()) {
     // We're done when we hit an explicit specialization.
     if (ClassTemplSpec->getSpecializationKind() == TSK_ExplicitSpecialization &&
         !isa<ClassTemplatePartialSpecializationDecl>(ClassTemplSpec))
       return Response::Done();
 
     Result.addOuterTemplateArguments(
         const_cast<ClassTemplateSpecializationDecl *>(ClassTemplSpec),
-        ClassTemplSpec->getTemplateInstantiationArgs().asArray());
+        ClassTemplSpec->getTemplateInstantiationArgs().asArray(),
+        /*Final=*/false);
 
     // If this class template specialization was instantiated from a
     // specialized member that is a class template, we're done.
     assert(ClassTemplSpec->getSpecializedTemplate() && "No class template?");
     if (ClassTemplSpec->getSpecializedTemplate()->isMemberSpecialization())
       return Response::Done();
   }
   return Response::UseNextDecl(ClassTemplSpec);
if (!RelativeToPrimary &&
     // This is an implicit instantiation of an explicit specialization. We
     // don't get any template arguments from this function but might get
     // some from an enclosing template.
     return Response::UseNextDecl(Function);
   } else if (const TemplateArgumentList *TemplateArgs =
                  Function->getTemplateSpecializationArgs()) {
     // Add the template arguments for this specialization.
     Result.addOuterTemplateArguments(const_cast<FunctionDecl *>(Function),
-                                     TemplateArgs->asArray());
+                                     TemplateArgs->asArray(),
+                                     /*Final=*/false);
 
     // If this function was instantiated from a specialized member that is
     // a function template, we're done.
     assert(Function->getPrimaryTemplate() && "No function template?");
     if (Function->getPrimaryTemplate()->isMemberSpecialization())
       return Response::Done();
 
     // If this function is a generic lambda specialization, we are done.
if (ClassTemplateDecl *ClassTemplate = Rec->getDescribedClassTemplate()) {
     if (ClassTemplate->isMemberSpecialization())
       return Response::Done();
     if (ForConstraintInstantiation) {
       QualType RecordType = Context.getTypeDeclType(Rec);
       QualType Injected = cast<InjectedClassNameType>(RecordType)
                               ->getInjectedSpecializationType();
       const auto *InjectedType = cast<TemplateSpecializationType>(Injected);
       Result.addOuterTemplateArguments(const_cast<CXXRecordDecl *>(Rec),
-                                       InjectedType->template_arguments());
+                                       InjectedType->template_arguments(),
+                                       /*Final=*/false);
     }
   }
 
   bool IsFriend = Rec->getFriendObjectKind() ||
                   (Rec->getDescribedClassTemplate() &&
                    Rec->getDescribedClassTemplate()->getFriendObjectKind());
   if (ForConstraintInstantiation && IsFriend &&
       Rec->getNonTransparentDeclContext()->isFileContext()) {
Response HandleRecordDecl(const CXXRecordDecl *Rec,
   return Response::UseNextDecl(Rec);
 }
 
 Response HandleImplicitConceptSpecializationDecl(
     const ImplicitConceptSpecializationDecl *CSD,
     MultiLevelTemplateArgumentList &Result) {
   Result.addOuterTemplateArguments(
       const_cast<ImplicitConceptSpecializationDecl *>(CSD),
-      CSD->getTemplateArguments());
+      CSD->getTemplateArguments(),
+      /*Final=*/false);
   return Response::UseNextDecl(CSD);
 }
 
 Response HandleGenericDeclContext(const Decl *CurDecl) {
   return Response::UseNextDecl(CurDecl);
 }
 } // namespace TemplateInstArgsHelpers
 } // namespace

 /// friend function template specializations.
 ///
 /// \param ForConstraintInstantiation when collecting arguments,
 /// ForConstraintInstantiation indicates we should continue looking when
 /// encountering a lambda generic call operator, and continue looking for
 /// arguments on an enclosing class template.
 
 MultiLevelTemplateArgumentList Sema::getTemplateInstantiationArgs(
-    const NamedDecl *ND, const TemplateArgumentList *Innermost,
+    const NamedDecl *ND, bool Sugared, const TemplateArgumentList *Innermost,
     bool RelativeToPrimary, const FunctionDecl *Pattern,
     bool ForConstraintInstantiation) {
   assert(ND && "Can't find arguments for a decl if one isn't provided");
   // Accumulate the set of template argument lists in this structure.
   MultiLevelTemplateArgumentList Result;
 
   if (Innermost)
     Result.addOuterTemplateArguments(const_cast<NamedDecl *>(ND),
-                                     Innermost->asArray());
+                                     Innermost->asArray(), Sugared);
 
   const Decl *CurDecl = ND;
 
   while (!CurDecl->isFileContextDecl()) {
     using namespace TemplateInstArgsHelpers;
     Response R;
     if (const auto *VarTemplSpec =
             dyn_cast<VarTemplateSpecializationDecl>(CurDecl)) {
void setBase(SourceLocation Loc, DeclarationName Entity) {
       this->Loc = Loc;
       this->Entity = Entity;
     }
 
     unsigned TransformTemplateDepth(unsigned Depth) {
       return TemplateArgs.getNewDepth(Depth);
     }
 
+    Optional<unsigned> getPackIndex(TemplateArgument Pack) {
+      int Index = getSema().ArgumentPackSubstitutionIndex;
+      if (Index == -1)
+        return None;
+      return Pack.pack_size() - 1 - Index;
+    }
+
     bool TryExpandParameterPacks(SourceLocation EllipsisLoc,
                                  SourceRange PatternRange,
                                  ArrayRef<UnexpandedParameterPack> Unexpanded,
                                  bool &ShouldExpand, bool &RetainExpansion,
                                  Optional<unsigned> &NumExpansions) {
       return getSema().CheckParameterPacksForExpansion(EllipsisLoc,
                                                        PatternRange, Unexpanded,
                                                        TemplateArgs,
QualType TransformFunctionProtoType(TypeLocBuilder &TLB,
                                         Qualifiers ThisTypeQuals,
                                         Fn TransformExceptionSpec);
 
     ParmVarDecl *TransformFunctionTypeParam(ParmVarDecl *OldParm,
                                             int indexAdjustment,
                                             Optional<unsigned> NumExpansions,
                                             bool ExpectParameterPack);
 
+    using inherited::TransformTemplateTypeParmType;
     /// Transforms a template type parameter type by performing
     /// substitution of the corresponding template type argument.
     QualType TransformTemplateTypeParmType(TypeLocBuilder &TLB,
-                                           TemplateTypeParmTypeLoc TL);
+                                           TemplateTypeParmTypeLoc TL,
+                                           bool SuppressObjCLifetime);
+
+    QualType BuildSubstTemplateTypeParmType(
+        TypeLocBuilder &TLB, bool SuppressObjCLifetime, bool Sugared,
+        Decl *AssociatedDecl, unsigned Index, Optional<unsigned> PackIndex,
+        TemplateArgument Arg, SourceLocation NameLoc);
 
     /// Transforms an already-substituted template type parameter pack
     /// into either itself (if we aren't substituting into its pack expansion)
     /// or the appropriate substituted argument.
-    QualType TransformSubstTemplateTypeParmPackType(TypeLocBuilder &TLB,
-                                           SubstTemplateTypeParmPackTypeLoc TL);
+    using inherited::TransformSubstTemplateTypeParmPackType;
+    QualType
+    TransformSubstTemplateTypeParmPackType(TypeLocBuilder &TLB,
+                                           SubstTemplateTypeParmPackTypeLoc TL,
+                                           bool SuppressObjCLifetime);
 
     ExprResult TransformLambdaExpr(LambdaExpr *E) {
       LocalInstantiationScope Scope(SemaRef, /*CombineWithOuterScope=*/true);
       Sema::ConstraintEvalRAII<TemplateInstantiator> RAII(*this);
       ExprResult Result = inherited::TransformLambdaExpr(E);
       if (Result.isInvalid())
         return Result;
 
if (TTP->getDepth() < TemplateArgs.getNumLevels()) {
                  "unexpected pack arguments in template rewrite");
           Arg = Arg.pack_begin()->getPackExpansionPattern();
         }
         assert(Arg.getKind() == TemplateArgument::Template &&
                "unexpected nontype template argument kind in template rewrite");
         return Arg.getAsTemplate();
       }
 
-      Decl *AssociatedDecl = TemplateArgs.getAssociatedDecl(TTP->getDepth());
+      auto [AssociatedDecl, Final] =
+          TemplateArgs.getAssociatedDecl(TTP->getDepth());
       Optional<unsigned> PackIndex;
       if (TTP->isParameterPack()) {
         assert(Arg.getKind() == TemplateArgument::Pack &&
                "Missing argument pack");
 
         if (getSema().ArgumentPackSubstitutionIndex == -1) {
           // We have the template argument pack to substitute, but we're not
           // actually expanding the enclosing pack expansion yet. So, just
           // keep the entire argument pack.
           return getSema().Context.getSubstTemplateTemplateParmPack(
-              Arg, AssociatedDecl, TTP->getIndex());
+              Arg, AssociatedDecl, TTP->getIndex(), Final);
         }
 
+        PackIndex = getPackIndex(Arg);
         Arg = getPackSubstitutedTemplateArgument(getSema(), Arg);
-        PackIndex = getSema().ArgumentPackSubstitutionIndex;
       }
 
       TemplateName Template = Arg.getAsTemplate().getNameToSubstitute();
       assert(!Template.isNull() && "Null template template argument");
       assert(!Template.getAsQualifiedTemplateName() &&
              "template decl to substitute is qualified?");
 
-      Template = getSema().Context.getSubstTemplateTemplateParm(
-          Template, AssociatedDecl, TTP->getIndex(), PackIndex);
+      if (Final)
         return Template;
+      return getSema().Context.getSubstTemplateTemplateParm(
+          Template, AssociatedDecl, TTP->getIndex(), PackIndex);
     }
   }
 
   if (SubstTemplateTemplateParmPackStorage *SubstPack
       = Name.getAsSubstTemplateTemplateParmPack()) {
     if (getSema().ArgumentPackSubstitutionIndex == -1)
       return Name;
 
-    TemplateArgument Arg = SubstPack->getArgumentPack();
-    Arg = getPackSubstitutedTemplateArgument(getSema(), Arg);
-    return Arg.getAsTemplate().getNameToSubstitute();
+    TemplateArgument Pack = SubstPack->getArgumentPack();
+    TemplateName Template =
+        getPackSubstitutedTemplateArgument(getSema(), Pack).getAsTemplate();
+    if (SubstPack->getFinal())
+      return Template;
+    return getSema().Context.getSubstTemplateTemplateParm(
+        Template.getNameToSubstitute(), SubstPack->getAssociatedDecl(),
+        SubstPack->getIndex(), getPackIndex(Pack));
   }
 
   return inherited::TransformTemplateName(SS, Name, NameLoc, ObjectType,
                                           FirstQualifierInScope,
                                           AllowInjectedClassName);
 }
 
 ExprResult
if (TemplateArgs.isRewrite()) {
     }
     assert(Arg.getKind() == TemplateArgument::Expression &&
            "unexpected nontype template argument kind in template rewrite");
     // FIXME: This can lead to the same subexpression appearing multiple times
     // in a complete expression.
     return Arg.getAsExpr();
   }
 
-  Decl *AssociatedDecl = TemplateArgs.getAssociatedDecl(NTTP->getDepth());
+  auto [AssociatedDecl, _] = TemplateArgs.getAssociatedDecl(NTTP->getDepth());
   Optional<unsigned> PackIndex;
   if (NTTP->isParameterPack()) {
     assert(Arg.getKind() == TemplateArgument::Pack &&
            "Missing argument pack");
 
     if (getSema().ArgumentPackSubstitutionIndex == -1) {
       // We have an argument pack, but we can't select a particular argument
       // out of it yet. Therefore, we'll build an expression to hold on to that
       // argument pack.
       QualType TargetType = SemaRef.SubstType(NTTP->getType(), TemplateArgs,
                                               E->getLocation(),
                                               NTTP->getDeclName());
       if (TargetType.isNull())
         return ExprError();
 
       QualType ExprType = TargetType.getNonLValueExprType(SemaRef.Context);
       if (TargetType->isRecordType())
         ExprType.addConst();
-
+      // FIXME: Pass in Final.
       return new (SemaRef.Context) SubstNonTypeTemplateParmPackExpr(
           ExprType, TargetType->isReferenceType() ? VK_LValue : VK_PRValue,
           E->getLocation(), Arg, AssociatedDecl, NTTP->getPosition());
     }
-    PackIndex = getSema().ArgumentPackSubstitutionIndex;
+    PackIndex = getPackIndex(Arg);
     Arg = getPackSubstitutedTemplateArgument(getSema(), Arg);
   }
-
+  // FIXME: Don't put subst node on Final replacement.
   return transformNonTypeTemplateParmRef(AssociatedDecl, NTTP, E->getLocation(),
                                          Arg, PackIndex);
 }
 
 const LoopHintAttr *
 TemplateInstantiator::TransformLoopHintAttr(const LoopHintAttr *LH) {
   Expr *TransformedExpr = getDerived().TransformExpr(LH->getValue()).get();
 
assert(result.isInvalid() ||
            SemaRef.Context.hasSameType(result.get()->getType(),
                                        arg.getIntegralType()));
   }
 
   if (result.isInvalid())
     return ExprError();
 
   Expr *resultExpr = result.get();
+  // FIXME: Don't put subst node on sugared replacement.
   return new (SemaRef.Context) SubstNonTypeTemplateParmExpr(
       resultExpr->getType(), resultExpr->getValueKind(), loc, resultExpr,
       AssociatedDecl, parm->getIndex(), PackIndex, refParam);
 }
 
 ExprResult
 TemplateInstantiator::TransformSubstNonTypeTemplateParmPackExpr(
                                           SubstNonTypeTemplateParmPackExpr *E) {
-  int PackIndex = getSema().ArgumentPackSubstitutionIndex;
-  if (PackIndex == -1) {
+  if (getSema().ArgumentPackSubstitutionIndex == -1) {
     // We aren't expanding the parameter pack, so just return ourselves.
     return E;
   }
 
-  TemplateArgument Arg = E->getArgumentPack();
-  Arg = getPackSubstitutedTemplateArgument(getSema(), Arg);
+  TemplateArgument Pack = E->getArgumentPack();
+  TemplateArgument Arg = getPackSubstitutedTemplateArgument(getSema(), Pack);
+  // FIXME: Don't put subst node on sugared replacement.
   return transformNonTypeTemplateParmRef(
       E->getAssociatedDecl(), E->getParameterPack(),
-      E->getParameterPackLocation(), Arg, PackIndex);
+      E->getParameterPackLocation(), Arg, getPackIndex(Pack));
 }
 
 ExprResult
 TemplateInstantiator::TransformSubstNonTypeTemplateParmExpr(
                                           SubstNonTypeTemplateParmExpr *E) {
   ExprResult SubstReplacement = E->getReplacement();
   if (!isa<ConstantExpr>(SubstReplacement.get()))
     SubstReplacement = TransformExpr(E->getReplacement());
TemplateInstantiator::TransformSubstNonTypeTemplateParmExpr(
   if (SemaRef
           .CheckTemplateArgument(E->getParameter(), SubstType,
                                  SubstReplacement.get(), SugaredConverted,
                                  CanonicalConverted, Sema::CTAK_Specified)
           .isInvalid())
     return true;
   return transformNonTypeTemplateParmRef(E->getAssociatedDecl(),
                                          E->getParameter(), E->getExprLoc(),
-                                         CanonicalConverted, E->getPackIndex());
+                                         SugaredConverted, E->getPackIndex());
 }
 
 ExprResult TemplateInstantiator::RebuildVarDeclRefExpr(VarDecl *PD,
                                                        SourceLocation Loc) {
   DeclarationNameInfo NameInfo(PD->getDeclName(), Loc);
   return getSema().BuildDeclarationNameExpr(CXXScopeSpec(), NameInfo, PD);
 }
 
TemplateInstantiator::TransformFunctionTypeParam(ParmVarDecl *OldParm,
   auto NewParm = SemaRef.SubstParmVarDecl(
       OldParm, TemplateArgs, indexAdjustment, NumExpansions,
       ExpectParameterPack, EvaluateConstraints);
   if (NewParm && SemaRef.getLangOpts().OpenCL)
     SemaRef.deduceOpenCLAddressSpace(NewParm);
   return NewParm;
 }
 
+QualType TemplateInstantiator::BuildSubstTemplateTypeParmType(
+    TypeLocBuilder &TLB, bool SuppressObjCLifetime, bool Final,
+    Decl *AssociatedDecl, unsigned Index, Optional<unsigned> PackIndex,
+    TemplateArgument Arg, SourceLocation NameLoc) {
+  QualType Replacement = Arg.getAsType();
+
+  // If the template parameter had ObjC lifetime qualifiers,
+  // then any such qualifiers on the replacement type are ignored.
+  if (SuppressObjCLifetime) {
+    Qualifiers RQs;
+    RQs = Replacement.getQualifiers();
+    RQs.removeObjCLifetime();
+    Replacement =
+        SemaRef.Context.getQualifiedType(Replacement.getUnqualifiedType(), RQs);
+  }
+
+  if (Final) {
+    TLB.pushTrivial(SemaRef.Context, Replacement, NameLoc);
+    return Replacement;
+  }
+  // TODO: only do this uniquing once, at the start of instantiation.
+  QualType Result = getSema().Context.getSubstTemplateTypeParmType(
+      Replacement, AssociatedDecl, Index, PackIndex);
+  SubstTemplateTypeParmTypeLoc NewTL =
+      TLB.push<SubstTemplateTypeParmTypeLoc>(Result);
+  NewTL.setNameLoc(NameLoc);
+  return Result;
+}
+
 QualType
 TemplateInstantiator::TransformTemplateTypeParmType(TypeLocBuilder &TLB,
-                                                TemplateTypeParmTypeLoc TL) {
+                                                    TemplateTypeParmTypeLoc TL,
+                                                    bool SuppressObjCLifetime) {
   const TemplateTypeParmType *T = TL.getTypePtr();
   if (T->getDepth() < TemplateArgs.getNumLevels()) {
     // Replace the template type parameter with its corresponding
     // template argument.
 
     // If the corresponding template argument is NULL or doesn't exist, it's
     // because we are performing instantiation from explicitly-specified
     // template arguments in a function template class, but there were some
if (TemplateArgs.isRewrite()) {
       QualType NewT = Arg.getAsType();
       assert(isa<TemplateTypeParmType>(NewT) &&
              "type parm not rewritten to type parm");
       auto NewTL = TLB.push<TemplateTypeParmTypeLoc>(NewT);
       NewTL.setNameLoc(TL.getNameLoc());
       return NewT;
     }
 
-    Decl *AssociatedDecl =
-        const_cast<Decl *>(TemplateArgs.getAssociatedDecl(T->getDepth()));
+    auto [AssociatedDecl, Final] =
+        TemplateArgs.getAssociatedDecl(T->getDepth());
     Optional<unsigned> PackIndex;
     if (T->isParameterPack()) {
       assert(Arg.getKind() == TemplateArgument::Pack &&
              "Missing argument pack");
 
       if (getSema().ArgumentPackSubstitutionIndex == -1) {
         // We have the template argument pack, but we're not expanding the
         // enclosing pack expansion yet. Just save the template argument
         // pack for later substitution.
         QualType Result = getSema().Context.getSubstTemplateTypeParmPackType(
-            AssociatedDecl, T->getIndex(), Arg);
+            AssociatedDecl, T->getIndex(), Final, Arg);
         SubstTemplateTypeParmPackTypeLoc NewTL
           = TLB.push<SubstTemplateTypeParmPackTypeLoc>(Result);
         NewTL.setNameLoc(TL.getNameLoc());
         return Result;
       }
 
       // PackIndex starts from last element.
-      PackIndex = Arg.pack_size() - 1 - getSema().ArgumentPackSubstitutionIndex;
+      PackIndex = getPackIndex(Arg);
       Arg = getPackSubstitutedTemplateArgument(getSema(), Arg);
     }
 
     assert(Arg.getKind() == TemplateArgument::Type &&
            "Template argument kind mismatch");
 
-    QualType Replacement = Arg.getAsType();
-
-    // TODO: only do this uniquing once, at the start of instantiation.
-    QualType Result = getSema().Context.getSubstTemplateTypeParmType(
-        Replacement, AssociatedDecl, T->getIndex(), PackIndex);
-    SubstTemplateTypeParmTypeLoc NewTL
-      = TLB.push<SubstTemplateTypeParmTypeLoc>(Result);
-    NewTL.setNameLoc(TL.getNameLoc());
-    return Result;
+    return BuildSubstTemplateTypeParmType(TLB, SuppressObjCLifetime, Final,
+                                          AssociatedDecl, T->getIndex(),
+                                          PackIndex, Arg, TL.getNameLoc());
   }
 
   // The template type parameter comes from an inner template (e.g.,
   // the template parameter list of a member template inside the
   // template we are instantiating). Create a new template type
   // parameter with the template "level" reduced by one.
   TemplateTypeParmDecl *NewTTPDecl = nullptr;
   if (TemplateTypeParmDecl *OldTTPDecl = T->getDecl())
     NewTTPDecl = cast_or_null<TemplateTypeParmDecl>(
         TransformDecl(TL.getNameLoc(), OldTTPDecl));
   QualType Result = getSema().Context.getTemplateTypeParmType(
       T->getDepth() - TemplateArgs.getNumSubstitutedLevels(), T->getIndex(),
       T->isParameterPack(), NewTTPDecl);
   TemplateTypeParmTypeLoc NewTL = TLB.push<TemplateTypeParmTypeLoc>(Result);
   NewTL.setNameLoc(TL.getNameLoc());
   return Result;
 }
 
 QualType TemplateInstantiator::TransformSubstTemplateTypeParmPackType(
-    TypeLocBuilder &TLB, SubstTemplateTypeParmPackTypeLoc TL) {
+    TypeLocBuilder &TLB, SubstTemplateTypeParmPackTypeLoc TL,
+    bool SuppressObjCLifetime) {
   const SubstTemplateTypeParmPackType *T = TL.getTypePtr();
 
   Decl *NewReplaced = TransformDecl(TL.getNameLoc(), T->getAssociatedDecl());
 
   if (getSema().ArgumentPackSubstitutionIndex == -1) {
     // We aren't expanding the parameter pack, so just return ourselves.
     QualType Result = TL.getType();
     if (NewReplaced != T->getAssociatedDecl())
       Result = getSema().Context.getSubstTemplateTypeParmPackType(
-          NewReplaced, T->getIndex(), T->getArgumentPack());
+          NewReplaced, T->getIndex(), T->getFinal(), T->getArgumentPack());
     SubstTemplateTypeParmPackTypeLoc NewTL =
         TLB.push<SubstTemplateTypeParmPackTypeLoc>(Result);
     NewTL.setNameLoc(TL.getNameLoc());
     return Result;
   }
 
   TemplateArgument Pack = T->getArgumentPack();
   TemplateArgument Arg = getPackSubstitutedTemplateArgument(getSema(), Pack);
-  // PackIndex starts from last element.
-  QualType Result = getSema().Context.getSubstTemplateTypeParmType(
-      Arg.getAsType(), NewReplaced, T->getIndex(),
-      Pack.pack_size() - 1 - getSema().ArgumentPackSubstitutionIndex);
-  SubstTemplateTypeParmTypeLoc NewTL =
-      TLB.push<SubstTemplateTypeParmTypeLoc>(Result);
-  NewTL.setNameLoc(TL.getNameLoc());
-  return Result;
+  return BuildSubstTemplateTypeParmType(
+      TLB, SuppressObjCLifetime, T->getFinal(), NewReplaced, T->getIndex(),
+      getPackIndex(Pack), Arg, TL.getNameLoc());
 }
 
 template<typename EntityPrinter>
 static concepts::Requirement::SubstitutionDiagnostic *
 createSubstDiag(Sema &S, TemplateDeductionInfo &Info, EntityPrinter Printer) {
   SmallString<128> Message;
   SourceLocation ErrorLoc;
   if (Info.hasSFINAEDiagnostic()) {

clang/lib/Sema/SemaTemplateInstantiateDecl.cpp

bool Sema::InstantiateDefaultArgument(SourceLocation CallLoc, FunctionDecl *FD,
   //   template<typename Tp>
   //   // bug: default argument A<T>::FooImpl() is evaluated with 2-level
   //   // template argument list [[T], [Tp]], should be [[Tp]].
   //   friend A<Tp> Foo(int a);
   // };
   //
   // template<typename T>
   // A<T> Foo(int a = A<T>::FooImpl());
-  MultiLevelTemplateArgumentList TemplateArgs
-    = getTemplateInstantiationArgs(FD, nullptr, /*RelativeToPrimary=*/true);
+  MultiLevelTemplateArgumentList TemplateArgs = getTemplateInstantiationArgs(
+      FD, /*Final=*/false, nullptr, /*RelativeToPrimary=*/true);
 
   if (SubstDefaultArgument(CallLoc, Param, TemplateArgs, /*ForCallExpr*/ true))
     return true;
 
   if (ASTMutationListener *L = getASTMutationListener())
     L->DefaultArgumentInstantiated(Param);
 
   return false;
if (Inst.isAlreadyInstantiating()) {
     return;
   }
 
   // Enter the scope of this instantiation. We don't use
   // PushDeclContext because we don't have a scope.
   Sema::ContextRAII savedContext(*this, Decl);
   LocalInstantiationScope Scope(*this);
 
-  MultiLevelTemplateArgumentList TemplateArgs =
-    getTemplateInstantiationArgs(Decl, nullptr, /*RelativeToPrimary*/true);
+  MultiLevelTemplateArgumentList TemplateArgs = getTemplateInstantiationArgs(
+      Decl, /*Final=*/false, nullptr, /*RelativeToPrimary*/ true);
 
   // FIXME: We can't use getTemplateInstantiationPattern(false) in general
   // here, because for a non-defining friend declaration in a class template,
   // we don't store enough information to map back to the friend declaration in
   // the template.
   FunctionDecl *Template = Proto->getExceptionSpecTemplate();
   if (addInstantiatedParametersToScope(Decl, Template, Scope, TemplateArgs)) {
     UpdateExceptionSpec(Decl, EST_None);
Sema::InstantiateFunctionDeclaration(FunctionTemplateDecl *FTD,
   sema::TemplateDeductionInfo Info(Loc);
   InstantiatingTemplate Inst(
       *this, Loc, FTD, Args->asArray(),
       CodeSynthesisContext::ExplicitTemplateArgumentSubstitution, Info);
   if (Inst.isInvalid())
     return nullptr;
 
   ContextRAII SavedContext(*this, FD);
-  MultiLevelTemplateArgumentList MArgs(FTD, Args->asArray());
+  MultiLevelTemplateArgumentList MArgs(FTD, Args->asArray(),
+                                       /*Final=*/false);
 
   return cast_or_null<FunctionDecl>(SubstDecl(FD, FD->getParent(), MArgs));
 }
 
 /// Instantiate the definition of the given function from its
 /// template.
 ///
 /// \param PointOfInstantiation the point at which the instantiation was
auto RebuildTypeSourceInfoForDefaultSpecialMembers = [&]() {
     Parm->setType(NewParmSI->getType());
     Parm->setTypeSourceInfo(NewParmSI);
   };
 
   if (PatternDecl->isDefaulted()) {
     RebuildTypeSourceInfoForDefaultSpecialMembers();
     SetDeclDefaulted(Function, PatternDecl->getLocation());
   } else {
-    MultiLevelTemplateArgumentList TemplateArgs =
-      getTemplateInstantiationArgs(Function, nullptr, false, PatternDecl);
+    MultiLevelTemplateArgumentList TemplateArgs = getTemplateInstantiationArgs(
+        Function, /*Final=*/false, nullptr, false, PatternDecl);
 
     // Substitute into the qualifier; we can get a substitution failure here
     // through evil use of alias templates.
     // FIXME: Is CurContext correct for this? Should we go to the (instantiation
     // of the) lexical context of the pattern?
     SubstQualifier(*this, PatternDecl, Function, TemplateArgs);
 
     ActOnStartOfFunctionDef(nullptr, Function);
VarTemplateSpecializationDecl *Sema::BuildVarTemplateInstantiation(
   // If we're instantiating an explicitly-specialized member template or member
   // partial specialization, don't do this. The member specialization completely
   // replaces the original declaration in this case.
   bool IsMemberSpec = false;
   MultiLevelTemplateArgumentList MultiLevelList;
   if (auto *PartialSpec =
           dyn_cast<VarTemplatePartialSpecializationDecl>(FromVar)) {
     IsMemberSpec = PartialSpec->isMemberSpecialization();
-    MultiLevelList.addOuterTemplateArguments(PartialSpec,
-                                             TemplateArgList.asArray());
+    MultiLevelList.addOuterTemplateArguments(
+        PartialSpec, TemplateArgList.asArray(), /*Final=*/false);
   } else {
     assert(VarTemplate == FromVar->getDescribedVarTemplate());
     IsMemberSpec = VarTemplate->isMemberSpecialization();
-    MultiLevelList.addOuterTemplateArguments(VarTemplate,
-                                             TemplateArgList.asArray());
+    MultiLevelList.addOuterTemplateArguments(
+        VarTemplate, TemplateArgList.asArray(), /*Final=*/false);
   }
   if (!IsMemberSpec)
     FromVar = FromVar->getFirstDecl();
 
   TemplateDeclInstantiator Instantiator(*this, FromVar->getDeclContext(),
                                         MultiLevelList);
 
   // TODO: Set LateAttrs and StartingScope ...

clang/lib/Sema/SemaType.cpp

static bool checkOmittedBlockReturnType(Sema &S, Declarator &declarator,
   declarator.getMutableDeclSpec().ClearTypeQualifiers();
 
   return true;
 }
 
 /// Apply Objective-C type arguments to the given type.
 static QualType applyObjCTypeArgs(Sema &S, SourceLocation loc, QualType type,
                                   ArrayRef<TypeSourceInfo *> typeArgs,
-                                  SourceRange typeArgsRange,
-                                  bool failOnError = false) {
+                                  SourceRange typeArgsRange, bool failOnError,
+                                  bool rebuilding) {
   // We can only apply type arguments to an Objective-C class type.
   const auto *objcObjectType = type->getAs<ObjCObjectType>();
   if (!objcObjectType || !objcObjectType->getInterface()) {
     S.Diag(loc, diag::err_objc_type_args_non_class)
       << type
       << typeArgsRange;
 
     if (failOnError)
if (TypeLoc qual = typeArgInfo->getTypeLoc().findExplicitQualifierLoc()) {
           typeArg = attr.getTypePtr()->getModifiedType();
           S.Diag(attr.getBeginLoc(),
                  diag::err_objc_type_arg_explicit_nullability)
               << typeArg << FixItHint::CreateRemoval(rangeToRemove);
           diagnosed = true;
         }
       }
 
-      if (!diagnosed) {
+      // When rebuilding, qualifiers might have gotten here through a
+      // sugared substitution.
+      if (!rebuilding && !diagnosed) {
         S.Diag(qual.getBeginLoc(), diag::err_objc_type_arg_qualified)
             << typeArg << typeArg.getQualifiers().getAsString()
             << FixItHint::CreateRemoval(rangeToRemove);
       }
     }
 
     // Remove qualifiers even if they're non-local.
     typeArg = typeArg.getUnqualifiedType();
if (!Protocols.empty()) {
     }
     if (FailOnError && Result.isNull())
       return QualType();
   }
 
   return Result;
 }
 
-QualType Sema::BuildObjCObjectType(QualType BaseType,
-                                   SourceLocation Loc,
-                                   SourceLocation TypeArgsLAngleLoc,
-                                   ArrayRef<TypeSourceInfo *> TypeArgs,
-                                   SourceLocation TypeArgsRAngleLoc,
-                                   SourceLocation ProtocolLAngleLoc,
-                                   ArrayRef<ObjCProtocolDecl *> Protocols,
-                                   ArrayRef<SourceLocation> ProtocolLocs,
-                                   SourceLocation ProtocolRAngleLoc,
-                                   bool FailOnError) {
+QualType Sema::BuildObjCObjectType(
+    QualType BaseType, SourceLocation Loc, SourceLocation TypeArgsLAngleLoc,
+    ArrayRef<TypeSourceInfo *> TypeArgs, SourceLocation TypeArgsRAngleLoc,
+    SourceLocation ProtocolLAngleLoc, ArrayRef<ObjCProtocolDecl *> Protocols,
+    ArrayRef<SourceLocation> ProtocolLocs, SourceLocation ProtocolRAngleLoc,
+    bool FailOnError, bool Rebuilding) {
   QualType Result = BaseType;
   if (!TypeArgs.empty()) {
-    Result = applyObjCTypeArgs(*this, Loc, Result, TypeArgs,
-                               SourceRange(TypeArgsLAngleLoc,
-                                           TypeArgsRAngleLoc),
-                               FailOnError);
+    Result =
+        applyObjCTypeArgs(*this, Loc, Result, TypeArgs,
+                          SourceRange(TypeArgsLAngleLoc, TypeArgsRAngleLoc),
+                          FailOnError, Rebuilding);
     if (FailOnError && Result.isNull())
       return QualType();
   }
 
   if (!Protocols.empty()) {
     bool HasError;
     Result = Context.applyObjCProtocolQualifiers(Result, Protocols,
                                                  HasError);
for (unsigned i = 0, n = TypeArgs.size(); i != n; ++i) {
     ActualTypeArgInfos.push_back(TypeArgInfo);
   }
 
   // Build the object type.
   QualType Result = BuildObjCObjectType(
       T, BaseTypeInfo->getTypeLoc().getSourceRange().getBegin(),
       TypeArgsLAngleLoc, ActualTypeArgInfos, TypeArgsRAngleLoc,
       ProtocolLAngleLoc,
-      llvm::makeArrayRef((ObjCProtocolDecl * const *)Protocols.data(),
+      llvm::makeArrayRef((ObjCProtocolDecl *const *)Protocols.data(),
                          Protocols.size()),
       ProtocolLocs, ProtocolRAngleLoc,
-      /*FailOnError=*/false);
+      /*FailOnError=*/false,
+      /*Rebuilding=*/false);
 
   if (Result == T)
     return BaseType;
 
   // Create source information for this type.
   TypeSourceInfo *ResultTInfo = Context.CreateTypeSourceInfo(Result);
   TypeLoc ResultTL = ResultTInfo->getTypeLoc();
 

clang/lib/Sema/TreeTransform.h

return SemaRef.Context.getTrivialTypeSourceInfo(T,
                        getDerived().getBaseLocation());
   }
 
 #define ABSTRACT_TYPELOC(CLASS, PARENT)
 #define TYPELOC(CLASS, PARENT)                                   \
   QualType Transform##CLASS##Type(TypeLocBuilder &TLB, CLASS##TypeLoc T);
 #include "clang/AST/TypeLocNodes.def"
 
+  QualType TransformTemplateTypeParmType(TypeLocBuilder &TLB,
+                                         TemplateTypeParmTypeLoc TL,
+                                         bool SuppressObjCLifetime);
+  QualType
+  TransformSubstTemplateTypeParmPackType(TypeLocBuilder &TLB,
+                                         SubstTemplateTypeParmPackTypeLoc TL,
+                                         bool SuppressObjCLifetime);
+
   template<typename Fn>
   QualType TransformFunctionProtoType(TypeLocBuilder &TLB,
                                       FunctionProtoTypeLoc TL,
                                       CXXRecordDecl *ThisContext,
                                       Qualifiers ThisTypeQuals,
                                       Fn TransformExceptionSpec);
 
   bool TransformExceptionSpec(SourceLocation Loc,
#include "llvm/Frontend/OpenMP/OMP.inc"
   /// Build a new template name given a template template parameter pack
   /// and the
   ///
   /// By default, performs semantic analysis to determine whether the name can
   /// be resolved to a specific template, then builds the appropriate kind of
   /// template name. Subclasses may override this routine to provide different
   /// behavior.
   TemplateName RebuildTemplateName(const TemplateArgument &ArgPack,
-                                   Decl *AssociatedDecl, unsigned Index) {
+                                   Decl *AssociatedDecl, unsigned Index,
+                                   bool Final) {
     return getSema().Context.getSubstTemplateTemplateParmPack(
-        ArgPack, AssociatedDecl, Index);
+        ArgPack, AssociatedDecl, Index, Final);
   }
 
   /// Build a new compound statement.
   ///
   /// By default, performs semantic analysis to build the new statement.
   /// Subclasses may override this routine to provide different behavior.
   StmtResult RebuildCompoundStmt(SourceLocation LBraceLoc,
                                        MultiStmtArg Statements,
Sema::ConditionResult TreeTransform<Derived>::TransformCondition(
   return Sema::ConditionResult();
 }
 
 template <typename Derived>
 NestedNameSpecifierLoc TreeTransform<Derived>::TransformNestedNameSpecifierLoc(
     NestedNameSpecifierLoc NNS, QualType ObjectType,
     NamedDecl *FirstQualifierInScope) {
   SmallVector<NestedNameSpecifierLoc, 4> Qualifiers;
+
+  auto insertNNS = [&Qualifiers](NestedNameSpecifierLoc NNS) {
     for (NestedNameSpecifierLoc Qualifier = NNS; Qualifier;
          Qualifier = Qualifier.getPrefix())
       Qualifiers.push_back(Qualifier);
+  };
+  insertNNS(NNS);
 
   CXXScopeSpec SS;
   while (!Qualifiers.empty()) {
     NestedNameSpecifierLoc Q = Qualifiers.pop_back_val();
     NestedNameSpecifier *QNNS = Q.getNestedNameSpecifier();
 
     switch (QNNS->getKind()) {
     case NestedNameSpecifier::Identifier: {
while (!Qualifiers.empty()) {
     case NestedNameSpecifier::TypeSpecWithTemplate:
     case NestedNameSpecifier::TypeSpec: {
       TypeLoc TL = TransformTypeInObjectScope(Q.getTypeLoc(), ObjectType,
                                               FirstQualifierInScope, SS);
 
       if (!TL)
         return NestedNameSpecifierLoc();
 
-      if (TL.getType()->isDependentType() || TL.getType()->isRecordType() ||
-          (SemaRef.getLangOpts().CPlusPlus11 &&
-           TL.getType()->isEnumeralType())) {
-        assert(!TL.getType().hasLocalQualifiers() &&
-               "Can't get cv-qualifiers here");
-        if (TL.getType()->isEnumeralType())
+      QualType T = TL.getType();
+      if (T->isDependentType() || T->isRecordType() ||
+          (SemaRef.getLangOpts().CPlusPlus11 && T->isEnumeralType())) {
+        if (T->isEnumeralType())
           SemaRef.Diag(TL.getBeginLoc(),
                        diag::warn_cxx98_compat_enum_nested_name_spec);
+
+        if (const auto ETL = TL.getAs<ElaboratedTypeLoc>()) {
+          SS.Adopt(ETL.getQualifierLoc());
+          TL = ETL.getNamedTypeLoc();
+        }
         SS.Extend(SemaRef.Context, /*FIXME:*/ SourceLocation(), TL,
                   Q.getLocalEndLoc());
         break;
       }
       // If the nested-name-specifier is an invalid type def, don't emit an
       // error because a previous error should have already been emitted.
       TypedefTypeLoc TTL = TL.getAsAdjusted<TypedefTypeLoc>();
       if (!TTL || !TTL.getTypedefNameDecl()->isInvalidDecl()) {
         SemaRef.Diag(TL.getBeginLoc(), diag::err_nested_name_spec_non_tag)
-            << TL.getType() << SS.getRange();
+            << T << SS.getRange();
       }
       return NestedNameSpecifierLoc();
     }
     }
 
     // The qualifier-in-scope and object type only apply to the leftmost entity.
     FirstQualifierInScope = nullptr;
     ObjectType = QualType();
if (!getDerived().AlwaysRebuild() &&
         TransTemplate == Template)
       return Name;
 
     return TemplateName(TransTemplate);
   }
 
   if (SubstTemplateTemplateParmPackStorage *SubstPack
       = Name.getAsSubstTemplateTemplateParmPack()) {
-    return getDerived().RebuildTemplateName(SubstPack->getArgumentPack(),
-                                            SubstPack->getAssociatedDecl(),
-                                            SubstPack->getIndex());
+    return getDerived().RebuildTemplateName(
+        SubstPack->getArgumentPack(), SubstPack->getAssociatedDecl(),
+        SubstPack->getIndex(), SubstPack->getFinal());
   }
 
   // These should be getting filtered out before they reach the AST.
   llvm_unreachable("overloaded function decl survived to here");
 }
 
 template<typename Derived>
 void TreeTransform<Derived>::InventTemplateArgumentLoc(
TreeTransform<Derived>::TransformTypeWithDeducedTST(TypeSourceInfo *DI) {
 
   return TLB.getTypeSourceInfo(SemaRef.Context, Result);
 }
 
 template<typename Derived>
 QualType
 TreeTransform<Derived>::TransformQualifiedType(TypeLocBuilder &TLB,
                                                QualifiedTypeLoc T) {
-  QualType Result = getDerived().TransformType(TLB, T.getUnqualifiedLoc());
+  QualType Result;
+  TypeLoc UnqualTL = T.getUnqualifiedLoc();
+  auto SuppressObjCLifetime =
+      T.getType().getLocalQualifiers().hasObjCLifetime();
+  if (auto TTP = UnqualTL.getAs<TemplateTypeParmTypeLoc>()) {
+    Result = getDerived().TransformTemplateTypeParmType(TLB, TTP,
+                                                        SuppressObjCLifetime);
+  } else if (auto STTP = UnqualTL.getAs<SubstTemplateTypeParmPackTypeLoc>()) {
+    Result = getDerived().TransformSubstTemplateTypeParmPackType(
+        TLB, STTP, SuppressObjCLifetime);
+  } else {
+    Result = getDerived().TransformType(TLB, UnqualTL);
+  }
+
   if (Result.isNull())
     return QualType();
 
   Result = getDerived().RebuildQualifiedType(Result, T);
 
   if (Result.isNull())
     return QualType();
 
QualType TreeTransform<Derived>::RebuildQualifiedType(QualType T,
   if (Quals.hasObjCLifetime()) {
     if (!T->isObjCLifetimeType() && !T->isDependentType())
       Quals.removeObjCLifetime();
     else if (T.getObjCLifetime()) {
       // Objective-C ARC:
       //   A lifetime qualifier applied to a substituted template parameter
       //   overrides the lifetime qualifier from the template argument.
       const AutoType *AutoTy;
-      if (const SubstTemplateTypeParmType *SubstTypeParam
-                                = dyn_cast<SubstTemplateTypeParmType>(T)) {
-        QualType Replacement = SubstTypeParam->getReplacementType();
-        Qualifiers Qs = Replacement.getQualifiers();
-        Qs.removeObjCLifetime();
-        Replacement = SemaRef.Context.getQualifiedType(
-            Replacement.getUnqualifiedType(), Qs);
-        T = SemaRef.Context.getSubstTemplateTypeParmType(
-            Replacement, SubstTypeParam->getAssociatedDecl(),
-            SubstTypeParam->getIndex(), SubstTypeParam->getPackIndex());
-      } else if ((AutoTy = dyn_cast<AutoType>(T)) && AutoTy->isDeduced()) {
+      if ((AutoTy = dyn_cast<AutoType>(T)) && AutoTy->isDeduced()) {
         // 'auto' types behave the same way as template parameters.
         QualType Deduced = AutoTy->getDeducedType();
         Qualifiers Qs = Deduced.getQualifiers();
         Qs.removeObjCLifetime();
         Deduced =
             SemaRef.Context.getQualifiedType(Deduced.getUnqualifiedType(), Qs);
         T = SemaRef.Context.getAutoType(Deduced, AutoTy->getKeyword(),
                                         AutoTy->isDependentType(),
QualType TreeTransform<Derived>::TransformInjectedClassNameType(
   TLB.pushTypeSpec(T).setNameLoc(TL.getNameLoc());
   return T;
 }
 
 template<typename Derived>
 QualType TreeTransform<Derived>::TransformTemplateTypeParmType(
                                                 TypeLocBuilder &TLB,
                                                 TemplateTypeParmTypeLoc TL) {
+  return getDerived().TransformTemplateTypeParmType(
+      TLB, TL,
+      /*SuppressObjCLifetime=*/false);
+}
+
+template <typename Derived>
+QualType TreeTransform<Derived>::TransformTemplateTypeParmType(
+    TypeLocBuilder &TLB, TemplateTypeParmTypeLoc TL, bool) {
   return TransformTypeSpecType(TLB, TL);
 }
 
 template<typename Derived>
 QualType TreeTransform<Derived>::TransformSubstTemplateTypeParmType(
                                          TypeLocBuilder &TLB,
                                          SubstTemplateTypeParmTypeLoc TL) {
   const SubstTemplateTypeParmType *T = TL.getTypePtr();
QualType TreeTransform<Derived>::TransformSubstTemplateTypeParmType(
   return Result;
 
 }
 
 template<typename Derived>
 QualType TreeTransform<Derived>::TransformSubstTemplateTypeParmPackType(
                                           TypeLocBuilder &TLB,
                                           SubstTemplateTypeParmPackTypeLoc TL) {
+  return getDerived().TransformSubstTemplateTypeParmPackType(
+      TLB, TL, /*SuppressObjCLifetime=*/false);
+}
+
+template <typename Derived>
+QualType TreeTransform<Derived>::TransformSubstTemplateTypeParmPackType(
+    TypeLocBuilder &TLB, SubstTemplateTypeParmPackTypeLoc TL, bool) {
   return TransformTypeSpecType(TLB, TL);
 }
 
 template<typename Derived>
 QualType TreeTransform<Derived>::TransformTemplateSpecializationType(
                                                         TypeLocBuilder &TLB,
                                            TemplateSpecializationTypeLoc TL) {
   const TemplateSpecializationType *T = TL.getTypePtr();
if (auto PackExpansionLoc = TypeArgLoc.getAs<PackExpansionTypeLoc>()) {
           TypeArgBuilder.getTypeSourceInfo(SemaRef.Context, NewTypeArg));
       }
 
       continue;
     }
 
     TypeLocBuilder TypeArgBuilder;
     TypeArgBuilder.reserve(TypeArgLoc.getFullDataSize());
-    QualType NewTypeArg = getDerived().TransformType(TypeArgBuilder, TypeArgLoc);
+    QualType NewTypeArg =
+        getDerived().TransformType(TypeArgBuilder, TypeArgLoc);
     if (NewTypeArg.isNull())
       return QualType();
 
     // If nothing changed, just keep the old TypeSourceInfo.
     if (NewTypeArg == TypeArg) {
       NewTypeArgInfos.push_back(TypeArgInfo);
       continue;
     }
QualType TreeTransform<Derived>::RebuildObjCObjectType(
            SourceLocation Loc,
            SourceLocation TypeArgsLAngleLoc,
            ArrayRef<TypeSourceInfo *> TypeArgs,
            SourceLocation TypeArgsRAngleLoc,
            SourceLocation ProtocolLAngleLoc,
            ArrayRef<ObjCProtocolDecl *> Protocols,
            ArrayRef<SourceLocation> ProtocolLocs,
            SourceLocation ProtocolRAngleLoc) {
-  return SemaRef.BuildObjCObjectType(BaseType, Loc, TypeArgsLAngleLoc,
-                                     TypeArgs, TypeArgsRAngleLoc,
-                                     ProtocolLAngleLoc, Protocols, ProtocolLocs,
-                                     ProtocolRAngleLoc,
-                                     /*FailOnError=*/true);
+  return SemaRef.BuildObjCObjectType(BaseType, Loc, TypeArgsLAngleLoc, TypeArgs,
+                                     TypeArgsRAngleLoc, ProtocolLAngleLoc,
+                                     Protocols, ProtocolLocs, ProtocolRAngleLoc,
+                                     /*FailOnError=*/true,
+                                     /*Rebuilding=*/true);
 }
 
 template<typename Derived>
 QualType TreeTransform<Derived>::RebuildObjCObjectPointerType(
            QualType PointeeType,
            SourceLocation Star) {
   return SemaRef.Context.getObjCObjectPointerType(PointeeType);
 }

clang/lib/Sema/TypeLocBuilder.h

if (Requested > Capacity)
       // For now, match the request exactly.
       grow(Requested);
   }
 
   /// Pushes a copy of the given TypeLoc onto this builder.  The builder
   /// must be empty for this to work.
   void pushFullCopy(TypeLoc L);
 
+  /// Pushes 'T' with all locations pointing to 'Loc'.
+  /// The builder must be empty for this to work.
+  void pushTrivial(ASTContext &Context, QualType T, SourceLocation Loc);
+
   /// Pushes space for a typespec TypeLoc.  Invalidates any TypeLocs
   /// previously retrieved from this builder.
   TypeSpecTypeLoc pushTypeSpec(QualType T) {
     size_t LocalSize = TypeSpecTypeLoc::LocalDataSize;
     unsigned LocalAlign = TypeSpecTypeLoc::LocalDataAlignment;
     return pushImpl(T, LocalSize, LocalAlign).castAs<TypeSpecTypeLoc>();
   }
 

clang/lib/Sema/TypeLocBuilder.cpp

case TypeLoc::CLASS: { \
       memcpy(NewTL.getOpaqueData(), CurTL.getOpaqueData(), NewTL.getLocalDataSize()); \
       break; \
     }
 #include "clang/AST/TypeLocNodes.def"
     }
   }
 }
 
+void TypeLocBuilder::pushTrivial(ASTContext &Context, QualType T,
+                                 SourceLocation Loc) {
+  auto L = TypeLoc(T, nullptr);
+  reserve(L.getFullDataSize());
+
+  SmallVector<TypeLoc, 4> TypeLocs;
+  for (auto CurTL = L; CurTL; CurTL = CurTL.getNextTypeLoc())
+    TypeLocs.push_back(CurTL);
+
+  for (const auto &CurTL : llvm::reverse(TypeLocs)) {
+    switch (CurTL.getTypeLocClass()) {
+#define ABSTRACT_TYPELOC(CLASS, PARENT)
+#define TYPELOC(CLASS, PARENT)                                                 \
+  case TypeLoc::CLASS: {                                                       \
+    auto NewTL = push<class CLASS##TypeLoc>(CurTL.getType());                  \
+    NewTL.initializeLocal(Context, Loc);                                       \
+    break;                                                                     \
+  }
+#include "clang/AST/TypeLocNodes.def"
+    }
+  }
+}
+
 void TypeLocBuilder::grow(size_t NewCapacity) {
   assert(NewCapacity > Capacity);
 
   // Allocate the new buffer and copy the old data into it.
   char *NewBuffer = new char[NewCapacity];
   unsigned NewIndex = Index + NewCapacity - Capacity;
   memcpy(&NewBuffer[NewIndex],
          &Buffer[Index],

clang/test/SemaTemplate/nested-name-spec-template.cpp

namespace PR9449 {
 
   template <typename T>
   void f() {
     int s<T>::template n<T>::* f; // expected-error{{implicit instantiation of undefined template 'PR9449::s<int>'}}
   }
 
   template void f<int>(); // expected-note{{in instantiation of}}
 }
+
+namespace sugared_template_instantiation {
+  // Test that we ignore local qualifiers.
+  template <class A1, class = typename A1::type1> struct A {};
+  struct B { typedef int type1; };
+  typedef A<const B> type2;
+} // namespace sugated_template_instantiation