[constexpr-lambda] Support parsing of constexpr specifier (and its inference) on lambda expressions

[faisalv]

Summary

This patch supports parsing of the constexpr specifier on lambdas - and its inference from the lambda call operator's body.

i.e.
auto L = [] () constexpr { return 5; };
static_assert(L() == 5); // OK

It does not support evaluation of lambda's within constant expressions, which should follow this patch, when approved.

[faisalv]

*ping*

[rsmith]

include/clang/Basic/DiagnosticParseKinds.td

772

are -> is

773

Should be CXXPre1zCompat, it's also incompatible with C++11.

775

Any reason not to allow this as an extension in earlier standards?

lib/Parse/ParseExprCXX.cpp

1049

This doesn't look like it'll handle duplicates very well. We should consume and diagnose them, and recover as if they weren't there.

1061

Move this into the caller. It's inconsistent (and doesn't match the function name) to handle this but not 'mutable' here.

lib/Sema/SemaLambda.cpp

1599

Ah, I now see why you were asking about this... :)

I would prefer a Diagnose flag here, but we can make that change later.

test/Parser/cxx1z-constexpr-lambdas.cpp

6

Maybe test __cplusplus > 201402L rather than passing a separate macro?

test/SemaCXX/cxx1z-constexpr-lambdas.cpp

5

You're not using these -D flags for anything; remove them.

[faisalv]

Incorporated Richard's feedback:

• constexpr now elicits an extension warning in pre-cxx1z
• refactored the function that parses constexpr/mutable and added recovery for duplicates (while emitting a diagnostic)

[faisalv]

*ping*

[rsmith]

lib/AST/ExprConstant.cpp

2050

What's the DeclContext of a variable we synthesize to represent an init-capture?

lib/Parse/ParseExprCXX.cpp

1045

No need to mark this inline.

1051

consume -> Consume

1056–1058

Any reason to have an array here rather than two separate variables?

1066

Please add a FixItHint::CreateRemoval(P.getCurToken().getLocation()) here.

1067

Why suppress the diagnostic in this case? If there are three mutable specifiers, I'd like two diagnostics and two FixItHints removing the extra ones. That's what we do for duplicated specifiers in other contexts.

1085

Please spell this as return; instead :)

1090

No need to mark this inline.

[faisalv]

Thanks for the review - will submit a updated patch with recommended changes, shortly.

[faisalv]

An updated patch that incorporates Richard's feedback.
Thanks!

[rsmith]

include/clang/Basic/DiagnosticParseKinds.td

769

declspecifier -> decl_specifier

lib/AST/ExprConstant.cpp

2049–2063

I'd prefer to phrase this if/assert arrangement as:

if (!Result) {
  assert(isLambdaCallOperator(...) && (VD->getDeclContext != ...

lib/Parse/ParseExprCXX.cpp

1077

No space before ;.

[faisalv]

Closed by r264513.
http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160321/153846.html

include/clang/Basic/DiagnosticASTKinds.td

 def note_constexpr_baa_value_insufficient_alignment : Note<
   "value of the aligned pointer (%0) is not a multiple of the asserted %1 "
   "%plural{1:byte|:bytes}1">;
 
 def warn_integer_constant_overflow : Warning<
   "overflow in expression; result is %0 with type %1">,
   InGroup<DiagGroup<"integer-overflow">>;
 
+// This is a temporary diagnostic, and shall be removed once our 
+// implementation is complete, and like the preceding constexpr notes belongs
+// in Sema.
+def note_unimplemented_constexpr_lambda_feature_ast : Note<
+    "unimplemented constexpr lambda feature: %0 (coming soon!)">;
+
 // inline asm related.
 let CategoryName = "Inline Assembly Issue" in {
   def err_asm_invalid_escape : Error<
     "invalid %% escape in inline assembly string">;
   def err_asm_unknown_symbolic_operand_name : Error<
     "unknown symbolic operand name in inline assembly string">;
 
   def err_asm_unterminated_symbolic_operand_name : Error<

include/clang/Basic/DiagnosticParseKinds.td

 def err_expected_capture : Error<
   "expected variable name or 'this' in lambda capture list">;
 def err_expected_lambda_body : Error<"expected body of lambda expression">;
 def warn_cxx98_compat_lambda : Warning<
   "lambda expressions are incompatible with C++98">,
   InGroup<CXX98Compat>, DefaultIgnore;
 def err_lambda_missing_parens : Error<
   "lambda requires '()' before %select{'mutable'|return type|"
-  "attribute specifier}0">;
+  "attribute specifier|'constexpr'}0">;
+def err_lambda_declspecifier_repeated : Error<
+  "%select{'mutable'|'constexpr'}0 cannot appear multiple times in a lambda declarator">;
+
+// C++1z constexpr lambda expressions
+def warn_cxx14_compat_constexpr_on_lambda : Warning<
+  "constexpr on lambda expressions is incompatible with C++ standards before C++1z">,
+  InGroup<CXXPre1zCompat>, DefaultIgnore;
+def ext_constexpr_on_lambda_cxx1z : ExtWarn<
+  "'constexpr' on lambda expressions is a C++1z extension">, InGroup<CXX1z>;
 
 // Availability attribute
 def err_expected_version : Error<
   "expected a version of the form 'major[.minor[.subminor]]'">;
 def warn_expected_consistent_version_separator : Warning<
   "use same version number separators '_' or '.'; as in "
   "'major[.minor[.subminor]]'">, InGroup<Availability>;
 def err_zero_version : Error<

include/clang/Sema/Sema.h

                                          bool KnownDependent, 
                                          LambdaCaptureDefault CaptureDefault);
 
   /// \brief Start the definition of a lambda expression.
   CXXMethodDecl *startLambdaDefinition(CXXRecordDecl *Class,
                                        SourceRange IntroducerRange,
                                        TypeSourceInfo *MethodType,
                                        SourceLocation EndLoc,
-                                       ArrayRef<ParmVarDecl *> Params);
+                                       ArrayRef<ParmVarDecl *> Params, 
+                                       bool IsConstexprSpecified);
 
   /// \brief Endow the lambda scope info with the relevant properties.
   void buildLambdaScope(sema::LambdaScopeInfo *LSI, 
                         CXXMethodDecl *CallOperator,
                         SourceRange IntroducerRange,
                         LambdaCaptureDefault CaptureDefault,
                         SourceLocation CaptureDefaultLoc,
                         bool ExplicitParams,

lib/AST/ExprConstant.cpp

 // return value (the expression could not be folded) and no diagnostic (the
 // expression is not necessarily non-constant).
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/AST/APValue.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/ASTDiagnostic.h"
+#include "clang/AST/ASTLambda.h"
 #include "clang/AST/CharUnits.h"
 #include "clang/AST/Expr.h"
 #include "clang/AST/RecordLayout.h"
 #include "clang/AST/StmtVisitor.h"
 #include "clang/AST/TypeLoc.h"
 #include "clang/Basic/Builtins.h"
 #include "clang/Basic/TargetInfo.h"
 #include "llvm/ADT/SmallString.h"

     }
     Result = &Frame->Arguments[PVD->getFunctionScopeIndex()];
     return true;
   }
 
   // If this is a local variable, dig out its value.
   if (Frame) {
     Result = Frame->getTemporary(VD);
+    if (!Result && isLambdaCallOperator(Frame->Callee) &&
+        (VD->getDeclContext() != Frame->Callee || VD->isInitCapture())) {
+      // Assume variables referenced within a lambda's call operator that were
+      // not declared within the call operator are captures and during checking
+      // of a potential constant expression, assume they are unknown constant
+      // expressions. 
+      if (Info.checkingPotentialConstantExpression())
+        return false;
+      // FIXME: implement capture evaluation during constant expr evaluation.
+      Info.Diag(E->getLocStart(),
+           diag::note_unimplemented_constexpr_lambda_feature_ast)
+          << "captures not currently allowed";
+      return false;
+    }
     assert(Result && "missing value for local variable");
     return true;
   }
 
   // Dig out the initializer, and use the declaration which it's attached to.
   const Expr *Init = VD->getAnyInitializer(VD);
   if (!Init || Init->isValueDependent()) {
     // If we're checking a potential constant expression, the variable could be
     // initialized later.

lib/Parse/ParseExprCXX.cpp

     assert(!DiagID && "parsing lambda-introducer failed on reparse");
     return false;
   }
 
   PA.Commit();
   return false;
 }
 
+static void
+tryConsumeMutableOrConstexprToken(Parser &P, SourceLocation &MutableLoc,
+                                  SourceLocation &ConstexprLoc,
+                                  SourceLocation &DeclEndLoc) {
+  assert(MutableLoc.isInvalid());
+  assert(ConstexprLoc.isInvalid());
+  // Consume constexpr-opt mutable-opt in any sequence, and set the DeclEndLoc
+  // to the final of those locations. Emit an error if we have multiple
+  // copies of those keywords and recover.
+
+  while (true) {
+    switch (P.getCurToken().getKind()) {
+    case tok::kw_mutable: {
+      if (MutableLoc.isValid()) {
+        P.Diag(P.getCurToken().getLocation(),
+               diag::err_lambda_declspecifier_repeated)
+            << 0 << FixItHint::CreateRemoval(P.getCurToken().getLocation());
+      }
+      MutableLoc = P.ConsumeToken();
+      DeclEndLoc = MutableLoc;
+      break /*switch*/;
+    }
+    case tok::kw_constexpr:
+      if (ConstexprLoc.isValid()) {
+        P.Diag(P.getCurToken().getLocation(),
+               diag::err_lambda_declspecifier_repeated)
+            << 1 << FixItHint::CreateRemoval(P.getCurToken().getLocation());
+      }
+      ConstexprLoc = P.ConsumeToken();
+      DeclEndLoc = ConstexprLoc;
+      break /*switch*/;
+    default:
+      return ;
+    }
+  }
+}
+
+static void
+addConstexprToLambdaDeclSpecifier(Parser &P, SourceLocation ConstexprLoc,
+                                  DeclSpec &DS) {
+  if (ConstexprLoc.isValid()) {
+    P.Diag(ConstexprLoc, !P.getLangOpts().CPlusPlus1z
+                             ? diag::ext_constexpr_on_lambda_cxx1z
+                             : diag::warn_cxx14_compat_constexpr_on_lambda);
+    const char *PrevSpec = nullptr;
+    unsigned DiagID = 0;
+    DS.SetConstexprSpec(ConstexprLoc, PrevSpec, DiagID);
+    assert(PrevSpec == nullptr && DiagID == 0 &&
+           "Constexpr cannot have been set previously!");
+  }
+}
+
 /// ParseLambdaExpressionAfterIntroducer - Parse the rest of a lambda
 /// expression.
 ExprResult Parser::ParseLambdaExpressionAfterIntroducer(
                      LambdaIntroducer &Intro) {
   SourceLocation LambdaBeginLoc = Intro.Range.getBegin();
   Diag(LambdaBeginLoc, diag::warn_cxx98_compat_lambda);
 
   PrettyStackTraceLoc CrashInfo(PP.getSourceManager(), LambdaBeginLoc,

     // GNU-style attributes must be parsed before the mutable specifier to be
     // compatible with GCC.
     MaybeParseGNUAttributes(Attr, &DeclEndLoc);
 
     // MSVC-style attributes must be parsed before the mutable specifier to be
     // compatible with MSVC.
     MaybeParseMicrosoftDeclSpecs(Attr, &DeclEndLoc);
 
-    // Parse 'mutable'[opt].
+    // Parse mutable-opt and/or constexpr-opt, and update the DeclEndLoc.
     SourceLocation MutableLoc;
-    if (TryConsumeToken(tok::kw_mutable, MutableLoc))
-      DeclEndLoc = MutableLoc;
+    SourceLocation ConstexprLoc;
+    tryConsumeMutableOrConstexprToken(*this, MutableLoc, ConstexprLoc,
+                                      DeclEndLoc);
+    
+    addConstexprToLambdaDeclSpecifier(*this, ConstexprLoc, DS);
 
     // Parse exception-specification[opt].
     ExceptionSpecificationType ESpecType = EST_None;
     SourceRange ESpecRange;
     SmallVector<ParsedType, 2> DynamicExceptions;
     SmallVector<SourceRange, 2> DynamicExceptionRanges;
     ExprResult NoexceptExpr;
     CachedTokens *ExceptionSpecTokens;

                                            DynamicExceptionRanges.data(),
                                            DynamicExceptions.size(),
                                            NoexceptExpr.isUsable() ?
                                              NoexceptExpr.get() : nullptr,
                                            /*ExceptionSpecTokens*/nullptr,
                                            LParenLoc, FunLocalRangeEnd, D,
                                            TrailingReturnType),
                   Attr, DeclEndLoc);
-  } else if (Tok.isOneOf(tok::kw_mutable, tok::arrow, tok::kw___attribute) ||
+  } else if (Tok.isOneOf(tok::kw_mutable, tok::arrow, tok::kw___attribute,
+                         tok::kw_constexpr) ||
              (Tok.is(tok::l_square) && NextToken().is(tok::l_square))) {
     // It's common to forget that one needs '()' before 'mutable', an attribute
     // specifier, or the result type. Deal with this.
     unsigned TokKind = 0;
     switch (Tok.getKind()) {
     case tok::kw_mutable: TokKind = 0; break;
     case tok::arrow: TokKind = 1; break;
     case tok::kw___attribute:
     case tok::l_square: TokKind = 2; break;
+    case tok::kw_constexpr: TokKind = 3; break;
     default: llvm_unreachable("Unknown token kind");
     }
 
     Diag(Tok, diag::err_lambda_missing_parens)
       << TokKind
       << FixItHint::CreateInsertion(Tok.getLocation(), "() ");
     SourceLocation DeclLoc = Tok.getLocation();
     SourceLocation DeclEndLoc = DeclLoc;

lib/Sema/SemaLambda.cpp

     MangleNumbering = Ctx.createMangleNumberingContext();
   return *MangleNumbering;
 }
 
 CXXMethodDecl *Sema::startLambdaDefinition(CXXRecordDecl *Class,
                                            SourceRange IntroducerRange,
                                            TypeSourceInfo *MethodTypeInfo,
                                            SourceLocation EndLoc,
-                                           ArrayRef<ParmVarDecl *> Params) {
+                                           ArrayRef<ParmVarDecl *> Params,
+                                           const bool IsConstexprSpecified) {
   QualType MethodType = MethodTypeInfo->getType();
   TemplateParameterList *TemplateParams = 
             getGenericLambdaTemplateParameterList(getCurLambda(), *this);
   // If a lambda appears in a dependent context or is a generic lambda (has
   // template parameters) and has an 'auto' return type, deduce it to a 
   // dependent type.
   if (Class->isDependentContext() || TemplateParams) {
     const FunctionProtoType *FPT = MethodType->castAs<FunctionProtoType>();

   CXXMethodDecl *Method
     = CXXMethodDecl::Create(Context, Class, EndLoc,
                             DeclarationNameInfo(MethodName, 
                                                 IntroducerRange.getBegin(),
                                                 MethodNameLoc),
                             MethodType, MethodTypeInfo,
                             SC_None,
                             /*isInline=*/true,
-                            /*isConstExpr=*/false,
+                            IsConstexprSpecified,
                             EndLoc);
   Method->setAccess(AS_public);
   
   // Temporarily set the lexical declaration context to the current
   // context, so that the Scope stack matches the lexical nesting.
   Method->setLexicalDeclContext(CurContext);  
   // Create a function template if we have a template parameter list
   FunctionTemplateDecl *const TemplateMethod = TemplateParams ?

     // Check for unexpanded parameter packs in the method type.
     if (MethodTyInfo->getType()->containsUnexpandedParameterPack())
       ContainsUnexpandedParameterPack = true;
   }
 
   CXXRecordDecl *Class = createLambdaClosureType(Intro.Range, MethodTyInfo,
                                                  KnownDependent, Intro.Default);
 
-  CXXMethodDecl *Method = startLambdaDefinition(Class, Intro.Range,
-                                                MethodTyInfo, EndLoc, Params);
+  CXXMethodDecl *Method =
+      startLambdaDefinition(Class, Intro.Range, MethodTyInfo, EndLoc, Params,
+                            ParamInfo.getDeclSpec().isConstexprSpecified());
   if (ExplicitParams)
     CheckCXXDefaultArguments(Method);
   
   // Attributes on the lambda apply to the method.  
   ProcessDeclAttributes(CurScope, Method, ParamInfo);
   
   // Introduce the function call operator as the current declaration context.
   PushDeclContext(CurScope, Method);

 
   if (LambdaExprNeedsCleanups)
     ExprNeedsCleanups = true;
   
   LambdaExpr *Lambda = LambdaExpr::Create(Context, Class, IntroducerRange, 
                                           CaptureDefault, CaptureDefaultLoc,
                                           Captures, 
                                           ExplicitParams, ExplicitResultType,
                                           CaptureInits, ArrayIndexVars, 
                                           ArrayIndexStarts, EndLoc,
                                           ContainsUnexpandedParameterPack);
+  // If the lambda expression's call operator is not explicitly marked constexpr
+  // and we are not in a dependent context, analyze the call operator to infer
+  // its constexpr-ness, supressing diagnostics while doing so.
+  if (getLangOpts().CPlusPlus1z && !CallOperator->isInvalidDecl() &&
+      !CallOperator->isConstexpr() &&
+      !Class->getDeclContext()->isDependentContext()) {
+    TentativeAnalysisScope DiagnosticScopeGuard(*this);
+    CallOperator->setConstexpr(
+        CheckConstexprFunctionDecl(CallOperator) &&
+        CheckConstexprFunctionBody(CallOperator, CallOperator->getBody()));
+  }
 
   if (!CurContext->isDependentContext()) {
     switch (ExprEvalContexts.back().Context) {
     // C++11 [expr.prim.lambda]p2:
     //   A lambda-expression shall not appear in an unevaluated operand
     //   (Clause 5).
     case Unevaluated:
     case UnevaluatedAbstract:

lib/Sema/TreeTransform.h

                                         /*KnownDependent=*/false,
                                         E->getCaptureDefault());
   getDerived().transformedLocalDecl(E->getLambdaClass(), Class);
 
   // Build the call operator.
   CXXMethodDecl *NewCallOperator = getSema().startLambdaDefinition(
       Class, E->getIntroducerRange(), NewCallOpTSI,
       E->getCallOperator()->getLocEnd(),
-      NewCallOpTSI->getTypeLoc().castAs<FunctionProtoTypeLoc>().getParams());
+      NewCallOpTSI->getTypeLoc().castAs<FunctionProtoTypeLoc>().getParams(),
+      E->getCallOperator()->isConstexpr());
+
   LSI->CallOperator = NewCallOperator;
 
   getDerived().transformAttrs(E->getCallOperator(), NewCallOperator);
   getDerived().transformedLocalDecl(E->getCallOperator(), NewCallOperator);
 
   // Introduce the context of the call operator.
   Sema::ContextRAII SavedContext(getSema(), NewCallOperator,
                                  /*NewThisContext*/false);

test/Parser/cxx1z-constexpr-lambdas.cpp

+// RUN: %clang_cc1 -std=c++1z %s -verify 
+// RUN: %clang_cc1 -std=c++14 %s -verify 
+// RUN: %clang_cc1 -std=c++11 %s -verify 
+
+
+auto XL0 = [] constexpr { }; //expected-error{{requires '()'}} expected-error{{expected body}}
+auto XL1 = [] () mutable mutable mutable { }; //expected-error{{cannot appear multiple times}}
+
+#if __cplusplus > 201402L
+auto XL2 = [] () constexpr mutable constexpr { }; //expected-error{{cannot appear multiple times}}
+auto L = []() mutable constexpr { };
+auto L2 = []() constexpr { };
+auto L4 = []() constexpr mutable { }; 
+#else
+auto L = []() mutable constexpr {return 0; }; //expected-warning{{is a C++1z extension}}
+auto L2 = []() constexpr { return 0;};//expected-warning{{is a C++1z extension}}
+auto L4 = []() constexpr mutable { return 0; }; //expected-warning{{is a C++1z extension}}
+#endif
+
+

test/SemaCXX/cxx1z-constexpr-lambdas.cpp

+// RUN: %clang_cc1 -std=c++1z -verify -fsyntax-only -fblocks %s
+// RUN: %clang_cc1 -std=c++1z -verify -fsyntax-only -fblocks -fdelayed-template-parsing %s 
+// RUN: %clang_cc1 -std=c++1z -verify -fsyntax-only -fblocks -fms-extensions %s 
+// RUN: %clang_cc1 -std=c++1z -verify -fsyntax-only -fblocks -fdelayed-template-parsing -fms-extensions %s 
+
+namespace test_constexpr_checking {
+
+namespace ns1 {
+  struct NonLit { ~NonLit(); };  //expected-note{{not literal}}
+  auto L = [](NonLit NL) constexpr { }; //expected-error{{not a literal type}}
+} // end ns1
+
+namespace ns2 {
+  auto L = [](int I) constexpr { static int J;  }; //expected-error{{not permitted in a constexpr function}}
+} // end ns1
+
+} // end ns test_constexpr_checking
+
+namespace test_constexpr_call {
+
+namespace ns1 {
+  auto L = [](int I) { return I; };
+  static_assert(L(3) == 3);
+} // end ns1
+
+} // end ns test_constexpr_call
 No newline at end of file