Introduce an 'external_source_symbol' attribute that describes the origin and the nature of a declaration

[arphaman]

Summary

This patch adds an external_source_symbol attribute to Clang. This attribute specifies that a declaration originates
from an external source and describes the nature of that source.

This attribute is useful for mixed-language projects or project that use auto-generated code. For instance, Xcode can use this attribute to provide a correct 'jump-to-definition' feature. For a concrete example, consider a protocol that's defined in a Swift file:

@objc public protocol SwiftProtocol {
  func method()
}

This protocol can be used from Objective-C code by including a header file that was generated by the Swift compiler. The declarations in that header can use the external_source_symbol attribute to make Clang aware of the fact that SwiftProtocol actually originates from a Swift module:

__attribute__((external_source_symbol(language=Swift,defined_in="module")))
@protocol SwiftProtocol
@required
- (void) method;
@end

Consequently, when 'jump-to-definition' is performed at a location that references SwiftProtocol, Xcode can jump to the original definition in the Swift source file rather than jumping to the Objective-C declaration in the auto-generated header file.

his attribute uses custom parsing to specify a varying number of clauses such as 'language', 'defined_in' or 'generated_declaration'. The 'language' clause specifies the source language from which the declaration originates. The 'defined_in' clause specifies the name of the source container in which the declaration was defined (the exact definition of source container is language-specific, e.g. Swift's source containers are modules, so 'defined_in' should specify the Swift module name). The 'generated_declaration' is a flag that specifies that whether this declaration was automatically generated by some tool or not.

[aaron.ballman]

include/clang/Basic/Attr.td

532 ↗

(On Diff #87981)

Is this supported by GCC? If not, it should be under the clang namespace instead of the gnu namespace. If it is supported by GCC, then you should just use a single GCC spelling.

include/clang/Basic/AttrDocs.td

1007 ↗

(On Diff #87981)

Would this hold something like a file name? If so, I worry about conflicts between the comma separator and a file name -- you might want to pick a separator that can't be used in a file name (like | or :).

1015 ↗

(On Diff #87981)

Are these clauses parsed in a strict order? If so, you may want to mention that the order matters (or doesn't).

Also, the code in SemaDeclAttr.cpp implies that some of these are optional. It should be made clear which (if any) arguments are optional.

include/clang/Basic/DiagnosticParseKinds.td

863 ↗

(On Diff #87981)

I would drop the e.g. and instead describe what's really expected: an identifier. The e.g. muddies the water because it suggests there's a list of supported languages, and it shows something that looks kind of like a string when it isn't one. Something like: expected an identifier representing the source language or some such?

lib/Parse/ParseDecl.cpp

1141 ↗

(On Diff #87981)

You may want to assert this.

lib/Sema/SemaDeclAttr.cpp

2414 ↗

(On Diff #87981)

You should also diagnose if there's more than 3 arguments, no?

2418 ↗

(On Diff #87981)

This isn't really the right diagnostic for a mismatched attribute subject. It should be using warn_attribute_wrong_decl_type instead so that the user is more clear on why the attribute is ignored.

2428 ↗

(On Diff #87981)

Rather than rely on the implicit conversion to bool, I think this is a case where it should be tested against nullptr explicitly.

test/Parser/attr-external-source-symbol-cxx11.cpp

6 ↗

(On Diff #87981)

Please put this directly below the RUN line.

test/Parser/attr-external-source-symbol.m

26 ↗

(On Diff #87981)

I think you can currently get away with writing external_source_symbol(generated_declaration, generated_declaration, generated_declaration, defined_in="module")) and other odd combinations.

There should be additional parser tests for totally crazy parsing scenarios. You may want to consider running a fuzzer to generate some of them.

test/Sema/attr-external-source-symbol.c

5 ↗

(On Diff #87981)

Should also have a sema test for when there are 2 args, and 4 args.

There should also be a test under Misc that also checks that the args are properly lowered into the AST in the correct way with differing argument orders. e.g, external_source_symbol(generated_declaration, language=Swift, defined_in="module") vs external_source_symbol(language=Swift, generated_declaration, defined_in="module") vs external_source_symbol(defined_in="module", language=Swift, generated_declaration), etc.

[arphaman]

Thanks for the feedback! I've addressed the majority of your comments.

include/clang/Basic/AttrDocs.td

1007 ↗

(On Diff #87981)

It could potentially include a filename, yes.
I don't quite follow your concerns though.. If a comma is in a string literal then it's wrapped in quotes. Wouldn't that be enough to distinguish the comma separator token from the comma in a filename?

lib/Sema/SemaDeclAttr.cpp

2414 ↗

(On Diff #87981)

I think an assert would be more appropriate since I only use up to 3 arguments when creating the attribute, so I wouldn't be able to test the diagnostic.

[arphaman]

Ping.

[aaron.ballman]

include/clang/Basic/AttrDocs.td

1005 ↗

(On Diff #88222)

This being an identifier makes me wonder about languages that aren't a single token. For instance, how do you specify Objective-C or Objective-C++? What about C++? Visual Basic .NET?

Perhaps this should also be a string literal.

1007 ↗

(On Diff #87981)

You're correct, I had a brain fart. :-)

include/clang/Basic/DiagnosticSemaKinds.td

2713 ↗

(On Diff #88222)

I'm not too keen on this entry, because I'm not certain how many users really know what a "named declaration" is ("don't all declarations have names?" is a valid question for this to raise). However, I don't know of a better term to use, so it may be fine.

include/clang/Parse/Parser.h

146 ↗

(On Diff #88222)

Can we handle identifiers that have spaces in the name? I'm thinking about the case where Ident_defined_in is something like "My Long File Name With Spaces On Windows.cpp"

We should have something like that as a test to make sure it's handled properly. Same with odd values for Ident_language, like 'c++' or 'Objective-C', etc.

lib/Parse/ParseDecl.cpp

1090 ↗

(On Diff #88222)

expectAndConsume() instead of manually diagnosing?

1160 ↗

(On Diff #88222)

As an interesting case: if the attribute has two defined_in arguments and the first one is invalid, they will not get a diagnostic about the second one being a duplicate. e.g., __attribute__((external_source_symbol(defined_in = "1234"udl, defined_in = "1234")))

lib/Parse/ParseDeclCXX.cpp

3818–3819 ↗

(On Diff #88222)

I don't really like hard-coding a list of attributes like this. I think you should handle clang-namespaced attributes with a separate helper function.

lib/Sema/SemaDeclAttr.cpp

2420 ↗

(On Diff #88222)

This is incorrect -- you need to update the enumeration at the end of AttributeList.h and use an enumerator rather than a magic literal value.

2414 ↗

(On Diff #87981)

An assert is fine by me, but please add a string literal after the assert to explain why it fails.

test/Parser/attr-external-source-symbol.m

26 ↗

(On Diff #87981)

The test is still missing the weird parsing situations (like failing to use an = for arguments, etc). Basically, you should look at your custom parsing code and make sure each case you want to diagnose is actually being diagnosed (so we don't accidentally regress if we ever refactor the parsing code).

[arphaman]

I've addressed Aaron's comments and made the language a string literal.

include/clang/Basic/AttrDocs.td

1005 ↗

(On Diff #88222)

Good point, I've changed it to a string literal.

[aaron.ballman]

lib/Parse/ParseDecl.cpp

1161 ↗

(On Diff #89298)

Add a string literal to the assert?

lib/Parse/ParseDeclCXX.cpp

3818–3819 ↗

(On Diff #88222)

This still isn't quite what I was looking for -- the helper function I was talking about was a replacement for ParseGNUAttributeArgs(). I'm sorry if I was unclear.

I think we should have a ParseClangAttributeArgs() that handles your custom parsing; it can then call through to ParseAttributeArgsCommon() for common argument handling in the same way ParseGNUAttributeArgs() does. So the predicate would be else if (ScopeName && ScopeName->getName() == "clang").

[arphaman]

Use ParseClangAttributeArgs and add a string to an assert.

lib/Parse/ParseDeclCXX.cpp

3818–3819 ↗

(On Diff #88222)

I see, I added ParseClangAttributeArgs in the updated patch.

[aaron.ballman]

LGTM, despite a long-ish comment.

lib/Parse/ParseDeclCXX.cpp

3830–3837 ↗

(On Diff #90042)

This code is equivalent to the old code, which is good. However, it points out what I think may be a latent bug -- for attributes in the clang namespace that are also standard attributes, I'm not certain we should be generating parse errors for the clang-namespaced spelling. This only impacts [[fallthrough]] vs [[clang::fallthrough]], according to the implementation of IsBuiltInOrStandardCXX11Attribute().

All of that is to say: I'm not certain we need to hoist NumArgs to get to the call to IsBuiltInOrStandardCXX11Attribute() below, because attributes in the clang namespace are not built-in or standard attributes.

This requires a bit further exploration, but I won't have time to do that for a few weeks. I don't want to hold your patch up for that, so I think this is fine for now -- I can modify the code later when I have the chance to finish my exploration.

[arphaman]

lib/Parse/ParseDeclCXX.cpp

3830–3837 ↗

(On Diff #90042)

Thanks!

I agree with the comment about NumArgs, I wasn't sure if we need it for clang:: attributes as well.

cfe/trunk/include/clang/Basic/Attr.td

return llvm::StringSwitch<llvm::StringRef>(Platform)
              .Default(llvm::StringRef());
 } }];
   let HasCustomParsing = 1;
   let DuplicatesAllowedWhileMerging = 1;
 //  let Subjects = SubjectList<[Named]>;
   let Documentation = [AvailabilityDocs];
 }
 
+def ExternalSourceSymbol : InheritableAttr {
+  let Spellings = [GNU<"external_source_symbol">,
+                   CXX11<"clang", "external_source_symbol">];
+  let Args = [StringArgument<"language", 1>,
+              StringArgument<"definedIn", 1>,
+              BoolArgument<"generatedDeclaration", 1>];
+  let HasCustomParsing = 1;
+//  let Subjects = SubjectList<[Named]>;
+  let Documentation = [ExternalSourceSymbolDocs];
+}
+
 def Blocks : InheritableAttr {
   let Spellings = [GNU<"blocks">];
   let Args = [EnumArgument<"Type", "BlockType", ["byref"], ["ByRef"]>];
   let Documentation = [Undocumented];
 }
 
 def Bounded : IgnoredAttr {
   let Spellings = [GNU<"bounded">];

cfe/trunk/include/clang/Basic/AttrDocs.td

.. code-block:: objc
 
   @interface B : A
   - (id)method __attribute__((availability(macos,introduced=10.3))); // okay: method moved into base class later
   - (id)method __attribute__((availability(macos,introduced=10.5))); // error: this method was available via the base class in 10.4
   @end
   }];
 }
 
+def ExternalSourceSymbolDocs : Documentation {
+  let Category = DocCatFunction;
+  let Content = [{
+The ``external_source_symbol`` attribute specifies that a declaration originates
+from an external source and describes the nature of that source.
+
+The fact that Clang is capable of recognizing declarations that were defined
+externally can be used to provide better tooling support for mixed-language
+projects or projects that rely on auto-generated code. For instance, an IDE that
+uses Clang and that supports mixed-language projects can use this attribute to
+provide a correct 'jump-to-definition' feature. For a concrete example,
+consider a protocol that's defined in a Swift file:
+
+.. code-block:: swift
+
+  @objc public protocol SwiftProtocol {
+    func method()
+  }
+
+This protocol can be used from Objective-C code by including a header file that
+was generated by the Swift compiler. The declarations in that header can use
+the ``external_source_symbol`` attribute to make Clang aware of the fact
+that ``SwiftProtocol`` actually originates from a Swift module:
+
+.. code-block:: objc
+
+  __attribute__((external_source_symbol(language=Swift,defined_in="module")))
+  @protocol SwiftProtocol
+  @required
+  - (void) method;
+  @end
+
+Consequently, when 'jump-to-definition' is performed at a location that
+references ``SwiftProtocol``, the IDE can jump to the original definition in
+the Swift source file rather than jumping to the Objective-C declaration in the
+auto-generated header file.
+
+The ``external_source_symbol`` attribute is a comma-separated list that includes
+clauses that describe the origin and the nature of the particular declaration.
+Those clauses can be:
+
+language=\ *string-literal*
+  The name of the source language in which this declaration was defined.
+
+defined_in=\ *string-literal*
+  The name of the source container in which the declaration was defined. The
+  exact definition of source container is language-specific, e.g. Swift's
+  source containers are modules, so ``defined_in`` should specify the Swift
+  module name.
+
+generated_declaration
+  This declaration was automatically generated by some tool.
+
+The clauses can be specified in any order. The clauses that are listed above are
+all optional, but the attribute has to have at least one clause.
+  }];
+}
 
 def RequireConstantInitDocs : Documentation {
   let Category = DocCatVariable;
   let Content = [{
 This attribute specifies that the variable to which it is attached is intended
 to have a `constant initializer <http://en.cppreference.com/w/cpp/language/constant_initialization>`_
 according to the rules of [basic.start.static]. The variable is required to
 have static or thread storage duration. If the initialization of the variable

cfe/trunk/include/clang/Basic/DiagnosticCommonKinds.td

 
 let CategoryName = "Lexical or Preprocessor Issue" in {
 
 def err_expected_colon_after_setter_name : Error<
   "method name referenced in property setter attribute "
   "must end with ':'">;
 def err_expected_string_literal : Error<"expected string literal "
   "%select{in %1|for diagnostic message in static_assert|"
-          "for optional message in 'availability' attribute}0">;
+          "for optional message in 'availability' attribute|"
+          "for %select{language|source container}1 name in "
+          "'external_source_symbol' attribute}0">;
 def err_invalid_string_udl : Error<
   "string literal with user-defined suffix cannot be used here">;
 def err_invalid_character_udl : Error<
   "character literal with user-defined suffix cannot be used here">;
 def err_invalid_numeric_udl : Error<
   "numeric literal with user-defined suffix cannot be used here">;
 
 }

cfe/trunk/include/clang/Basic/DiagnosticParseKinds.td

def err_avail_query_unrecognized_platform_name : Error<
   "unrecognized platform name %0">;
 def err_availability_query_wildcard_required: Error<
   "must handle potential future platforms with '*'">;
 def err_availability_query_repeated_platform: Error<
   "version for '%0' already specified">;
 def err_availability_query_repeated_star : Error<
   "'*' query has already been specified">;
 
+// External source symbol attribute
+def err_external_source_symbol_expected_keyword : Error<
+  "expected 'language', 'defined_in', or 'generated_declaration'">;
+def err_external_source_symbol_duplicate_clause : Error<
+  "duplicate %0 clause in an 'external_source_symbol' attribute">;
+
 // Type safety attributes
 def err_type_safety_unknown_flag : Error<
   "invalid comparison flag %0; use 'layout_compatible' or 'must_be_null'">;
 
 // Type traits
 def err_type_trait_arity : Error<
   "type trait requires %0%select{| or more}1 argument%select{|s}2; have "
   "%3 argument%s3">;

cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td

def warn_attribute_wrong_decl_type : Warning<
   "|structs and typedefs"
   "|interface or protocol declarations"
   "|kernel functions"
   "|non-K&R-style functions"
   "|variables, enums, fields and typedefs"
   "|functions, methods, enums, and classes"
   "|structs, classes, variables, functions, and inline namespaces"
   "|variables, functions, methods, types, enumerations, enumerators, labels, and non-static data members"
-  "|classes and enumerations}1">,
+  "|classes and enumerations"
+  "|named declarations}1">,
   InGroup<IgnoredAttributes>;
 def err_attribute_wrong_decl_type : Error<warn_attribute_wrong_decl_type.Text>;
 def warn_type_attribute_wrong_type : Warning<
   "'%0' only applies to %select{function|pointer|"
   "Objective-C object or block pointer}1 types; type here is %2">,
   InGroup<IgnoredAttributes>;
 def warn_incomplete_encoded_type : Warning<
   "encoding of %0 type is incomplete because %1 component has unknown encoding">,

cfe/trunk/include/clang/Parse/Parser.h

class Parser : public CodeCompletionHandler {
   IdentifierInfo *Ident_message;
 
   /// \brief Identifier for "strict".
   IdentifierInfo *Ident_strict;
 
   /// \brief Identifier for "replacement".
   IdentifierInfo *Ident_replacement;
 
+  /// Identifiers used by the 'external_source_symbol' attribute.
+  IdentifierInfo *Ident_language, *Ident_defined_in,
+      *Ident_generated_declaration;
+
   /// C++0x contextual keywords.
   mutable IdentifierInfo *Ident_final;
   mutable IdentifierInfo *Ident_GNU_final;
   mutable IdentifierInfo *Ident_override;
 
   // C++ type trait keywords that can be reverted to identifiers and still be
   // used as type traits.
   llvm::SmallDenseMap<IdentifierInfo *, tok::TokenKind> RevertibleTypeTraits;
void ParseGNUAttributeArgs(IdentifierInfo *AttrName,
                              ParsedAttributes &Attrs,
                              SourceLocation *EndLoc,
                              IdentifierInfo *ScopeName,
                              SourceLocation ScopeLoc,
                              AttributeList::Syntax Syntax,
                              Declarator *D);
   IdentifierLoc *ParseIdentifierLoc();
 
+  unsigned
+  ParseClangAttributeArgs(IdentifierInfo *AttrName, SourceLocation AttrNameLoc,
+                          ParsedAttributes &Attrs, SourceLocation *EndLoc,
+                          IdentifierInfo *ScopeName, SourceLocation ScopeLoc,
+                          AttributeList::Syntax Syntax);
+
   void MaybeParseCXX11Attributes(Declarator &D) {
     if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
       ParsedAttributesWithRange attrs(AttrFactory);
       SourceLocation endLoc;
       ParseCXX11Attributes(attrs, &endLoc);
       D.takeAttributes(attrs, endLoc);
     }
   }
void ParseAvailabilityAttribute(IdentifierInfo &Availability,
                                   SourceLocation *endLoc,
                                   IdentifierInfo *ScopeName,
                                   SourceLocation ScopeLoc,
                                   AttributeList::Syntax Syntax);
 
   Optional<AvailabilitySpec> ParseAvailabilitySpec();
   ExprResult ParseAvailabilityCheckExpr(SourceLocation StartLoc);
 
+  void ParseExternalSourceSymbolAttribute(IdentifierInfo &ExternalSourceSymbol,
+                                          SourceLocation Loc,
+                                          ParsedAttributes &Attrs,
+                                          SourceLocation *EndLoc,
+                                          IdentifierInfo *ScopeName,
+                                          SourceLocation ScopeLoc,
+                                          AttributeList::Syntax Syntax);
+
   void ParseObjCBridgeRelatedAttribute(IdentifierInfo &ObjCBridgeRelated,
                                        SourceLocation ObjCBridgeRelatedLoc,
                                        ParsedAttributes &attrs,
                                        SourceLocation *endLoc,
                                        IdentifierInfo *ScopeName,
                                        SourceLocation ScopeLoc,
                                        AttributeList::Syntax Syntax);
 

cfe/trunk/include/clang/Sema/AttributeList.h

enum AttributeDeclKind {
   ExpectedObjectiveCInterfaceOrProtocol,
   ExpectedKernelFunction,
   ExpectedFunctionWithProtoType,
   ExpectedVariableEnumFieldOrTypedef,
   ExpectedFunctionMethodEnumOrClass,
   ExpectedStructClassVariableFunctionOrInlineNamespace,
   ExpectedForMaybeUnused,
   ExpectedEnumOrClass,
+  ExpectedNamedDecl,
 };
 
 }  // end namespace clang
 
 #endif

cfe/trunk/lib/Parse/ParseDecl.cpp

void Parser::ParseGNUAttributeArgs(IdentifierInfo *AttrName,
 
   AttributeList::Kind AttrKind =
       AttributeList::getKind(AttrName, ScopeName, Syntax);
 
   if (AttrKind == AttributeList::AT_Availability) {
     ParseAvailabilityAttribute(*AttrName, AttrNameLoc, Attrs, EndLoc, ScopeName,
                                ScopeLoc, Syntax);
     return;
+  } else if (AttrKind == AttributeList::AT_ExternalSourceSymbol) {
+    ParseExternalSourceSymbolAttribute(*AttrName, AttrNameLoc, Attrs, EndLoc,
+                                       ScopeName, ScopeLoc, Syntax);
+    return;
   } else if (AttrKind == AttributeList::AT_ObjCBridgeRelated) {
     ParseObjCBridgeRelatedAttribute(*AttrName, AttrNameLoc, Attrs, EndLoc,
                                     ScopeName, ScopeLoc, Syntax);
     return;
   } else if (AttrKind == AttributeList::AT_TypeTagForDatatype) {
     ParseTypeTagForDatatypeAttribute(*AttrName, AttrNameLoc, Attrs, EndLoc,
                                      ScopeName, ScopeLoc, Syntax);
     return;
for (unsigned i = 0; i != FTI.NumParams; ++i) {
       Actions.ActOnReenterCXXMethodParameter(getCurScope(), Param);
     }
   }
 
   ParseAttributeArgsCommon(AttrName, AttrNameLoc, Attrs, EndLoc, ScopeName,
                            ScopeLoc, Syntax);
 }
 
+unsigned Parser::ParseClangAttributeArgs(
+    IdentifierInfo *AttrName, SourceLocation AttrNameLoc,
+    ParsedAttributes &Attrs, SourceLocation *EndLoc, IdentifierInfo *ScopeName,
+    SourceLocation ScopeLoc, AttributeList::Syntax Syntax) {
+  assert(Tok.is(tok::l_paren) && "Attribute arg list not starting with '('");
+
+  AttributeList::Kind AttrKind =
+      AttributeList::getKind(AttrName, ScopeName, Syntax);
+
+  if (AttrKind == AttributeList::AT_ExternalSourceSymbol) {
+    ParseExternalSourceSymbolAttribute(*AttrName, AttrNameLoc, Attrs, EndLoc,
+                                       ScopeName, ScopeLoc, Syntax);
+    return Attrs.getList() ? Attrs.getList()->getNumArgs() : 0;
+  }
+
+  return ParseAttributeArgsCommon(AttrName, AttrNameLoc, Attrs, EndLoc,
+                                  ScopeName, ScopeLoc, Syntax);
+}
+
 bool Parser::ParseMicrosoftDeclSpecArgs(IdentifierInfo *AttrName,
                                         SourceLocation AttrNameLoc,
                                         ParsedAttributes &Attrs) {
   // If the attribute isn't known, we will not attempt to parse any
   // arguments.
   if (!hasAttribute(AttrSyntax::Declspec, nullptr, AttrName,
                     getTargetInfo(), getLangOpts())) {
     // Eat the left paren, then skip to the ending right paren.
attrs.addNew(&Availability,
                Platform,
                Changes[Introduced],
                Changes[Deprecated],
                Changes[Obsoleted],
                UnavailableLoc, MessageExpr.get(),
                Syntax, StrictLoc, ReplacementExpr.get());
 }
 
+/// \brief Parse the contents of the "external_source_symbol" attribute.
+///
+/// external-source-symbol-attribute:
+///   'external_source_symbol' '(' keyword-arg-list ')'
+///
+/// keyword-arg-list:
+///   keyword-arg
+///   keyword-arg ',' keyword-arg-list
+///
+/// keyword-arg:
+///   'language' '=' <string>
+///   'defined_in' '=' <string>
+///   'generated_declaration'
+void Parser::ParseExternalSourceSymbolAttribute(
+    IdentifierInfo &ExternalSourceSymbol, SourceLocation Loc,
+    ParsedAttributes &Attrs, SourceLocation *EndLoc, IdentifierInfo *ScopeName,
+    SourceLocation ScopeLoc, AttributeList::Syntax Syntax) {
+  // Opening '('.
+  BalancedDelimiterTracker T(*this, tok::l_paren);
+  if (T.expectAndConsume())
+    return;
+
+  // Initialize the pointers for the keyword identifiers when required.
+  if (!Ident_language) {
+    Ident_language = PP.getIdentifierInfo("language");
+    Ident_defined_in = PP.getIdentifierInfo("defined_in");
+    Ident_generated_declaration = PP.getIdentifierInfo("generated_declaration");
+  }
+
+  ExprResult Language;
+  bool HasLanguage = false;
+  ExprResult DefinedInExpr;
+  bool HasDefinedIn = false;
+  IdentifierLoc *GeneratedDeclaration = nullptr;
+
+  // Parse the language/defined_in/generated_declaration keywords
+  do {
+    if (Tok.isNot(tok::identifier)) {
+      Diag(Tok, diag::err_external_source_symbol_expected_keyword);
+      SkipUntil(tok::r_paren, StopAtSemi);
+      return;
+    }
+
+    SourceLocation KeywordLoc = Tok.getLocation();
+    IdentifierInfo *Keyword = Tok.getIdentifierInfo();
+    if (Keyword == Ident_generated_declaration) {
+      if (GeneratedDeclaration) {
+        Diag(Tok, diag::err_external_source_symbol_duplicate_clause) << Keyword;
+        SkipUntil(tok::r_paren, StopAtSemi);
+        return;
+      }
+      GeneratedDeclaration = ParseIdentifierLoc();
+      continue;
+    }
+
+    if (Keyword != Ident_language && Keyword != Ident_defined_in) {
+      Diag(Tok, diag::err_external_source_symbol_expected_keyword);
+      SkipUntil(tok::r_paren, StopAtSemi);
+      return;
+    }
+
+    ConsumeToken();
+    if (ExpectAndConsume(tok::equal, diag::err_expected_after,
+                         Keyword->getName())) {
+      SkipUntil(tok::r_paren, StopAtSemi);
+      return;
+    }
+
+    bool HadLanguage = HasLanguage, HadDefinedIn = HasDefinedIn;
+    if (Keyword == Ident_language)
+      HasLanguage = true;
+    else
+      HasDefinedIn = true;
+
+    if (Tok.isNot(tok::string_literal)) {
+      Diag(Tok, diag::err_expected_string_literal)
+          << /*Source='external_source_symbol attribute'*/ 3
+          << /*language | source container*/ (Keyword != Ident_language);
+      SkipUntil(tok::comma, tok::r_paren, StopAtSemi | StopBeforeMatch);
+      continue;
+    }
+    if (Keyword == Ident_language) {
+      if (HadLanguage) {
+        Diag(KeywordLoc, diag::err_external_source_symbol_duplicate_clause)
+            << Keyword;
+        ParseStringLiteralExpression();
+        continue;
+      }
+      Language = ParseStringLiteralExpression();
+    } else {
+      assert(Keyword == Ident_defined_in && "Invalid clause keyword!");
+      if (HadDefinedIn) {
+        Diag(KeywordLoc, diag::err_external_source_symbol_duplicate_clause)
+            << Keyword;
+        ParseStringLiteralExpression();
+        continue;
+      }
+      DefinedInExpr = ParseStringLiteralExpression();
+    }
+  } while (TryConsumeToken(tok::comma));
+
+  // Closing ')'.
+  if (T.consumeClose())
+    return;
+  if (EndLoc)
+    *EndLoc = T.getCloseLocation();
+
+  ArgsUnion Args[] = {Language.get(), DefinedInExpr.get(),
+                      GeneratedDeclaration};
+  Attrs.addNew(&ExternalSourceSymbol, SourceRange(Loc, T.getCloseLocation()),
+               ScopeName, ScopeLoc, Args, llvm::array_lengthof(Args), Syntax);
+}
+
 /// \brief Parse the contents of the "objc_bridge_related" attribute.
 /// objc_bridge_related '(' related_class ',' opt-class_method ',' opt-instance_method ')'
 /// related_class:
 ///     Identifier
 ///
 /// opt-class_method:
 ///     Identifier: | <empty>
 ///

cfe/trunk/lib/Parse/ParseDeclCXX.cpp

bool Parser::ParseCXX11AttributeArgs(IdentifierInfo *AttrName,
   if (!hasAttribute(AttrSyntax::CXX, ScopeName, AttrName,
                     getTargetInfo(), getLangOpts())) {
     // Eat the left paren, then skip to the ending right paren.
     ConsumeParen();
     SkipUntil(tok::r_paren);
     return false;
   }
 
-  if (ScopeName && ScopeName->getName() == "gnu")
+  if (ScopeName && ScopeName->getName() == "gnu") {
     // GNU-scoped attributes have some special cases to handle GNU-specific
     // behaviors.
     ParseGNUAttributeArgs(AttrName, AttrNameLoc, Attrs, EndLoc, ScopeName,
                           ScopeLoc, AttributeList::AS_CXX11, nullptr);
-  else {
-    unsigned NumArgs =
+    return true;
+  }
+
+  unsigned NumArgs;
+  // Some Clang-scoped attributes have some special parsing behavior.
+  if (ScopeName && ScopeName->getName() == "clang")
+    NumArgs =
+        ParseClangAttributeArgs(AttrName, AttrNameLoc, Attrs, EndLoc, ScopeName,
+                                ScopeLoc, AttributeList::AS_CXX11);
+  else
+    NumArgs =
         ParseAttributeArgsCommon(AttrName, AttrNameLoc, Attrs, EndLoc,
                                  ScopeName, ScopeLoc, AttributeList::AS_CXX11);
-    
-    const AttributeList *Attr = Attrs.getList();
-    if (Attr && IsBuiltInOrStandardCXX11Attribute(AttrName, ScopeName)) {
-      // If the attribute is a standard or built-in attribute and we are
-      // parsing an argument list, we need to determine whether this attribute
-      // was allowed to have an argument list (such as [[deprecated]]), and how
-      // many arguments were parsed (so we can diagnose on [[deprecated()]]).
-      if (Attr->getMaxArgs() && !NumArgs) {
-        // The attribute was allowed to have arguments, but none were provided
-        // even though the attribute parsed successfully. This is an error.
-        Diag(LParenLoc, diag::err_attribute_requires_arguments) << AttrName;
-        Attr->setInvalid(true);
-      } else if (!Attr->getMaxArgs()) {
-        // The attribute parsed successfully, but was not allowed to have any
-        // arguments. It doesn't matter whether any were provided -- the
-        // presence of the argument list (even if empty) is diagnosed.
-        Diag(LParenLoc, diag::err_cxx11_attribute_forbids_arguments)
-            << AttrName
-            << FixItHint::CreateRemoval(SourceRange(LParenLoc, *EndLoc));
-        Attr->setInvalid(true);
-      }
-    }
-  }
   return true;
 }
 
 /// ParseCXX11AttributeSpecifier - Parse a C++11 attribute-specifier.
 ///
 /// [C++11] attribute-specifier:
 ///         '[' '[' attribute-list ']' ']'
 ///         alignment-specifier

cfe/trunk/lib/Parse/Parser.cpp

void Parser::Initialize() {
 
   Ident_introduced = nullptr;
   Ident_deprecated = nullptr;
   Ident_obsoleted = nullptr;
   Ident_unavailable = nullptr;
   Ident_strict = nullptr;
   Ident_replacement = nullptr;
 
+  Ident_language = Ident_defined_in = Ident_generated_declaration = nullptr;
+
   Ident__except = nullptr;
 
   Ident__exception_code = Ident__exception_info = nullptr;
   Ident__abnormal_termination = Ident___exception_code = nullptr;
   Ident___exception_info = Ident___abnormal_termination = nullptr;
   Ident_GetExceptionCode = Ident_GetExceptionInfo = nullptr;
   Ident_AbnormalTermination = nullptr;
 

cfe/trunk/lib/Sema/SemaDeclAttr.cpp

if (NewII) {
                                                             Sema::AMK_None,
                                                             Index);
         if (NewAttr)
           D->addAttr(NewAttr);
       }
   }
 }
 
+static void handleExternalSourceSymbolAttr(Sema &S, Decl *D,
+                                           const AttributeList &Attr) {
+  if (!checkAttributeAtLeastNumArgs(S, Attr, 1))
+    return;
+  assert(checkAttributeAtMostNumArgs(S, Attr, 3) &&
+         "Invalid number of arguments in an external_source_symbol attribute");
+
+  if (!isa<NamedDecl>(D)) {
+    S.Diag(Attr.getLoc(), diag::warn_attribute_wrong_decl_type)
+        << Attr.getName() << ExpectedNamedDecl;
+    return;
+  }
+
+  StringRef Language;
+  if (const auto *SE = dyn_cast_or_null<StringLiteral>(Attr.getArgAsExpr(0)))
+    Language = SE->getString();
+  StringRef DefinedIn;
+  if (const auto *SE = dyn_cast_or_null<StringLiteral>(Attr.getArgAsExpr(1)))
+    DefinedIn = SE->getString();
+  bool IsGeneratedDeclaration = Attr.getArgAsIdent(2) != nullptr;
+
+  D->addAttr(::new (S.Context) ExternalSourceSymbolAttr(
+      Attr.getRange(), S.Context, Language, DefinedIn, IsGeneratedDeclaration,
+      Attr.getAttributeSpellingListIndex()));
+}
+
 template <class T>
 static T *mergeVisibilityAttr(Sema &S, Decl *D, SourceRange range,
                               typename T::VisibilityType value,
                               unsigned attrSpellingListIndex) {
   T *existingAttr = D->getAttr<T>();
   if (existingAttr) {
     typename T::VisibilityType existingValue = existingAttr->getVisibility();
     if (existingValue == value)
case AttributeList::AT_EnableIf:
     handleEnableIfAttr(S, D, Attr);
     break;
   case AttributeList::AT_DiagnoseIf:
     handleDiagnoseIfAttr(S, D, Attr);
     break;
   case AttributeList::AT_ExtVectorType:
     handleExtVectorTypeAttr(S, scope, D, Attr);
     break;
+  case AttributeList::AT_ExternalSourceSymbol:
+    handleExternalSourceSymbolAttr(S, D, Attr);
+    break;
   case AttributeList::AT_MinSize:
     handleMinSizeAttr(S, D, Attr);
     break;
   case AttributeList::AT_OptimizeNone:
     handleOptimizeNoneAttr(S, D, Attr);
     break;
   case AttributeList::AT_FlagEnum:
     handleSimpleAttribute<FlagEnumAttr>(S, D, Attr);

cfe/trunk/test/Misc/ast-dump-attr.cpp

void f() {
   // CHECK: IndirectFieldDecl{{.*}}Test 'int'
   // CHECK: DeprecatedAttr
 }
 }
 
 struct __attribute__((objc_bridge_related(NSParagraphStyle,,))) TestBridgedRef;
 // CHECK: CXXRecordDecl{{.*}} struct TestBridgedRef
 // CHECK-NEXT: ObjCBridgeRelatedAttr{{.*}} NSParagraphStyle
+
+void TestExternalSourceSymbolAttr1()
+__attribute__((external_source_symbol(language="Swift", defined_in="module", generated_declaration)));
+// CHECK: FunctionDecl{{.*}} TestExternalSourceSymbolAttr1
+// CHECK-NEXT: ExternalSourceSymbolAttr{{.*}} "Swift" "module" GeneratedDeclaration
+
+void TestExternalSourceSymbolAttr2()
+__attribute__((external_source_symbol(defined_in="module", language="Swift")));
+// CHECK: FunctionDecl{{.*}} TestExternalSourceSymbolAttr2
+// CHECK-NEXT: ExternalSourceSymbolAttr{{.*}} "Swift" "module"{{$}}
+
+void TestExternalSourceSymbolAttr3()
+__attribute__((external_source_symbol(generated_declaration, language="Objective-C++", defined_in="module")));
+// CHECK: FunctionDecl{{.*}} TestExternalSourceSymbolAttr3
+// CHECK-NEXT: ExternalSourceSymbolAttr{{.*}} "Objective-C++" "module" GeneratedDeclaration
+
+void TestExternalSourceSymbolAttr4()
+__attribute__((external_source_symbol(defined_in="Some external file.cs", generated_declaration, language="C Sharp")));
+// CHECK: FunctionDecl{{.*}} TestExternalSourceSymbolAttr4
+// CHECK-NEXT: ExternalSourceSymbolAttr{{.*}} "C Sharp" "Some external file.cs" GeneratedDeclaration
+
+void TestExternalSourceSymbolAttr5()
+__attribute__((external_source_symbol(generated_declaration, defined_in="module", language="Swift")));
+// CHECK: FunctionDecl{{.*}} TestExternalSourceSymbolAttr5
+// CHECK-NEXT: ExternalSourceSymbolAttr{{.*}} "Swift" "module" GeneratedDeclaration

cfe/trunk/test/Parser/attr-external-source-symbol-cxx11.cpp

+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 %s
+// expected-no-diagnostics
+
+[[clang::external_source_symbol(language="Swift", defined_in="module")]]
+void function() { }

cfe/trunk/test/Parser/attr-external-source-symbol.m

+// RUN: %clang_cc1 -fsyntax-only -verify %s
+
+void function() __attribute__((external_source_symbol(language="Swift", defined_in="module", generated_declaration)));
+
+__attribute__((external_source_symbol(language="Swift", defined_in="module")))
+@interface I
+
+- (void)method __attribute__((external_source_symbol(defined_in= "module")));
+
+@end
+
+enum E {
+  CaseA __attribute__((external_source_symbol(generated_declaration))),
+  CaseB __attribute__((external_source_symbol(generated_declaration, language="Swift")))
+} __attribute__((external_source_symbol(language = "Swift")));
+
+void f2()
+__attribute__((external_source_symbol())); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+void f3()
+__attribute__((external_source_symbol(invalid))); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+void f4()
+__attribute__((external_source_symbol(language))); // expected-error {{expected '=' after language}}
+void f5()
+__attribute__((external_source_symbol(language=))); // expected-error {{expected string literal for language name in 'external_source_symbol' attribute}}
+void f6()
+__attribute__((external_source_symbol(defined_in=20))); // expected-error {{expected string literal for source container name in 'external_source_symbol' attribute}}
+
+void f7()
+__attribute__((external_source_symbol(generated_declaration, generated_declaration))); // expected-error {{duplicate 'generated_declaration' clause in an 'external_source_symbol' attribute}}
+void f8()
+__attribute__((external_source_symbol(language="Swift", language="Swift"))); // expected-error {{duplicate 'language' clause in an 'external_source_symbol' attribute}}
+void f9()
+__attribute__((external_source_symbol(defined_in="module", language="Swift", defined_in="foo"))); // expected-error {{duplicate 'defined_in' clause in an 'external_source_symbol' attribute}}
+
+void f10()
+__attribute__((external_source_symbol(generated_declaration, language="Swift", defined_in="foo", generated_declaration, generated_declaration, language="Swift"))); // expected-error {{duplicate 'generated_declaration' clause in an 'external_source_symbol' attribute}}
+
+void f11()
+__attribute__((external_source_symbol(language="Objective-C++", defined_in="Some file with spaces")));
+
+void f12()
+__attribute__((external_source_symbol(language="C Sharp", defined_in="file:////Hello world with spaces. cs")));
+
+void f13()
+__attribute__((external_source_symbol(language=Swift))); // expected-error {{expected string literal for language name in 'external_source_symbol' attribute}}
+
+void f14()
+__attribute__((external_source_symbol(=))); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+
+void f15()
+__attribute__((external_source_symbol(="Swift"))); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+
+void f16()
+__attribute__((external_source_symbol("Swift", "module", generated_declaration))); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+
+void f17()
+__attribute__((external_source_symbol(language="Swift", "generated_declaration"))); // expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}
+
+void f18()
+__attribute__((external_source_symbol(language= =))); // expected-error {{expected string literal for language name in 'external_source_symbol' attribute}}
+
+void f19()
+__attribute__((external_source_symbol(defined_in="module" language="swift"))); // expected-error {{expected ')'}} expected-note {{to match this '('}}
+
+void f20()
+__attribute__((external_source_symbol(defined_in="module" language="swift" generated_declaration))); // expected-error {{expected ')'}} expected-note {{to match this '('}}
+
+void f21()
+__attribute__((external_source_symbol(defined_in= language="swift"))); // expected-error {{expected string literal for source container name in 'external_source_symbol' attribute}}
+
+void f22()
+__attribute__((external_source_symbol)); // expected-error {{'external_source_symbol' attribute takes at least 1 argument}}
+
+void f23()
+__attribute__((external_source_symbol(defined_in=, language="swift" generated_declaration))); // expected-error {{expected string literal for source container name in 'external_source_symbol' attribute}} expected-error{{expected ')'}} expected-note{{to match this '('}}
+
+void f24()
+__attribute__((external_source_symbol(language = generated_declaration))); // expected-error {{expected string literal for language name in 'external_source_symbol' attribute}}
+
+void f25()
+__attribute__((external_source_symbol(defined_in=123, defined_in="module"))); // expected-error {{expected string literal for source container name in 'external_source_symbol'}} expected-error {{duplicate 'defined_in' clause in an 'external_source_symbol' attribute}}
+
+void f26()
+__attribute__((external_source_symbol(language=Swift, language="Swift", error))); // expected-error {{expected string literal for language name in 'external_source_symbol'}} expected-error {{duplicate 'language' clause in an 'external_source_symbol' attribute}} expected-error {{expected 'language', 'defined_in', or 'generated_declaration'}}

cfe/trunk/test/Sema/attr-external-source-symbol.c

+// RUN: %clang_cc1 -fsyntax-only -fblocks -verify %s
+
+void threeClauses() __attribute__((external_source_symbol(language="Swift", defined_in="module", generated_declaration)));
+
+void twoClauses() __attribute__((external_source_symbol(language="Swift", defined_in="module")));
+
+void fourClauses() __attribute__((external_source_symbol(language="Swift", defined_in="module", generated_declaration, generated_declaration))); // expected-error {{duplicate 'generated_declaration' clause in an 'external_source_symbol' attribute}}
+
+void oneClause() __attribute__((external_source_symbol(generated_declaration)));
+
+void noArguments()
+__attribute__((external_source_symbol)); // expected-error {{'external_source_symbol' attribute takes at least 1 argument}}
+
+void namedDeclsOnly() {
+  int (^block)(void) = ^ (void)
+    __attribute__((external_source_symbol(language="Swift"))) { // expected-warning {{'external_source_symbol' attribute only applies to named declarations}}
+      return 1;
+  };
+}