APINotes: add APINotesYAMLCompiler

[compnerd]

Summary

This adds the skeleton of the YAML Compiler for APINotes. This change
only adds the YAML IO model for the API Notes along with a new testing
tool apinotes-test which can be used to verify that can round trip the
YAML content properly. It provides the basis for the future work which
will add a binary serialization and deserialization format to the data
model.

[compnerd]

Ping

[martong]

Seems like this patch handles already existing attributes in Clang, so this might not be affected by the concerns brought up here by James:
http://lists.llvm.org/pipermail/cfe-dev/2020-October/066996.html

clang/include/clang/APINotes/APINotesYAMLCompiler.h

17

It would be useful to have more documentation here.

clang/include/clang/APINotes/Types.h

2

So we are mapping existing attributes here, I am missing this to be documented. Why do we support only these attributes?
Would be consistent to put SwiftPrivate here as well, instead of implicitly interpreting that as bool in APINotesYAMLCompiler.cpp, wouldn't it?
I think we should list all processed attributes here, or we should just use Attrs.inc (see below my comment as well).

26

This seems a bit redundant to Attrs.td. I'd prefer to have the structure of an attribute defined only in one place. Why can't we directly reuse the types in the generated Attrs.inc? In this case

class EnumExtensibilityAttr : public InheritableAttr {
public:
  enum Kind {
    Closed,
    Open
  };

could perfectly fit, wouldn't it?

33

Should this be rather SwiftNewTypeKind? Since swift_wrapper is deprecated according to the docs: https://clang.llvm.org/docs/AttributeReference.html#swift-newtype

clang/lib/APINotes/APINotesYAMLCompiler.cpp

241

Why are these classes in a unnamed namespace? I'd expect them to be in the header under the clang::api_notes namespace, so users of the APINotesYAMLCompiler could use these as the structured form of the YAML content. Isn't that the point of the whole stuff?

440

Hmm, why do we need "none"? Can't we interpret the non-existence as "none"?

622

I think the stream (llvm::outs) should not be written in stone. And why not llvm::errs (we dump to errs usually) ? Could it be a parameter?

clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes

2

I am pretty sure this does not provide a full coverage. What about e.g Functions? I'd like to see them tested as well.

clang/test/APINotes/yaml-roundtrip.test

5

Why do we have this difference? This seems odd and as a superfluous complexity.

clang/tools/apinotes-test/APINotesTest.cpp

23

auto -> std::string

[compnerd]

Thanks for the feedback!

clang/include/clang/APINotes/Types.h

2

These are the ones that are currently needed and can be safely merged. I really wouldn't want to extend the support to all the attributes in the initial attempt to merge the functionality as it is something which is actually in production already. However, once the functionality is in a shared state, it is much easier to revise and co-evolve other consumers.

clang/lib/APINotes/APINotesYAMLCompiler.cpp

241

There will be follow up types that provide structured access to the data. These types are strictly for the serialization to and from YAML via YAML::IO.

440

At the very least we need it for compatibility - this is already a shipping feature. However, nullability is also not completely annotated. So, there is some benefit in tracking the explicit none vs missing.

622

Sure, that is reasonable, I should be able to add a llvm::raw_ostream & parameter.

clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes

2

Correct, it isn't meant to provide full coverage at all. It is merely meant to be enough to ensure that we can load the YAML and process it. The full coverage will come with the follow up patches as they will require filling out more of the infrastructure. I am trying to keep the patch at a reasonable size. I can add additional test cases if you feel strongly that they should be added now, but, I do worry about the size of the patch ballooning.

clang/test/APINotes/yaml-roundtrip.test

5

The difference is needed for compatibility. Richard wanted the fully spelt out names, but that requires backwards compatibility, so we need the difference here.

[martong]

as it is something which is actually in production already
...
At the very least we need it for compatibility - this is already a shipping feature.

I value that you guys have a prototype on a fork that is being used in production. But, upstreaming and these reviews give us the chance to evolve and to create an implementation that serves well the whole Clang community. I am having a hard time to accept "this is how it is implemented in our fork" as a technical argument. Besides, I am not sure how could the Clang community benefit about being backward compatible with a specialized fork and thus making superfluous complications. This feature could be really valuable, but I'd like to have it landed in a high quality form which serves the whole community (and the static analyzer developers). I think that a simple copy-paste of the fork will not do it.

clang/include/clang/APINotes/Types.h

2

I understand that an initial implementation may not support all attributes. What's more, an evolved implementation may not do that either.

Including Attrs.inc does not put any requirement to support all attributes, but we could reuse the types and enums there. This way we could avoid the need to mirror the types in Attrs.inc.

clang/lib/APINotes/APINotesYAMLCompiler.cpp

241

Sounds good. Could you please describe in a nutshell which are the following steps for the upstreaming? I mean it could make the review easier if we could see a kind of road-map for the upcoming patches.

440

Optional<EnumExtensibilityAttr::Kind> ?

clang/test/APINotes/yaml-roundtrip.test

5

I have concerns about making things more complicated just to be compatible with a downstream fork.

[martong]

Adding @NoQ as a reviewer, he might have some valuable comments, as he is very keen on this feature.

[compnerd]

In D88859#2339159, @martong wrote:

I value that you guys have a prototype on a fork that is being used in production. But, upstreaming and these reviews give us the chance to evolve and to create an implementation that serves well the whole Clang community. I am having a hard time to accept "this is how it is implemented in our fork" as a technical argument. Besides, I am not sure how could the Clang community benefit about being backward compatible with a specialized fork and thus making superfluous complications. This feature could be really valuable, but I'd like to have it landed in a high quality form which serves the whole community (and the static analyzer developers). I think that a simple copy-paste of the fork will not do it.

I'm not against evolving the implementation. However, compatibility with production code is important. I don't think that it is possible to ignore that, especially when the impact is non-trivial (it is not a small localized use). The large scale usage is valuable in terms of demonstrating the value of the functionality, so it is valuable to clang community. I am not suggesting just a copy from the Swift implementation, and do care about it being a high quality implementation. But that cannot come at the cost of compatibility with the existing usage.

clang/include/clang/APINotes/Types.h

2

From that description, I think I am not necessarily following what you are suggesting. Could you be a bit more concrete in what you are suggesting in terms of the reuse?

clang/lib/APINotes/APINotesYAMLCompiler.cpp

241

I am still digesting the complete implementation, which is extremely large.

Currently, my thought is to split up it up with this change which only deals with the YAML aspect, then follow that up with the processing that is required:

• conversion to bitcode
• conversion from bitcode

And then the work to associate the attributes from the deserialized form to the AST.

I am trying to split this up into smaller chunks so that it can be adequately reviewed rather than just be a dump of a feature and then have to rework it to be something that makes sense in the upstream tree.

I also feel uncomfortable with the current version in Swift being merged as I can't seem to understand the testing aspect well enough, which is why I introduced the apinotes-test-tool so that we can be sure that the pieces can be tested.

440

That representation could work, let me see if I can get YAML::IO to provide something which would be compatible.

clang/test/APINotes/yaml-roundtrip.test

5

The compatibility is extremely important unfortunately. We could go with just the terse version for the time being if you are strongly opposed to the two spellings.

[gribozavr2]

I am having a hard time to accept "this is how it is implemented in our fork" as a technical argument. Besides, I am not sure how could the Clang community benefit about being backward compatible with a specialized fork and thus making superfluous complications.

Being compatible with Apple's SDKs is quite important for some part of the community.

This is similar to Clang implementing GCC and MSVC attributes, for example. Some might call them specialized attributes -- they are non-standard and are certainly not used by every C++ user -- but there exists a sizeable population of downstream consumers that rely on them. Apple's SDKs are in the same class.

This feature could be really valuable, but I'd like to have it landed in a high quality form which serves the whole community (and the static analyzer developers). I think that a simple copy-paste of the fork will not do it.

I completely agree that the feature should be useful for static analysis. Do compatibility aliases hurt these goals?

[compnerd]

clang/lib/APINotes/APINotesYAMLCompiler.cpp

440

The representation is already Optional<EnumExtensibilityAttr::Kind>, but we need to map the user providing none back to a llvm::None so this will still need to be listed as is.

[compnerd]

Address review comments.

[compnerd]

Add more testing coverage

[compnerd]

clang/include/clang/APINotes/Types.h

26

The none-case here is not the same as missing - it tracks the explicitly audited case. I suppose we can change the internal enum case from None to Audited if you like.

[martong]

In D88859#2354377, @gribozavr2 wrote:

I am having a hard time to accept "this is how it is implemented in our fork" as a technical argument. Besides, I am not sure how could the Clang community benefit about being backward compatible with a specialized fork and thus making superfluous complications.

Being compatible with Apple's SDKs is quite important for some part of the community.

This is similar to Clang implementing GCC and MSVC attributes, for example. Some might call them specialized attributes -- they are non-standard and are certainly not used by every C++ user -- but there exists a sizeable population of downstream consumers that rely on them. Apple's SDKs are in the same class.

Fair enough. But I don't think that Clang developers just copied the implementation from GCC or from MSVC.
I accept to keep the interface of the feature (i.e. the YAML format) as it is in the fork, okay. But, I'd like to have maximum flexibility from the submitter by willing to change the details of the implementation. I don't think that the implementation in the fork is necessarily the best realization, we can do better, after all that's the purpose of this review, isn't it?

This feature could be really valuable, but I'd like to have it landed in a high quality form which serves the whole community (and the static analyzer developers). I think that a simple copy-paste of the fork will not do it.

I completely agree that the feature should be useful for static analysis. Do compatibility aliases hurt these goals?

No. But they make the tests and the whole system more complicated, IMHO.

clang/include/clang/APINotes/Types.h

2

I was thinking about to simply
#include "clang/AST/Attr.h
(which further includes Attrs.inc).

This way the class clang::EnumExtensibilityAttr and many other attribute classes are immediately available for your use. And you don't have to define very similar classes here again (unless I misunderstand something).

26

I am not sure I can follow. Could you please elaborate what is an "explicit y audited" case?
https://clang.llvm.org/docs/AttributeReference.html#enum-extensibility mentions only open/closed ...

clang/test/APINotes/yaml-roundtrip.test

5

I'd rather stick with the terse version if possible (do we have to pursue rsmith about this?) If this is really a hassle then let's forget about it.

[gribozavr2]

Fair enough. But I don't think that Clang developers just copied the implementation from GCC or from MSVC.

No (nor we could copy the implementation for a number of reasons). However, we did have to be bug-for-bug compatible sometimes for compatibility.

[compnerd]

I'd like to have maximum flexibility from the submitter by willing to change the details of the implementation

I don't think that is a fair expectation - there are plenty of times where this is technical disagreement on functionality, and it doesn't get immediately resolved. A high quality implementation doesn't require that everything be changed immediately, nor does it mean that everything must be completely flexible. There are trade-offs, and the goal is to strike a balance between simplicity and the needs of the project.

clang/include/clang/APINotes/Types.h

26

There are three states consider:

enum Unaudited {
};
enum __attribute__((__enum_extensibility__(open))) Open {
};
enum __attribute__((__enum_extensibility__(closed))) Closed {
};

The optionality of the value indicates whether the value is present or not in the APINotes, not the tri-state nature of the attribute.

[martong]

In D88859#2359399, @compnerd wrote:

I'd like to have maximum flexibility from the submitter by willing to change the details of the implementation

I don't think that is a fair expectation - there are plenty of times where this is technical disagreement on functionality, and it doesn't get immediately resolved. A high quality implementation doesn't require that everything be changed immediately, nor does it mean that everything must be completely flexible. There are trade-offs, and the goal is to strike a balance between simplicity and the needs of the project.

Yes, absolutely, flexibility is needed from both sides (submitter and reviewer). It is our common interest to get an extensible implementation upstreamed. I hope I didn't push it too hard, I didn't mean any offence.

clang/include/clang/APINotes/Types.h

26

Ok. Thanks for the explanation.

But how could we describe then flag_enum? As a possible extension in the future (not in this patch).
E.g.:

enum __attribute__((enum_extensibility(open),flag_enum)) OpenFlagEnum {
  D0 = 1 << 0, D1 = 1 << 1
};

Or here

enum __attribute__((flag_enum)) Foo;

should Foo have an Audited or None Kind here?

If we were to add support to flag_enum would you create a new enum class for that here?

Note that I am asking all these questions because I am planning to extend and add more attributes in the future once the whole APINotes stuff is landed.

[martong]

clang/lib/APINotes/APINotesYAMLCompiler.cpp

34

Could you please clang-format the code? The Lint: Pre-merge checks are all over the code and makes the review a bit harder.

[compnerd]

clang/include/clang/APINotes/Types.h

26

The questions are reasonable - and we should understand what is missing and what can be done and what cannot be done.

I think that the use of the metadata is extremely important. The question is, are there any users who care about recovering that information? IMO, we should try to represent the input as faithfully as possible, even if that means that we need to replicate the enumerations. However, if the consumers do not care about the inferred state vs the written state, I don't think that it is an immediate concern if we cannot recover that from the representation, we can simplify and extend as and when needed, especially when we have the feature merged (I think that it is easier once merged only because this is a large feature, and there is a large user base that is using this, which complicates things. Really, this functionality should have been merged a long time ago, but that is both my own opinion and past), and hopefully in many cases we can do it during review time.

The tricky bit to remember that there is an additional state that is implicitly being added here, and optionality should reference the contents of the APINotes, not the state in the declaration that is being mutated.

Is there a good place to write this down? I am willing to add comments explaining this so that the next person (ha, no, not the next person, but me, because I *will* forget) is not tripped up on this nuance.

[compnerd]

clang/lib/APINotes/APINotesYAMLCompiler.cpp

34

Sure, though, I do think that some of the extra whitespace makes it easier to read.

[martong]

clang/include/clang/APINotes/Types.h

26

Thanks for clarifying this. It was not clear for me that these types were to represent the YAML file contents. It makes much more sense now.

Is there a good place to write this down?

I think a file comment right after the license could do it.

[compnerd]

rebase, clang-format, add comments

[martong]

Looks good to me now! Thanks for addressing my concerns.

clang/tools/apinotes-test/APINotesTest.cpp

16

We still have: clang-format: please reformat the code. (Mabe you already know, but if you use arc then all these formatting issues are handled automatically and it will upload a linted patch to Phabricator.)

[compnerd]

Thanks for the reviews! I'll reformat the rest of the code before merging. Im going to be working on merging this downstream first before merging this upstream, so it will take a couple of days still.

[thakis]

Looks like the test doesn't pass on Windows: http://45.33.8.238/win/27342/step_7.txt

Please take a look, and revert if it takes a while to fix.

Here's the output of the diff on that Win bot:

thakis@thakis6-w MINGW64 /c/src/llvm-project (merge)
$ diff --strip-trailing-cr out/gn/obj/clang/test/APINotes/Output/yaml-roundtrip.test.tmp.result clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes
1d0
< ---
8c7
<         Nullability:     Nonnull
---
>         Nullability:     N
14c13
<         Nullability:     Optional
---
>         Nullability:     O
20c19
<         Nullability:     Unspecified
---
>         Nullability:     U
26c25
<         Nullability:     Unspecified
---
>         Nullability:     S
29,30c28
<         Nullability:     Unspecified
< ...
---
>         Nullability:     Scalar

[compnerd]

@thakis, thanks for the heads up, I've disabled the test on Windows - I should've disabled it on Windows in the first place.

clang/include/clang/APINotes/APINotesYAMLCompiler.h

+//===-- APINotesYAMLCompiler.h - API Notes YAML Format Reader ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_APINOTES_APINOTESYAMLCOMPILER_H
+#define LLVM_CLANG_APINOTES_APINOTESYAMLCOMPILER_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace clang {
+namespace api_notes {
+/// Parses the APINotes YAML content and writes the representation back to the
+/// specified stream.  This provides a means of testing the YAML processing of
+/// the APINotes format.
+bool parseAndDumpAPINotes(llvm::StringRef YI, llvm::raw_ostream &OS);
+} // namespace api_notes
+} // namespace clang
+
+#endif

clang/include/clang/APINotes/Types.h

+//===-- Types.h - API Notes Data Types --------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_APINOTES_TYPES_H
+#define LLVM_CLANG_APINOTES_TYPES_H
+
+namespace clang {
+namespace api_notes {
+enum class RetainCountConventionKind {
+  None,
+  CFReturnsRetained,
+  CFReturnsNotRetained,
+  NSReturnsRetained,
+  NSReturnsNotRetained,
+};
+
+/// The payload for an enum_extensibility attribute. This is a tri-state rather
+/// than just a boolean because the presence of the attribute indicates
+/// auditing.
+enum class EnumExtensibilityKind {
+  None,
+  Open,
+  Closed,
+};
+
+/// The kind of a swift_wrapper/swift_newtype.
+enum class SwiftNewTypeKind {
+  None,
+  Struct,
+  Enum,
+};
+} // namespace api_notes
+} // namespace clang
+
+#endif

clang/lib/APINotes/APINotesYAMLCompiler.cpp

+//===-- APINotesYAMLCompiler.cpp - API Notes YAML Format Reader -*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// The types defined locally are designed to represent the YAML state, which
+// adds an additional bit of state: e.g. a tri-state boolean attribute (yes, no,
+// not applied) becomes a tri-state boolean + present.  As a result, while these
+// enumerations appear to be redefining constants from the attributes table
+// data, they are distinct.
+//
+
+#include "clang/APINotes/APINotesYAMLCompiler.h"
+#include "clang/APINotes/Types.h"
+#include "clang/Basic/LLVM.h"
+#include "clang/Basic/Specifiers.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/Support/VersionTuple.h"
+#include "llvm/Support/YAMLParser.h"
+#include "llvm/Support/YAMLTraits.h"
+#include <vector>
+using namespace clang;
+using namespace api_notes;
+
+namespace {
+enum class APIAvailability {
+  Available = 0,
+  OSX,
+  IOS,
+  None,
+  NonSwift,
+};
+} // namespace
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<APIAvailability> {
+  static void enumeration(IO &IO, APIAvailability &AA) {
+    IO.enumCase(AA, "OSX", APIAvailability::OSX);
+    IO.enumCase(AA, "iOS", APIAvailability::IOS);
+    IO.enumCase(AA, "none", APIAvailability::None);
+    IO.enumCase(AA, "nonswift", APIAvailability::NonSwift);
+    IO.enumCase(AA, "available", APIAvailability::Available);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+enum class MethodKind {
+  Class,
+  Instance,
+};
+} // namespace
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<MethodKind> {
+  static void enumeration(IO &IO, MethodKind &MK) {
+    IO.enumCase(MK, "Class", MethodKind::Class);
+    IO.enumCase(MK, "Instance", MethodKind::Instance);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Param {
+  unsigned Position;
+  Optional<bool> NoEscape = false;
+  Optional<NullabilityKind> Nullability;
+  Optional<RetainCountConventionKind> RetainCountConvention;
+  StringRef Type;
+};
+
+typedef std::vector<Param> ParamsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Param)
+LLVM_YAML_IS_FLOW_SEQUENCE_VECTOR(NullabilityKind)
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<NullabilityKind> {
+  static void enumeration(IO &IO, NullabilityKind &NK) {
+    IO.enumCase(NK, "Nonnull", NullabilityKind::NonNull);
+    IO.enumCase(NK, "Optional", NullabilityKind::Nullable);
+    IO.enumCase(NK, "Unspecified", NullabilityKind::Unspecified);
+    // TODO: Mapping this to it's own value would allow for better cross
+    // checking. Also the default should be Unknown.
+    IO.enumCase(NK, "Scalar", NullabilityKind::Unspecified);
+
+    // Aliases for compatibility with existing APINotes.
+    IO.enumCase(NK, "N", NullabilityKind::NonNull);
+    IO.enumCase(NK, "O", NullabilityKind::Nullable);
+    IO.enumCase(NK, "U", NullabilityKind::Unspecified);
+    IO.enumCase(NK, "S", NullabilityKind::Unspecified);
+  }
+};
+
+template <> struct ScalarEnumerationTraits<RetainCountConventionKind> {
+  static void enumeration(IO &IO, RetainCountConventionKind &RCCK) {
+    IO.enumCase(RCCK, "none", RetainCountConventionKind::None);
+    IO.enumCase(RCCK, "CFReturnsRetained",
+                RetainCountConventionKind::CFReturnsRetained);
+    IO.enumCase(RCCK, "CFReturnsNotRetained",
+                RetainCountConventionKind::CFReturnsNotRetained);
+    IO.enumCase(RCCK, "NSReturnsRetained",
+                RetainCountConventionKind::NSReturnsRetained);
+    IO.enumCase(RCCK, "NSReturnsNotRetained",
+                RetainCountConventionKind::NSReturnsNotRetained);
+  }
+};
+
+template <> struct MappingTraits<Param> {
+  static void mapping(IO &IO, Param &P) {
+    IO.mapRequired("Position", P.Position);
+    IO.mapOptional("Nullability", P.Nullability, llvm::None);
+    IO.mapOptional("RetainCountConvention", P.RetainCountConvention);
+    IO.mapOptional("NoEscape", P.NoEscape);
+    IO.mapOptional("Type", P.Type, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+typedef std::vector<NullabilityKind> NullabilitySeq;
+
+struct AvailabilityItem {
+  APIAvailability Mode = APIAvailability::Available;
+  StringRef Msg;
+};
+
+/// Old attribute deprecated in favor of SwiftName.
+enum class FactoryAsInitKind {
+  /// Infer based on name and type (the default).
+  Infer,
+  /// Treat as a class method.
+  AsClassMethod,
+  /// Treat as an initializer.
+  AsInitializer,
+};
+
+struct Method {
+  StringRef Selector;
+  MethodKind Kind;
+  ParamsSeq Params;
+  NullabilitySeq Nullability;
+  Optional<NullabilityKind> NullabilityOfRet;
+  Optional<RetainCountConventionKind> RetainCountConvention;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+  FactoryAsInitKind FactoryAsInit = FactoryAsInitKind::Infer;
+  bool DesignatedInit = false;
+  bool Required = false;
+  StringRef ResultType;
+};
+
+typedef std::vector<Method> MethodsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Method)
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<FactoryAsInitKind> {
+  static void enumeration(IO &IO, FactoryAsInitKind &FIK) {
+    IO.enumCase(FIK, "A", FactoryAsInitKind::Infer);
+    IO.enumCase(FIK, "C", FactoryAsInitKind::AsClassMethod);
+    IO.enumCase(FIK, "I", FactoryAsInitKind::AsInitializer);
+  }
+};
+
+template <> struct MappingTraits<Method> {
+  static void mapping(IO &IO, Method &M) {
+    IO.mapRequired("Selector", M.Selector);
+    IO.mapRequired("MethodKind", M.Kind);
+    IO.mapOptional("Parameters", M.Params);
+    IO.mapOptional("Nullability", M.Nullability);
+    IO.mapOptional("NullabilityOfRet", M.NullabilityOfRet, llvm::None);
+    IO.mapOptional("RetainCountConvention", M.RetainCountConvention);
+    IO.mapOptional("Availability", M.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", M.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", M.SwiftPrivate);
+    IO.mapOptional("SwiftName", M.SwiftName, StringRef(""));
+    IO.mapOptional("FactoryAsInit", M.FactoryAsInit, FactoryAsInitKind::Infer);
+    IO.mapOptional("DesignatedInit", M.DesignatedInit, false);
+    IO.mapOptional("Required", M.Required, false);
+    IO.mapOptional("ResultType", M.ResultType, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Property {
+  StringRef Name;
+  llvm::Optional<MethodKind> Kind;
+  llvm::Optional<NullabilityKind> Nullability;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+  Optional<bool> SwiftImportAsAccessors;
+  StringRef Type;
+};
+
+typedef std::vector<Property> PropertiesSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Property)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<Property> {
+  static void mapping(IO &IO, Property &P) {
+    IO.mapRequired("Name", P.Name);
+    IO.mapOptional("PropertyKind", P.Kind);
+    IO.mapOptional("Nullability", P.Nullability, llvm::None);
+    IO.mapOptional("Availability", P.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", P.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", P.SwiftPrivate);
+    IO.mapOptional("SwiftName", P.SwiftName, StringRef(""));
+    IO.mapOptional("SwiftImportAsAccessors", P.SwiftImportAsAccessors);
+    IO.mapOptional("Type", P.Type, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Class {
+  StringRef Name;
+  bool AuditedForNullability = false;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+  Optional<StringRef> SwiftBridge;
+  Optional<StringRef> NSErrorDomain;
+  Optional<bool> SwiftImportAsNonGeneric;
+  Optional<bool> SwiftObjCMembers;
+  MethodsSeq Methods;
+  PropertiesSeq Properties;
+};
+
+typedef std::vector<Class> ClassesSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Class)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<Class> {
+  static void mapping(IO &IO, Class &C) {
+    IO.mapRequired("Name", C.Name);
+    IO.mapOptional("AuditedForNullability", C.AuditedForNullability, false);
+    IO.mapOptional("Availability", C.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", C.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", C.SwiftPrivate);
+    IO.mapOptional("SwiftName", C.SwiftName, StringRef(""));
+    IO.mapOptional("SwiftBridge", C.SwiftBridge);
+    IO.mapOptional("NSErrorDomain", C.NSErrorDomain);
+    IO.mapOptional("SwiftImportAsNonGeneric", C.SwiftImportAsNonGeneric);
+    IO.mapOptional("SwiftObjCMembers", C.SwiftObjCMembers);
+    IO.mapOptional("Methods", C.Methods);
+    IO.mapOptional("Properties", C.Properties);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Function {
+  StringRef Name;
+  ParamsSeq Params;
+  NullabilitySeq Nullability;
+  Optional<NullabilityKind> NullabilityOfRet;
+  Optional<api_notes::RetainCountConventionKind> RetainCountConvention;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+  StringRef Type;
+  StringRef ResultType;
+};
+
+typedef std::vector<Function> FunctionsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Function)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<Function> {
+  static void mapping(IO &IO, Function &F) {
+    IO.mapRequired("Name", F.Name);
+    IO.mapOptional("Parameters", F.Params);
+    IO.mapOptional("Nullability", F.Nullability);
+    IO.mapOptional("NullabilityOfRet", F.NullabilityOfRet, llvm::None);
+    IO.mapOptional("RetainCountConvention", F.RetainCountConvention);
+    IO.mapOptional("Availability", F.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", F.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", F.SwiftPrivate);
+    IO.mapOptional("SwiftName", F.SwiftName, StringRef(""));
+    IO.mapOptional("ResultType", F.ResultType, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct GlobalVariable {
+  StringRef Name;
+  llvm::Optional<NullabilityKind> Nullability;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+  StringRef Type;
+};
+
+typedef std::vector<GlobalVariable> GlobalVariablesSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(GlobalVariable)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<GlobalVariable> {
+  static void mapping(IO &IO, GlobalVariable &GV) {
+    IO.mapRequired("Name", GV.Name);
+    IO.mapOptional("Nullability", GV.Nullability, llvm::None);
+    IO.mapOptional("Availability", GV.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", GV.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", GV.SwiftPrivate);
+    IO.mapOptional("SwiftName", GV.SwiftName, StringRef(""));
+    IO.mapOptional("Type", GV.Type, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct EnumConstant {
+  StringRef Name;
+  AvailabilityItem Availability;
+  Optional<bool> SwiftPrivate;
+  StringRef SwiftName;
+};
+
+typedef std::vector<EnumConstant> EnumConstantsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumConstant)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<EnumConstant> {
+  static void mapping(IO &IO, EnumConstant &EC) {
+    IO.mapRequired("Name", EC.Name);
+    IO.mapOptional("Availability", EC.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", EC.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", EC.SwiftPrivate);
+    IO.mapOptional("SwiftName", EC.SwiftName, StringRef(""));
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+/// Syntactic sugar for EnumExtensibility and FlagEnum
+enum class EnumConvenienceAliasKind {
+  /// EnumExtensibility: none, FlagEnum: false
+  None,
+  /// EnumExtensibility: open, FlagEnum: false
+  CFEnum,
+  /// EnumExtensibility: open, FlagEnum: true
+  CFOptions,
+  /// EnumExtensibility: closed, FlagEnum: false
+  CFClosedEnum
+};
+} // namespace
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<EnumConvenienceAliasKind> {
+  static void enumeration(IO &IO, EnumConvenienceAliasKind &ECAK) {
+    IO.enumCase(ECAK, "none", EnumConvenienceAliasKind::None);
+    IO.enumCase(ECAK, "CFEnum", EnumConvenienceAliasKind::CFEnum);
+    IO.enumCase(ECAK, "NSEnum", EnumConvenienceAliasKind::CFEnum);
+    IO.enumCase(ECAK, "CFOptions", EnumConvenienceAliasKind::CFOptions);
+    IO.enumCase(ECAK, "NSOptions", EnumConvenienceAliasKind::CFOptions);
+    IO.enumCase(ECAK, "CFClosedEnum", EnumConvenienceAliasKind::CFClosedEnum);
+    IO.enumCase(ECAK, "NSClosedEnum", EnumConvenienceAliasKind::CFClosedEnum);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Tag {
+  StringRef Name;
+  AvailabilityItem Availability;
+  StringRef SwiftName;
+  Optional<bool> SwiftPrivate;
+  Optional<StringRef> SwiftBridge;
+  Optional<StringRef> NSErrorDomain;
+  Optional<EnumExtensibilityKind> EnumExtensibility;
+  Optional<bool> FlagEnum;
+  Optional<EnumConvenienceAliasKind> EnumConvenienceKind;
+};
+
+typedef std::vector<Tag> TagsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Tag)
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<EnumExtensibilityKind> {
+  static void enumeration(IO &IO, EnumExtensibilityKind &EEK) {
+    IO.enumCase(EEK, "none", EnumExtensibilityKind::None);
+    IO.enumCase(EEK, "open", EnumExtensibilityKind::Open);
+    IO.enumCase(EEK, "closed", EnumExtensibilityKind::Closed);
+  }
+};
+
+template <> struct MappingTraits<Tag> {
+  static void mapping(IO &IO, Tag &T) {
+    IO.mapRequired("Name", T.Name);
+    IO.mapOptional("Availability", T.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", T.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", T.SwiftPrivate);
+    IO.mapOptional("SwiftName", T.SwiftName, StringRef(""));
+    IO.mapOptional("SwiftBridge", T.SwiftBridge);
+    IO.mapOptional("NSErrorDomain", T.NSErrorDomain);
+    IO.mapOptional("EnumExtensibility", T.EnumExtensibility);
+    IO.mapOptional("FlagEnum", T.FlagEnum);
+    IO.mapOptional("EnumKind", T.EnumConvenienceKind);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Typedef {
+  StringRef Name;
+  AvailabilityItem Availability;
+  StringRef SwiftName;
+  Optional<bool> SwiftPrivate;
+  Optional<StringRef> SwiftBridge;
+  Optional<StringRef> NSErrorDomain;
+  Optional<SwiftNewTypeKind> SwiftType;
+};
+
+typedef std::vector<Typedef> TypedefsSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Typedef)
+
+namespace llvm {
+namespace yaml {
+template <> struct ScalarEnumerationTraits<SwiftNewTypeKind> {
+  static void enumeration(IO &IO, SwiftNewTypeKind &SWK) {
+    IO.enumCase(SWK, "none", SwiftNewTypeKind::None);
+    IO.enumCase(SWK, "struct", SwiftNewTypeKind::Struct);
+    IO.enumCase(SWK, "enum", SwiftNewTypeKind::Enum);
+  }
+};
+
+template <> struct MappingTraits<Typedef> {
+  static void mapping(IO &IO, Typedef &T) {
+    IO.mapRequired("Name", T.Name);
+    IO.mapOptional("Availability", T.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", T.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftPrivate", T.SwiftPrivate);
+    IO.mapOptional("SwiftName", T.SwiftName, StringRef(""));
+    IO.mapOptional("SwiftBridge", T.SwiftBridge);
+    IO.mapOptional("NSErrorDomain", T.NSErrorDomain);
+    IO.mapOptional("SwiftWrapper", T.SwiftType);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct TopLevelItems {
+  ClassesSeq Classes;
+  ClassesSeq Protocols;
+  FunctionsSeq Functions;
+  GlobalVariablesSeq Globals;
+  EnumConstantsSeq EnumConstants;
+  TagsSeq Tags;
+  TypedefsSeq Typedefs;
+};
+} // namespace
+
+namespace llvm {
+namespace yaml {
+static void mapTopLevelItems(IO &IO, TopLevelItems &TLI) {
+  IO.mapOptional("Classes", TLI.Classes);
+  IO.mapOptional("Protocols", TLI.Protocols);
+  IO.mapOptional("Functions", TLI.Functions);
+  IO.mapOptional("Globals", TLI.Globals);
+  IO.mapOptional("Enumerators", TLI.EnumConstants);
+  IO.mapOptional("Tags", TLI.Tags);
+  IO.mapOptional("Typedefs", TLI.Typedefs);
+}
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Versioned {
+  VersionTuple Version;
+  TopLevelItems Items;
+};
+
+typedef std::vector<Versioned> VersionedSeq;
+} // namespace
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(Versioned)
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<Versioned> {
+  static void mapping(IO &IO, Versioned &V) {
+    IO.mapRequired("Version", V.Version);
+    mapTopLevelItems(IO, V.Items);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+namespace {
+struct Module {
+  StringRef Name;
+  AvailabilityItem Availability;
+  TopLevelItems TopLevel;
+  VersionedSeq SwiftVersions;
+
+  llvm::Optional<bool> SwiftInferImportAsMember = {llvm::None};
+
+  LLVM_DUMP_METHOD void dump() /*const*/;
+};
+} // namespace
+
+namespace llvm {
+namespace yaml {
+template <> struct MappingTraits<Module> {
+  static void mapping(IO &IO, Module &M) {
+    IO.mapRequired("Name", M.Name);
+    IO.mapOptional("Availability", M.Availability.Mode,
+                   APIAvailability::Available);
+    IO.mapOptional("AvailabilityMsg", M.Availability.Msg, StringRef(""));
+    IO.mapOptional("SwiftInferImportAsMember", M.SwiftInferImportAsMember);
+    mapTopLevelItems(IO, M.TopLevel);
+    IO.mapOptional("SwiftVersions", M.SwiftVersions);
+  }
+};
+} // namespace yaml
+} // namespace llvm
+
+void Module::dump() {
+  llvm::yaml::Output OS(llvm::errs());
+  OS << *this;
+}
+
+namespace {
+bool parseAPINotes(StringRef YI, Module &M, llvm::SourceMgr::DiagHandlerTy Diag,
+                   void *DiagContext) {
+  llvm::yaml::Input IS(YI, nullptr, Diag, DiagContext);
+  IS >> M;
+  return static_cast<bool>(IS.error());
+}
+} // namespace
+
+bool clang::api_notes::parseAndDumpAPINotes(StringRef YI,
+                                            llvm::raw_ostream &OS) {
+  Module M;
+  if (parseAPINotes(YI, M, nullptr, nullptr))
+    return true;
+
+  llvm::yaml::Output YOS(OS);
+  YOS << M;
+
+  return false;
+}

clang/lib/APINotes/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Support)
+add_clang_library(clangAPINotes
+  APINotesYAMLCompiler.cpp
+  LINK_LIBS
+    clangBasic)

clang/lib/CMakeLists.txt

 add_subdirectory(Headers)
 add_subdirectory(Basic)
+add_subdirectory(APINotes)
 add_subdirectory(Lex)
 add_subdirectory(Parse)
 add_subdirectory(AST)
 add_subdirectory(ASTMatchers)
 add_subdirectory(CrossTU)
 add_subdirectory(Sema)
 add_subdirectory(CodeGen)
 add_subdirectory(Analysis)

clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.h

+#ifndef SIMPLE_H
+#define SIMPLE_H
+
+__attribute__((__objc_root__))
+@interface I
+@property(class, nonatomic, readonly) id nonnullProperty;
+@property(class, nonatomic, readonly) id nonnullNewProperty;
+
+@property(class, nonatomic, readonly) id optionalProperty;
+@property(class, nonatomic, readonly) id optionalNewProperty;
+
+@property(nonatomic, readonly) id unspecifiedProperty;
+@property(nonatomic, readonly) id unspecifiedNewProperty;
+
+@property(nonatomic, readonly) id scalarProperty;
+@property(nonatomic, readonly) id scalarNewProperty;
+@end
+
+#endif

clang/test/APINotes/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes

+Name:            SimpleKit
+Classes:
+  - Name:            I
+    Properties:
+      - Name:            nonnullProperty
+        PropertyKind:    Class
+        Nullability:     N
+      - Name:            nonnullNewProperty
+        PropertyKind:    Class
+        Nullability:     Nonnull
+      - Name:            optionalProperty
+        PropertyKind:    Class
+        Nullability:     O
+      - Name:            optionalNewProperty
+        PropertyKind:    Class
+        Nullability:     Optional
+      - Name:            unspecifiedProperty
+        PropertyKind:    Instance
+        Nullability:     U
+      - Name:            unspecifiedNewProperty
+        PropertyKind:    Instance
+        Nullability:     Unspecified
+      - Name:            scalarProperty
+        PropertyKind:    Instance
+        Nullability:     S
+      - Name:            scalarNewProperty
+        PropertyKind:    Instance
+        Nullability:     Scalar

clang/test/APINotes/Inputs/Frameworks/SimpleKit.framework/Headers/SimpleKit.h

+struct RenamedAgainInAPINotesA {
+  int field;
+} __attribute__((__swift_name__("bad")));
+
+struct __attribute__((__swift_name__("bad"))) RenamedAgainInAPINotesB {
+  int field;
+};
+
+void *getCFOwnedToUnowned(void) __attribute__((__cf_returns_retained__));
+void *getCFUnownedToOwned(void) __attribute__((__cf_returns_not_retained__));
+void *getCFOwnedToNone(void) __attribute__((__cf_returns_retained__));
+id getObjCOwnedToUnowned(void) __attribute__((__ns_returns_retained__));
+id getObjCUnownedToOwned(void) __attribute__((__ns_returns_not_retained__));
+
+int indirectGetCFOwnedToUnowned(void **out __attribute__((__cf_returns_retained__)));
+int indirectGetCFUnownedToOwned(void **out __attribute__((__cf_returns_not_retained__)));
+int indirectGetCFOwnedToNone(void **out __attribute__((__cf_returns_retained__)));
+int indirectGetCFNoneToOwned(void **out);
+
+#pragma clang arc_cf_code_audited begin
+void *getCFAuditedToUnowned_DUMP(void);
+void *getCFAuditedToOwned_DUMP(void);
+void *getCFAuditedToNone_DUMP(void);
+#pragma clang arc_cf_code_audited end
+
+@interface MethodTest
+- (id)getOwnedToUnowned __attribute__((__ns_returns_retained__));
+- (id)getUnownedToOwned __attribute__((__ns_returns_not_retained__));
+@end

clang/test/APINotes/Inputs/Frameworks/SimpleKit.framework/Headers/SimpleKit.apinotes

+Name:            SimpleKit
+Classes:
+  - Name:            MethodTest
+    Methods:
+      - Selector:        getOwnedToUnowned
+        MethodKind:      Instance
+        RetainCountConvention: NSReturnsNotRetained
+      - Selector:        getUnownedToOwned
+        MethodKind:      Instance
+        RetainCountConvention: NSReturnsRetained
+Functions:
+  - Name:            getCFOwnedToUnowned
+    RetainCountConvention: CFReturnsNotRetained
+  - Name:            getCFUnownedToOwned
+    RetainCountConvention: CFReturnsRetained
+  - Name:            getCFOwnedToNone
+    RetainCountConvention: none
+  - Name:            getObjCOwnedToUnowned
+    RetainCountConvention: NSReturnsNotRetained
+  - Name:            getObjCUnownedToOwned
+    RetainCountConvention: NSReturnsRetained
+  - Name:            indirectGetCFOwnedToUnowned
+    Parameters:
+      - Position:        0
+        RetainCountConvention: CFReturnsNotRetained
+  - Name:            indirectGetCFUnownedToOwned
+    Parameters:
+      - Position:        0
+        RetainCountConvention: CFReturnsRetained
+  - Name:            indirectGetCFOwnedToNone
+    Parameters:
+      - Position:        0
+        RetainCountConvention: none
+  - Name:            indirectGetCFNoneToOwned
+    Parameters:
+      - Position:        0
+        RetainCountConvention: CFReturnsNotRetained
+  - Name:            getCFAuditedToUnowned_DUMP
+    RetainCountConvention: CFReturnsNotRetained
+  - Name:            getCFAuditedToOwned_DUMP
+    RetainCountConvention: CFReturnsRetained
+  - Name:            getCFAuditedToNone_DUMP
+    RetainCountConvention: none
+Tags:
+  - Name:            RenamedAgainInAPINotesA
+    SwiftName:       SuccessfullyRenamedA
+  - Name:            RenamedAgainInAPINotesB
+    SwiftName:       SuccessfullyRenamedB

clang/test/APINotes/Inputs/Frameworks/SimpleKit.framework/Headers/module.modulemap

+framework module SimpleKit {
+  umbrella header "SimpleKit.h"
+  export *
+  module * { export * }
+}

clang/test/APINotes/yaml-roundtrip-2.test

+RUN: apinotes-test %S/Inputs/Frameworks/SimpleKit.framework/Headers/SimpleKit.apinotes > %t.result
+RUN: not diff --strip-trailing-cr --ed %t.result %S/Inputs/Frameworks/SimpleKit.framework/Headers/SimpleKit.apinotes | FileCheck %s
+
+The `--ed` parameter to `diff` is not implemented in the builtin diff, assume
+that we have a GNU compatible diff when we have a shell.
+REQUIRES: shell
+
+We expect only the document markers to be emitted
+
+CHECK: 50d
+CHECK: 1d

clang/test/APINotes/yaml-roundtrip.test

+RUN: apinotes-test %S/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes > %t.result
+RUN: not diff --strip-trailing-cr %S/Inputs/Frameworks/Simple.framework/Headers/Simple.apinotes %t.result | FileCheck %s
+
+We expect only the nullability to be different as it is canonicalized during the
+roudtrip.
+
+CHECK:      7c8
+CHECK-NEXT: <         Nullability:     N
+CHECK-NEXT: ---
+CHECK-NEXT: >         Nullability:     Nonnull
+CHECK-NEXT: 13c14
+CHECK-NEXT: <         Nullability:     O
+CHECK-NEXT: ---
+CHECK-NEXT: >         Nullability:     Optional
+CHECK-NEXT: 19c20
+CHECK-NEXT: <         Nullability:     U
+CHECK-NEXT: ---
+CHECK-NEXT: >         Nullability:     Unspecified
+CHECK-NEXT: 25c26
+CHECK-NEXT: <         Nullability:     S
+CHECK-NEXT: ---
+CHECK-NEXT: >         Nullability:     Unspecified
+CHECK-NEXT: 28c29,30
+CHECK-NEXT: <         Nullability:     Scalar
+CHECK-NEXT: ---
+CHECK-NEXT: >         Nullability:     Unspecified

clang/test/CMakeLists.txt

configure_lit_site_cfg(
   )
 
 option(CLANG_TEST_USE_VG "Run Clang tests under Valgrind" OFF)
 if(CLANG_TEST_USE_VG)
   set(CLANG_TEST_EXTRA_ARGS ${CLANG_TEST_EXTRA_ARGS} "--vg")
 endif ()
 
 list(APPEND CLANG_TEST_DEPS
+  apinotes-test
   c-index-test
   clang
   clang-resource-headers
   clang-format
   clang-tblgen
   clang-offload-bundler
   clang-import-test
   clang-rename

clang/test/lit.cfg.py

 
 # For each occurrence of a clang tool name, replace it with the full path to
 # the build directory holding that tool.  We explicitly specify the directories
 # to search to ensure that we get the tools just built and not some random
 # tools that might happen to be in the user's PATH.
 tool_dirs = [config.clang_tools_dir, config.llvm_tools_dir]
 
 tools = [
-    'c-index-test', 'clang-diff', 'clang-format', 'clang-tblgen', 'opt', 'llvm-ifs',
+    'apinotes-test', 'c-index-test', 'clang-diff', 'clang-format',
+    'clang-tblgen', 'opt', 'llvm-ifs',
     ToolSubst('%clang_extdef_map', command=FindTool(
         'clang-extdef-mapping'), unresolved='ignore'),
 ]
 
 if config.clang_examples:
     config.available_features.add('examples')
     tools.append('clang-interpreter')
 

clang/tools/CMakeLists.txt

 create_subdirectory_options(CLANG TOOL)
 
 add_clang_subdirectory(diagtool)
 add_clang_subdirectory(driver)
+add_clang_subdirectory(apinotes-test)
 add_clang_subdirectory(clang-diff)
 add_clang_subdirectory(clang-format)
 add_clang_subdirectory(clang-format-vs)
 add_clang_subdirectory(clang-fuzzer)
 add_clang_subdirectory(clang-import-test)
 add_clang_subdirectory(clang-offload-bundler)
 add_clang_subdirectory(clang-offload-wrapper)
 add_clang_subdirectory(clang-scan-deps)

clang/tools/apinotes-test/APINotesTest.cpp

+//===-- APINotesTest.cpp - API Notes Testing Tool ------------------ C++ --===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/APINotes/APINotesYAMLCompiler.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/WithColor.h"
+
+static llvm::cl::list<std::string> APINotes(llvm::cl::Positional,
+                                            llvm::cl::desc("[<apinotes> ...]"),
+                                            llvm::cl::Required);
+
+static llvm::cl::opt<std::string>
+    OutputFileName("o", llvm::cl::desc("output filename"),
+                   llvm::cl::value_desc("filename"), llvm::cl::init("-"));
+
+int main(int argc, const char **argv) {
+  const bool DisableCrashReporting = true;
+  llvm::sys::PrintStackTraceOnErrorSignal(argv[0], DisableCrashReporting);
+  llvm::cl::ParseCommandLineOptions(argc, argv);
+
+  auto Error = [](const llvm::Twine &Msg) {
+    llvm::WithColor::error(llvm::errs(), "apinotes-test") << Msg << '\n';
+  };
+
+  std::error_code EC;
+  auto Out = std::make_unique<llvm::ToolOutputFile>(OutputFileName, EC,
+                                                    llvm::sys::fs::OF_None);
+  if (EC) {
+    Error("failed to open '" + OutputFileName + "': " + EC.message());
+    return EXIT_FAILURE;
+  }
+
+  for (const std::string &Notes : APINotes) {
+    llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> NotesOrError =
+        llvm::MemoryBuffer::getFileOrSTDIN(Notes);
+    if (std::error_code EC = NotesOrError.getError()) {
+      llvm::errs() << EC.message() << '\n';
+      return EXIT_FAILURE;
+    }
+
+    clang::api_notes::parseAndDumpAPINotes((*NotesOrError)->getBuffer(),
+                                           Out->os());
+  }
+
+  return EXIT_SUCCESS;
+}

clang/tools/apinotes-test/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Support)
+add_clang_executable(apinotes-test
+  APINotesTest.cpp)
+clang_target_link_libraries(apinotes-test PRIVATE
+  clangAPINotes)