[clang-doc] Add support for explicitly typed enums

[brettw]

Summary

Add support for explicitly typed enums:

enum Foo : unsigned { ... };

to the internal representation and to the YAML output.

Add support for getting the value of an enum constant, as well as accessing the original expression that produced it. This changes the YAML output of enums from an array of strings for the enum members to an array of dictionaries. These dictionaries now report the name, value, and original expression.

The markdown and HTML outputs are unchanged, they still output the name from the new enhanced internal schema.

[paulkirth]

Can you make the first line of the summary the commit title? It's a bit more descriptive.

clang-tools-extra/clang-doc/BitcodeReader.cpp

59

do you need to do all of this? APSInt already supports to/from string methods, as well as converting to/from integers. can you use that here and in the writer to avoid some complexity?

221

Don't you still need this? if it's now unused, we should probably remove it from the RecordId enum too.

clang-tools-extra/clang-doc/Serialize.cpp

185

We normally try to avoid returning small string by value. see https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallstring-h

Normally in the case that we need to return a string from a method, we just use std::string. In other cases, it may be more appropriate to use an output parameter instead of a return.

316

why was 128 chosen? Aren't we storing it into a SmallString<16> in EnumValueInfo? is there some external reason that we expect this to be the right size?

Do you have an idea for how long these are likely to be? if we expect them to be large or completely unpredictable, it may make more sense to use a std::string and avoid wasting stack space.

[brettw]

clang-tools-extra/clang-doc/BitcodeReader.cpp

59

I don't think converting to an integer is a good idea because people sometimes use min/max values for enum values and since this could be signed or unsigned, it gets kind of complicated.

Serializing a number as a string to bitcode also seemed wrong to me.

The simplest thing would be to store the value as a string in the EnumValueInfo and then always treat this as a string from then on. If you want it simplified, I think that's the thing to do. But I thought you would want the numeric value stored in clang-doc's "ast" because some backends may want to treat this as a number, check its signed/unsignedness, etc. I happy to change this if you want.

221

I think this is only used for properties and not the child members, so it isn't used. I removed the enum.

clang-tools-extra/clang-doc/Serialize.cpp

185

I was just copying some other code in this file, I changed this to std::string.

316

I changed this and the EnumValueInfo struct to std::string. I think the usage of SmallString in these records is over the top but I was trying to copy the existing style.

[paulkirth]

I think maybe you made the title the first line of the summary instead of the other way around. I was looking for this as the title: [clang-doc] Add support for explicitly typed enums

clang-tools-extra/clang-doc/BitcodeReader.cpp

59

Those are fair points, and I think I misread/misunderstood a bit of what's going on here.

As for encoding/decoding integers, BitcodeWriter already has integer support, so if you need to convert to/from those, it should already work, right?

Regardless, you may want to look at the bitcode reader/writer in llvm to see how they serialize/deserialize APInt, as their implementation seems a bit more straightforward IMO.

https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L1636
https://github.com/llvm/llvm-project/blob/main/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp#L2520
https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L2843
https://github.com/llvm/llvm-project/blob/9050a59c6678789831b7286b8b68d18b966c4694/llvm/lib/Bitcode/Reader/BitcodeReader.cpp#L2743

71

You can avoid the resize w/ SmallVector(Size) constructor, right?

clang-tools-extra/clang-doc/Representation.h

425

Sorry to nitpick, but SmallString was the correct choice to use in this type. We avoid its use as a return value, because it tends to be brittle, and stops us from assigning into arbitrary sized SmallStrings. In the struct, it's a reasonable choice, especially if you expect most uses to be small. for these expressions, I expect most to either be the number itself, or some shift operation, so SmallString<16> was probably more than sufficient.

clang-tools-extra/clang-doc/Serialize.cpp

316

small buffer optimizations significantly improve performance in the compiler and related tools. switching to a default string implementation that can use those optimizations reduced the total number of heap allocations in a normal run of clang by almost half.

That's why LLVM's programmer's manual and contribution guidelines suggest using those whenever possible.

[brettw]

clang-tools-extra/clang-doc/BitcodeReader.cpp

59

I just converted everything to a string which allowed most of this code to be deleted. I don't think any theoretical benefit of keeping the APSInt is worth the complexity.

71

This is now deleted.

clang-tools-extra/clang-doc/Representation.h

425

In the common case there will be will be empty, so std::string actually seems more efficient. Also, I personally think the current quantity of SmallString in this code is over-optimization.

[paulkirth]

LGTM.

[paulkirth]

Sorry, it looks like a pre-submit formatting check failed. can you git clang-format HEAD~ and re-upload? I can land your change after that.

[brettw]

Clang-formatted

clang-tools-extra/clang-doc/BitcodeReader.cpp

 #include "llvm/Support/Error.h"
 #include "llvm/Support/raw_ostream.h"
 
 namespace clang {
 namespace doc {
 
 using Record = llvm::SmallVector<uint64_t, 1024>;
 
+// This implements decode for SmallString.
 llvm::Error decodeRecord(const Record &R, llvm::SmallVectorImpl<char> &Field,
                          llvm::StringRef Blob) {
   Field.assign(Blob.begin(), Blob.end());
   return llvm::Error::success();
 }
 
+llvm::Error decodeRecord(const Record &R, std::string &Field,
+                         llvm::StringRef Blob) {
+  Field.assign(Blob.begin(), Blob.end());
+  return llvm::Error::success();
+}
+
 llvm::Error decodeRecord(const Record &R, SymbolID &Field,
                          llvm::StringRef Blob) {
   if (R[0] != BitCodeConstants::USRHashSize)
     return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                    "incorrect USR size");
 
   // First position in the record is the length of the following array, so we
   // copy the following elements to the field.

 llvm::Error decodeRecord(const Record &R, int &Field, llvm::StringRef Blob) {
   if (R[0] > INT_MAX)
     return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                    "integer too large to parse");
   Field = (int)R[0];
   return llvm::Error::success();
 }
 
+llvm::Error decodeRecord(const Record &R, llvm::APSInt &Field, llvm::StringRef Blob) {
+  auto ByteWidth = R[0];
+  // The writer only ever writes out the number of bytes in the whole words stored by APSInt.
+  assert(ByteWidth % llvm::APSInt::APINT_WORD_SIZE == 0);
+  assert(Blob.size() == ByteWidth);
+
+  auto WordWidth = ByteWidth / llvm::APSInt::APINT_WORD_SIZE;
+  assert(WordWidth > 0);
+  auto BitWidth = ByteWidth * 8;
+
+  auto IsUnsigned = R[1];
+
+  llvm::SmallVector<uint64_t, 1024> AsWords;
+  AsWords.resize(WordWidth);
+  memcpy(AsWords.data(), Blob.data(), ByteWidth);
+
+  llvm::APInt IntValue(static_cast<unsigned>(BitWidth), makeArrayRef(AsWords));
+  Field = llvm::APSInt(IntValue, IsUnsigned);
+  return llvm::Error::success();
+}
+
 llvm::Error decodeRecord(const Record &R, AccessSpecifier &Field,
                          llvm::StringRef Blob) {
   switch (R[0]) {
   case AS_public:
   case AS_private:
   case AS_protected:
   case AS_none:
     Field = (AccessSpecifier)R[0];
llvm::Error parseRecord(const Record &R, unsigned ID, llvm::StringRef Blob,
   case ENUM_USR:
     return decodeRecord(R, I->USR, Blob);
   case ENUM_NAME:
     return decodeRecord(R, I->Name, Blob);
   case ENUM_DEFLOCATION:
     return decodeRecord(R, I->DefLoc, Blob);
   case ENUM_LOCATION:
     return decodeRecord(R, I->Loc, Blob);
-  case ENUM_MEMBER:
-    return decodeRecord(R, I->Members, Blob);
   case ENUM_SCOPED:
     return decodeRecord(R, I->Scoped, Blob);
   default:
     return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                    "invalid field for EnumInfo");
   }
 }
 
 llvm::Error parseRecord(const Record &R, unsigned ID, llvm::StringRef Blob,
+                        EnumValueInfo *I) {
+  switch (ID) {
+  case ENUM_VALUE_NAME:
+    return decodeRecord(R, I->Name, Blob);
+  case ENUM_VALUE_VALUE:
+    return decodeRecord(R, I->Value, Blob);
+  case ENUM_VALUE_EXPR:
+    return decodeRecord(R, I->ValueExpr, Blob);
+  default:
+    return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                   "invalid field for EnumValueInfo");
+  }
+}
+
+llvm::Error parseRecord(const Record &R, unsigned ID, llvm::StringRef Blob,
                         FunctionInfo *I) {
   switch (ID) {
   case FUNCTION_USR:
     return decodeRecord(R, I->USR, Blob);
   case FUNCTION_NAME:
     return decodeRecord(R, I->Name, Blob);
   case FUNCTION_DEFLOCATION:
     return decodeRecord(R, I->DefLoc, Blob);
template <> llvm::Expected<CommentInfo *> getCommentInfo(CommentInfo *I) {
   return I->Children.back().get();
 }
 
 template <>
 llvm::Expected<CommentInfo *> getCommentInfo(std::unique_ptr<CommentInfo> &I) {
   return getCommentInfo(I.get());
 }
 
+// When readSubBlock encounters a TypeInfo sub-block, it calls addTypeInfo on
+// the parent block to set it. The template specializations define what to do
+// for each supported parent block.
 template <typename T, typename TTypeInfo>
 llvm::Error addTypeInfo(T I, TTypeInfo &&TI) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                  "invalid type cannot contain TypeInfo");
 }
 
 template <> llvm::Error addTypeInfo(RecordInfo *I, MemberTypeInfo &&T) {
   I->Members.emplace_back(std::move(T));
template <> llvm::Error addTypeInfo(FunctionInfo *I, TypeInfo &&T) {
   return llvm::Error::success();
 }
 
 template <> llvm::Error addTypeInfo(FunctionInfo *I, FieldTypeInfo &&T) {
   I->Params.emplace_back(std::move(T));
   return llvm::Error::success();
 }
 
+template <> llvm::Error addTypeInfo(EnumInfo *I, TypeInfo &&T) {
+  I->BaseType = std::move(T);
+  return llvm::Error::success();
+}
+
 template <typename T> llvm::Error addReference(T I, Reference &&R, FieldId F) {
   return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                  "invalid type cannot contain Reference");
 }
 
 template <> llvm::Error addReference(TypeInfo *I, Reference &&R, FieldId F) {
   switch (F) {
   case FieldId::F_type:

 template <> void addChild(RecordInfo *I, FunctionInfo &&R) {
   I->ChildFunctions.emplace_back(std::move(R));
 }
 
 template <> void addChild(RecordInfo *I, EnumInfo &&R) {
   I->ChildEnums.emplace_back(std::move(R));
 }
 
+template <> void addChild(EnumInfo *I, EnumValueInfo &&R) {
+  I->Members.emplace_back(std::move(R));
+}
+
 template <> void addChild(RecordInfo *I, BaseRecordInfo &&R) {
   I->Bases.emplace_back(std::move(R));
 }
 
 template <> void addChild(BaseRecordInfo *I, FunctionInfo &&R) {
   I->ChildFunctions.emplace_back(std::move(R));
 }
 
while (true) {
     if (auto Err = readRecord(BlockOrCode, I))
       return Err;
   }
 }
 
 template <typename T>
 llvm::Error ClangDocBitcodeReader::readSubBlock(unsigned ID, T I) {
   switch (ID) {
-  // Blocks can only have Comment, Reference, TypeInfo, FunctionInfo, or
-  // EnumInfo subblocks
+  // Blocks can only have certain types of sub blocks.
   case BI_COMMENT_BLOCK_ID: {
     auto Comment = getCommentInfo(I);
     if (!Comment)
       return Comment.takeError();
     if (auto Err = readBlock(ID, Comment.get()))
       return Err;
     return llvm::Error::success();
   }
llvm::Error ClangDocBitcodeReader::readSubBlock(unsigned ID, T I) {
   }
   case BI_ENUM_BLOCK_ID: {
     EnumInfo E;
     if (auto Err = readBlock(ID, &E))
       return Err;
     addChild(I, std::move(E));
     return llvm::Error::success();
   }
+  case BI_ENUM_VALUE_BLOCK_ID: {
+    EnumValueInfo EV;
+    if (auto Err = readBlock(ID, &EV))
+      return Err;
+    addChild(I, std::move(EV));
+    return llvm::Error::success();
+  }
   default:
     return llvm::createStringError(llvm::inconvertibleErrorCode(),
                                    "invalid subblock type");
   }
 }
 
 ClangDocBitcodeReader::Cursor
 ClangDocBitcodeReader::skipUntilRecordOrBlock(unsigned &BlockOrRecordID) {

clang-tools-extra/clang-doc/BitcodeWriter.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_DOC_BITCODEWRITER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANG_DOC_BITCODEWRITER_H
 
 #include "Representation.h"
 #include "clang/AST/AST.h"
+#include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Bitstream/BitstreamWriter.h"
 #include <initializer_list>
 #include <vector>
 
 namespace clang {

 };
 
 // New Ids need to be added to both the enum here and the relevant IdNameMap in
 // the implementation file.
 enum BlockId {
   BI_VERSION_BLOCK_ID = llvm::bitc::FIRST_APPLICATION_BLOCKID,
   BI_NAMESPACE_BLOCK_ID,
   BI_ENUM_BLOCK_ID,
+  BI_ENUM_VALUE_BLOCK_ID,
   BI_TYPE_BLOCK_ID,
   BI_FIELD_TYPE_BLOCK_ID,
   BI_MEMBER_TYPE_BLOCK_ID,
   BI_RECORD_BLOCK_ID,
   BI_BASE_RECORD_BLOCK_ID,
   BI_FUNCTION_BLOCK_ID,
   BI_COMMENT_BLOCK_ID,
   BI_REFERENCE_BLOCK_ID,
enum RecordId {
   MEMBER_TYPE_ACCESS,
   NAMESPACE_USR,
   NAMESPACE_NAME,
   NAMESPACE_PATH,
   ENUM_USR,
   ENUM_NAME,
   ENUM_DEFLOCATION,
   ENUM_LOCATION,
-  ENUM_MEMBER,
   ENUM_SCOPED,
+  ENUM_VALUE_NAME,
+  ENUM_VALUE_VALUE,
+  ENUM_VALUE_EXPR,
   RECORD_USR,
   RECORD_NAME,
   RECORD_PATH,
   RECORD_DEFLOCATION,
   RECORD_LOCATION,
   RECORD_TAG_TYPE,
   RECORD_IS_TYPE_DEF,
   BASE_RECORD_USR,
public:
   bool dispatchInfoForWrite(Info *I);
 
   // Block emission of different info types.
   void emitBlock(const NamespaceInfo &I);
   void emitBlock(const RecordInfo &I);
   void emitBlock(const BaseRecordInfo &I);
   void emitBlock(const FunctionInfo &I);
   void emitBlock(const EnumInfo &I);
+  void emitBlock(const EnumValueInfo &I);
   void emitBlock(const TypeInfo &B);
   void emitBlock(const FieldTypeInfo &B);
   void emitBlock(const MemberTypeInfo &B);
   void emitBlock(const CommentInfo &B);
   void emitBlock(const Reference &B, FieldId F);
 
 private:
   class AbbreviationMap {
private:
   // Emission of individual record types.
   void emitRecord(StringRef Str, RecordId ID);
   void emitRecord(const SymbolID &Str, RecordId ID);
   void emitRecord(const Location &Loc, RecordId ID);
   void emitRecord(const Reference &Ref, RecordId ID);
   void emitRecord(bool Value, RecordId ID);
   void emitRecord(int Value, RecordId ID);
   void emitRecord(unsigned Value, RecordId ID);
+  void emitRecord(llvm::APSInt Value, RecordId ID);
   bool prepRecordData(RecordId ID, bool ShouldEmit = true);
 
   // Emission of appropriate abbreviation type.
   void emitAbbrev(RecordId ID, BlockId Block);
 
   // Static size is the maximum length of the block/record names we're pushing
   // to this + 1. Longest is currently `MemberTypeBlock` at 15 chars.
   SmallVector<uint32_t, BitCodeConstants::RecordSize> Record;
   llvm::BitstreamWriter &Stream;
   AbbreviationMap Abbrevs;
 };
 
 } // namespace doc
 } // namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_DOC_BITCODEWRITER_H

clang-tools-extra/clang-doc/BitcodeWriter.cpp

 using AbbrevDsc = void (*)(std::shared_ptr<llvm::BitCodeAbbrev> &Abbrev);
 
 static void AbbrevGen(std::shared_ptr<llvm::BitCodeAbbrev> &Abbrev,
                       const std::initializer_list<llvm::BitCodeAbbrevOp> Ops) {
   for (const auto &Op : Ops)
     Abbrev->Add(Op);
 }
 
+static void ApsIntAbbrev(std::shared_ptr<llvm::BitCodeAbbrev> &Abbrev) {
+  AbbrevGen(Abbrev,
+            {// 0. Fixed-size integer (length of the following bytes)
+             llvm::BitCodeAbbrevOp(llvm::BitCodeAbbrevOp::Fixed,
+                                   BitCodeConstants::StringLengthSize),
+             // 1. Type unsigned flag.
+             llvm::BitCodeAbbrevOp(llvm::BitCodeAbbrevOp::Fixed,
+                                   BitCodeConstants::BoolSize),
+             // 2. The APSInt value blob.
+             llvm::BitCodeAbbrevOp(llvm::BitCodeAbbrevOp::Blob)});
+}
+
 static void BoolAbbrev(std::shared_ptr<llvm::BitCodeAbbrev> &Abbrev) {
   AbbrevGen(Abbrev,
             {// 0. Boolean
              llvm::BitCodeAbbrevOp(llvm::BitCodeAbbrevOp::Fixed,
                                    BitCodeConstants::BoolSize)});
 }
 
 static void IntAbbrev(std::shared_ptr<llvm::BitCodeAbbrev> &Abbrev) {
BlockIdNameMap = []() {
       BlockIdNameMap.resize(BlockIdCount);
 
       // There is no init-list constructor for the IndexedMap, so have to
       // improvise
       static const std::vector<std::pair<BlockId, const char *const>> Inits = {
           {BI_VERSION_BLOCK_ID, "VersionBlock"},
           {BI_NAMESPACE_BLOCK_ID, "NamespaceBlock"},
           {BI_ENUM_BLOCK_ID, "EnumBlock"},
+          {BI_ENUM_VALUE_BLOCK_ID, "EnumValueBlock"},
           {BI_TYPE_BLOCK_ID, "TypeBlock"},
           {BI_FIELD_TYPE_BLOCK_ID, "FieldTypeBlock"},
           {BI_MEMBER_TYPE_BLOCK_ID, "MemberTypeBlock"},
           {BI_RECORD_BLOCK_ID, "RecordBlock"},
           {BI_BASE_RECORD_BLOCK_ID, "BaseRecordBlock"},
           {BI_FUNCTION_BLOCK_ID, "FunctionBlock"},
           {BI_COMMENT_BLOCK_ID, "CommentBlock"},
           {BI_REFERENCE_BLOCK_ID, "ReferenceBlock"}};
RecordIdNameMap = []() {
           {MEMBER_TYPE_ACCESS, {"Access", &IntAbbrev}},
           {NAMESPACE_USR, {"USR", &SymbolIDAbbrev}},
           {NAMESPACE_NAME, {"Name", &StringAbbrev}},
           {NAMESPACE_PATH, {"Path", &StringAbbrev}},
           {ENUM_USR, {"USR", &SymbolIDAbbrev}},
           {ENUM_NAME, {"Name", &StringAbbrev}},
           {ENUM_DEFLOCATION, {"DefLocation", &LocationAbbrev}},
           {ENUM_LOCATION, {"Location", &LocationAbbrev}},
-          {ENUM_MEMBER, {"Member", &StringAbbrev}},
           {ENUM_SCOPED, {"Scoped", &BoolAbbrev}},
+          {ENUM_VALUE_NAME, {"Name", &StringAbbrev}},
+          {ENUM_VALUE_VALUE, {"Value", &ApsIntAbbrev}},
+          {ENUM_VALUE_EXPR, {"Expr", &StringAbbrev}},
           {RECORD_USR, {"USR", &SymbolIDAbbrev}},
           {RECORD_NAME, {"Name", &StringAbbrev}},
           {RECORD_PATH, {"Path", &StringAbbrev}},
           {RECORD_DEFLOCATION, {"DefLocation", &LocationAbbrev}},
           {RECORD_LOCATION, {"Location", &LocationAbbrev}},
           {RECORD_TAG_TYPE, {"TagType", &IntAbbrev}},
           {RECORD_IS_TYPE_DEF, {"IsTypeDef", &BoolAbbrev}},
           {BASE_RECORD_USR, {"USR", &SymbolIDAbbrev}},
RecordsByBlock{
         // Type Block
         {BI_TYPE_BLOCK_ID, {}},
         // FieldType Block
         {BI_FIELD_TYPE_BLOCK_ID, {FIELD_TYPE_NAME, FIELD_DEFAULT_VALUE}},
         // MemberType Block
         {BI_MEMBER_TYPE_BLOCK_ID, {MEMBER_TYPE_NAME, MEMBER_TYPE_ACCESS}},
         // Enum Block
         {BI_ENUM_BLOCK_ID,
-         {ENUM_USR, ENUM_NAME, ENUM_DEFLOCATION, ENUM_LOCATION, ENUM_MEMBER,
-          ENUM_SCOPED}},
+         {ENUM_USR, ENUM_NAME, ENUM_DEFLOCATION, ENUM_LOCATION, ENUM_SCOPED}},
+        // Enum Value Block
+        {BI_ENUM_VALUE_BLOCK_ID,
+         {ENUM_VALUE_NAME, ENUM_VALUE_VALUE, ENUM_VALUE_EXPR}},
         // Namespace Block
         {BI_NAMESPACE_BLOCK_ID,
          {NAMESPACE_USR, NAMESPACE_NAME, NAMESPACE_PATH}},
         // Record Block
         {BI_RECORD_BLOCK_ID,
          {RECORD_USR, RECORD_NAME, RECORD_PATH, RECORD_DEFLOCATION,
           RECORD_LOCATION, RECORD_TAG_TYPE, RECORD_IS_TYPE_DEF}},
         // BaseRecord Block
void ClangDocBitcodeWriter::emitRecord(unsigned Val, RecordId ID) {
   assert(RecordIdNameMap[ID].Abbrev == &IntAbbrev && "Abbrev type mismatch.");
   if (!prepRecordData(ID, Val))
     return;
   assert(Val < (1U << BitCodeConstants::IntSize));
   Record.push_back(Val);
   Stream.EmitRecordWithAbbrev(Abbrevs.get(ID), Record);
 }
 
+void ClangDocBitcodeWriter::emitRecord(llvm::APSInt Value, RecordId ID) {
+  assert(RecordIdNameMap[ID] && "Unknown RecordId.");
+  assert(RecordIdNameMap[ID].Abbrev == &ApsIntAbbrev && "Abbrev type mismatch.");
+  if (!prepRecordData(ID, true))
+    return;
+
+  auto ByteSize = Value.getNumWords() * llvm::APSInt::APINT_WORD_SIZE;
+  assert(ByteSize < (1U << BitCodeConstants::StringLengthSize));
+  Record.push_back(ByteSize);
+  Record.push_back(Value.isUnsigned());
+  Stream.EmitRecordWithBlob(Abbrevs.get(ID), Record, reinterpret_cast<const char*>(Value.getRawData()), ByteSize);
+}
+
 bool ClangDocBitcodeWriter::prepRecordData(RecordId ID, bool ShouldEmit) {
   assert(RecordIdNameMap[ID] && "Unknown RecordId.");
   if (!ShouldEmit)
     return false;
   Record.clear();
   Record.push_back(ID);
   return true;
 }
for (const auto &N : I.Namespace)
     emitBlock(N, FieldId::F_namespace);
   for (const auto &CI : I.Description)
     emitBlock(CI);
   if (I.DefLoc)
     emitRecord(*I.DefLoc, ENUM_DEFLOCATION);
   for (const auto &L : I.Loc)
     emitRecord(L, ENUM_LOCATION);
   emitRecord(I.Scoped, ENUM_SCOPED);
+  if (I.BaseType)
+    emitBlock(*I.BaseType);
   for (const auto &N : I.Members)
-    emitRecord(N, ENUM_MEMBER);
+    emitBlock(N);
+}
+
+void ClangDocBitcodeWriter::emitBlock(const EnumValueInfo &I) {
+  StreamSubBlockGuard Block(Stream, BI_ENUM_VALUE_BLOCK_ID);
+  emitRecord(I.Name, ENUM_VALUE_NAME);
+  emitRecord(I.Value, ENUM_VALUE_VALUE);
+  emitRecord(I.ValueExpr, ENUM_VALUE_EXPR);
 }
 
 void ClangDocBitcodeWriter::emitBlock(const RecordInfo &I) {
   StreamSubBlockGuard Block(Stream, BI_RECORD_BLOCK_ID);
   emitRecord(I.USR, RECORD_USR);
   emitRecord(I.Name, RECORD_NAME);
   emitRecord(I.Path, RECORD_PATH);
   for (const auto &N : I.Namespace)

clang-tools-extra/clang-doc/HTMLGenerator.cpp

genEnumsBlock(const std::vector<EnumInfo> &Enums,
   for (const auto &E : Enums) {
     std::vector<std::unique_ptr<TagNode>> Nodes = genHTML(E, CDCtx);
     AppendVector(std::move(Nodes), DivBody->Children);
   }
   return Out;
 }
 
 static std::unique_ptr<TagNode>
-genEnumMembersBlock(const llvm::SmallVector<SmallString<16>, 4> &Members) {
+genEnumMembersBlock(const llvm::SmallVector<EnumValueInfo, 4> &Members) {
   if (Members.empty())
     return nullptr;
 
   auto List = std::make_unique<TagNode>(HTMLTag::TAG_UL);
   for (const auto &M : Members)
-    List->Children.emplace_back(std::make_unique<TagNode>(HTMLTag::TAG_LI, M));
+    List->Children.emplace_back(std::make_unique<TagNode>(HTMLTag::TAG_LI, M.Name));
   return List;
 }
 
 static std::vector<std::unique_ptr<TagNode>>
 genFunctionsBlock(const std::vector<FunctionInfo> &Functions,
                   const ClangDocContext &CDCtx, StringRef ParentInfoDir) {
   if (Functions.empty())
     return {};

clang-tools-extra/clang-doc/MDGenerator.cpp

static void genMarkdown(const ClangDocContext &CDCtx, const EnumInfo &I,
   else
     writeLine("| enum " + I.Name + " |", OS);
   writeLine("--", OS);
 
   std::string Buffer;
   llvm::raw_string_ostream Members(Buffer);
   if (!I.Members.empty())
     for (const auto &N : I.Members)
-      Members << "| " << N << " |\n";
+      Members << "| " << N.Name << " |\n";
   writeLine(Members.str(), OS);
   if (I.DefLoc)
     writeFileDefinition(CDCtx, *I.DefLoc, OS);
 
   for (const auto &C : I.Description)
     writeDescription(C, OS);
 }
 

clang-tools-extra/clang-doc/Representation.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANG_DOC_REPRESENTATION_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANG_DOC_REPRESENTATION_H
 
 #include "clang/AST/Type.h"
 #include "clang/Basic/Specifiers.h"
 #include "clang/Tooling/StandaloneExecution.h"
+#include "llvm/ADT/APSInt.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringExtras.h"
 #include <array>
 #include <string>
 
 namespace clang {
 namespace doc {
struct BaseRecordInfo : public RecordInfo {
   // Indicates if base corresponds to a virtual inheritance
   bool IsVirtual = false;
   // Access level associated with this inherited info (public, protected,
   // private).
   AccessSpecifier Access = AccessSpecifier::AS_public;
   bool IsParent = false; // Indicates if this base is a direct parent
 };
 
+// Information for a single possible value of an enumeration.
+struct EnumValueInfo {
+  explicit EnumValueInfo(StringRef Name = StringRef(),
+                         llvm::APSInt Value = llvm::APSInt(),
+                         StringRef ValueExpr = StringRef())
+      : Name(Name), Value(Value), ValueExpr(ValueExpr) {}
+
+  bool operator==(const EnumValueInfo &Other) const {
+    // Be permissive in comparison for tests so compare the values rather than
+    // their exact representation (which is what operator== does).
+    if (llvm::APSInt::compareValues(Value, Other.Value) != 0)
+      return false;
+    return std::tie(Name, ValueExpr) == std::tie(Other.Name, Other.ValueExpr);
+  }
+
+  SmallString<16> Name;
+
+  // The computed value of the enumeration constant. This could be the result of
+  // evaluating the ValueExpr, or it could be automatically generated according
+  // to C rules.
+  llvm::APSInt Value;
+
+  // Stores the user-supplied initialization expression for this enumeration
+  // constant. This will be empty for implicit enumeration values.
+  std::string ValueExpr;
+};
+
 // TODO: Expand to allow for documenting templating.
 // Info for types.
 struct EnumInfo : public SymbolInfo {
   EnumInfo() : SymbolInfo(InfoType::IT_enum) {}
   EnumInfo(SymbolID USR) : SymbolInfo(InfoType::IT_enum, USR) {}
 
   void merge(EnumInfo &&I);
 
-  bool Scoped =
-      false; // Indicates whether this enum is scoped (e.g. enum class).
-  llvm::SmallVector<SmallString<16>, 4> Members; // List of enum members.
+  // Indicates whether this enum is scoped (e.g. enum class).
+  bool Scoped = false;
+
+  // Set to nonempty to the type when this is an explicitly typed enum. For
+  //   enum Foo : short { ... };
+  // this will be "short".
+  llvm::Optional<TypeInfo> BaseType;
+
+  llvm::SmallVector<EnumValueInfo, 4> Members; // List of enum members.
 };
 
 struct Index : public Reference {
   Index() = default;
   Index(StringRef Name) : Reference(Name) {}
   Index(StringRef Name, StringRef JumpToSection)
       : Reference(Name), JumpToSection(JumpToSection) {}
   Index(SymbolID USR, StringRef Name, InfoType IT, StringRef Path)

clang-tools-extra/clang-doc/Serialize.cpp

std::string ClangDocCommentVisitor::getCommandName(unsigned CommandID) const {
   if (Info)
     return Info->Name;
   // TODO: Add parsing for \file command.
   return "<not a builtin command>";
 }
 
 // Serializing functions.
 
+std::string getSourceCode(const Decl* D, const SourceRange& R) {
+  return Lexer::getSourceText(
+      CharSourceRange::getTokenRange(R),
+      D->getASTContext().getSourceManager(),
+      D->getASTContext().getLangOpts()).str();
+}
+
 template <typename T> static std::string serialize(T &I) {
   SmallString<2048> Buffer;
   llvm::BitstreamWriter Stream(Buffer);
   ClangDocBitcodeWriter Writer(Stream);
   Writer.emitBlock(I);
   return Buffer.str().str();
 }
 
for (const FieldDecl *F : D->fields()) {
     auto& member = I.Members.emplace_back(
         F->getTypeSourceInfo()->getType().getAsString(), F->getNameAsString(),
         getFinalAccessSpecifier(Access, F->getAccessUnsafe()));
     populateMemberTypeInfo(member, F);
   }
 }
 
 static void parseEnumerators(EnumInfo &I, const EnumDecl *D) {
-  for (const EnumConstantDecl *E : D->enumerators())
-    I.Members.emplace_back(E->getNameAsString());
+  for (const EnumConstantDecl *E : D->enumerators()) {
+    std::string ValueExpr;
+    if (const Expr* InitExpr = E->getInitExpr())
+      ValueExpr = getSourceCode(D, InitExpr->getSourceRange());
+    I.Members.emplace_back(E->getNameAsString(), E->getInitVal(), ValueExpr);
+  }
 }
 
 static void parseParameters(FunctionInfo &I, const FunctionDecl *D) {
   for (const ParmVarDecl *P : D->parameters()) {
     FieldTypeInfo *FieldInfo = nullptr;
     if (const auto *T = getDeclForType(P->getOriginalType())) {
       if (const auto *N = dyn_cast<EnumDecl>(T)) {
         FieldInfo = &I.Params.emplace_back(
             getUSRForDecl(N), N->getNameAsString(), InfoType::IT_enum,
             getInfoRelativePath(N), P->getNameAsString());
       } else if (const auto *N = dyn_cast<RecordDecl>(T)) {
         FieldInfo = &I.Params.emplace_back(
             getUSRForDecl(N), N->getNameAsString(), InfoType::IT_record,
             getInfoRelativePath(N), P->getNameAsString());
       }
       // Otherwise fall through to the default case below.
     }
 
     if (!FieldInfo) {
       FieldInfo = &I.Params.emplace_back(P->getOriginalType().getAsString(),
                                          P->getNameAsString());
     }
 
     if (const Expr *DefaultArg = P->getDefaultArg()) {
-      FieldInfo->DefaultValue = Lexer::getSourceText(
-          CharSourceRange::getTokenRange(DefaultArg->getSourceRange()),
-          D->getASTContext().getSourceManager(),
-          D->getASTContext().getLangOpts());
+      FieldInfo->DefaultValue = getSourceCode(D, DefaultArg->getSourceRange());
     }
   }
 }
 
 // TODO: Remove the serialization of Parents and VirtualParents, this
 // information is also extracted in the other definition of parseBases.
 static void parseBases(RecordInfo &I, const CXXRecordDecl *D) {
   // Don't parse bases if this isn't a definition.
emitInfo(const EnumDecl *D, const FullComment *FC, int LineNumber,
   EnumInfo Enum;
   bool IsInAnonymousNamespace = false;
   populateSymbolInfo(Enum, D, FC, LineNumber, File, IsFileInRootDir,
                      IsInAnonymousNamespace);
   if (!shouldSerializeInfo(PublicOnly, IsInAnonymousNamespace, D))
     return {};
 
   Enum.Scoped = D->isScoped();
+  if (D->isFixed())
+    Enum.BaseType = TypeInfo(D->getIntegerType().getAsString());
   parseEnumerators(Enum, D);
 
   // Put in global namespace
   if (Enum.Namespace.empty()) {
     auto ParentI = std::make_unique<NamespaceInfo>();
     ParentI->USR = SymbolID();
     ParentI->ChildEnums.emplace_back(std::move(Enum));
     ParentI->Path = getInfoRelativePath(ParentI->Namespace);

clang-tools-extra/clang-doc/YAMLGenerator.cpp

 //===-- YAMLGenerator.cpp - ClangDoc YAML -----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 // Implementation of the YAML generator, converting decl info into YAML output.
 //===----------------------------------------------------------------------===//
 
 #include "Generators.h"
 #include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace clang::doc;
 
+// These define YAML traits for decoding the listed values within a vector.
 LLVM_YAML_IS_SEQUENCE_VECTOR(FieldTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(MemberTypeInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Reference)
 LLVM_YAML_IS_SEQUENCE_VECTOR(Location)
 LLVM_YAML_IS_SEQUENCE_VECTOR(CommentInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(FunctionInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(EnumInfo)
+LLVM_YAML_IS_SEQUENCE_VECTOR(EnumValueInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(BaseRecordInfo)
 LLVM_YAML_IS_SEQUENCE_VECTOR(std::unique_ptr<CommentInfo>)
 LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::SmallString<16>)
 
 namespace llvm {
 namespace yaml {
 
 // Enumerations to YAML output.
static void mapping(IO &IO, BaseRecordInfo &I) {
     // clang::AccessSpecifier::AS_none is used as the default here because it's
     // the AS that shouldn't be part of the output. Even though AS_public is the
     // default in the struct, it should be displayed in the YAML output.
     IO.mapOptional("Access", I.Access, clang::AccessSpecifier::AS_none);
     IO.mapOptional("IsParent", I.IsParent, false);
   }
 };
 
+template <> struct MappingTraits<EnumValueInfo> {
+  static void mapping(IO &IO, EnumValueInfo &I) {
+    IO.mapOptional("Name", I.Name);
+
+    SmallString<16> ValueStr;
+    I.Value.toString(ValueStr);
+    IO.mapOptional("Value", ValueStr);
+    IO.mapOptional("Expr", I.ValueExpr, std::string());
+  }
+};
+
 template <> struct MappingTraits<EnumInfo> {
   static void mapping(IO &IO, EnumInfo &I) {
     SymbolInfoMapping(IO, I);
     IO.mapOptional("Scoped", I.Scoped, false);
+    IO.mapOptional("BaseType", I.BaseType);
     IO.mapOptional("Members", I.Members);
   }
 };
 
 template <> struct MappingTraits<FunctionInfo> {
   static void mapping(IO &IO, FunctionInfo &I) {
     SymbolInfoMapping(IO, I);
     IO.mapOptional("IsMethod", I.IsMethod, false);

clang-tools-extra/unittests/clang-doc/SerializeTest.cpp

TEST(SerializeTest, emitEnumInfo) {
   ExtractInfosFromCode("enum E { X, Y }; enum class G { A, B };", 2,
                        /*Public=*/false, Infos);
 
   NamespaceInfo *NamespaceWithEnum = InfoAsNamespace(Infos[0].get());
   NamespaceInfo ExpectedNamespaceWithEnum(EmptySID);
   EnumInfo E;
   E.Name = "E";
   E.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  E.Members.emplace_back("X");
-  E.Members.emplace_back("Y");
+  E.Members.emplace_back("X", llvm::APSInt::get(0));
+  E.Members.emplace_back("Y", llvm::APSInt::get(1));
   ExpectedNamespaceWithEnum.ChildEnums.emplace_back(std::move(E));
   CheckNamespaceInfo(&ExpectedNamespaceWithEnum, NamespaceWithEnum);
 
   NamespaceInfo *NamespaceWithScopedEnum = InfoAsNamespace(Infos[1].get());
   NamespaceInfo ExpectedNamespaceWithScopedEnum(EmptySID);
   EnumInfo G;
   G.Name = "G";
   G.Scoped = true;
   G.DefLoc = Location(0, llvm::SmallString<16>{"test.cpp"});
-  G.Members.emplace_back("A");
-  G.Members.emplace_back("B");
+  G.Members.emplace_back("A", llvm::APSInt::get(0));
+  G.Members.emplace_back("B", llvm::APSInt::get(1));
   ExpectedNamespaceWithScopedEnum.ChildEnums.emplace_back(std::move(G));
   CheckNamespaceInfo(&ExpectedNamespaceWithScopedEnum, NamespaceWithScopedEnum);
 }
 
 TEST(SerializeTest, emitUndefinedRecordInfo) {
   EmittedInfoList Infos;
   ExtractInfosFromCode("class E;", 2, /*Public=*/false, Infos);
 

clang-tools-extra/unittests/clang-doc/YAMLGeneratorTest.cpp

ReturnType:
   Type:
     Name:            'void'
     Path:            'path/to/void'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
 }
 
-TEST(YAMLGeneratorTest, emitEnumYAML) {
+// Tests the equivalent of:
+// namespace A {
+// enum e { X };
+// }
+TEST(YAMLGeneratorTest, emitSimpleEnumYAML) {
   EnumInfo I;
   I.Name = "e";
   I.Namespace.emplace_back(EmptySID, "A", InfoType::IT_namespace);
 
   I.DefLoc = Location(10, llvm::SmallString<16>{"test.cpp"});
   I.Loc.emplace_back(12, llvm::SmallString<16>{"test.cpp"});
 
   I.Members.emplace_back("X");
-  I.Scoped = true;
+  I.Scoped = false;
 
   auto G = getYAMLGenerator();
   assert(G);
   std::string Buffer;
   llvm::raw_string_ostream Actual(Buffer);
   auto Err = G->generateDocForInfo(&I, Actual, ClangDocContext());
   assert(!Err);
   std::string Expected =
       R"raw(---
 USR:             '0000000000000000000000000000000000000000'
 Name:            'e'
 Namespace:
   - Type:            Namespace
     Name:            'A'
 DefLocation:
   LineNumber:      10
   Filename:        'test.cpp'
 Location:
   - LineNumber:      12
     Filename:        'test.cpp'
+Members:
+  - Name:            'X'
+    Value:           '0'
+...
+)raw";
+  EXPECT_EQ(Expected, Actual.str());
+}
+
+// Tests the equivalent of:
+// enum class e : short { X = FOO_BAR + 2 };
+TEST(YAMLGeneratorTest, enumTypedScopedEnumYAML) {
+  EnumInfo I;
+  I.Name = "e";
+
+  I.Members.emplace_back("X", llvm::APSInt::get(-9876), "FOO_BAR + 2");
+  I.Scoped = true;
+  I.BaseType = TypeInfo("short");
+
+  auto G = getYAMLGenerator();
+  assert(G);
+  std::string Buffer;
+  llvm::raw_string_ostream Actual(Buffer);
+  auto Err = G->generateDocForInfo(&I, Actual, ClangDocContext());
+  assert(!Err);
+  std::string Expected =
+      R"raw(---
+USR:             '0000000000000000000000000000000000000000'
+Name:            'e'
 Scoped:          true
+BaseType:
+  Type:
+    Name:            'short'
 Members:
-  - 'X'
+  - Name:            'X'
+    Value:           '-9876'
+    Expr:            'FOO_BAR + 2'
 ...
 )raw";
   EXPECT_EQ(Expected, Actual.str());
 }
 
 TEST(YAMLGeneratorTest, emitCommentYAML) {
   FunctionInfo I;
   I.Name = "f";