[clang][NFC] Rearrange Comment Token and Lexer fields to reduce padding

[njames93]

Summary

Rearrange the fields to reduce the size of the classes

[gribozavr2]

clang/include/clang/AST/CommentLexer.h

244

I'm not a fan of this change to Lexer because it breaks the grouping of fields: VerbatimBlockEndCommandName is no longer next to State.

There is only ever one Lexer class instance anyway, so any memory savings are not important I think.

[njames93]

Keep VerbatimBlockEndCommandName after LexerState while preserving smaller size

[njames93]

clang/include/clang/AST/CommentLexer.h

244

Fair point, I've fixed that grouping of fields while keeping the smaller size, but if you think its still not worth it, I'll revert that change.
The token class size change would be worth it as they appear in vectors.

[njames93]

Fix field initialisation order in constructor

[gribozavr2]

clang/include/clang/AST/CommentLexer.h

70

Could you also swap Length and IntVal? It would be nice to keep Length and TextPtr next to each other.

clang/include/clang/AST/CommentLexer.h

class Token {
   friend class TextTokenRetokenizer;
 
   /// The location of the token.
   SourceLocation Loc;
 
   /// The actual kind of the token.
   tok::TokenKind Kind;
 
-  /// Length of the token spelling in comment.  Can be 0 for synthenized
-  /// tokens.
-  unsigned Length;
-
-  /// Contains text value associated with a token.
-  const char *TextPtr;
-
   /// Integer value associated with a token.
   ///
   /// If the token is a known command, contains command ID and TextPtr is
   /// unused (command spelling can be found with CommandTraits).  Otherwise,
   /// contains the length of the string that starts at TextPtr.
   unsigned IntVal;
 
+  /// Length of the token spelling in comment.  Can be 0 for synthenized
+  /// tokens.
+  unsigned Length;
+
+  /// Contains text value associated with a token.
+  const char *TextPtr;
+
 public:
   SourceLocation getLocation() const LLVM_READONLY { return Loc; }
   void setLocation(SourceLocation SL) { Loc = SL; }
 
   SourceLocation getEndLocation() const LLVM_READONLY {
     if (Length == 0 || Length == 1)
       return Loc;
     return Loc.getLocWithOffset(Length - 1);
private:
   llvm::BumpPtrAllocator &Allocator;
 
   DiagnosticsEngine &Diags;
 
   const CommandTraits &Traits;
 
   const char *const BufferStart;
   const char *const BufferEnd;
-  SourceLocation FileLoc;
 
   const char *BufferPtr;
 
   /// One past end pointer for the current comment.  For BCPL comments points
   /// to newline or BufferEnd, for C comments points to star in '*/'.
   const char *CommentEnd;
 
-  enum LexerCommentState {
+  SourceLocation FileLoc;
+
+  /// If true, the commands, html tags, etc will be parsed and reported as
+  /// separate tokens inside the comment body. If false, the comment text will
+  /// be parsed into text and newline tokens.
+  bool ParseCommands;
+
+  enum LexerCommentState : uint8_t {
     LCS_BeforeComment,
     LCS_InsideBCPLComment,
     LCS_InsideCComment,
     LCS_BetweenComments
   };
 
   /// Low-level lexer state, track if we are inside or outside of comment.
   LexerCommentState CommentState;
 
-  enum LexerState {
+  enum LexerState : uint8_t {
     /// Lexing normal comment text
     LS_Normal,
 
     /// Finished lexing verbatim block beginning command, will lex first body
     /// line.
     LS_VerbatimBlockFirstLine,
 
     /// Lexing verbatim block body line-by-line, skipping line-starting
private:
 
   /// Current lexing mode.
   LexerState State;
 
   /// If State is LS_VerbatimBlock, contains the name of verbatim end
   /// command, including command marker.
   SmallString<16> VerbatimBlockEndCommandName;
 
-  /// If true, the commands, html tags, etc will be parsed and reported as
-  /// separate tokens inside the comment body. If false, the comment text will
-  /// be parsed into text and newline tokens.
-  bool ParseCommands;
-
   /// Given a character reference name (e.g., "lt"), return the character that
   /// it stands for (e.g., "<").
   StringRef resolveHTMLNamedCharacterReference(StringRef Name) const;
 
   /// Given a Unicode codepoint as base-10 integer, return the character.
   StringRef resolveHTMLDecimalCharacterReference(StringRef Name) const;
 
   /// Given a Unicode codepoint as base-16 integer, return the character.

clang/lib/AST/CommentLexer.cpp

void Lexer::lexHTMLEndTag(Token &T) {
   assert(BufferPtr != CommentEnd && *BufferPtr == '>');
 
   formTokenWithChars(T, BufferPtr + 1, tok::html_greater);
   State = LS_Normal;
 }
 
 Lexer::Lexer(llvm::BumpPtrAllocator &Allocator, DiagnosticsEngine &Diags,
              const CommandTraits &Traits, SourceLocation FileLoc,
-             const char *BufferStart, const char *BufferEnd,
-             bool ParseCommands)
+             const char *BufferStart, const char *BufferEnd, bool ParseCommands)
     : Allocator(Allocator), Diags(Diags), Traits(Traits),
-      BufferStart(BufferStart), BufferEnd(BufferEnd), FileLoc(FileLoc),
-      BufferPtr(BufferStart), CommentState(LCS_BeforeComment), State(LS_Normal),
-      ParseCommands(ParseCommands) {}
+      BufferStart(BufferStart), BufferEnd(BufferEnd), BufferPtr(BufferStart),
+      FileLoc(FileLoc), ParseCommands(ParseCommands),
+      CommentState(LCS_BeforeComment), State(LS_Normal) {}
 
 void Lexer::lex(Token &T) {
 again:
   switch (CommentState) {
   case LCS_BeforeComment:
     if (BufferPtr == BufferEnd) {
       formTokenWithChars(T, BufferPtr, tok::eof);
       return;