[clang] Increase the bitness of data length in ASTDeclContextNameLookup

[danlark]

Summary

We faced the assert of too many declarations for serialized lookup for a very huge generated C++ target. We
think moving the type to uint32_t helps us. As I am not sure in the fix, please validate the correctness.

I don't have commit rights. Email: [e-mail]. Name: Danila Kutenin.

[rsmith]

Thanks for the diagnosis and the patch!

We spend a significant amount of our AST file size on these lookup tables, so I think the extra two bytes per lookup entry is best avoided if possible. We also have the same problem for other on-disk hash tables. I went ahead and generalized this to cover all of our on-disk hash tables and switched from fixed-width to variable-width encoding in rGe0589d70fb8e29842fab228df716f73b378710a6.

[danlark]

Thanks!

clang/lib/Serialization/ASTReader.cpp

ASTDeclContextNameLookupTrait::ReadFileRef(const unsigned char *&d) {
   return Reader.getLocalModuleFile(F, ModuleFileID);
 }
 
 std::pair<unsigned, unsigned>
 ASTDeclContextNameLookupTrait::ReadKeyDataLength(const unsigned char *&d) {
   using namespace llvm::support;
 
   unsigned KeyLen = endian::readNext<uint16_t, little, unaligned>(d);
-  unsigned DataLen = endian::readNext<uint16_t, little, unaligned>(d);
+  unsigned DataLen = endian::readNext<uint32_t, little, unaligned>(d);
   return std::make_pair(KeyLen, DataLen);
 }
 
 ASTDeclContextNameLookupTrait::internal_key_type
 ASTDeclContextNameLookupTrait::ReadKey(const unsigned char *d, unsigned) {
   using namespace llvm::support;
 
   auto Kind = (DeclarationName::NameKind)*d++;

clang/lib/Serialization/ASTWriter.cpp

std::pair<unsigned, unsigned> EmitKeyDataLength(raw_ostream &Out,
     case DeclarationName::CXXConversionFunctionName:
     case DeclarationName::CXXUsingDirective:
       break;
     }
     LE.write<uint16_t>(KeyLen);
 
     // 4 bytes for each DeclID.
     unsigned DataLen = 4 * (Lookup.second - Lookup.first);
-    assert(uint16_t(DataLen) == DataLen &&
-           "too many decls for serialized lookup result");
-    LE.write<uint16_t>(DataLen);
+    LE.write<uint32_t>(DataLen);
 
     return std::make_pair(KeyLen, DataLen);
   }
 
   void EmitKey(raw_ostream &Out, DeclarationNameKey Name, unsigned) {
     using namespace llvm::support;
 
     endian::Writer LE(Out, little);