[CodeView] Fix random access of types names

[zturner]

Summary

Suppose we had a type index offsets array with a boundary at type index N. Then you request the name of the type with index N+1, and that name requires the name of index N-1 (think a parameter list, for example). We didn't handle this, and we would print something like (<unknown UDT>, <unknown UDT>).

The fix for this is not entirely trivial, and speaks to a larger problem. I think we need to kill TypeDatabase, or at the very least kill TypeDatabaseVisitor. We need a thing that doesn't do any caching whatsoever, just given a type index it can compute the type name "the slow way". The reason for the bug is that we don't have anything like that. Everything goes through the type database, and if we've visited a record, then we're "done". It doesn't know how to do the expensive thing of re-visiting dependent records if they've not yet been visited.

What I've done here is more or less copied the code (albeit greatly simplified) from TypeDatabaseVisitor, but wrapped it in an interface that just returns a std::string. The logic of caching the name is now in LazyRandomTypeCollection. Eventually I'd like to move the record database here as well and the visited record bitfield here as well, at which point we can actually just delete TypeDatabase. I don't see any reason for it if a "sequential" collection is just a special case of a random access collection with an empty partial offsets array.

[rnk]

lgtm

I'm not really that concerned with the efficiency of calculating type names. We don't recurse through aggregate types, so the biggest most complex type we could have is probably a function type.

llvm/trunk/include/llvm/DebugInfo/CodeView/LazyRandomTypeCollection.h

private:
   const TypeDatabase &database() const { return Database; }
   Error ensureTypeExists(TypeIndex Index);
 
   Error visitRangeForType(TypeIndex TI);
   Error fullScanForType(TypeIndex TI);
   Error visitRange(TypeIndex Begin, uint32_t BeginOffset, TypeIndex End);
   Error visitOneRecord(TypeIndex TI, uint32_t Offset, CVType &Record);
 
+  BumpPtrAllocator Allocator;
+  StringSaver NameStorage;
+
+  SmallVector<StringRef, 10> TypeNames;
+
   /// Visited records get automatically added to the type database.
   TypeDatabase Database;
 
   /// The type array to allow random access visitation of.
   CVTypeArray Types;
 
   /// The database visitor which adds new records to the database.
   TypeDatabaseVisitor DatabaseVisitor;

llvm/trunk/include/llvm/DebugInfo/CodeView/TypeName.h

+//===- TypeName.h --------------------------------------------- *- C++ --*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_CODEVIEW_TYPENAME_H
+#define LLVM_DEBUGINFO_CODEVIEW_TYPENAME_H
+
+#include "llvm/DebugInfo/CodeView/TypeCollection.h"
+#include "llvm/DebugInfo/CodeView/TypeIndex.h"
+
+namespace llvm {
+namespace codeview {
+std::string computeTypeName(TypeCollection &Types, TypeIndex Index);
+}
+} // namespace llvm
+
+#endif

llvm/trunk/lib/DebugInfo/CodeView/CMakeLists.txt

add_llvm_library(LLVMDebugInfoCodeView
   SymbolRecordMapping.cpp
   SymbolDumper.cpp
   SymbolSerializer.cpp
   TypeDatabase.cpp
   TypeDatabaseVisitor.cpp
   TypeDumpVisitor.cpp
   TypeIndex.cpp
   TypeIndexDiscovery.cpp
+  TypeName.cpp
   TypeRecordMapping.cpp
   TypeSerializer.cpp
   TypeStreamMerger.cpp
   TypeTableCollection.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/DebugInfo/CodeView
   )

llvm/trunk/lib/DebugInfo/CodeView/LazyRandomTypeCollection.cpp

 //===- LazyRandomTypeCollection.cpp ---------------------------- *- C++--*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/CodeView/LazyRandomTypeCollection.h"
 
 #include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
 #include "llvm/DebugInfo/CodeView/CodeViewError.h"
 #include "llvm/DebugInfo/CodeView/TypeDatabase.h"
+#include "llvm/DebugInfo/CodeView/TypeName.h"
 #include "llvm/DebugInfo/CodeView/TypeServerHandler.h"
 #include "llvm/DebugInfo/CodeView/TypeVisitorCallbacks.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 
 static void error(Error &&EC) {
   assert(!static_cast<bool>(EC));
   if (EC)
     consumeError(std::move(EC));
 }
 
 LazyRandomTypeCollection::LazyRandomTypeCollection(uint32_t RecordCountHint)
     : LazyRandomTypeCollection(CVTypeArray(), RecordCountHint,
                                PartialOffsetArray()) {}
 
 LazyRandomTypeCollection::LazyRandomTypeCollection(
     const CVTypeArray &Types, uint32_t RecordCountHint,
     PartialOffsetArray PartialOffsets)
-    : Database(RecordCountHint), Types(Types), DatabaseVisitor(Database),
-      PartialOffsets(PartialOffsets) {
+    : NameStorage(Allocator), Database(RecordCountHint), Types(Types),
+      DatabaseVisitor(Database), PartialOffsets(PartialOffsets) {
   KnownOffsets.resize(Database.capacity());
 }
 
 LazyRandomTypeCollection::LazyRandomTypeCollection(ArrayRef<uint8_t> Data,
                                                    uint32_t RecordCountHint)
     : LazyRandomTypeCollection(RecordCountHint) {
   reset(Data);
 }

 }
 
 CVType LazyRandomTypeCollection::getType(TypeIndex Index) {
   error(ensureTypeExists(Index));
   return Database.getTypeRecord(Index);
 }
 
 StringRef LazyRandomTypeCollection::getTypeName(TypeIndex Index) {
-  if (!Index.isSimple()) {
-    // Try to make sure the type exists.  Even if it doesn't though, it may be
-    // because we're dumping a symbol stream with no corresponding type stream
-    // present, in which case we still want to be able to print <unknown UDT>
-    // for the type names.
-    consumeError(ensureTypeExists(Index));
-  }
+  if (Index.isNoneType() || Index.isSimple())
+    return TypeIndex::simpleTypeName(Index);
 
-  return Database.getTypeName(Index);
+  uint32_t I = Index.toArrayIndex();
+  if (I >= TypeNames.size())
+    TypeNames.resize(I + 1);
+
+  if (TypeNames[I].data() == nullptr) {
+    StringRef Result = NameStorage.save(computeTypeName(*this, Index));
+    TypeNames[I] = Result;
+  }
+  return TypeNames[I];
 }
 
 bool LazyRandomTypeCollection::contains(TypeIndex Index) {
   return Database.contains(Index);
 }
 
 uint32_t LazyRandomTypeCollection::size() { return Database.size(); }
 

llvm/trunk/lib/DebugInfo/CodeView/TypeName.cpp

+//===- TypeName.cpp ------------------------------------------- *- C++ --*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/CodeView/TypeName.h"
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
+#include "llvm/DebugInfo/CodeView/TypeVisitorCallbacks.h"
+#include "llvm/Support/FormatVariadic.h"
+
+using namespace llvm;
+using namespace llvm::codeview;
+
+namespace {
+class TypeNameComputer : public TypeVisitorCallbacks {
+  /// The type collection.  Used to calculate names of nested types.
+  TypeCollection &Types;
+  TypeIndex CurrentTypeIndex = TypeIndex::None();
+
+  /// Name of the current type. Only valid before visitTypeEnd.
+  SmallString<256> Name;
+
+public:
+  explicit TypeNameComputer(TypeCollection &Types) : Types(Types) {}
+
+  StringRef name() const { return Name; }
+
+  /// Paired begin/end actions for all types. Receives all record data,
+  /// including the fixed-length record prefix.
+  Error visitTypeBegin(CVType &Record) override;
+  Error visitTypeBegin(CVType &Record, TypeIndex Index) override;
+  Error visitTypeEnd(CVType &Record) override;
+
+#define TYPE_RECORD(EnumName, EnumVal, Name)                                   \
+  Error visitKnownRecord(CVType &CVR, Name##Record &Record) override;
+#define TYPE_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
+#define MEMBER_RECORD(EnumName, EnumVal, Name)
+#include "llvm/DebugInfo/CodeView/CodeViewTypes.def"
+};
+} // namespace
+
+Error TypeNameComputer::visitTypeBegin(CVType &Record) {
+  llvm_unreachable("Must call visitTypeBegin with a TypeIndex!");
+  return Error::success();
+}
+
+Error TypeNameComputer::visitTypeBegin(CVType &Record, TypeIndex Index) {
+  // Reset Name to the empty string. If the visitor sets it, we know it.
+  Name = "";
+  CurrentTypeIndex = Index;
+  return Error::success();
+}
+
+Error TypeNameComputer::visitTypeEnd(CVType &CVR) { return Error::success(); }
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         FieldListRecord &FieldList) {
+  Name = "<field list>";
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVRecord<TypeLeafKind> &CVR,
+                                         StringIdRecord &String) {
+  Name = String.getString();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, ArgListRecord &Args) {
+  auto Indices = Args.getIndices();
+  uint32_t Size = Indices.size();
+  Name = "(";
+  for (uint32_t I = 0; I < Size; ++I) {
+    assert(Indices[I] < CurrentTypeIndex);
+
+    Name.append(Types.getTypeName(Indices[I]));
+    if (I + 1 != Size)
+      Name.append(", ");
+  }
+  Name.push_back(')');
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         StringListRecord &Strings) {
+  auto Indices = Strings.getIndices();
+  uint32_t Size = Indices.size();
+  Name = "\"";
+  for (uint32_t I = 0; I < Size; ++I) {
+    Name.append(Types.getTypeName(Indices[I]));
+    if (I + 1 != Size)
+      Name.append("\" \"");
+  }
+  Name.push_back('\"');
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, ClassRecord &Class) {
+  Name = Class.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, UnionRecord &Union) {
+  Name = Union.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, EnumRecord &Enum) {
+  Name = Enum.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, ArrayRecord &AT) {
+  Name = AT.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, VFTableRecord &VFT) {
+  Name = VFT.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, MemberFuncIdRecord &Id) {
+  Name = Id.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, ProcedureRecord &Proc) {
+  StringRef Ret = Types.getTypeName(Proc.getReturnType());
+  StringRef Params = Types.getTypeName(Proc.getArgumentList());
+  Name = formatv("{0} {1}", Ret, Params).sstr<256>();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         MemberFunctionRecord &MF) {
+  StringRef Ret = Types.getTypeName(MF.getReturnType());
+  StringRef Class = Types.getTypeName(MF.getClassType());
+  StringRef Params = Types.getTypeName(MF.getArgumentList());
+  Name = formatv("{0} {1}::{2}", Ret, Class, Params).sstr<256>();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, FuncIdRecord &Func) {
+  Name = Func.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, TypeServer2Record &TS) {
+  Name = TS.getName();
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, PointerRecord &Ptr) {
+
+  if (Ptr.isPointerToMember()) {
+    const MemberPointerInfo &MI = Ptr.getMemberInfo();
+
+    StringRef Pointee = Types.getTypeName(Ptr.getReferentType());
+    StringRef Class = Types.getTypeName(MI.getContainingType());
+    Name = formatv("{0} {1}::*", Pointee, Class);
+  } else {
+    if (Ptr.isConst())
+      Name.append("const ");
+    if (Ptr.isVolatile())
+      Name.append("volatile ");
+    if (Ptr.isUnaligned())
+      Name.append("__unaligned ");
+
+    Name.append(Types.getTypeName(Ptr.getReferentType()));
+
+    if (Ptr.getMode() == PointerMode::LValueReference)
+      Name.append("&");
+    else if (Ptr.getMode() == PointerMode::RValueReference)
+      Name.append("&&");
+    else if (Ptr.getMode() == PointerMode::Pointer)
+      Name.append("*");
+  }
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, ModifierRecord &Mod) {
+  uint16_t Mods = static_cast<uint16_t>(Mod.getModifiers());
+
+  SmallString<256> TypeName;
+  if (Mods & uint16_t(ModifierOptions::Const))
+    Name.append("const ");
+  if (Mods & uint16_t(ModifierOptions::Volatile))
+    Name.append("volatile ");
+  if (Mods & uint16_t(ModifierOptions::Unaligned))
+    Name.append("__unaligned ");
+  Name.append(Types.getTypeName(Mod.getModifiedType()));
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         VFTableShapeRecord &Shape) {
+  Name = formatv("<vftable {0} methods>", Shape.getEntryCount());
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(
+    CVType &CVR, UdtModSourceLineRecord &ModSourceLine) {
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         UdtSourceLineRecord &SourceLine) {
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, BitFieldRecord &BF) {
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR,
+                                         MethodOverloadListRecord &Overloads) {
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, BuildInfoRecord &BI) {
+  return Error::success();
+}
+
+Error TypeNameComputer::visitKnownRecord(CVType &CVR, LabelRecord &R) {
+  return Error::success();
+}
+
+std::string llvm::codeview::computeTypeName(TypeCollection &Types,
+                                            TypeIndex Index) {
+  TypeNameComputer Computer(Types);
+  CVType Record = Types.getType(Index);
+  if (auto EC = visitTypeRecord(Record, Index, Computer)) {
+    consumeError(std::move(EC));
+    return "<unknown UDT>";
+  }
+  return Computer.name();
+}

llvm/trunk/unittests/DebugInfo/CodeView/RandomAccessVisitorTest.cpp

TEST_F(RandomAccessVisitorTest, InnerChunk) {
   for (uint32_t I = 4; I < 9; ++I)
     EXPECT_TRUE(ValidateDatabaseRecord(Types, I));
 
   // 5, 7
   EXPECT_EQ(2u, TestState->Callbacks.count());
   for (auto &I : enumerate(IndicesToVisit))
     EXPECT_TRUE(ValidateVisitedRecord(I.index(), I.value()));
 }
+
+TEST_F(RandomAccessVisitorTest, CrossChunkName) {
+  TypeTableBuilder Builder(GlobalState->Allocator);
+
+  // TypeIndex 0
+  ClassRecord Class(TypeRecordKind::Class);
+  Class.Name = "FooClass";
+  Class.Options = ClassOptions::None;
+  Class.MemberCount = 0;
+  Class.DerivationList = TypeIndex::fromArrayIndex(0);
+  Class.FieldList = TypeIndex::fromArrayIndex(0);
+  Class.VTableShape = TypeIndex::fromArrayIndex(0);
+  TypeIndex IndexZero = Builder.writeKnownType(Class);
+
+  // TypeIndex 1 refers to type index 0.
+  ModifierRecord Modifier(TypeRecordKind::Modifier);
+  Modifier.ModifiedType = TypeIndex::fromArrayIndex(0);
+  Modifier.Modifiers = ModifierOptions::Const;
+  TypeIndex IndexOne = Builder.writeKnownType(Modifier);
+
+  // set up a type stream that refers to the above two serialized records.
+  std::vector<CVType> TypeArray;
+  TypeArray.push_back(
+      CVType(static_cast<TypeLeafKind>(Class.Kind), Builder.records()[0]));
+  TypeArray.push_back(
+      CVType(static_cast<TypeLeafKind>(Modifier.Kind), Builder.records()[1]));
+  BinaryItemStream<CVType> ItemStream(llvm::support::little);
+  ItemStream.setItems(TypeArray);
+  VarStreamArray<CVType> TypeStream(ItemStream);
+
+  // Figure out the byte offset of the second item.
+  auto ItemOneIter = TypeStream.begin();
+  ++ItemOneIter;
+
+  // Set up a partial offsets buffer that contains the first and second items
+  // in separate chunks.
+  std::vector<TypeIndexOffset> TIO;
+  TIO.push_back({IndexZero, ulittle32_t(0u)});
+  TIO.push_back({IndexOne, ulittle32_t(ItemOneIter.offset())});
+  ArrayRef<uint8_t> Buffer(reinterpret_cast<const uint8_t *>(TIO.data()),
+                           TIO.size() * sizeof(TypeIndexOffset));
+
+  BinaryStreamReader Reader(Buffer, llvm::support::little);
+  FixedStreamArray<TypeIndexOffset> PartialOffsets;
+  ASSERT_THAT_ERROR(Reader.readArray(PartialOffsets, 2), Succeeded());
+
+  LazyRandomTypeCollection Types(TypeStream, 2, PartialOffsets);
+
+  StringRef Name = Types.getTypeName(IndexOne);
+  EXPECT_EQ("const FooClass", Name);
+}
 No newline at end of file