[PDB] Support following Type server records

[zturner]

Summary

[pdb] Add the ability to resolve TypeServer PDBs.

Some PDBs or object files can contain references to other PDBs
where the real type information lives.  When this happens,
all type indices in the original PDB are meaningless because
their records are not there.

With this patch we add the ability to pull type info from those
secondary PDBs, and update LLD to do so when it merges.

[ruiu]

llvm/include/llvm/DebugInfo/CodeView/TypeServerHandler.h

20

I think you want to define a virtual dtor to silence -Wnon-virtual-dtor.

llvm/lib/DebugInfo/PDB/Native/PDBTypeServerHandler.cpp

7

Can you write a file comment what we are doing in this file? E.g. a LF_TYPESERVER2 record referes an external pdb file, so we read that file and replace TypeServer2Record with the file contents.

33

Calling this function more than once with the same argument seems to add the same element to to SearchPaths. Is this a desired behavior?

40

You want to change this to Error and return Error::success() on success.

80

I believe that evaluating EC as a boolean value clears "checked" bits of the object, so you can safely ignore errors without calling this.

92

Please add a comment -- GUID must be the same.

[zturner]

llvm/lib/DebugInfo/PDB/Native/PDBTypeServerHandler.cpp

40

Error is not quite the same. Here I want to return 3 possible pieces of information:

1. Caller should assume the type server record has been handled.
2. Caller should assume the type server record has not been handled.
3. An error occurred. (For example the file was corrupt).

1 and 2 are distinct non-error conditions though. In case 1, the caller stops processing this record and goes to the next one in the stream. In case 2, caller keeps searching through the list of handlers in case another one can handle it. But in case 3, something bad happened and the caller probably has to exit and report an error.

[zturner]

llvm/lib/DebugInfo/PDB/Native/PDBTypeServerHandler.cpp

33

I don't feel strongly either way, if you think it's better to change it I can.

80

Error only clears checked bit if it was in a success state. You have to check it regardless of if it's success or failure, but if it's an error condition you also have to handle it.

/// Bool conversion. Returns true if this Error is in a failure state,
/// and false if it is in an accept state. If the error is in a Success state
/// it will be considered checked.
explicit operator bool() {
  setChecked(getPtr() == nullptr);
  return getPtr() != nullptr;
}

[ruiu]

LGTM

llvm/lib/DebugInfo/PDB/Native/PDBTypeServerHandler.cpp

17

no corresponding '('?

33

I don't have a preference, so this is fine.

lld/COFF/PDB.cpp

 #include "llvm/DebugInfo/MSF/MSFBuilder.h"
 #include "llvm/DebugInfo/MSF/MSFCommon.h"
 #include "llvm/DebugInfo/PDB/Native/DbiStream.h"
 #include "llvm/DebugInfo/PDB/Native/DbiStreamBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStreamBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFileBuilder.h"
+#include "llvm/DebugInfo/PDB/Native/PDBTypeServerHandler.h"
 #include "llvm/DebugInfo/PDB/Native/StringTableBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStream.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStreamBuilder.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include <memory>
 
 using namespace lld;
 using namespace lld::coff;
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::support;
static std::vector<uint8_t> mergeDebugT(SymbolTable *Symtab) {
   for (ObjectFile *File : Symtab->ObjectFiles) {
     ArrayRef<uint8_t> Data = getDebugSection(File, ".debug$T");
     if (Data.empty())
       continue;
 
     msf::ByteStream Stream(Data);
     codeview::CVTypeArray Types;
     msf::StreamReader Reader(Stream);
+    // Follow type servers.  If the same type server is encountered more than
+    // once for this instance of `PDBTypeServerHandler` (for example if many
+    // object files reference the same TypeServer), the types from the
+    // TypeServer will only be visited once.
+    pdb::PDBTypeServerHandler Handler;
+    Handler.addSearchPath(llvm::sys::path::parent_path(File->getName()));
     if (auto EC = Reader.readArray(Types, Reader.getLength()))
       fatal(EC, "Reader::readArray failed");
-    if (auto Err = codeview::mergeTypeStreams(Builder, Types))
+    if (auto Err = codeview::mergeTypeStreams(Builder, &Handler, Types))
       fatal(Err, "codeview::mergeTypeStreams failed");
   }
 
   // Construct section contents.
   std::vector<uint8_t> V;
   Builder.ForEachRecord([&](TypeIndex TI, ArrayRef<uint8_t> Rec) {
     V.insert(V.end(), Rec.begin(), Rec.end());
   });
   return V;
 }
 
 static void dumpDebugT(ScopedPrinter &W, ObjectFile *File) {
   ListScope LS(W, "DebugT");
   ArrayRef<uint8_t> Data = getDebugSection(File, ".debug$T");
   if (Data.empty())
     return;
 
   TypeDatabase TDB;
   TypeDumpVisitor TDV(TDB, &W, false);
+  // Use a default implementation that does not follow type servers and instead
+  // just dumps the contents of the TypeServer2 record.
   CVTypeDumper TypeDumper(TDB);
   if (auto EC = TypeDumper.dump(Data, TDV))
     fatal(EC, "CVTypeDumper::dump failed");
 }
 
 static void dumpDebugS(ScopedPrinter &W, ObjectFile *File) {
   ListScope LS(W, "DebugS");
   ArrayRef<uint8_t> Data = getDebugSection(File, ".debug$S");

llvm/include/llvm/DebugInfo/CodeView/CVTypeDumper.h

 #include "llvm/DebugInfo/CodeView/TypeRecord.h"
 #include "llvm/DebugInfo/CodeView/TypeVisitorCallbacks.h"
 #include "llvm/Support/ScopedPrinter.h"
 
 namespace llvm {
 
 namespace codeview {
 
+class TypeServerHandler;
+
 /// Dumper for CodeView type streams found in COFF object files and PDB files.
 class CVTypeDumper {
 public:
-  explicit CVTypeDumper(TypeDatabase &TypeDB) : TypeDB(TypeDB) {}
+  explicit CVTypeDumper(TypeDatabase &TypeDB,
+                        TypeServerHandler *Handler = nullptr)
+      : TypeDB(TypeDB), Handler(Handler) {}
 
   /// Dumps one type record.  Returns false if there was a type parsing error,
   /// and true otherwise.  This should be called in order, since the dumper
   /// maintains state about previous records which are necessary for cross
   /// type references.
   Error dump(const CVType &Record, TypeVisitorCallbacks &Dumper);
 
   /// Dumps the type records in Types. Returns false if there was a type stream
   /// parse error, and true otherwise.
   Error dump(const CVTypeArray &Types, TypeVisitorCallbacks &Dumper);
 
   /// Dumps the type records in Data. Returns false if there was a type stream
   /// parse error, and true otherwise. Use this method instead of the
   /// CVTypeArray overload when type records are laid out contiguously in
   /// memory.
   Error dump(ArrayRef<uint8_t> Data, TypeVisitorCallbacks &Dumper);
 
   static void printTypeIndex(ScopedPrinter &Printer, StringRef FieldName,
                              TypeIndex TI, TypeDatabase &DB);
 
 private:
   TypeDatabase &TypeDB;
+  TypeServerHandler *Handler;
 };
 
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_TYPEDUMPER_H

llvm/include/llvm/DebugInfo/CodeView/CVTypeVisitor.h

 //===- CVTypeVisitor.h ------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_CODEVIEW_CVTYPEVISITOR_H
 #define LLVM_DEBUGINFO_CODEVIEW_CVTYPEVISITOR_H
 
-#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/TinyPtrVector.h"
 #include "llvm/DebugInfo/CodeView/CVRecord.h"
 #include "llvm/DebugInfo/CodeView/TypeRecord.h"
+#include "llvm/DebugInfo/CodeView/TypeServerHandler.h"
 #include "llvm/DebugInfo/CodeView/TypeVisitorCallbacks.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace codeview {
 
 class CVTypeVisitor {
 public:
   explicit CVTypeVisitor(TypeVisitorCallbacks &Callbacks);
 
+  void addTypeServerHandler(TypeServerHandler &Handler);
+
   Error visitTypeRecord(CVType &Record);
   Error visitMemberRecord(CVMemberRecord &Record);
 
   /// Visits the type records in Data. Sets the error flag on parse failures.
   Error visitTypeStream(const CVTypeArray &Types);
+  Error visitTypeStream(CVTypeRange Types);
 
   Error visitFieldListMemberStream(ArrayRef<uint8_t> FieldList);
   Error visitFieldListMemberStream(msf::StreamReader Reader);
 
 private:
   /// The interface to the class that gets notified of each visitation.
   TypeVisitorCallbacks &Callbacks;
+
+  TinyPtrVector<TypeServerHandler *> Handlers;
 };
 
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_CVTYPEVISITOR_H

llvm/include/llvm/DebugInfo/CodeView/CodeViewError.h

 
 namespace llvm {
 namespace codeview {
 enum class cv_error_code {
   unspecified = 1,
   insufficient_buffer,
   operation_unsupported,
   corrupt_record,
+  no_records,
   unknown_member_record,
 };
 
 /// Base class for errors originating when parsing raw PDB files
 class CodeViewError : public ErrorInfo<CodeViewError> {
 public:
   static char ID;
   CodeViewError(cv_error_code C);

llvm/include/llvm/DebugInfo/CodeView/TypeRecord.h

 
 typedef CVRecord<TypeLeafKind> CVType;
 
 struct CVMemberRecord {
   TypeLeafKind Kind;
   ArrayRef<uint8_t> Data;
 };
 typedef msf::VarStreamArray<CVType> CVTypeArray;
+typedef iterator_range<CVTypeArray::Iterator> CVTypeRange;
 
 /// Equvalent to CV_fldattr_t in cvinfo.h.
 struct MemberAttributes {
   uint16_t Attrs = 0;
   enum {
     MethodKindShift = 2,
   };
   MemberAttributes() = default;

llvm/include/llvm/DebugInfo/CodeView/TypeServerHandler.h

+//===- TypeServerHandler.h --------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_CODEVIEW_TYPESERVERHANDLER_H
+#define LLVM_DEBUGINFO_CODEVIEW_TYPESERVERHANDLER_H
+
+#include "llvm/DebugInfo/CodeView/TypeRecord.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+namespace codeview {
+class TypeVisitorCallbacks;
+
+class TypeServerHandler {
+public:
+  /// Handle a TypeServer record.  If the implementation returns true
+  /// the record will not be processed by the top-level visitor.  If
+  /// it returns false, it will be processed.  If it returns an Error,
+  /// then the top-level visitor will fail.
+  virtual Expected<bool> handle(TypeServer2Record &TS,
+                                TypeVisitorCallbacks &Callbacks) {
+    return false;
+  }
+};
+}
+}
+
+#endif

llvm/include/llvm/DebugInfo/CodeView/TypeStreamMerger.h

 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/DebugInfo/CodeView/TypeTableBuilder.h"
 #include "llvm/Support/Error.h"
 
 namespace llvm {
 namespace codeview {
 
+class TypeServerHandler;
+
 /// Merges one type stream into another. Returns true on success.
-Error mergeTypeStreams(TypeTableBuilder &DestStream, const CVTypeArray &Types);
+Error mergeTypeStreams(TypeTableBuilder &DestStream, TypeServerHandler *Handler,
+                       const CVTypeArray &Types);
 
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_TYPESTREAMMERGER_H

llvm/include/llvm/DebugInfo/PDB/Native/PDBFile.h

 class PublicsStream;
 class SymbolStream;
 class TpiStream;
 
 class PDBFile : public msf::IMSFFile {
   friend PDBFileBuilder;
 
 public:
-  PDBFile(std::unique_ptr<msf::ReadableStream> PdbFileBuffer,
+  PDBFile(StringRef Path, std::unique_ptr<msf::ReadableStream> PdbFileBuffer,
           BumpPtrAllocator &Allocator);
   ~PDBFile() override;
 
+  StringRef getFileDirectory() const;
+  StringRef getFilePath() const;
+
   uint32_t getFreeBlockMapBlock() const;
   uint32_t getUnknown1() const;
 
   uint32_t getBlockSize() const override;
   uint32_t getBlockCount() const override;
   uint32_t getNumDirectoryBytes() const;
   uint32_t getBlockMapIndex() const;
   uint32_t getNumDirectoryBlocks() const;
public:
   bool hasStringTable();
 
 private:
   Expected<std::unique_ptr<msf::MappedBlockStream>>
   safelyCreateIndexedStream(const msf::MSFLayout &Layout,
                             const msf::ReadableStream &MsfData,
                             uint32_t StreamIndex) const;
 
+  std::string FilePath;
   BumpPtrAllocator &Allocator;
 
   std::unique_ptr<msf::ReadableStream> Buffer;
 
   std::vector<uint32_t> FpmPages;
   msf::MSFLayout ContainerLayout;
 
   std::unique_ptr<GlobalsStream> Globals;

llvm/include/llvm/DebugInfo/PDB/Native/PDBTypeServerHandler.h

+//===- PDBTypeServerHandler.h -----------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_PDB_PDBTYPESERVERHANDLER_H
+#define LLVM_DEBUGINFO_PDB_PDBTYPESERVERHANDLER_H
+
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
+#include "llvm/DebugInfo/CodeView/TypeRecord.h"
+#include "llvm/DebugInfo/CodeView/TypeServerHandler.h"
+#include "llvm/DebugInfo/PDB/Native/NativeSession.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+
+#include <memory>
+#include <string>
+
+namespace llvm {
+namespace pdb {
+class NativeSession;
+
+class PDBTypeServerHandler : public codeview::TypeServerHandler {
+public:
+  PDBTypeServerHandler(bool RevisitAlways = false);
+
+  void addSearchPath(StringRef Path);
+  Expected<bool> handle(codeview::TypeServer2Record &TS,
+                        codeview::TypeVisitorCallbacks &Callbacks) override;
+
+private:
+  Expected<bool> handleInternal(PDBFile &File,
+                                codeview::TypeVisitorCallbacks &Callbacks);
+
+  bool RevisitAlways;
+  std::unique_ptr<NativeSession> Session;
+  SmallVector<SmallString<64>, 4> SearchPaths;
+};
+}
+}
+
+#endif

llvm/include/llvm/DebugInfo/PDB/Native/TpiStream.h

public:
   uint16_t getTypeHashStreamAuxIndex() const;
 
   uint32_t getHashKeySize() const;
   uint32_t NumHashBuckets() const;
   msf::FixedStreamArray<support::ulittle32_t> getHashValues() const;
   msf::FixedStreamArray<TypeIndexOffset> getTypeIndexOffsets() const;
   HashTable &getHashAdjusters();
 
-  iterator_range<codeview::CVTypeArray::Iterator> types(bool *HadError) const;
+  codeview::CVTypeRange types(bool *HadError) const;
 
   Error commit();
 
 private:
   Error verifyHashValues();
 
   const PDBFile &Pdb;
   std::unique_ptr<msf::MappedBlockStream> Stream;

llvm/lib/DebugInfo/CodeView/CVTypeDumper.cpp

Error CVTypeDumper::dump(const CVType &Record, TypeVisitorCallbacks &Dumper) {
   TypeDatabaseVisitor DBV(TypeDB);
   TypeDeserializer Deserializer;
   TypeVisitorCallbackPipeline Pipeline;
   Pipeline.addCallbackToPipeline(Deserializer);
   Pipeline.addCallbackToPipeline(DBV);
   Pipeline.addCallbackToPipeline(Dumper);
 
   CVTypeVisitor Visitor(Pipeline);
+  if (Handler)
+    Visitor.addTypeServerHandler(*Handler);
 
   CVType RecordCopy = Record;
   if (auto EC = Visitor.visitTypeRecord(RecordCopy))
     return EC;
   return Error::success();
 }
 
 Error CVTypeDumper::dump(const CVTypeArray &Types,
                          TypeVisitorCallbacks &Dumper) {
   TypeDatabaseVisitor DBV(TypeDB);
   TypeDeserializer Deserializer;
   TypeVisitorCallbackPipeline Pipeline;
   Pipeline.addCallbackToPipeline(Deserializer);
   Pipeline.addCallbackToPipeline(DBV);
   Pipeline.addCallbackToPipeline(Dumper);
 
   CVTypeVisitor Visitor(Pipeline);
+  if (Handler)
+    Visitor.addTypeServerHandler(*Handler);
 
   if (auto EC = Visitor.visitTypeStream(Types))
     return EC;
   return Error::success();
 }
 
 Error CVTypeDumper::dump(ArrayRef<uint8_t> Data, TypeVisitorCallbacks &Dumper) {
   msf::ByteStream Stream(Data);

llvm/lib/DebugInfo/CodeView/CVTypeVisitor.cpp

 //===- CVTypeVisitor.cpp ----------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
 
 #include "llvm/DebugInfo/CodeView/CodeViewError.h"
+#include "llvm/DebugInfo/CodeView/TypeDatabase.h"
+#include "llvm/DebugInfo/CodeView/TypeDatabaseVisitor.h"
 #include "llvm/DebugInfo/CodeView/TypeDeserializer.h"
+#include "llvm/DebugInfo/CodeView/TypeRecordMapping.h"
+#include "llvm/DebugInfo/CodeView/TypeServerHandler.h"
 #include "llvm/DebugInfo/CodeView/TypeVisitorCallbackPipeline.h"
 #include "llvm/DebugInfo/MSF/ByteStream.h"
+#include "llvm/DebugInfo/MSF/StreamReader.h"
 
 using namespace llvm;
 using namespace llvm::codeview;
 
 CVTypeVisitor::CVTypeVisitor(TypeVisitorCallbacks &Callbacks)
     : Callbacks(Callbacks) {}
 
 template <typename T>
-static Error visitKnownRecord(CVType &Record, TypeVisitorCallbacks &Callbacks) {
+static Error visitKnownRecord(CVTypeVisitor &Visitor, CVType &Record,
+                              TypeVisitorCallbacks &Callbacks) {
   TypeRecordKind RK = static_cast<TypeRecordKind>(Record.Type);
   T KnownRecord(RK);
   if (auto EC = Callbacks.visitKnownRecord(Record, KnownRecord))
     return EC;
   return Error::success();
 }
 
 template <typename T>
 static Error visitKnownMember(CVMemberRecord &Record,
                               TypeVisitorCallbacks &Callbacks) {
   TypeRecordKind RK = static_cast<TypeRecordKind>(Record.Kind);
   T KnownRecord(RK);
   if (auto EC = Callbacks.visitKnownMember(Record, KnownRecord))
     return EC;
   return Error::success();
 }
 
+static Expected<TypeServer2Record> deserializeTypeServerRecord(CVType &Record) {
+  class StealTypeServerVisitor : public TypeVisitorCallbacks {
+  public:
+    explicit StealTypeServerVisitor(TypeServer2Record &TR) : TR(TR) {}
+
+    Error visitKnownRecord(CVType &CVR, TypeServer2Record &Record) override {
+      TR = Record;
+      return Error::success();
+    }
+
+  private:
+    TypeServer2Record &TR;
+  };
+
+  TypeServer2Record R(TypeRecordKind::TypeServer2);
+  TypeDeserializer Deserializer;
+  StealTypeServerVisitor Thief(R);
+  TypeVisitorCallbackPipeline Pipeline;
+  Pipeline.addCallbackToPipeline(Deserializer);
+  Pipeline.addCallbackToPipeline(Thief);
+  CVTypeVisitor Visitor(Pipeline);
+  if (auto EC = Visitor.visitTypeRecord(Record))
+    return std::move(EC);
+
+  return R;
+}
+
+void CVTypeVisitor::addTypeServerHandler(TypeServerHandler &Handler) {
+  Handlers.push_back(&Handler);
+}
+
 Error CVTypeVisitor::visitTypeRecord(CVType &Record) {
+  if (Record.Type == TypeLeafKind::LF_TYPESERVER2 && !Handlers.empty()) {
+    auto TS = deserializeTypeServerRecord(Record);
+    if (!TS)
+      return TS.takeError();
+
+    for (auto Handler : Handlers) {
+      auto ExpectedResult = Handler->handle(*TS, Callbacks);
+      // If there was an error, return the error.
+      if (!ExpectedResult)
+        return ExpectedResult.takeError();
+
+      // If the handler processed the record, return success.
+      if (*ExpectedResult)
+        return Error::success();
+
+      // Otherwise keep searching for a handler, eventually falling out and
+      // using the default record handler.
+    }
+  }
+
   if (auto EC = Callbacks.visitTypeBegin(Record))
     return EC;
 
   switch (Record.Type) {
   default:
     if (auto EC = Callbacks.visitUnknownType(Record))
       return EC;
     break;
 #define TYPE_RECORD(EnumName, EnumVal, Name)                                   \
   case EnumName: {                                                             \
-    if (auto EC = visitKnownRecord<Name##Record>(Record, Callbacks))           \
+    if (auto EC = visitKnownRecord<Name##Record>(*this, Record, Callbacks))    \
       return EC;                                                               \
     break;                                                                     \
   }
 #define TYPE_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)                  \
   TYPE_RECORD(EnumVal, EnumVal, AliasName)
 #define MEMBER_RECORD(EnumName, EnumVal, Name)
 #define MEMBER_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
 #include "llvm/DebugInfo/CodeView/TypeRecords.def"

 Error CVTypeVisitor::visitTypeStream(const CVTypeArray &Types) {
   for (auto I : Types) {
     if (auto EC = visitTypeRecord(I))
       return EC;
   }
   return Error::success();
 }
 
+Error CVTypeVisitor::visitTypeStream(CVTypeRange Types) {
+  for (auto I : Types) {
+    if (auto EC = visitTypeRecord(I))
+      return EC;
+  }
+  return Error::success();
+}
+
 Error CVTypeVisitor::visitFieldListMemberStream(msf::StreamReader Reader) {
   FieldListDeserializer Deserializer(Reader);
   TypeVisitorCallbackPipeline Pipeline;
   Pipeline.addCallbackToPipeline(Deserializer);
   Pipeline.addCallbackToPipeline(Callbacks);
 
   TypeLeafKind Leaf;
   while (!Reader.empty()) {

llvm/lib/DebugInfo/CodeView/CodeViewError.cpp

std::string message(int Condition) const override {
     switch (static_cast<cv_error_code>(Condition)) {
     case cv_error_code::unspecified:
       return "An unknown error has occurred.";
     case cv_error_code::insufficient_buffer:
       return "The buffer is not large enough to read the requested number of "
              "bytes.";
     case cv_error_code::corrupt_record:
       return "The CodeView record is corrupted.";
+    case cv_error_code::no_records:
+      return "There are no records";
     case cv_error_code::operation_unsupported:
       return "The requested operation is not supported.";
     case cv_error_code::unknown_member_record:
       return "The member record is of an unknown type.";
     }
     llvm_unreachable("Unrecognized cv_error_code");
   }
 };

llvm/lib/DebugInfo/CodeView/TypeStreamMerger.cpp

 /// - If the new type record is not already present in the destination stream
 ///   hash table, append it to the destination type stream, assign it the next
 ///   type index, and update the two hash tables.
 /// - If the type record already exists in the destination stream, discard it
 ///   and update the type index map to forward the source type index to the
 ///   existing destination type index.
 class TypeStreamMerger : public TypeVisitorCallbacks {
 public:
-  TypeStreamMerger(TypeTableBuilder &DestStream)
-      : DestStream(DestStream), FieldListBuilder(DestStream) {}
+  TypeStreamMerger(TypeTableBuilder &DestStream, TypeServerHandler *Handler)
+      : DestStream(DestStream), FieldListBuilder(DestStream), Handler(Handler) {
+  }
 
 /// TypeVisitorCallbacks overrides.
 #define TYPE_RECORD(EnumName, EnumVal, Name)                                   \
   Error visitKnownRecord(CVType &CVR, Name##Record &Record) override;
 #define MEMBER_RECORD(EnumName, EnumVal, Name)                                 \
   Error visitKnownMember(CVMemberRecord &CVR, Name##Record &Record) override;
 #define TYPE_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
 #define MEMBER_RECORD_ALIAS(EnumName, EnumVal, Name, AliasName)
private:
   }
 
   Optional<Error> LastError;
 
   BumpPtrAllocator Allocator;
 
   TypeTableBuilder &DestStream;
   FieldListRecordBuilder FieldListBuilder;
+  TypeServerHandler *Handler;
 
   bool IsInFieldList{false};
   size_t BeginIndexMapSize = 0;
 
   /// Map from source type index to destination type index. Indexed by source
   /// type index minus 0x1000.
   SmallVector<TypeIndex, 0> IndexMap;
 };
Error TypeStreamMerger::mergeStream(const CVTypeArray &Types) {
   TypeVisitorCallbackPipeline Pipeline;
   LastError = Error::success();
 
   TypeDeserializer Deserializer;
   Pipeline.addCallbackToPipeline(Deserializer);
   Pipeline.addCallbackToPipeline(*this);
 
   CVTypeVisitor Visitor(Pipeline);
+  if (Handler)
+    Visitor.addTypeServerHandler(*Handler);
 
   if (auto EC = Visitor.visitTypeStream(Types))
     return EC;
   IndexMap.clear();
 
   Error Ret = std::move(*LastError);
   LastError.reset();
   return Ret;
 }
 
 Error llvm::codeview::mergeTypeStreams(TypeTableBuilder &DestStream,
+                                       TypeServerHandler *Handler,
                                        const CVTypeArray &Types) {
-  return TypeStreamMerger(DestStream).mergeStream(Types);
+  return TypeStreamMerger(DestStream, Handler).mergeStream(Types);
 }

llvm/lib/DebugInfo/PDB/CMakeLists.txt

add_pdb_impl_folder(Native
   Native/InfoStreamBuilder.cpp
   Native/ModInfo.cpp
   Native/ModStream.cpp
   Native/NativeRawSymbol.cpp
   Native/NamedStreamMap.cpp
   Native/NativeSession.cpp
   Native/PDBFile.cpp
   Native/PDBFileBuilder.cpp
+  Native/PDBTypeServerHandler.cpp
   Native/PublicsStream.cpp
   Native/RawError.cpp
   Native/StringTable.cpp
   Native/StringTableBuilder.cpp
   Native/SymbolStream.cpp
   Native/TpiHashing.cpp
   Native/TpiStream.cpp
   Native/TpiStreamBuilder.cpp)

llvm/lib/DebugInfo/PDB/Native/NativeSession.cpp

ErrorOr<std::unique_ptr<MemoryBuffer>> ErrorOrBuffer =
                                    /*RequiresNullTerminator=*/false);
   if (!ErrorOrBuffer)
     return make_error<GenericError>(generic_error_code::invalid_path);
 
   std::unique_ptr<MemoryBuffer> Buffer = std::move(*ErrorOrBuffer);
   auto Stream = llvm::make_unique<MemoryBufferByteStream>(std::move(Buffer));
 
   auto Allocator = llvm::make_unique<BumpPtrAllocator>();
-  auto File = llvm::make_unique<PDBFile>(std::move(Stream), *Allocator);
+  auto File = llvm::make_unique<PDBFile>(Path, std::move(Stream), *Allocator);
   if (auto EC = File->parseFileHeaders())
     return EC;
   if (auto EC = File->parseStreamData())
     return EC;
 
   Session =
       llvm::make_unique<NativeSession>(std::move(File), std::move(Allocator));
 

llvm/lib/DebugInfo/PDB/Native/PDBFile.cpp

 #include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Native/PublicsStream.h"
 #include "llvm/DebugInfo/PDB/Native/RawError.h"
 #include "llvm/DebugInfo/PDB/Native/StringTable.h"
 #include "llvm/DebugInfo/PDB/Native/SymbolStream.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStream.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/Path.h"
 #include <algorithm>
 #include <cassert>
 #include <cstdint>
 
 using namespace llvm;
 using namespace llvm::codeview;
 using namespace llvm::msf;
 using namespace llvm::pdb;
 
 namespace {
 typedef FixedStreamArray<support::ulittle32_t> ulittle_array;
 } // end anonymous namespace
 
-PDBFile::PDBFile(std::unique_ptr<ReadableStream> PdbFileBuffer,
+PDBFile::PDBFile(StringRef Path, std::unique_ptr<ReadableStream> PdbFileBuffer,
                  BumpPtrAllocator &Allocator)
-    : Allocator(Allocator), Buffer(std::move(PdbFileBuffer)) {}
+    : FilePath(Path), Allocator(Allocator), Buffer(std::move(PdbFileBuffer)) {}
 
 PDBFile::~PDBFile() = default;
 
+StringRef PDBFile::getFilePath() const { return FilePath; }
+
+StringRef PDBFile::getFileDirectory() const {
+  return sys::path::parent_path(FilePath);
+}
+
 uint32_t PDBFile::getBlockSize() const { return ContainerLayout.SB->BlockSize; }
 
 uint32_t PDBFile::getFreeBlockMapBlock() const {
   return ContainerLayout.SB->FreeBlockMapBlock;
 }
 
 uint32_t PDBFile::getBlockCount() const {
   return ContainerLayout.SB->NumBlocks;

llvm/lib/DebugInfo/PDB/Native/PDBTypeServerHandler.cpp

+//===- PDBTypeServerHandler.cpp ---------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/PDB/Native/PDBTypeServerHandler.h"
+
+#include "llvm/DebugInfo/CodeView/CodeViewError.h"
+#include "llvm/DebugInfo/PDB/GenericError.h"
+#include "llvm/DebugInfo/PDB/Native/InfoStream.h"
+#include "llvm/DebugInfo/PDB/Native/NativeSession.h"
+#include "llvm/DebugInfo/PDB/Native/PDBFile.h"
+#include "llvm/DebugInfo/PDB/Native/TpiStream.h"
+#include "llvm/DebugInfo/PDB/PDB.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+
+using namespace llvm;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+static void ignoreErrors(Error EC) {
+  llvm::handleAllErrors(std::move(EC), [&](ErrorInfoBase &EIB) {});
+}
+
+PDBTypeServerHandler::PDBTypeServerHandler(bool RevisitAlways)
+    : RevisitAlways(RevisitAlways) {}
+
+void PDBTypeServerHandler::addSearchPath(StringRef Path) {
+  if (Path.empty() || !sys::fs::is_directory(Path))
+    return;
+
+  SearchPaths.push_back(Path);
+}
+
+Expected<bool>
+PDBTypeServerHandler::handleInternal(PDBFile &File,
+                                     TypeVisitorCallbacks &Callbacks) {
+  auto ExpectedTpi = File.getPDBTpiStream();
+  if (!ExpectedTpi)
+    return ExpectedTpi.takeError();
+  CVTypeVisitor Visitor(Callbacks);
+
+  if (auto EC = Visitor.visitTypeStream(ExpectedTpi->types(nullptr)))
+    return std::move(EC);
+
+  return true;
+}
+
+Expected<bool> PDBTypeServerHandler::handle(TypeServer2Record &TS,
+                                            TypeVisitorCallbacks &Callbacks) {
+  if (Session) {
+    // If we've already handled this TypeServer and we only want to handle each
+    // TypeServer once, consume the record without doing anything.
+    if (!RevisitAlways)
+      return true;
+
+    return handleInternal(Session->getPDBFile(), Callbacks);
+  }
+
+  StringRef File = sys::path::filename(TS.Name);
+  if (File.empty())
+    return make_error<CodeViewError>(
+        cv_error_code::corrupt_record,
+        "TypeServer2Record does not contain filename!");
+
+  for (auto Path : SearchPaths) {
+    sys::path::append(Path, File);
+    if (!sys::fs::exists(Path))
+      continue;
+
+    std::unique_ptr<IPDBSession> ThisSession;
+    if (auto EC = loadDataForPDB(PDB_ReaderType::Native, Path, ThisSession)) {
+      // It is not an error if this PDB fails to load, it just means that it
+      // doesn't match and we should continue searching.
+      ignoreErrors(std::move(EC));
+      continue;
+    }
+
+    std::unique_ptr<NativeSession> NS(
+        static_cast<NativeSession *>(ThisSession.release()));
+    PDBFile &File = NS->getPDBFile();
+    auto ExpectedInfo = File.getPDBInfoStream();
+    // All PDB Files should have an Info stream.
+    if (!ExpectedInfo)
+      return ExpectedInfo.takeError();
+
+    ArrayRef<uint8_t> GuidBytes(ExpectedInfo->getGuid().Guid);
+    StringRef GuidStr(reinterpret_cast<const char *>(GuidBytes.begin()),
+                      GuidBytes.size());
+    if (GuidStr != TS.Guid)
+      continue;
+
+    Session = std::move(NS);
+    return handleInternal(File, Callbacks);
+  }
+
+  // We couldn't find a matching PDB, so let it be handled by someone else.
+  return false;
+}

llvm/lib/DebugInfo/PDB/Native/TpiStream.cpp

 #include "llvm/ADT/iterator_range.h"
 #include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
 #include "llvm/DebugInfo/CodeView/TypeDeserializer.h"
 #include "llvm/DebugInfo/CodeView/TypeRecord.h"
 #include "llvm/DebugInfo/CodeView/TypeVisitorCallbackPipeline.h"
 #include "llvm/DebugInfo/MSF/MappedBlockStream.h"
 #include "llvm/DebugInfo/MSF/StreamReader.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
+#include "llvm/DebugInfo/PDB/Native/PDBTypeServerHandler.h"
 #include "llvm/DebugInfo/PDB/Native/RawConstants.h"
 #include "llvm/DebugInfo/PDB/Native/RawError.h"
 #include "llvm/DebugInfo/PDB/Native/RawTypes.h"
 #include "llvm/DebugInfo/PDB/Native/TpiHashing.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 #include <algorithm>
 #include <cstdint>

 }
 
 FixedStreamArray<TypeIndexOffset> TpiStream::getTypeIndexOffsets() const {
   return TypeIndexOffsets;
 }
 
 HashTable &TpiStream::getHashAdjusters() { return HashAdjusters; }
 
-iterator_range<CVTypeArray::Iterator> TpiStream::types(bool *HadError) const {
+CVTypeRange TpiStream::types(bool *HadError) const {
   return make_range(TypeRecords.begin(HadError), TypeRecords.end());
 }
 
 Error TpiStream::commit() { return Error::success(); }

llvm/tools/llvm-readobj/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   DebugInfoCodeView
   Object
   Support
   DebugInfoCodeView
   DebugInfoMSF
+  DebugInfoPDB
   )
 
 add_llvm_tool(llvm-readobj
   ARMWinEHPrinter.cpp
   COFFDumper.cpp
   COFFImportDumper.cpp
   ELFDumper.cpp
   Error.cpp
   llvm-readobj.cpp
   MachODumper.cpp
   ObjDumper.cpp
   WasmDumper.cpp
   Win64EHDumper.cpp
   )

llvm/tools/llvm-readobj/COFFDumper.cpp

 #include "llvm/DebugInfo/MSF/ByteStream.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/COFF.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/DataExtractor.h"
 #include "llvm/Support/Format.h"
+#include "llvm/Support/Path.h"
 #include "llvm/Support/ScopedPrinter.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/Win64EH.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cstring>
 #include <system_error>
 #include <time.h>
if (SectionName == ".debug$T") {
       CVTypeArray Types;
       StreamReader Reader(Stream);
       if (auto EC = Reader.readArray(Types, Reader.getLength())) {
         consumeError(std::move(EC));
         W.flush();
         error(object_error::parse_failed);
       }
 
-      if (auto EC = mergeTypeStreams(CVTypes, Types)) {
+      if (auto EC = mergeTypeStreams(CVTypes, nullptr, Types)) {
         consumeError(std::move(EC));
         return error(object_error::parse_failed);
       }
     }
   }
 }
 
 void COFFDumper::printCodeViewTypeSection(StringRef SectionName,

llvm/tools/llvm-readobj/LLVMBuild.txt

 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [component_0]
 type = Tool
 name = llvm-readobj
 parent = Tools
-required_libraries = all-targets BitReader Object DebugInfoCodeView DebugInfoMSF
+required_libraries = all-targets BitReader Object DebugInfoCodeView DebugInfoPDB DebugInfoMSF

llvm/unittests/DebugInfo/PDB/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   DebugInfoCodeView
   DebugInfoMSF
   DebugInfoPDB
   )
 
 set(DebugInfoPDBSources
   HashTableTest.cpp
   MappedBlockStreamTest.cpp
   StringTableBuilderTest.cpp
   MSFBuilderTest.cpp
   PDBApiTest.cpp
+  TypeServerHandlerTest.cpp
   )
 
 add_llvm_unittest(DebugInfoPDBTests
   ${DebugInfoPDBSources}
   )

llvm/unittests/DebugInfo/PDB/TypeServerHandlerTest.cpp

+//===- llvm/unittest/DebugInfo/PDB/TypeServerHandlerTest.cpp --------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ErrorChecking.h"
+
+#include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
+#include "llvm/DebugInfo/CodeView/TypeRecord.h"
+#include "llvm/DebugInfo/CodeView/TypeRecordMapping.h"
+#include "llvm/DebugInfo/CodeView/TypeSerializer.h"
+#include "llvm/DebugInfo/CodeView/TypeServerHandler.h"
+#include "llvm/DebugInfo/CodeView/TypeTableBuilder.h"
+#include "llvm/DebugInfo/CodeView/TypeVisitorCallbackPipeline.h"
+#include "llvm/DebugInfo/CodeView/TypeVisitorCallbacks.h"
+#include "llvm/DebugInfo/PDB/Native/RawTypes.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/Error.h"
+
+#include "gtest/gtest.h"
+
+using namespace llvm;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+namespace {
+
+constexpr uint8_t Guid[] = {0x2a, 0x2c, 0x1c, 0x2a, 0xcb, 0x9e, 0x48, 0x18,
+                            0x82, 0x82, 0x7a, 0x87, 0xc3, 0xfe, 0x16, 0xe8};
+StringRef GuidStr(reinterpret_cast<const char *>(Guid),
+                  llvm::array_lengthof(Guid));
+
+constexpr const char *Name = "Test Name";
+constexpr int Age = 1;
+
+class MockTypeServerHandler : public TypeServerHandler {
+public:
+  explicit MockTypeServerHandler(bool HandleAlways)
+      : HandleAlways(HandleAlways) {}
+
+  Expected<bool> handle(TypeServer2Record &TS,
+                        TypeVisitorCallbacks &Callbacks) override {
+    if (TS.Age != Age || TS.Guid != GuidStr || TS.Name != Name)
+      return make_error<CodeViewError>(cv_error_code::corrupt_record,
+                                       "Invalid TypeServer record!");
+
+    if (Handled && !HandleAlways)
+      return false;
+
+    Handled = true;
+    return true;
+  }
+
+  bool Handled = false;
+  bool HandleAlways;
+};
+
+class MockTypeVisitorCallbacks : public TypeVisitorCallbacks {
+public:
+  enum class State {
+    Ready,
+    VisitTypeBegin,
+    VisitKnownRecord,
+    VisitTypeEnd,
+  };
+  Error visitTypeBegin(CVType &CVT) override {
+    if (S != State::Ready)
+      return make_error<CodeViewError>(cv_error_code::unspecified,
+                                       "Invalid visitor state!");
+
+    S = State::VisitTypeBegin;
+    return Error::success();
+  }
+
+  Error visitKnownRecord(CVType &CVT, TypeServer2Record &TS) override {
+    if (S != State::VisitTypeBegin)
+      return make_error<CodeViewError>(cv_error_code::unspecified,
+                                       "Invalid visitor state!");
+
+    S = State::VisitKnownRecord;
+    return Error::success();
+  }
+
+  Error visitTypeEnd(CVType &CVT) override {
+    if (S != State::VisitKnownRecord)
+      return make_error<CodeViewError>(cv_error_code::unspecified,
+                                       "Invalid visitor state!");
+
+    S = State::VisitTypeEnd;
+    return Error::success();
+  }
+
+  State S = State::Ready;
+};
+
+class TypeServerHandlerTest : public testing::Test {
+public:
+  void SetUp() override {
+    TypeServer2Record R(TypeRecordKind::TypeServer2);
+    R.Age = Age;
+    R.Guid = GuidStr;
+    R.Name = Name;
+
+    TypeTableBuilder Builder(Allocator);
+    Builder.writeKnownType(R);
+    TypeServerRecord.RecordData = Builder.records().front();
+    TypeServerRecord.Type = TypeLeafKind::LF_TYPESERVER2;
+  }
+
+protected:
+  BumpPtrAllocator Allocator;
+  CVType TypeServerRecord;
+};
+
+// Test that when no type server handler is registered, it gets handled by the
+// normal
+// visitor callbacks.
+TEST_F(TypeServerHandlerTest, VisitRecordNoTypeServer) {
+  MockTypeVisitorCallbacks C2;
+  MockTypeVisitorCallbacks C1;
+  TypeVisitorCallbackPipeline Pipeline;
+
+  Pipeline.addCallbackToPipeline(C1);
+  Pipeline.addCallbackToPipeline(C2);
+  CVTypeVisitor Visitor(Pipeline);
+  EXPECT_NO_ERROR(Visitor.visitTypeRecord(TypeServerRecord));
+
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::VisitTypeEnd, C1.S);
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::VisitTypeEnd, C2.S);
+}
+
+// Test that when a TypeServerHandler is registered, it gets consumed by the
+// handler if and only if the handler returns true.
+TEST_F(TypeServerHandlerTest, VisitRecordWithTypeServerOnce) {
+  MockTypeServerHandler Handler(false);
+
+  MockTypeVisitorCallbacks C1;
+  CVTypeVisitor Visitor(C1);
+  Visitor.addTypeServerHandler(Handler);
+
+  // Our mock server returns true the first time.
+  EXPECT_NO_ERROR(Visitor.visitTypeRecord(TypeServerRecord));
+  EXPECT_TRUE(Handler.Handled);
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::Ready, C1.S);
+
+  // And false the second time.
+  EXPECT_NO_ERROR(Visitor.visitTypeRecord(TypeServerRecord));
+  EXPECT_TRUE(Handler.Handled);
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::VisitTypeEnd, C1.S);
+}
+
+// Test that when a type server handler is registered, if the handler keeps
+// returning true, it will keep getting consumed by the handler and not go
+// to the default processor.
+TEST_F(TypeServerHandlerTest, VisitRecordWithTypeServerAlways) {
+  MockTypeServerHandler Handler(true);
+
+  MockTypeVisitorCallbacks C1;
+  CVTypeVisitor Visitor(C1);
+  Visitor.addTypeServerHandler(Handler);
+
+  EXPECT_NO_ERROR(Visitor.visitTypeRecord(TypeServerRecord));
+  EXPECT_TRUE(Handler.Handled);
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::Ready, C1.S);
+
+  EXPECT_NO_ERROR(Visitor.visitTypeRecord(TypeServerRecord));
+  EXPECT_TRUE(Handler.Handled);
+  EXPECT_EQ(MockTypeVisitorCallbacks::State::Ready, C1.S);
+}
+
+} // end anonymous namespace