Introduce NativeEnumModules and NativeCompilandSymbol

[amccarth]

Summary

Together, these allow lldb-pdbdump to list all the modules from a PDB using a native reader (rather than DIA).

Note that I'll probably be specializing NativeRawSymbol in a subsequent patch.

[amccarth]

Cope with the backwards naming of the getName and getLibraryName methods.

[amccarth]

Added a -native option to the llvm-pdbdump pretty subcommand to select the reader, and added a test that checks the compilands list it produces.

Note that the -native option is not yet ready for usage beyond enumerating the compilands.

[zturner]

llvm/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h

2–3

The line wraps here, need to find a way to make it shorter.

llvm/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

24

For the longest time I've wondered what they mean by EC in the native PDB code, and now it just hit me that it's edit and continue. Perhaps you could just return Module.Info.hasECInfo(); here?

26

In the future, this will be the unique (within the context of this PDB session) of the Exe symbol.

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

35–37

Not as important as it is when you have a shared_ptr, but I think as a matter of style we prefer make_unique instead of explicitly constructing a unique_ptr with the result of a new expression.

llvm/test/DebugInfo/PDB/Native/pdb-native-compilands.test

19

Should this prefix be removed as well? (as per the comment above)?

66

newline

[amccarth]

llvm/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

26

Acknowledged. I'm going to make a specialization next for the Exe, so this should become clearer soon.

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

35–37

I generally do use make_unique, but I don't know of another way to have make_unique create a unique_ptr of the base class that actually points to an instance of a subclass.

llvm/test/DebugInfo/PDB/Native/pdb-native-compilands.test

19

I missed that one. It probably doesn't matter, since the path is fixed for whoever built the PDB originally, but I've done it to be consistent.

[amccarth]

Fixes per comments.

[inglorion]

llvm/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h

3

This comment seems broken. It's spread over multiple lines and the opening "-*-" is missing. Is the problem here that this really didn't fit on one line? If so, since Emacs likes its magic markers on the first line, perhaps we can put the description on the second line?

llvm/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

2

s/\.h/.cpp/

28

Instead of "counter-intuitive", can you specify what they actually mean? That would make it easier to verify that these methods are indeed doing the right thing.

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

33

You're choosing to return nullptr here, instead of something else, e.g. crashing or asserting. I'm curious why you prefer to return nullptr. Could you explain your rationale?

[amccarth]

llvm/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

28

Zach and I were having a partially offline discussion about this. We might rename these methods in a future patch, but I'll take a crack at re-wording this comment in the mean time.

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

33

I'm following the lead set by the DIA versions of the enumerators I've looked at. For example, DIAEnumDebugStreams.cpp returns nullptr if anything goes wrong querying DIA for the desired item, and calling with an invalid index is one possibility.

I'd be happy to assert instead if you think it's important.

[amccarth]

Addresses the header line mistakes caught by inglorion and re-words the comment on the mappings for getName and getLibraryName methods.

[inglorion]

lgtm2

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

33

I'm perfectly happy with us doing the same thing DIA does. Thanks for the explanation!

llvm/include/llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h

+//===- NativeCompilandSymbol.h - native impl for compiland syms -*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_PDB_NATIVE_NATIVECOMPILANDSYMBOL_H
+#define LLVM_DEBUGINFO_PDB_NATIVE_NATIVECOMPILANDSYMBOL_H
+
+#include "llvm/DebugInfo/PDB/Native/ModInfo.h"
+#include "llvm/DebugInfo/PDB/Native/NativeRawSymbol.h"
+
+namespace llvm {
+namespace pdb {
+
+class NativeCompilandSymbol : public NativeRawSymbol {
+public:
+  NativeCompilandSymbol(NativeSession &Session, const ModuleInfoEx &MI);
+  PDB_SymType getSymTag() const override;
+  bool isEditAndContinueEnabled() const override;
+  uint32_t getLexicalParentId() const override;
+  std::string getLibraryName() const override;
+  std::string getName() const override;
+
+private:
+  ModuleInfoEx Module;
+};
+
+} // namespace pdb
+} // namespace llvm
+
+#endif

llvm/include/llvm/DebugInfo/PDB/Native/NativeEnumModules.h

+//==- NativeEnumModules.h - Native Module Enumerator impl --------*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_PDB_NATIVE_NATIVEENUMMODULES_H
+#define LLVM_DEBUGINFO_PDB_NATIVE_NATIVEENUMMODULES_H
+
+#include "llvm/DebugInfo/PDB/IPDBEnumChildren.h"
+#include "llvm/DebugInfo/PDB/Native/ModInfo.h"
+#include "llvm/DebugInfo/PDB/PDBSymbol.h"
+namespace llvm {
+namespace pdb {
+
+class NativeSession;
+
+class NativeEnumModules : public IPDBEnumChildren<PDBSymbol> {
+public:
+  explicit NativeEnumModules(NativeSession &Session,
+                             ArrayRef<ModuleInfoEx> Modules,
+                             uint32_t Index = 0);
+
+  uint32_t getChildCount() const override;
+  std::unique_ptr<PDBSymbol> getChildAtIndex(uint32_t Index) const override;
+  std::unique_ptr<PDBSymbol> getNext() override;
+  void reset() override;
+  NativeEnumModules *clone() const override;
+
+private:
+  NativeSession &Session;
+  ArrayRef<ModuleInfoEx> Modules;
+  uint32_t Index;
+};
+}
+}
+
+#endif

llvm/lib/DebugInfo/PDB/CMakeLists.txt

add_pdb_impl_folder(Native
   Native/GlobalsStream.cpp
   Native/GSI.cpp
   Native/Hash.cpp
   Native/HashTable.cpp
   Native/InfoStream.cpp
   Native/InfoStreamBuilder.cpp
   Native/ModInfo.cpp
   Native/ModStream.cpp
+  Native/NativeCompilandSymbol.cpp
+  Native/NativeEnumModules.cpp
   Native/NativeRawSymbol.cpp
   Native/NamedStreamMap.cpp
   Native/NativeSession.cpp
   Native/PDBFile.cpp
   Native/PDBFileBuilder.cpp
   Native/PDBTypeServerHandler.cpp
   Native/PublicsStream.cpp
   Native/RawError.cpp

llvm/lib/DebugInfo/PDB/Native/NativeCompilandSymbol.cpp

+//===- NativeCompilandSymbol.h - Native impl of PDBCompilandSymbol -C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h"
+
+namespace llvm {
+namespace pdb {
+
+NativeCompilandSymbol::NativeCompilandSymbol(NativeSession &Session,
+                                             const ModuleInfoEx &MI)
+    : NativeRawSymbol(Session), Module(MI) {}
+
+PDB_SymType NativeCompilandSymbol::getSymTag() const {
+  return PDB_SymType::Compiland;
+}
+
+bool NativeCompilandSymbol::isEditAndContinueEnabled() const {
+  return Module.Info.hasECInfo();
+}
+
+uint32_t NativeCompilandSymbol::getLexicalParentId() const { return 0; }
+
+// DIA, which this API was modeled after, uses counter-intuitive meanings for
+// IDiaSymbol::get_name and IDiaSymbol::get_libraryName, which is why these
+// methods may appear to be cross-mapped.
+
+std::string NativeCompilandSymbol::getLibraryName() const {
+  return Module.Info.getObjFileName();
+}
+
+std::string NativeCompilandSymbol::getName() const {
+  return Module.Info.getModuleName();
+}
+
+} // namespace pdb
+} // namespace llvm

llvm/lib/DebugInfo/PDB/Native/NativeEnumModules.cpp

+//==- NativeEnumModules.cpp - Native Symbol Enumerator impl ------*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/PDB/Native/NativeEnumModules.h"
+
+#include "llvm/DebugInfo/PDB/IPDBEnumChildren.h"
+#include "llvm/DebugInfo/PDB/Native/NativeCompilandSymbol.h"
+#include "llvm/DebugInfo/PDB/Native/NativeSession.h"
+#include "llvm/DebugInfo/PDB/PDBSymbol.h"
+#include "llvm/DebugInfo/PDB/PDBSymbolCompiland.h"
+
+namespace llvm {
+namespace pdb {
+
+NativeEnumModules::NativeEnumModules(NativeSession &PDBSession,
+                                     ArrayRef<ModuleInfoEx> Modules,
+                                     uint32_t Index)
+    : Session(PDBSession), Modules(Modules), Index(Index) {}
+
+uint32_t NativeEnumModules::getChildCount() const {
+  return static_cast<uint32_t>(Modules.size());
+}
+
+std::unique_ptr<PDBSymbol>
+NativeEnumModules::getChildAtIndex(uint32_t Index) const {
+  if (Index >= Modules.size())
+    return nullptr;
+  return std::unique_ptr<PDBSymbol>(new PDBSymbolCompiland(Session,
+      std::unique_ptr<IPDBRawSymbol>(
+          new NativeCompilandSymbol(Session, Modules[Index]))));
+}
+
+std::unique_ptr<PDBSymbol> NativeEnumModules::getNext() {
+  if (Index >= Modules.size())
+    return nullptr;
+  return getChildAtIndex(Index++);
+}
+
+void NativeEnumModules::reset() { Index = 0; }
+
+NativeEnumModules *NativeEnumModules::clone() const {
+  return new NativeEnumModules(Session, Modules, Index);
+}
+
+}
+}

llvm/lib/DebugInfo/PDB/Native/NativeRawSymbol.cpp

 //===- NativeRawSymbol.cpp - Native implementation of IPDBRawSymbol -*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/PDB/Native/NativeRawSymbol.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/STLExtras.h"
-#include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/IPDBEnumChildren.h"
+#include "llvm/DebugInfo/PDB/Native/DbiStream.h"
+#include "llvm/DebugInfo/PDB/Native/InfoStream.h"
+#include "llvm/DebugInfo/PDB/Native/NativeEnumModules.h"
+#include "llvm/DebugInfo/PDB/Native/NativeSession.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
 #include "llvm/DebugInfo/PDB/PDBExtras.h"
-#include "llvm/DebugInfo/PDB/Native/NativeSession.h"
 #include "llvm/Support/ConvertUTF.h"
 #include "llvm/Support/raw_ostream.h"
 
 using namespace llvm;
 using namespace llvm::pdb;
 
 NativeRawSymbol::NativeRawSymbol(NativeSession &PDBSession)
   : Session(PDBSession) {}
 
 void NativeRawSymbol::dump(raw_ostream &OS, int Indent) const {}
 
 std::unique_ptr<IPDBEnumSymbols>
 NativeRawSymbol::findChildren(PDB_SymType Type) const {
+  switch (Type) {
+  case PDB_SymType::Compiland: {
+    auto &File = Session.getPDBFile();
+    auto Dbi = File.getPDBDbiStream();
+    if (Dbi) {
+      const auto Modules = Dbi->modules();
+      return std::unique_ptr<IPDBEnumSymbols>(
+          new NativeEnumModules(Session, Modules));
+    }
+    consumeError(Dbi.takeError());
+    break;
+  }
+  default:
+    break;
+  }
   return nullptr;
 }
 
 std::unique_ptr<IPDBEnumSymbols>
 NativeRawSymbol::findChildren(PDB_SymType Type, StringRef Name,
     PDB_NameSearchFlags Flags) const {
   return nullptr;
 }

llvm/test/DebugInfo/PDB/Native/pdb-native-compilands.test

+; Test that the native PDB reader can enumerate the compilands.
+; RUN: llvm-pdbdump pretty -native -compilands %p/../Inputs/empty.pdb \
+; RUN:   | FileCheck -check-prefix=EMPTY %s
+; RUN: llvm-pdbdump pretty -native -compilands %p/../Inputs/big-read.pdb \
+; RUN:   | FileCheck -check-prefix=BIGREAD %s
+
+; Reference output was generated with the DIA reader to ensure that the
+; `-native` option produces identical output.  The paths output will have
+; backslashes even on non-Windows platforms because they are from PDBs built
+; on Windows.  The path prefixes have been elided because those may be
+; machine-specific.
+
+EMPTY:---COMPILANDS---
+EMPTY:  \llvm\test\DebugInfo\PDB\Inputs\empty.obj
+EMPTY:  * Linker *
+
+BIGREAD:---COMPILANDS---
+BIGREAD:  \llvm\test\tools\llvm-symbolizer\pdb\Inputs\test.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\_cpu_disp_.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\_initsect_.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\_sehprolg4_.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\_chandler4gs_.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\_secchk_.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\gs_cookie.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\gs_report.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\gs_support.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\checkcfg.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\guard_support.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\loadcfg.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\dyn_tls_dtor.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\dyn_tls_init.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\matherr_detection.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\ucrt_detection.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\argv_mode.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\commit_mode.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\default_local_stdio_options.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\denormal_control.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\env_mode.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\file_mode.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\invalid_parameter_handler.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\matherr.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\new_mode.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\thread_locale.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\tncleanup.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\exe_main.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\initializers.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\utility.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\ucrt_stubs.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\utility_desktop.obj
+BIGREAD:  f:\dd\vctools\crt\vcstartup\build\md\msvcrt_kernel32\obj1r\i386\default_precision.obj
+BIGREAD:  Import:KERNEL32.dll
+BIGREAD:  KERNEL32.dll
+BIGREAD:  Import:VCRUNTIME140.dll
+BIGREAD:  VCRUNTIME140.dll
+BIGREAD:  Import:api-ms-win-crt-stdio-l1-1-0.dll
+BIGREAD:  api-ms-win-crt-stdio-l1-1-0.dll
+BIGREAD:  Import:api-ms-win-crt-runtime-l1-1-0.dll
+BIGREAD:  api-ms-win-crt-runtime-l1-1-0.dll
+BIGREAD:  Import:api-ms-win-crt-math-l1-1-0.dll
+BIGREAD:  api-ms-win-crt-math-l1-1-0.dll
+BIGREAD:  Import:api-ms-win-crt-locale-l1-1-0.dll
+BIGREAD:  api-ms-win-crt-locale-l1-1-0.dll
+BIGREAD:  Import:api-ms-win-crt-heap-l1-1-0.dll
+BIGREAD:  api-ms-win-crt-heap-l1-1-0.dll
+BIGREAD:  * Linker *

llvm/tools/llvm-pdbdump/llvm-pdbdump.cpp

 cl::opt<bool>
     All("all", cl::desc("Implies all other options in 'Symbol Types' category"),
         cl::cat(TypeCategory), cl::sub(PrettySubcommand));
 
 cl::opt<uint64_t> LoadAddress(
     "load-address",
     cl::desc("Assume the module is loaded at the specified address"),
     cl::cat(OtherOptions), cl::sub(PrettySubcommand));
+cl::opt<bool> Native("native", cl::desc("Use native PDB reader instead of DIA"),
+  cl::cat(OtherOptions), cl::sub(PrettySubcommand));
+
 cl::list<std::string> ExcludeTypes(
     "exclude-types", cl::desc("Exclude types by regular expression"),
     cl::ZeroOrMore, cl::cat(FilterCategory), cl::sub(PrettySubcommand));
 cl::list<std::string> ExcludeSymbols(
     "exclude-symbols", cl::desc("Exclude symbols by regular expression"),
     cl::ZeroOrMore, cl::cat(FilterCategory), cl::sub(PrettySubcommand));
 cl::list<std::string> ExcludeCompilands(
     "exclude-compilands", cl::desc("Exclude compilands by regular expression"),
static void dumpAnalysis(StringRef Path) {
   auto O = llvm::make_unique<AnalysisStyle>(File);
 
   ExitOnErr(O->dump());
 }
 
 static void dumpPretty(StringRef Path) {
   std::unique_ptr<IPDBSession> Session;
 
-  ExitOnErr(loadDataForPDB(PDB_ReaderType::DIA, Path, Session));
+  const auto ReaderType =
+      opts::pretty::Native ? PDB_ReaderType::Native : PDB_ReaderType::DIA;
+  ExitOnErr(loadDataForPDB(ReaderType, Path, Session));
 
   if (opts::pretty::LoadAddress)
     Session->setLoadAddress(opts::pretty::LoadAddress);
 
   LinePrinter Printer(2, outs());
 
   auto GlobalScope(Session->getGlobalScope());
   std::string FileName(GlobalScope->getSymbolsFileName());