[llvm-pdbutil] Improve output of llvm-pdbutil diff mode

[zturner]

Summary

We're getting to the point that some MS tools (e.g. DIA) can recognize our PDBs but others (e.g. link.exe) cannot. I think the way forward is to improve our tooling to help us find differences more easily. For example, if we can compile the same program with clang-cl and cl and have a tool tell us all the places where the PDBs differ, this could tell us what we're doing wrong. It's tricky though, because there are a lot of "benign" differences in a PDB. For example, if the string table in one PDB consists of "foo" followed by "bar" and in the other PDB it consists of "bar" followed by "foo", this is not necessarily a critical difference, as long as the uses of these strings also refer to the correct location. On the other hand, if the second PDB doesn't even contain the string "foo" at all, this is a critical difference.

diff mode has been in llvm-pdbutil for quite a while, but because of the above challenge along with some others, it's been hard to make it useful. I think this patch addresses that. It looks for all the same things, but it now prints the output in tabular format (carefully formatted and aligned into tables and fields), and it highlights critical differences in red, non-critical differences in yellow, and identical fields in green. Example:

This makes it easy to spot the places we differ, and the general concept of outputting arbitrary fields in tabular format can be extended to provide analysis into many of the different types of information that show up in a PDB.

[rnk]

llvm/tools/llvm-pdbutil/DiffPrinter.cpp

1

Add the annoying LLVM banner for consistency.

llvm/tools/llvm-pdbutil/DiffPrinter.h

43

I don't quite get this. Basically we have a comparator which says "if they're different, no big deal, they're actually equivalent". Is that more or less the idea? Maybe instead of asking the caller to pass a comparator we should have a different printEquivalent method? It seems shorter and less magical.

50–51

Having a different entry point for "equivalent" number comparisons would let us hoist these format calls up to the next level, since they don't depend on the comparator.

[zturner]

llvm/tools/llvm-pdbutil/DiffPrinter.h

43

Yea. Like, In pdb 1 the stream for module foo has stream index 12, but in pdb 2 it has stream index 15. These are equivalent because a stream exists in both pdbs. The number doesn't matter. If both of them had kInvalidStreamIndex, they would be identical. If one had kInvalidStreamIndex and another had a valid stream index, these would be different.

I think we need some notion of a comparator because sometimes the comparisons get rather obtuse, and we want to wrap that up in an abstraction that can be re-used over and over. For example, the stream number comparison described above. But it even gets worse. If it's a name index, then we actually have to go look in the PDB file, get the corresponding string out, and compare the strings themselves rather than the numbers.

I have some enhancements to this model in a forthcoming patch (not up for review yet, but depends on all 3 of these changes) where the class has both a format method and a compare method. Consider the case where you have two module names foo.obj and bar.obj. You want to print Module "foo.obj" and Module "bar.obj" but you only want to compare the actual file paths, but you can't compare the strings directly because for really long paths you *also* need to truncate part of the module name so that it displays like Module "...o.obj" rather than ...dule "foo.obj". So yea, it's kinda gross.

Note that the caller doesn't have to pass anything, it gets a default template argument of Identical. In this patch, the caller does have to instantiate a default-constructed instance and pass that through, but in the forthcoming patch I mentioned, all you have to do is write:

print<Equivalent>("Foo", A, B);

or

print<CustomComparator>("Foo", A, B);

and it just works.

llvm/include/llvm/DebugInfo/PDB/Native/NamedStreamMap.h

public:
   Error load(BinaryStreamReader &Stream);
   Error commit(BinaryStreamWriter &Writer) const;
   uint32_t finalize();
 
   uint32_t size() const;
   bool get(StringRef Stream, uint32_t &StreamNo) const;
   void set(StringRef Stream, uint32_t StreamNo);
   void remove(StringRef Stream);
-
+  const StringMap<uint32_t> &getStringMap() const { return Mapping; }
   iterator_range<StringMapConstIterator<uint32_t>> entries() const;
 
 private:
   Optional<FinalizationInfo> FinalizedInfo;
   HashTable FinalizedHashTable;
   StringMap<uint32_t> Mapping;
 };
 
 } // end namespace pdb
 
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_NATIVE_NAMEDSTREAMMAP_H

llvm/include/llvm/DebugInfo/PDB/Native/PDBStringTable.h

private:
   Error readHeader(BinaryStreamReader &Reader);
   Error readStrings(BinaryStreamReader &Reader);
   Error readHashTable(BinaryStreamReader &Reader);
   Error readEpilogue(BinaryStreamReader &Reader);
 
   const PDBStringTableHeader *Header = nullptr;
   codeview::DebugStringTableSubsectionRef Strings;
   FixedStreamArray<support::ulittle32_t> IDs;
-  uint32_t ByteSize = 0;
   uint32_t NameCount = 0;
 };
 
 } // end namespace pdb
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_PDB_RAW_STRINGTABLE_H

llvm/lib/DebugInfo/PDB/Native/PDBStringTable.cpp

 #include "llvm/DebugInfo/PDB/Native/RawTypes.h"
 #include "llvm/Support/BinaryStreamReader.h"
 #include "llvm/Support/Endian.h"
 
 using namespace llvm;
 using namespace llvm::support;
 using namespace llvm::pdb;
 
-uint32_t PDBStringTable::getByteSize() const { return ByteSize; }
+uint32_t PDBStringTable::getByteSize() const { return Header->ByteSize; }
 uint32_t PDBStringTable::getNameCount() const { return NameCount; }
 uint32_t PDBStringTable::getHashVersion() const { return Header->HashVersion; }
 uint32_t PDBStringTable::getSignature() const { return Header->Signature; }
 
 Error PDBStringTable::readHeader(BinaryStreamReader &Reader) {
   if (auto EC = Reader.readObject(Header))
     return EC;
 

llvm/tools/llvm-pdbutil/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   DebugInfoCodeView
   DebugInfoMSF
   DebugInfoPDB
   Object
   ObjectYAML
   Support
   )
 
 add_llvm_tool(llvm-pdbutil
   Analyze.cpp
   BytesOutputStyle.cpp
   Diff.cpp
+  DiffPrinter.cpp
   DumpOutputStyle.cpp
   llvm-pdbutil.cpp
   FormatUtil.cpp
   LinePrinter.cpp
   MinimalSymbolDumper.cpp
   MinimalTypeDumper.cpp
   PdbYaml.cpp
   PrettyBuiltinDumper.cpp

llvm/tools/llvm-pdbutil/Diff.cpp

 //===- Diff.cpp - PDB diff utility ------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "Diff.h"
 
+#include "DiffPrinter.h"
+#include "FormatUtil.h"
 #include "StreamUtil.h"
 #include "llvm-pdbutil.h"
 
+#include "llvm/ADT/StringSet.h"
+
 #include "llvm/DebugInfo/PDB/Native/Formatters.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Native/PDBStringTable.h"
 #include "llvm/DebugInfo/PDB/Native/RawConstants.h"
 
 #include "llvm/Support/FormatAdapters.h"
 #include "llvm/Support/FormatProviders.h"
case PdbRaw_FeatureSig::VC140:
       break;
     }
   }
 };
 }
 
 template <typename R> using ValueOfRange = llvm::detail::ValueOfRange<R>;
 
-template <typename Range, typename Comp>
-static void set_differences(Range &&R1, Range &&R2,
-                            SmallVectorImpl<ValueOfRange<Range>> *OnlyLeft,
-                            SmallVectorImpl<ValueOfRange<Range>> *OnlyRight,
-                            SmallVectorImpl<ValueOfRange<Range>> *Intersection,
-                            Comp Comparator) {
-
-  std::sort(R1.begin(), R1.end(), Comparator);
-  std::sort(R2.begin(), R2.end(), Comparator);
-
-  if (OnlyLeft) {
-    OnlyLeft->reserve(R1.size());
-    auto End = std::set_difference(R1.begin(), R1.end(), R2.begin(), R2.end(),
-                                   OnlyLeft->begin(), Comparator);
-    OnlyLeft->set_size(std::distance(OnlyLeft->begin(), End));
-  }
-  if (OnlyRight) {
-    OnlyLeft->reserve(R2.size());
-    auto End = std::set_difference(R2.begin(), R2.end(), R1.begin(), R1.end(),
-                                   OnlyRight->begin(), Comparator);
-    OnlyRight->set_size(std::distance(OnlyRight->begin(), End));
-  }
-  if (Intersection) {
-    Intersection->reserve(std::min(R1.size(), R2.size()));
-    auto End = std::set_intersection(R1.begin(), R1.end(), R2.begin(), R2.end(),
-                                     Intersection->begin(), Comparator);
-    Intersection->set_size(std::distance(Intersection->begin(), End));
-  }
-}
-
-template <typename Range>
-static void
-set_differences(Range &&R1, Range &&R2,
-                SmallVectorImpl<ValueOfRange<Range>> *OnlyLeft,
-                SmallVectorImpl<ValueOfRange<Range>> *OnlyRight,
-                SmallVectorImpl<ValueOfRange<Range>> *Intersection = nullptr) {
-  std::less<ValueOfRange<Range>> Comp;
-  set_differences(std::forward<Range>(R1), std::forward<Range>(R2), OnlyLeft,
-                  OnlyRight, Intersection, Comp);
-}
-
 DiffStyle::DiffStyle(PDBFile &File1, PDBFile &File2)
     : File1(File1), File2(File2) {}
 
 Error DiffStyle::dump() {
   if (auto EC = diffSuperBlock())
     return EC;
 
   if (auto EC = diffFreePageMap())
if (auto EC = diffPublics())
     return EC;
 
   if (auto EC = diffGlobals())
     return EC;
 
   return Error::success();
 }
 
-template <typename T>
-static bool diffAndPrint(StringRef Label, PDBFile &File1, PDBFile &File2, T V1,
-                         T V2) {
-  if (V1 == V2) {
-    outs() << formatv("  {0}: No differences detected!\n", Label);
-    return false;
-  }
-
-  outs().indent(2) << Label << "\n";
-  outs().indent(4) << formatv("{0}: {1}\n", File1.getFilePath(), V1);
-  outs().indent(4) << formatv("{0}: {1}\n", File2.getFilePath(), V2);
-  return true;
-}
-
-template <typename T>
-static bool diffAndPrint(StringRef Label, PDBFile &File1, PDBFile &File2,
-                         ArrayRef<T> V1, ArrayRef<T> V2) {
-  if (V1 == V2) {
-    outs() << formatv("  {0}: No differences detected!\n", Label);
-    return false;
-  }
-
-  outs().indent(2) << Label << "\n";
-  outs().indent(4) << formatv("{0}: {1}\n", File1.getFilePath(),
-                              make_range(V1.begin(), V1.end()));
-  outs().indent(4) << formatv("{0}: {1}\n", File2.getFilePath(),
-                              make_range(V2.begin(), V2.end()));
-  return true;
-}
-
-template <typename T>
-static bool printSymmetricDifferences(PDBFile &File1, PDBFile &File2,
-                                      T &&OnlyRange1, T &&OnlyRange2,
-                                      StringRef Label) {
-  bool HasDiff = false;
-  if (!OnlyRange1.empty()) {
-    HasDiff = true;
-    outs() << formatv("  {0} {1}(s) only in ({2})\n", OnlyRange1.size(), Label,
-                      File1.getFilePath());
-    for (const auto &Item : OnlyRange1)
-      outs() << formatv("    {0}\n", Label, Item);
-  }
-  if (!OnlyRange2.empty()) {
-    HasDiff = true;
-    outs() << formatv("  {0} {1}(s) only in ({2})\n", OnlyRange2.size(),
-                      File2.getFilePath());
-    for (const auto &Item : OnlyRange2)
-      outs() << formatv("    {0}\n", Item);
-  }
-  return HasDiff;
+static std::string shortFilePath(StringRef Path, uint32_t Width) {
+  if (Path.size() <= Width)
+    return Path;
+  Path = Path.take_back(Width - 3);
+  return std::string("...") + Path.str();
 }
 
 Error DiffStyle::diffSuperBlock() {
-  outs() << "MSF Super Block: Searching for differences...\n";
-  bool Diffs = false;
-
-  Diffs |= diffAndPrint("Block Size", File1, File2, File1.getBlockSize(),
-                        File2.getBlockSize());
-  Diffs |= diffAndPrint("Block Count", File1, File2, File1.getBlockCount(),
-                        File2.getBlockCount());
-  Diffs |= diffAndPrint("Unknown 1", File1, File2, File1.getUnknown1(),
-                        File2.getUnknown1());
-  if (!Diffs)
-    outs() << "MSF Super Block: No differences detected...\n";
+  DiffPrinter D(2, "MSF Super Block", 16, 20, outs());
+  D.printExplicit("File", DiffResult::UNSPECIFIED,
+                  shortFilePath(File1.getFilePath(), 18),
+                  shortFilePath(File2.getFilePath(), 18));
+  D.print("Block Size", File1.getBlockSize(), File2.getBlockSize());
+  D.print("Block Count", File1.getBlockCount(), File2.getBlockCount());
+  D.print("Unknown 1", File1.getUnknown1(), File2.getUnknown1());
+  D.print("Directory Size", File1.getNumDirectoryBytes(),
+          File2.getNumDirectoryBytes());
   return Error::success();
 }
 
 Error DiffStyle::diffStreamDirectory() {
+  DiffPrinter D(2, "Stream Directory", 30, 20, outs());
+  D.printExplicit("File", DiffResult::UNSPECIFIED,
+                  shortFilePath(File1.getFilePath(), 18),
+                  shortFilePath(File2.getFilePath(), 18));
+
   SmallVector<std::string, 32> P;
   SmallVector<std::string, 32> Q;
-  discoverStreamPurposes(File1, P);
-  discoverStreamPurposes(File2, Q);
-  outs() << "Stream Directory: Searching for differences...\n";
-
-  bool HasDifferences = false;
+  discoverStreamPurposes(File1, P, 28);
+  discoverStreamPurposes(File2, Q, 28);
+  D.print("Stream Count", File1.getNumStreams(), File2.getNumStreams());
   auto PI = to_vector<32>(enumerate(P));
   auto QI = to_vector<32>(enumerate(Q));
 
+  // Scan all streams in the left hand side, looking for ones that are also
+  // in the right.  Each time we find one, remove it.  When we're done, Q
+  // should contain all the streams that are in the right but not in the left.
+  DiffPrinter::Equivalent<uint32_t, uint32_t> Comp;
+  for (const auto &P : PI) {
     typedef decltype(PI) ContainerType;
     typedef typename ContainerType::value_type value_type;
 
-  auto Comparator = [](const value_type &I1, const value_type &I2) {
-    return I1.value() < I2.value();
-  };
+    auto Iter = llvm::find_if(
+        QI, [P](const value_type &V) { return V.value() == P.value(); });
 
-  decltype(PI) OnlyP;
-  decltype(QI) OnlyQ;
-  decltype(PI) Common;
-
-  set_differences(PI, QI, &OnlyP, &OnlyQ, &Common, Comparator);
-
-  if (!OnlyP.empty()) {
-    HasDifferences = true;
-    outs().indent(2) << formatv("{0} Stream(s) only in ({1})\n", OnlyP.size(),
-                                File1.getFilePath());
-    for (auto &Item : OnlyP) {
-      outs().indent(4) << formatv("Stream {0} - {1}\n", Item.index(),
-                                  Item.value());
-    }
+    if (Iter == QI.end()) {
+      D.printExplicit(P.value(), DiffResult::DIFFERENT, P.index(),
+                      "(not present)");
+      continue;
     }
 
-  if (!OnlyQ.empty()) {
-    HasDifferences = true;
-    outs().indent(2) << formatv("{0} Streams(s) only in ({1})\n", OnlyQ.size(),
-                                File2.getFilePath());
-    for (auto &Item : OnlyQ) {
-      outs().indent(4) << formatv("Stream {0} - {1}\n", Item.index(),
-                                  Item.value());
-    }
-  }
-  if (!Common.empty()) {
-    outs().indent(2) << formatv("Found {0} common streams.  Searching for "
-                                "intra-stream differences.\n",
-                                Common.size());
-    bool HasCommonDifferences = false;
-    for (const auto &Left : Common) {
-      // Left was copied from the first range so its index refers to a stream
-      // index in the first file.  Find the corresponding stream index in the
-      // second file.
-      auto Range =
-          std::equal_range(QI.begin(), QI.end(), Left,
-                           [](const value_type &L, const value_type &R) {
-                             return L.value() < R.value();
-                           });
-      const auto &Right = *Range.first;
-      assert(Left.value() == Right.value());
-      uint32_t LeftSize = File1.getStreamByteSize(Left.index());
-      uint32_t RightSize = File2.getStreamByteSize(Right.index());
-      if (LeftSize != RightSize) {
-        HasDifferences = true;
-        HasCommonDifferences = true;
-        outs().indent(4) << formatv("{0} ({1}: {2} bytes, {3}: {4} bytes)\n",
-                                    Left.value(), File1.getFilePath(), LeftSize,
-                                    File2.getFilePath(), RightSize);
-      }
-    }
-    if (!HasCommonDifferences)
-      outs().indent(2) << "Common Streams:  No differences detected!\n";
+    D.print(P.value(), P.index(), Iter->index(), Comp);
+    QI.erase(Iter);
+
+  for (const auto &Q : QI) {
+    D.printExplicit(Q.value(), DiffResult::DIFFERENT, "(not present)",
+                    Q.index());
   }
-  if (!HasDifferences)
-    outs() << "Stream Directory: No differences detected!\n";
 
   return Error::success();
 }
 
 Error DiffStyle::diffStringTable() {
+  DiffPrinter D(2, "String Table", 30, 20, outs());
+  D.printExplicit("File", DiffResult::UNSPECIFIED,
+                  shortFilePath(File1.getFilePath(), 18),
+                  shortFilePath(File2.getFilePath(), 18));
+
   auto ExpectedST1 = File1.getStringTable();
   auto ExpectedST2 = File2.getStringTable();
-  outs() << "String Table: Searching for differences...\n";
   bool Has1 = !!ExpectedST1;
   bool Has2 = !!ExpectedST2;
-  if (!(Has1 && Has2)) {
-    // If one has a string table and the other doesn't, we can print less
-    // output.
-    if (Has1 != Has2) {
-      if (Has1) {
-        outs() << formatv("  {0}: ({1} strings)\n", File1.getFilePath(),
-                          ExpectedST1->getNameCount());
-        outs() << formatv("  {0}: (string table not present)\n",
-                          File2.getFilePath());
-      } else {
-        outs() << formatv("  {0}: (string table not present)\n",
-                          File1.getFilePath());
-        outs() << formatv("  {0}: ({1})\n", File2.getFilePath(),
-                          ExpectedST2->getNameCount());
-      }
-    }
+  std::string Count1 = Has1 ? llvm::utostr(ExpectedST1->getNameCount())
+                            : "(string table not present)";
+  std::string Count2 = Has2 ? llvm::utostr(ExpectedST2->getNameCount())
+                            : "(string table not present)";
+  D.print("Number of Strings", Count1, Count2);
+
+  if (!Has1 || !Has2) {
     consumeError(ExpectedST1.takeError());
     consumeError(ExpectedST2.takeError());
     return Error::success();
   }
 
-  bool HasDiff = false;
   auto &ST1 = *ExpectedST1;
   auto &ST2 = *ExpectedST2;
 
-  if (ST1.getByteSize() != ST2.getByteSize()) {
-    outs() << "  Stream Size\n";
-    outs() << formatv("    {0} - {1} byte(s)\n", File1.getFilePath(),
-                      ST1.getByteSize());
-    outs() << formatv("    {0} - {1} byte(s)\n", File2.getFilePath(),
-                      ST2.getByteSize());
-    outs() << formatv("    Difference: {0} bytes\n",
-                      AbsoluteDifference(ST1.getByteSize(), ST2.getByteSize()));
-    HasDiff = true;
-  }
-  HasDiff |= diffAndPrint("Hash Version", File1, File2, ST1.getHashVersion(),
-                          ST1.getHashVersion());
-  HasDiff |= diffAndPrint("Signature", File1, File2, ST1.getSignature(),
-                          ST1.getSignature());
+  D.print("Hash Version", ST1.getHashVersion(), ST2.getHashVersion());
+  D.print("Byte Size", ST1.getByteSize(), ST2.getByteSize());
+  D.print("Signature", ST1.getSignature(), ST2.getSignature());
 
   // Both have a valid string table, dive in and compare individual strings.
 
   auto IdList1 = ST1.name_ids();
   auto IdList2 = ST2.name_ids();
-  std::vector<StringRef> Strings1, Strings2;
-  Strings1.reserve(IdList1.size());
-  Strings2.reserve(IdList2.size());
+  StringSet<> LS;
+  StringSet<> RS;
+  uint32_t Empty1 = 0;
+  uint32_t Empty2 = 0;
   for (auto ID : IdList1) {
     auto S = ST1.getStringForID(ID);
     if (!S)
       return S.takeError();
-    Strings1.push_back(*S);
+    if (S->empty())
+      ++Empty1;
+    else
+      LS.insert(*S);
   }
   for (auto ID : IdList2) {
     auto S = ST2.getStringForID(ID);
     if (!S)
       return S.takeError();
-    Strings2.push_back(*S);
+    if (S->empty())
+      ++Empty2;
+    else
+      RS.insert(*S);
+  }
+  D.print("Empty Strings", Empty1, Empty2);
+
+  DiffPrinter::Equivalent<uint32_t, uint32_t> Comp;
+  for (const auto &S : LS) {
+    auto R = RS.find(S.getKey());
+    std::string Truncated = truncateStringMiddle(S.getKey(), 28);
+    uint32_t I = cantFail(ST1.getIDForString(S.getKey()));
+    if (R == RS.end()) {
+      D.printExplicit(Truncated, DiffResult::DIFFERENT, I, "(not present)");
+      continue;
+    }
+
+    uint32_t J = cantFail(ST2.getIDForString(R->getKey()));
+    D.print(Truncated, I, J, Comp);
+    RS.erase(R);
+  }
+
+  for (const auto &S : RS) {
+    auto L = LS.find(S.getKey());
+    std::string Truncated = truncateStringMiddle(S.getKey(), 28);
+    uint32_t J = cantFail(ST2.getIDForString(S.getKey()));
+    if (L == LS.end()) {
+      D.printExplicit(Truncated, DiffResult::DIFFERENT, "(not present)", J);
+      continue;
     }
 
-  SmallVector<StringRef, 64> OnlyP;
-  SmallVector<StringRef, 64> OnlyQ;
-  auto End1 = std::remove(Strings1.begin(), Strings1.end(), "");
-  auto End2 = std::remove(Strings2.begin(), Strings2.end(), "");
-  uint32_t Empty1 = std::distance(End1, Strings1.end());
-  uint32_t Empty2 = std::distance(End2, Strings2.end());
-  Strings1.erase(End1, Strings1.end());
-  Strings2.erase(End2, Strings2.end());
-  set_differences(Strings1, Strings2, &OnlyP, &OnlyQ);
-  printSymmetricDifferences(File1, File2, OnlyP, OnlyQ, "String");
-
-  if (Empty1 != Empty2) {
-    PDBFile &MoreF = (Empty1 > Empty2) ? File1 : File2;
-    PDBFile &LessF = (Empty1 < Empty2) ? File1 : File2;
-    uint32_t Difference = AbsoluteDifference(Empty1, Empty2);
-    outs() << formatv("  {0} had {1} more empty strings than {2}\n",
-                      MoreF.getFilePath(), Difference, LessF.getFilePath());
+    uint32_t I = cantFail(ST1.getIDForString(L->getKey()));
+    D.print(Truncated, I, J, Comp);
   }
-  if (!HasDiff)
-    outs() << "String Table: No differences detected!\n";
   return Error::success();
 }
 
 Error DiffStyle::diffFreePageMap() { return Error::success(); }
 
 Error DiffStyle::diffInfoStream() {
+  DiffPrinter D(2, "PDB Stream", 22, 40, outs());
+  D.printExplicit("File", DiffResult::UNSPECIFIED,
+                  shortFilePath(File1.getFilePath(), 38),
+                  shortFilePath(File2.getFilePath(), 38));
+
   auto ExpectedInfo1 = File1.getPDBInfoStream();
   auto ExpectedInfo2 = File2.getPDBInfoStream();
 
-  outs() << "PDB Stream: Searching for differences...\n";
   bool Has1 = !!ExpectedInfo1;
   bool Has2 = !!ExpectedInfo2;
   if (!(Has1 && Has2)) {
-    if (Has1 != Has2)
-      outs() << formatv("{0} does not have a PDB Stream!\n",
-                        Has1 ? File1.getFilePath() : File2.getFilePath());
+    std::string L = Has1 ? "(present)" : "(not present)";
+    std::string R = Has2 ? "(present)" : "(not present)";
+    D.print("Stream", L, R);
+
     consumeError(ExpectedInfo2.takeError());
     consumeError(ExpectedInfo2.takeError());
     return Error::success();
   }
 
-  bool HasDiff = false;
   auto &IS1 = *ExpectedInfo1;
   auto &IS2 = *ExpectedInfo2;
-  if (IS1.getStreamSize() != IS2.getStreamSize()) {
-    outs() << "  Stream Size\n";
-    outs() << formatv("    {0} - {1} byte(s)\n", File1.getFilePath(),
-                      IS1.getStreamSize());
-    outs() << formatv("    {0} - {1} byte(s)\n", File2.getFilePath(),
-                      IS2.getStreamSize());
-    outs() << formatv(
-        "    Difference: {0} bytes\n",
-        AbsoluteDifference(IS1.getStreamSize(), IS2.getStreamSize()));
-    HasDiff = true;
-  }
-  HasDiff |= diffAndPrint("Age", File1, File2, IS1.getAge(), IS2.getAge());
-  HasDiff |= diffAndPrint("Guid", File1, File2, IS1.getGuid(), IS2.getGuid());
-  HasDiff |= diffAndPrint("Signature", File1, File2, IS1.getSignature(),
-                          IS2.getSignature());
-  HasDiff |=
-      diffAndPrint("Version", File1, File2, IS1.getVersion(), IS2.getVersion());
-  HasDiff |= diffAndPrint("Features", File1, File2, IS1.getFeatureSignatures(),
-                          IS2.getFeatureSignatures());
-  HasDiff |= diffAndPrint("Named Stream Byte Size", File1, File2,
-                          IS1.getNamedStreamMapByteSize(),
-                          IS2.getNamedStreamMapByteSize());
-  SmallVector<StringRef, 4> NS1;
-  SmallVector<StringRef, 4> NS2;
-  for (const auto &X : IS1.getNamedStreams().entries())
-    NS1.push_back(X.getKey());
-  for (const auto &X : IS2.getNamedStreams().entries())
-    NS2.push_back(X.getKey());
-  SmallVector<StringRef, 4> OnlyP;
-  SmallVector<StringRef, 4> OnlyQ;
-  set_differences(NS1, NS2, &OnlyP, &OnlyQ);
-  printSymmetricDifferences(File1, File2, OnlyP, OnlyQ, "Named Streams");
-  if (!HasDiff)
-    outs() << "PDB Stream: No differences detected!\n";
-
+  D.print("Stream Size", IS1.getStreamSize(), IS2.getStreamSize());
+  D.print("Age", IS1.getAge(), IS2.getAge());
+  D.print("Guid", IS1.getGuid(), IS2.getGuid());
+  D.print("Signature", IS1.getSignature(), IS2.getSignature());
+  D.print("Version", IS1.getVersion(), IS2.getVersion());
+  D.diffUnorderedArray("Feature", IS1.getFeatureSignatures(),
+  D.print("Named Stream Size", IS1.getNamedStreamMapByteSize(),
+  StringMap<uint32_t> NSL = IS1.getNamedStreams().getStringMap();
+  StringMap<uint32_t> NSR = IS2.getNamedStreams().getStringMap();
+  DiffPrinter::Equivalent<uint32_t, uint32_t> Comp;
+  D.diffUnorderedMap("Named Stream", NSL, NSR, Comp);
   return Error::success();
 }
 
 Error DiffStyle::diffDbiStream() { return Error::success(); }
 
 Error DiffStyle::diffSectionContribs() { return Error::success(); }
 
 Error DiffStyle::diffSectionMap() { return Error::success(); }

llvm/tools/llvm-pdbutil/DiffPrinter.h

+//===- DiffPrinter.h ------------------------------------------ *- C++ --*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVMPDBDUMP_DIFFPRINTER_H
+#define LLVM_TOOLS_LLVMPDBDUMP_DIFFPRINTER_H
+
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <list>
+#include <unordered_set>
+
+namespace llvm {
+namespace pdb {
+
+class PDBFile;
+
+enum class DiffResult { UNSPECIFIED, IDENTICAL, EQUIVALENT, DIFFERENT };
+
+class DiffPrinter {
+public:
+  DiffPrinter(uint32_t Indent, StringRef Header, uint32_t PropertyWidth,
+              uint32_t FieldWidth, raw_ostream &Stream);
+  ~DiffPrinter();
+
+  template <typename T, typename U> struct Identical {
+    DiffResult operator()(const T &Left, const U &Right) const {
+      return (Left == Right) ? DiffResult::IDENTICAL : DiffResult::DIFFERENT;
+    }
+  };
+
+  template <typename T, typename U> struct Equivalent {
+    DiffResult operator()(const T &Left, const U &Right) const {
+      return (Left == Right) ? DiffResult::IDENTICAL : DiffResult::EQUIVALENT;
+    }
+  };
+
+  template <typename T, typename U, typename Comparator>
+  void print(StringRef Property, const T &Left, const U &Right,
+             Comparator Comp) {
+    std::string L = formatv("{0}", Left).str();
+    std::string R = formatv("{0}", Right).str();
+
+    DiffResult Result = Comp(Left, Right);
+    printExplicit(Property, Result, L, R);
+  }
+
+  template <typename T, typename U>
+  void print(StringRef Property, const T &Left, const U &Right) {
+    Identical<T, U> Comp;
+    print(Property, Left, Right, Comp);
+  }
+
+  void printExplicit(StringRef Property, DiffResult C, StringRef Left,
+                     StringRef Right);
+
+  template <typename T, typename U>
+  void printExplicit(StringRef Property, DiffResult C, const T &Left,
+                     const U &Right) {
+    std::string L = formatv("{0}", Left).str();
+    std::string R = formatv("{0}", Right).str();
+    printExplicit(Property, C, StringRef(L), StringRef(R));
+  }
+
+  template <typename T, typename U>
+  void diffUnorderedArray(StringRef Property, ArrayRef<T> Left,
+                          ArrayRef<U> Right) {
+    std::unordered_set<T> LS(Left.begin(), Left.end());
+    std::unordered_set<U> RS(Right.begin(), Right.end());
+    std::string Count1 = formatv("{0} element(s)", Left.size());
+    std::string Count2 = formatv("{0} element(s)", Right.size());
+    print(std::string(Property) + "s (set)", Count1, Count2);
+    for (const auto &L : LS) {
+      auto Iter = RS.find(L);
+      std::string Text = formatv("{0}", L).str();
+      if (Iter == RS.end()) {
+        print(Property, Text, "(not present)");
+        continue;
+      }
+      print(Property, Text, Text);
+      RS.erase(Iter);
+    }
+    for (const auto &R : RS) {
+      auto Iter = LS.find(R);
+      std::string Text = formatv("{0}", R).str();
+      if (Iter == LS.end()) {
+        print(Property, "(not present)", Text);
+        continue;
+      }
+      print(Property, Text, Text);
+    }
+  }
+
+  template <typename T, typename U, typename ValueComparator>
+  void diffUnorderedMap(StringRef Property, const StringMap<T> &Left,
+                        const StringMap<U> &Right, ValueComparator Comp) {
+    StringMap<U> RightCopy(Right);
+
+    std::string Count1 = formatv("{0} element(s)", Left.size());
+    std::string Count2 = formatv("{0} element(s)", Right.size());
+    print(std::string(Property) + "s (map)", Count1, Count2);
+
+    for (const auto &L : Left) {
+      auto Iter = RightCopy.find(L.getKey());
+      if (Iter == RightCopy.end()) {
+        printExplicit(L.getKey(), DiffResult::DIFFERENT, L.getValue(),
+                      "(not present)");
+        continue;
+      }
+
+      print(L.getKey(), L.getValue(), Iter->getValue(), Comp);
+      RightCopy.erase(Iter);
+    }
+
+    for (const auto &R : RightCopy) {
+      printExplicit(R.getKey(), DiffResult::DIFFERENT, "(not present)",
+                    R.getValue());
+    }
+  }
+
+private:
+  void printSame(StringRef Property, StringRef Value);
+  void printDifferent(StringRef Property, StringRef Left, StringRef Right);
+
+  void printHeaderRow();
+  void printSeparatorRow();
+  void newLine(char InitialChar = '|');
+  void printField(StringRef Value, DiffResult C, AlignStyle Style,
+                  uint32_t Width);
+
+  uint32_t Indent;
+  uint32_t PropertyWidth;
+  uint32_t FieldWidth;
+  raw_ostream &OS;
+};
+} // namespace pdb
+} // namespace llvm
+
+#endif

llvm/tools/llvm-pdbutil/DiffPrinter.cpp

+
+#include "DiffPrinter.h"
+
+#include "llvm/Support/FormatAdapters.h"
+
+using namespace llvm;
+using namespace llvm::pdb;
+
+static void setColor(llvm::raw_ostream &OS, DiffResult Result) {
+  switch (Result) {
+  case DiffResult::IDENTICAL:
+    OS.changeColor(raw_ostream::Colors::GREEN, false);
+    break;
+  case DiffResult::EQUIVALENT:
+    OS.changeColor(raw_ostream::Colors::YELLOW, true);
+    break;
+  default:
+    OS.changeColor(raw_ostream::Colors::RED, false);
+    break;
+  }
+}
+
+DiffPrinter::DiffPrinter(uint32_t Indent, StringRef Header,
+                         uint32_t PropertyWidth, uint32_t FieldWidth,
+                         raw_ostream &Stream)
+    : Indent(Indent), PropertyWidth(PropertyWidth), FieldWidth(FieldWidth),
+      OS(Stream) {
+  printHeaderRow();
+  newLine();
+  printField(Header, DiffResult::UNSPECIFIED, AlignStyle::Center,
+             PropertyWidth + 1 + FieldWidth + 1 + FieldWidth);
+  printSeparatorRow();
+}
+
+DiffPrinter::~DiffPrinter() {}
+
+void DiffPrinter::printSeparatorRow() {
+  newLine();
+  OS << formatv("{0}", fmt_repeat('-', PropertyWidth));
+  OS << '+';
+  OS << formatv("{0}", fmt_repeat('-', FieldWidth));
+  OS << '+';
+  OS << formatv("{0}", fmt_repeat('-', FieldWidth));
+  OS << '|';
+}
+
+void DiffPrinter::printHeaderRow() {
+  newLine('-');
+  OS << formatv("{0}", fmt_repeat('-', PropertyWidth + 2 * FieldWidth + 3));
+}
+
+void DiffPrinter::newLine(char InitialChar) {
+  OS << "\n";
+  OS.indent(Indent) << InitialChar;
+}
+
+void DiffPrinter::printExplicit(StringRef Property, DiffResult C,
+                                StringRef Left, StringRef Right) {
+  newLine();
+  printField(Property, DiffResult::UNSPECIFIED, AlignStyle::Right,
+             PropertyWidth);
+  printField(Left, C, AlignStyle::Center, FieldWidth);
+  printField(Right, C, AlignStyle::Center, FieldWidth);
+  printSeparatorRow();
+}
+
+void DiffPrinter::printSame(StringRef Property, StringRef Value) {
+  newLine();
+  printField(Property, DiffResult::UNSPECIFIED, AlignStyle::Right,
+             PropertyWidth);
+  printField(Value, DiffResult::IDENTICAL, AlignStyle::Center,
+             FieldWidth + 1 + FieldWidth);
+  printSeparatorRow();
+}
+
+void DiffPrinter::printDifferent(StringRef Property, StringRef Left,
+                                 StringRef Right) {
+  newLine();
+  printField(Property, DiffResult::UNSPECIFIED, AlignStyle::Right,
+             PropertyWidth);
+  printField(Left, DiffResult::DIFFERENT, AlignStyle::Center, FieldWidth);
+  printField(Right, DiffResult::DIFFERENT, AlignStyle::Center, FieldWidth);
+  printSeparatorRow();
+}
+
+void DiffPrinter::printField(StringRef Value, DiffResult C, AlignStyle Style,
+                             uint32_t Width) {
+  if (Style == AlignStyle::Right)
+    --Width;
+
+  std::string FormattedItem =
+      formatv("{0}", fmt_align(Value, Style, Width)).str();
+  if (C != DiffResult::UNSPECIFIED) {
+    setColor(OS, C);
+    OS << FormattedItem;
+    OS.resetColor();
+  } else
+    OS << FormattedItem;
+  if (Style == AlignStyle::Right)
+    OS << ' ';
+  OS << '|';
+}

llvm/tools/llvm-pdbutil/FormatUtil.h

 
 #include <string>
 #include <type_traits>
 
 namespace llvm {
 namespace pdb {
 
 std::string truncateStringBack(StringRef S, uint32_t MaxLen);
+std::string truncateStringMiddle(StringRef S, uint32_t MaxLen);
 static std::string truncateStringFront(StringRef S, uint32_t MaxLen);
 std::string truncateQuotedNameFront(StringRef Label, StringRef Name,
                                     uint32_t MaxLen);
 std::string truncateQuotedNameBack(StringRef Label, StringRef Name,
                                    uint32_t MaxLen);
 
 #define PUSH_MASKED_FLAG(Enum, Mask, TheOpt, Value, Text)                      \
   if (Enum::TheOpt == (Value & Mask))                                          \

llvm/tools/llvm-pdbutil/FormatUtil.cpp

if (MaxLen == 0 || S.size() <= MaxLen || S.size() <= 3)
     return S;
 
   assert(MaxLen >= 3);
   uint32_t FinalLen = std::min<size_t>(S.size(), MaxLen - 3);
   S = S.take_front(FinalLen);
   return std::string(S) + std::string("...");
 }
 
+std::string llvm::pdb::truncateStringMiddle(StringRef S, uint32_t MaxLen) {
+  if (MaxLen == 0 || S.size() <= MaxLen || S.size() <= 3)
+    return S;
+
+  assert(MaxLen >= 3);
+  uint32_t FinalLen = std::min<size_t>(S.size(), MaxLen - 3);
+  StringRef Front = S.take_front(FinalLen / 2);
+  StringRef Back = S.take_back(Front.size());
+  return std::string(Front) + std::string("...") + std::string(Back);
+}
+
 std::string llvm::pdb::truncateStringFront(StringRef S, uint32_t MaxLen) {
   if (MaxLen == 0 || S.size() <= MaxLen || S.size() <= 3)
     return S;
 
   assert(MaxLen >= 3);
   S = S.take_back(MaxLen - 3);
   return std::string("...") + std::string(S);
 }