lld-link: Set PDB GUID to hash of PDB contents instead of to a random byte sequence.

[thakis]

Summary

This is not quite ready for real review. I still need to collect performance data of this, and of the more naive approach that just hashes in a second pass after writing the file.

So feel free to ignore for now, but I figured I'd upload what I have. It's probably done enough that I can use it for benchmarking.

Actual patch description:

Previously, lld-link would use a random byte sequence as the PDB GUID. Instead, use a hash of the PDB file contents.

To compute it, introduce HashingFileBufferByteStream that computes a running hash of the bytes it writes.

Since we already use xxhash, add its streaming parts to llvm//Support/xxhash. xxhash gives only 8 bytes of content hash, so put a fixed string in the other 8 bytes available in the PDB GUID.

To not disturb llvm-pdbutil pdb2yaml, make the hash generation an opt-in feature on InfoStreamBuilder and let ldb/COFF/PDB.cpp always set it.

Since writing the PDB computes this ID which also goes in the exe, the PDB writing code now must be called before writeBuildId(). writeBuildId() for that reason is no longer included in the "Code Layout" timer.

Since the PDB GUID is now a function of the PDB contents, the PDB Age is always set to 1. There was a long comment above loadExistingBuildId (now gone) about how not changing the GUID and only incrementing the age was important, but according to the discussion in PR35914 that comment was incorrect.

[ruiu]

I wonder if you can just compute a hash for a mmap'ed PDB file instead of defining a new type of Stream. That's what we do for ELF when computing a build-id, and I found that's easy to do. It is also easy to parallelize hash computation by making it a tree hash.

[rnk]

In D51887#1229530, @ruiu wrote:

I wonder if you can just compute a hash for a mmap'ed PDB file instead of defining a new type of Stream. That's what we do for ELF when computing a build-id, and I found that's easy to do. It is also easy to parallelize hash computation by making it a tree hash.

If we use a streaming interface to write the file in the first place, hashing as we write will probably provide better temporal locality to the data being hashed. If the hash computation is cheap, fetching the PDB data from RAM into cache will dominate. It's hard to say whether parallelizing the hash will be more of a win than the locality of doing it in the streaming interface, but given that it's not clearly better, I would lean towards doing this because it's convenient.

[thakis]

In D51887#1229530, @ruiu wrote:

I wonder if you can just compute a hash for a mmap'ed PDB file instead of defining a new type of Stream. That's what we do for ELF when computing a build-id, and I found that's easy to do. It is also easy to parallelize hash computation by making it a tree hash.

I considered this, but I figured that's much worse cache-ing wise. Data passed to the stream is in memory, while walking the file again will likely cause lots of page faults, given how bad Windows's file system cache is. Since the new stream turned out to be very little code, I'm pretty happy with this approach.

If you want, I can prototype the other version too and see how much slower it is and how the code looks there. Most of the patch will stay the same in both cases.

[ruiu]

Sequential file access might be fast enough, so I don't know which is better performance-wise. In such situation, I'd probably try to implement an easier one (hashing a mmap'ed buffer) to see if it is satisfactory, or try to implement both to compare if not, to keep it as simple as possible.

[rnk]

In D51887#1229702, @ruiu wrote:

Sequential file access might be fast enough, so I don't know which is better performance-wise. In such situation, I'd probably try to implement an easier one (hashing a mmap'ed buffer) to see if it is satisfactory, or try to implement both to compare if not, to keep it as simple as possible.

I guess we have different ideas about what is simple. :)

[ruiu]

Oh, I read "this" as my suggestion. :)

But computing a hash of a continuous region in memory is much easier than defining a new Stream, no? If you take a look at the code in ELF that computes a build-id, I think you'll notice that's pretty short. It also eliminates the need to modify xxhash.h.

[thakis]

(Landed r341945 to address the 2 differing bytes that caused the guid for rsds.test to be different. It's still different due to different /pdbaltpath: flags, but the pdb contents are now identical except for guid and timestamp. I don't understand yet how /pdbaltpath: makes it into the hash, but for the same /pdbaltpath: it seems to be deterministic now. Looking more…)

[thakis]

In D51887#1230485, @thakis wrote:

(I don't understand yet how /pdbaltpath: makes it into the hash)

(That's because EnvBlockSym's cmd stores the the linker args under the "cmd" entry, so all flags appear in the pdb. So rsds.test can't use different /pdbaltpath:s if it expects the guids to match. All that's left now is to collect perf data with both approaches.)

[thakis]

tests pass

[thakis]

Did some measurements on my linux box (didn't have access to my win box). There, just naively calling xxHash64 on the 1.4GB chrome.dll.pdb file after creating it, doing the parallel xxHas64 computation, this patch, and the old nondeterministic guid code all take about the same amount of time (around 32s, min-of-4 links with each approach is within 0.2s of that). I'll try this on Windows tomorrow.

(Naive: https://reviews.llvm.org/D51956 Parallel: https://reviews.llvm.org/D51957)

[ruiu]

Did some measurements on my linux box (didn't have access to my win box). There, just naively calling xxHash64 on the 1.4GB chrome.dll.pdb file after creating it, doing the parallel xxHas64 computation, this patch, and the old nondeterministic guid code all take about the same amount of time (around 32s, min-of-4 links with each approach is within 0.2s of that). I'll try this on Windows tomorrow.

If it is so cheap to compute a hash, maybe we should use MD5 instead of xxhash to eliminate the possibility of hash collision?

[zturner]

In D51887#1231332, @thakis wrote:

Did some measurements on my linux box (didn't have access to my win box). There, just naively calling xxHash64 on the 1.4GB chrome.dll.pdb file after creating it, doing the parallel xxHas64 computation, this patch, and the old nondeterministic guid code all take about the same amount of time (around 32s, min-of-4 links with each approach is within 0.2s of that). I'll try this on Windows tomorrow.

(Naive: https://reviews.llvm.org/D51956 Parallel: https://reviews.llvm.org/D51957)

I'm a little surprised it's this fast. Are you sure it's even doing anything? What kind of hard drive are you using? A SATA III bus interface, which is still probably the single most common interface used by SSDs, has a theoretical maximum transfer rate of 600MB/s, so you'd be adding at least 2.5 seconds to the link, and that's under optimal circumstances. A SATA II interface, which is also still common enough, is 300MB/s. nVME and PCIe interfaces can reach gigabytes / second but we definitely shoudln't be basing measurements off of those. I don't really have a super strong opinion, but it's something to think about.

[ruiu]

When you are reading back a file that you just created, that read !access doesn't actually hit the disk, no?

[zturner]

Not if it’s small, but a 1.4GB file I would imagine could not fit in cache.
Still though, it’s a good point

[thakis]

In D51887#1232119, @zturner wrote:

In D51887#1231332, @thakis wrote:

Did some measurements on my linux box (didn't have access to my win box). There, just naively calling xxHash64 on the 1.4GB chrome.dll.pdb file after creating it, doing the parallel xxHas64 computation, this patch, and the old nondeterministic guid code all take about the same amount of time (around 32s, min-of-4 links with each approach is within 0.2s of that). I'll try this on Windows tomorrow.

(Naive: https://reviews.llvm.org/D51956 Parallel: https://reviews.llvm.org/D51957)

I'm a little surprised it's this fast. Are you sure it's even doing anything? What kind of hard drive are you using? A SATA III bus interface, which is still probably the single most common interface used by SSDs, has a theoretical maximum transfer rate of 600MB/s, so you'd be adding at least 2.5 seconds to the link, and that's under optimal circumstances. A SATA II interface, which is also still common enough, is 300MB/s. nVME and PCIe interfaces can reach gigabytes / second but we definitely shoudln't be basing measurements off of those. I don't really have a super strong opinion, but it's something to think about.

I'm surprised too. I did check that the guid in the pdb was computed by the new algorithm though, and that the "size" var in front of the xxhash64 call contains 1.4GB. The linux disk cache is pretty good though, so I'm guessing it was all in memory. I'm doing Windows next, where I expect the disk cache to be worse.

[thakis]

The situation on Windows is like on Linux. I built chrome.dll with our pinned lld (which was compiled with clang), with a locally-built lld (built by msvc 2017), and then with this patch and the other two patches linked from here. Min-of-5 times are comparable (full data in parens):

pinned lld, chromedll: 41.2s (42.0s, 45.0s, 41.9s, 43.5s, 41.2s)
trunk lld, chrome.dll: 48.1s (48.3, 48.1, 49.0, 51.0, 49.3)
patched lld, chrome.dll: 48.6s (48.6s, 50.7s, 51.1s, 48.8s, 48.8s)
patched lld, naive hash, chrome.dll: 47.4s (48.8s, 51.0s, 51.5s, 51.0s, 47.4s)
patched lld, parallel hash, chrome.dll: 46.7s (50.2s, 47.9s, 47.8s, 46.7s, 48.8s)

After looking at this data again, the "naive hash" line looked a bit slower than the rest so i re-ran those 5 links and got

patched lld, naive hash, chrome.dll, again: 45.8s (46.9s, 45.8s, 46.1s, 45.8s, 48.5s)

So measurement noise is pretty high :-/ Pinned lld is consistently faster (due to being built by clang), but how exactly the hash is computed doesn't matter. So I suppose https://reviews.llvm.org/D51956 is the way to go. I'll give that a real patch description and send it out.

To get the time, I first built all of chrome_dll, then deleted chrome.dll and got the build target like so: ninja -C out\gn chrome_dll ; del out\gn\chrome.dll; ninja -C out\gn chrome_dll -v -d keeprsp

Then I ran the link command with my locally-built lld like so: C:\src\chrome\src\out\gn>..\..\..\..\tim\tim ninja -t msvc -e environment.x64 -- ../../../../llvm-mono/out/gn/bin/lld-link.exe /nologo /IMPLIB:./chrome.dll.lib /DLL /OUT:./chrome.dll /PDB:./chrome.dll.pdb @./chrome.dll.rsp

(tim is https://github.com/sgraham/tim)

(re md5: Let's do that in a follow-up since it needs re-measuring and will be a single-file change then. The 8 byte hash still gives decent hash collision resistance for up to 2**32 different pdb files, and snce pdbs are keyed by executable name on the symbol server that's per binary. Projects tend to have far fewer revisions than 4 billion.)

[ruiu]

Thank you for sharing the result of the benchmark!

lld/COFF/PDB.h

 namespace coff {
 class OutputSection;
 class SectionChunk;
 class SymbolTable;
 
 void createPDB(SymbolTable *Symtab,
                llvm::ArrayRef<OutputSection *> OutputSections,
                llvm::ArrayRef<uint8_t> SectionTable,
-               const llvm::codeview::DebugInfo &BuildId);
+               llvm::codeview::DebugInfo *BuildId);
 
 std::pair<llvm::StringRef, uint32_t> getFileLine(const SectionChunk *C,
                                                  uint32_t Addr);
 }
 }
 
 #endif

lld/COFF/PDB.cpp

PDBLinker(SymbolTable *Symtab)
         IDTable(Alloc), GlobalTypeTable(Alloc), GlobalIDTable(Alloc) {
     // This isn't strictly necessary, but link.exe usually puts an empty string
     // as the first "valid" string in the string table, so we do the same in
     // order to maintain as much byte-for-byte compatibility as possible.
     PDBStrTab.insert("");
   }
 
   /// Emit the basic PDB structure: initial streams, headers, etc.
-  void initialize(const llvm::codeview::DebugInfo &BuildId);
+  void initialize(llvm::codeview::DebugInfo *BuildId);
 
   /// Add natvis files specified on the command line.
   void addNatvisFiles();
 
   /// Link CodeView from each object file in the symbol table into the PDB.
   void addObjectsToPDB();
 
   /// Link CodeView from a single object file into the PDB.
public:
 
   Expected<const CVIndexMap&> maybeMergeTypeServerPDB(ObjFile *File,
                                                       TypeServer2Record &TS);
 
   /// Add the section map and section contributions to the PDB.
   void addSections(ArrayRef<OutputSection *> OutputSections,
                    ArrayRef<uint8_t> SectionTable);
 
-  /// Write the PDB to disk.
-  void commit();
+  /// Write the PDB to disk and return the Guid generated for it.
+  void commit(codeview::GUID *Guid);
 
 private:
   BumpPtrAllocator Alloc;
 
   SymbolTable *Symtab;
 
   pdb::PDBFileBuilder Builder;
 
tryToLoadPDB(const GUID &GuidFromObj, StringRef TSPath) {
   // PDB file doesn't mean it matches.  For it to match the InfoStream's GUID
   // must match the GUID specified in the TypeServer2 record.
   if (ExpectedInfo->getGuid() != GuidFromObj)
     return make_error<pdb::PDBError>(pdb::pdb_error_code::signature_out_of_date);
 
   return std::move(NS);
 }
 
-Expected<const CVIndexMap&> PDBLinker::maybeMergeTypeServerPDB(ObjFile *File,
-                                                               TypeServer2Record &TS) {
+Expected<const CVIndexMap &>
+PDBLinker::maybeMergeTypeServerPDB(ObjFile *File, TypeServer2Record &TS) {
   const GUID &TSId = TS.getGuid();
   StringRef TSPath = TS.getName();
 
   // First, check if the PDB has previously failed to load.
   auto PrevErr = MissingTypeServerPDBs.find(TSId);
   if (PrevErr != MissingTypeServerPDBs.end())
     return createFileError(
         TSPath,
void PDBLinker::addObjFile(ObjFile *File) {
   }
 
   // Make a new file checksum table that refers to offsets in the PDB-wide
   // string table. Generally the string table subsection appears after the
   // checksum table, so we have to do this after looping over all the
   // subsections.
   auto NewChecksums = make_unique<DebugChecksumsSubsection>(PDBStrTab);
   for (FileChecksumEntry &FC : Checksums) {
-    SmallString<128> FileName = ExitOnErr(CVStrTab.getString(FC.FileNameOffset));
+    SmallString<128> FileName =
+        ExitOnErr(CVStrTab.getString(FC.FileNameOffset));
     if (!sys::path::is_absolute(FileName) &&
         !Config->PDBSourcePath.empty()) {
       SmallString<128> AbsoluteFileName = Config->PDBSourcePath;
       sys::path::append(AbsoluteFileName, FileName);
       sys::path::native(AbsoluteFileName);
       sys::path::remove_dots(AbsoluteFileName, /*remove_dot_dots=*/true);
       FileName = std::move(AbsoluteFileName);
     }
static void addLinkerModuleSectionSymbol(pdb::DbiModuleDescriptorBuilder &Mod,
   Mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
       Sym, Allocator, CodeViewContainer::Pdb));
 }
 
 // Creates a PDB file.
 void coff::createPDB(SymbolTable *Symtab,
                      ArrayRef<OutputSection *> OutputSections,
                      ArrayRef<uint8_t> SectionTable,
-                     const llvm::codeview::DebugInfo &BuildId) {
+                     llvm::codeview::DebugInfo *BuildId) {
   ScopedTimer T1(TotalPdbLinkTimer);
   PDBLinker PDB(Symtab);
 
   PDB.initialize(BuildId);
   PDB.addObjectsToPDB();
   PDB.addSections(OutputSections, SectionTable);
   PDB.addNatvisFiles();
 
   ScopedTimer T2(DiskCommitTimer);
-  PDB.commit();
+  codeview::GUID Guid;
+  PDB.commit(&Guid);
+  memcpy(&BuildId->PDB70.Signature, &Guid, 16);
 }
 
-void PDBLinker::initialize(const llvm::codeview::DebugInfo &BuildId) {
+void PDBLinker::initialize(llvm::codeview::DebugInfo *BuildId) {
   ExitOnErr(Builder.initialize(4096)); // 4096 is blocksize
 
+  BuildId->Signature.CVSignature = OMF::Signature::PDB70;
+  // Signature is set to a hash of the PDB contents when the PDB is done.
+  memset(BuildId->PDB70.Signature, 0, 16);
+  BuildId->PDB70.Age = 1;
+
   // Create streams in MSF for predefined streams, namely
   // PDB, TPI, DBI and IPI.
   for (int I = 0; I < (int)pdb::kSpecialStreamCount; ++I)
     ExitOnErr(Builder.getMsfBuilder().addStream(0));
 
   // Add an Info stream.
   auto &InfoBuilder = Builder.getInfoBuilder();
-  GUID uuid;
-  memcpy(&uuid, &BuildId.PDB70.Signature, sizeof(uuid));
-  InfoBuilder.setAge(BuildId.PDB70.Age);
-  InfoBuilder.setGuid(uuid);
   InfoBuilder.setVersion(pdb::PdbRaw_ImplVer::PdbImplVC70);
+  InfoBuilder.setHashPDBContentsToGUID(true);
 
   // Add an empty DBI stream.
   pdb::DbiStreamBuilder &DbiBuilder = Builder.getDbiBuilder();
-  DbiBuilder.setAge(BuildId.PDB70.Age);
+  DbiBuilder.setAge(BuildId->PDB70.Age);
   DbiBuilder.setVersionHeader(pdb::PdbDbiV70);
   DbiBuilder.setMachineType(Config->Machine);
   // Technically we are not link.exe 14.11, but there are known cases where
   // debugging tools on Windows expect Microsoft-specific version numbers or
   // they fail to work at all.  Since we know we produce PDBs that are
   // compatible with LINK 14.11, we set that version number here.
   DbiBuilder.setBuildNumber(14, 11);
 }
void PDBLinker::addSections(ArrayRef<OutputSection *> OutputSections,
   SectionMap = pdb::DbiStreamBuilder::createSectionMap(Sections);
   DbiBuilder.setSectionMap(SectionMap);
 
   // Add COFF section header stream.
   ExitOnErr(
       DbiBuilder.addDbgStream(pdb::DbgHeaderType::SectionHdr, SectionTable));
 }
 
-void PDBLinker::commit() {
+void PDBLinker::commit(codeview::GUID *Guid) {
   // Write to a file.
-  ExitOnErr(Builder.commit(Config->PDBPath));
+  ExitOnErr(Builder.commit(Config->PDBPath, Guid));
 }
 
 static Expected<StringRef>
 getFileName(const DebugStringTableSubsectionRef &Strings,
             const DebugChecksumsSubsectionRef &Checksums, uint32_t FileID) {
   auto Iter = Checksums.getArray().at(FileID);
   if (Iter == Checksums.getArray().end())
     return make_error<CodeViewError>(cv_error_code::no_records);

lld/COFF/Writer.cpp

private:
   IdataContents Idata;
   DelayLoadContents DelayIdata;
   EdataContents Edata;
   bool SetNoSEHCharacteristic = false;
 
   DebugDirectoryChunk *DebugDirectory = nullptr;
   std::vector<Chunk *> DebugRecords;
   CVDebugRecordChunk *BuildId = nullptr;
-  Optional<codeview::DebugInfo> PreviousBuildId;
   ArrayRef<uint8_t> SectionTable;
 
   uint64_t FileSize;
   uint32_t PointerToSymbolTable = 0;
   uint64_t SizeOfImage;
   uint64_t SizeOfHeaders;
 
   OutputSection *TextSec;
if (StringTableOff) {
     strncpy(Hdr->Name, Name.data(),
             std::min(Name.size(), (size_t)COFF::NameSize));
   }
 }
 
 } // namespace coff
 } // namespace lld
 
-// PDBs are matched against executables using a build id which consists of three
-// components:
-//   1. A 16-bit GUID
-//   2. An age
-//   3. A time stamp.
-//
-// Debuggers and symbol servers match executables against debug info by checking
-// each of these components of the EXE/DLL against the corresponding value in
-// the PDB and failing a match if any of the components differ.  In the case of
-// symbol servers, symbols are cached in a folder that is a function of the
-// GUID.  As a result, in order to avoid symbol cache pollution where every
-// incremental build copies a new PDB to the symbol cache, we must try to re-use
-// the existing GUID if one exists, but bump the age.  This way the match will
-// fail, so the symbol cache knows to use the new PDB, but the GUID matches, so
-// it overwrites the existing item in the symbol cache rather than making a new
-// one.
-static Optional<codeview::DebugInfo> loadExistingBuildId(StringRef Path) {
-  // We don't need to incrementally update a previous build id if we're not
-  // writing codeview debug info.
-  if (!Config->Debug)
-    return None;
-
-  auto ExpectedBinary = llvm::object::createBinary(Path);
-  if (!ExpectedBinary) {
-    consumeError(ExpectedBinary.takeError());
-    return None;
-  }
-
-  auto Binary = std::move(*ExpectedBinary);
-  if (!Binary.getBinary()->isCOFF())
-    return None;
-
-  std::error_code EC;
-  COFFObjectFile File(Binary.getBinary()->getMemoryBufferRef(), EC);
-  if (EC)
-    return None;
-
-  // If the machine of the binary we're outputting doesn't match the machine
-  // of the existing binary, don't try to re-use the build id.
-  if (File.is64() != Config->is64() || File.getMachine() != Config->Machine)
-    return None;
-
-  for (const auto &DebugDir : File.debug_directories()) {
-    if (DebugDir.Type != IMAGE_DEBUG_TYPE_CODEVIEW)
-      continue;
-
-    const codeview::DebugInfo *ExistingDI = nullptr;
-    StringRef PDBFileName;
-    if (auto EC = File.getDebugPDBInfo(ExistingDI, PDBFileName)) {
-      (void)EC;
-      return None;
-    }
-    // We only support writing PDBs in v70 format.  So if this is not a build
-    // id that we recognize / support, ignore it.
-    if (ExistingDI->Signature.CVSignature != OMF::Signature::PDB70)
-      return None;
-    return *ExistingDI;
-  }
-  return None;
-}
-
 // The main function of the writer.
 void Writer::run() {
   ScopedTimer T1(CodeLayoutTimer);
 
   createSections();
   createMiscChunks();
   createImportTables();
   createExportTable();
   mergeSections();
   assignAddresses();
   removeEmptySections();
   setSectionPermissions();
   createSymbolAndStringTable();
 
   if (FileSize > UINT32_MAX)
     fatal("image size (" + Twine(FileSize) + ") " +
         "exceeds maximum allowable size (" + Twine(UINT32_MAX) + ")");
 
-  // We must do this before opening the output file, as it depends on being able
-  // to read the contents of the existing output file.
-  PreviousBuildId = loadExistingBuildId(Config->OutputFile);
   openFile(Config->OutputFile);
   if (Config->is64()) {
     writeHeader<pe32plus_header>();
   } else {
     writeHeader<pe32_header>();
   }
   writeSections();
   sortExceptionTable();
-  writeBuildId();
 
   T1.stop();
 
   if (!Config->PDBPath.empty() && Config->Debug) {
     assert(BuildId);
-    createPDB(Symtab, OutputSections, SectionTable, *BuildId->BuildId);
+    createPDB(Symtab, OutputSections, SectionTable, BuildId->BuildId);
   }
+  writeBuildId();
 
   writeMapFile(OutputSections);
 
   ScopedTimer T2(DiskCommitTimer);
   if (auto E = Buffer->commit())
     fatal("failed to write the output file: " + toString(std::move(E)));
 }
 

 }
 
 void Writer::writeBuildId() {
   // There are two important parts to the build ID.
   // 1) If building with debug info, the COFF debug directory contains a
   //    timestamp as well as a Guid and Age of the PDB.
   // 2) In all cases, the PE COFF file header also contains a timestamp.
   // For reproducibility, instead of a timestamp we want to use a hash of the
-  // binary, however when building with debug info the hash needs to take into
-  // account the debug info, since it's possible to add blank lines to a file
-  // which causes the debug info to change but not the generated code.
-  //
-  // To handle this, we first set the Guid and Age in the debug directory (but
-  // only if we're doing a debug build).  Then, we hash the binary (thus causing
-  // the hash to change if only the debug info changes, since the Age will be
-  // different).  Finally, we write that hash into the debug directory (if
-  // present) as well as the COFF file header (always).
+  // PE contents.
   if (Config->Debug) {
     assert(BuildId && "BuildId is not set!");
-    if (PreviousBuildId.hasValue()) {
-      *BuildId->BuildId = *PreviousBuildId;
-      BuildId->BuildId->PDB70.Age = BuildId->BuildId->PDB70.Age + 1;
-    } else {
-      BuildId->BuildId->Signature.CVSignature = OMF::Signature::PDB70;
-      BuildId->BuildId->PDB70.Age = 1;
-      llvm::getRandomBytes(BuildId->BuildId->PDB70.Signature, 16);
-    }
+    // BuildId->BuildId was filled in when the PDB was written.
   }
 
   // At this point the only fields in the COFF file which remain unset are the
   // "timestamp" in the COFF file header, and the ones in the coff debug
   // directory.  Now we can hash the file and write that hash to the various
   // timestamp fields in the file.
   StringRef OutputFileData(
       reinterpret_cast<const char *>(Buffer->getBufferStart()),

lld/test/COFF/rsds.test

 # RUN: yaml2obj %s > %t.obj
 
 # RUN: rm -f %t.dll %t.pdb
-# RUN: lld-link /debug /pdbaltpath:test1.pdb /dll /out:%t.dll /entry:DllMain %t.obj
+# RUN: lld-link /debug /pdbaltpath:test.pdb /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll > %t.1.txt
-# RUN: lld-link /debug /pdbaltpath:test2.pdb /dll /out:%t.dll /entry:DllMain %t.obj
+# RUN: lld-link /debug /pdbaltpath:test.pdb /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll > %t.2.txt
 # RUN: cat %t.1.txt %t.2.txt | FileCheck %s
 
 # RUN: rm -f %t.dll %t.pdb
 # RUN: lld-link /debug /pdb:%t1.pdb /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll > %t.3.txt
 # RUN: lld-link /debug /pdb:%t2.pdb /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll > %t.4.txt
-# RUN: cat %t.3.txt %t.4.txt | FileCheck %s
+# RUN: cat %t.3.txt %t.4.txt | FileCheck --check-prefix TWOPDBS %s
 
 # RUN: rm -f %t.dll %t.pdb
 # RUN: lld-link /Brepro /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll | FileCheck --check-prefix REPRO %s
 
 # RUN: rm -f %t.dll %t.pdb
 # RUN: lld-link /Brepro /debug /dll /out:%t.dll /entry:DllMain %t.obj
 # RUN: llvm-readobj -coff-debug-directory %t.dll | FileCheck --check-prefix REPRODEBUG %s
 
 # CHECK: File: [[FILE:.*]].dll
 # CHECK: DebugDirectory [
 # CHECK:   DebugEntry {
 # CHECK:     Characteristics: 0x0
 # CHECK:     TimeDateStamp: 
 # CHECK:     MajorVersion: 0x0
 # CHECK:     MinorVersion: 0x0
 # CHECK:     Type: CodeView (0x2)
 # CHECK:     SizeOfData: 0x{{[^0]}}
 # CHECK:     AddressOfRawData: 0x{{[^0]}}
 # CHECK:     PointerToRawData: 0x{{[^0]}}
 # CHECK:     PDBInfo {
 # CHECK:       PDBSignature: 0x53445352
 # CHECK:       PDBGUID: [[GUID:\(([A-Za-z0-9]{2} ?){16}\)]]
 # CHECK:       PDBAge: 1
-# CHECK:       PDBFileName: {{.*}}1.pdb
+# CHECK:       PDBFileName: {{.*}}.pdb
 # CHECK:     }
 # CHECK:   }
 # CHECK: ]
 # CHECK: File: [[FILE]].dll
 # CHECK: DebugDirectory [
 # CHECK:   DebugEntry {
 # CHECK:     Characteristics: 0x0
 # CHECK:     TimeDateStamp: 
 # CHECK:     MajorVersion: 0x0
 # CHECK:     MinorVersion: 0x0
 # CHECK:     Type: CodeView (0x2)
 # CHECK:     SizeOfData: 0x{{[^0]}}
 # CHECK:     AddressOfRawData: 0x{{[^0]}}
 # CHECK:     PointerToRawData: 0x{{[^0]}}
 # CHECK:     PDBInfo {
 # CHECK:       PDBSignature: 0x53445352
 # CHECK:       PDBGUID: [[GUID]]
-# CHECK:       PDBAge: 2
-# CHECK:       PDBFileName: {{.*}}2.pdb
+# CHECK:       PDBAge: 1
+# CHECK:       PDBFileName: {{.*}}.pdb
 # CHECK:     }
 # CHECK:   }
 # CHECK: ]
 
+# TWOPDBS: File: [[FILE:.*]].dll
+# TWOPDBS: DebugDirectory [
+# TWOPDBS:   DebugEntry {
+# TWOPDBS:     Characteristics: 0x0
+# TWOPDBS:     TimeDateStamp: 
+# TWOPDBS:     MajorVersion: 0x0
+# TWOPDBS:     MinorVersion: 0x0
+# TWOPDBS:     Type: CodeView (0x2)
+# TWOPDBS:     SizeOfData: 0x{{[^0]}}
+# TWOPDBS:     AddressOfRawData: 0x{{[^0]}}
+# TWOPDBS:     PointerToRawData: 0x{{[^0]}}
+# TWOPDBS:     PDBInfo {
+# TWOPDBS:       PDBSignature: 0x53445352
+# TWOPDBS:       PDBGUID: [[GUID:\(([A-Za-z0-9]{2} ?){16}\)]]
+# TWOPDBS:       PDBAge: 1
+# TWOPDBS:       PDBFileName: {{.*}}.pdb
+# TWOPDBS:     }
+# TWOPDBS:   }
+# TWOPDBS: ]
+# TWOPDBS: File: [[FILE]].dll
+# TWOPDBS: DebugDirectory [
+# TWOPDBS:   DebugEntry {
+# TWOPDBS:     Characteristics: 0x0
+# TWOPDBS:     TimeDateStamp: 
+# TWOPDBS:     MajorVersion: 0x0
+# TWOPDBS:     MinorVersion: 0x0
+# TWOPDBS:     Type: CodeView (0x2)
+# TWOPDBS:     SizeOfData: 0x{{[^0]}}
+# TWOPDBS:     AddressOfRawData: 0x{{[^0]}}
+# TWOPDBS:     PointerToRawData: 0x{{[^0]}}
+# TWOPDBS:     PDBInfo {
+# TWOPDBS:       PDBSignature: 0x53445352
+# TWOPDBS-NOT:       PDBGUID: [[GUID]]
+# TWOPDBS:       PDBAge: 1
+# TWOPDBS:       PDBFileName: {{.*}}.pdb
+# TWOPDBS:     }
+# TWOPDBS:   }
+# TWOPDBS: ]
+
 # REPRO: File: {{.*}}.dll
 # REPRO: DebugDirectory [
 # REPRO:   DebugEntry {
 # REPRO:     Characteristics: 0x0
 # REPRO:     TimeDateStamp: 
 # REPRO:     MajorVersion: 0x0
 # REPRO:     MinorVersion: 0x0
 # REPRO:     Type: Repro (0x10)

llvm/include/llvm/DebugInfo/MSF/MSFBuilder.h

 //===- MSFBuilder.h - MSF Directory & Metadata Builder ----------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEBUGINFO_MSF_MSFBUILDER_H
 #define LLVM_DEBUGINFO_MSF_MSFBUILDER_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/BitVector.h"
 #include "llvm/DebugInfo/MSF/MSFCommon.h"
 #include "llvm/Support/Allocator.h"
+#include "llvm/Support/BinaryByteStream.h"
 #include "llvm/Support/Error.h"
+#include "llvm/Support/xxhash.h"
 #include <cstdint>
 #include <utility>
 #include <vector>
 
 namespace llvm {
-class FileBufferByteStream;
+
+class HashingFileBufferByteStream final : public FileBufferByteStream {
+public:
+  HashingFileBufferByteStream(std::unique_ptr<FileOutputBuffer> Buffer,
+                              llvm::support::endianness Endian)
+      : FileBufferByteStream(std::move(Buffer), Endian),
+        HashState(XXH64_createState()) {
+    XXH64_reset(HashState, 0);
+  }
+
+  HashingFileBufferByteStream(HashingFileBufferByteStream &&RHS)
+      : FileBufferByteStream(std::move(RHS)) {
+    HashState = RHS.HashState;
+    RHS.HashState = nullptr;
+  }
+
+  ~HashingFileBufferByteStream() {
+    XXH64_freeState(HashState);
+  }
+
+  Error writeBytes(uint32_t Offset, ArrayRef<uint8_t> Data) override {
+    XXH64_update(HashState, Data.data(), Data.size());
+    return FileBufferByteStream::writeBytes(Offset, Data);
+  }
+
+  XXH64_state_t *HashState;
+};
+
 class WritableBinaryStream;
 namespace msf {
 
 class MSFBuilder {
 public:
   /// Create a new `MSFBuilder`.
   ///
   /// \param BlockSize The internal block size used by the PDB file.  See
public:
   /// Check whether a particular block is allocated or free.
   bool isBlockFree(uint32_t Idx) const;
 
   /// Finalize the layout and build the headers and structures that describe the
   /// MSF layout and can be written directly to the MSF file.
   Expected<MSFLayout> generateLayout();
 
   /// Write the MSF layout to the underlying file.
-  Expected<FileBufferByteStream> commit(StringRef Path, MSFLayout &Layout);
+  Expected<HashingFileBufferByteStream> commit(StringRef Path,
+                                               MSFLayout &Layout);
 
   BumpPtrAllocator &getAllocator() { return Allocator; }
 
 private:
   MSFBuilder(uint32_t BlockSize, uint32_t MinBlockCount, bool CanGrow,
              BumpPtrAllocator &Allocator);
 
   Error allocateBlocks(uint32_t NumBlocks, MutableArrayRef<uint32_t> Blocks);

llvm/include/llvm/DebugInfo/PDB/Native/InfoStreamBuilder.h

 
 class InfoStreamBuilder {
 public:
   InfoStreamBuilder(msf::MSFBuilder &Msf, NamedStreamMap &NamedStreams);
   InfoStreamBuilder(const InfoStreamBuilder &) = delete;
   InfoStreamBuilder &operator=(const InfoStreamBuilder &) = delete;
 
   void setVersion(PdbRaw_ImplVer V);
+  void addFeature(PdbRaw_FeatureSig Sig);
+
+  // If this is true, the PDB contents are hashed and this hash is used as
+  // PDB GUID and as Signature. The age is always 1.
+  void setHashPDBContentsToGUID(bool B);
+
+  // These only have an effect if hashPDBContentsToGUID() is false.
   void setSignature(uint32_t S);
   void setAge(uint32_t A);
   void setGuid(codeview::GUID G);
-  void addFeature(PdbRaw_FeatureSig Sig);
 
+  bool hashPDBContentsToGUID() const { return HashPDBContentsToGUID; }
   uint32_t getAge() const { return Age; }
   codeview::GUID getGuid() const { return Guid; }
   Optional<uint32_t> getSignature() const { return Signature; }
 
   uint32_t finalize();
 
   Error finalizeMsfLayout();
 
   Error commit(const msf::MSFLayout &Layout,
                WritableBinaryStreamRef Buffer) const;
 
 private:
   msf::MSFBuilder &Msf;
 
   std::vector<PdbRaw_FeatureSig> Features;
   PdbRaw_ImplVer Ver;
   uint32_t Age;
   Optional<uint32_t> Signature;
   codeview::GUID Guid;
 
+  bool HashPDBContentsToGUID = false;
+
   NamedStreamMap &NamedStreams;
 };
 }
 }
 
 #endif

llvm/include/llvm/DebugInfo/PDB/Native/PDBFileBuilder.h

public:
   msf::MSFBuilder &getMsfBuilder();
   InfoStreamBuilder &getInfoBuilder();
   DbiStreamBuilder &getDbiBuilder();
   TpiStreamBuilder &getTpiBuilder();
   TpiStreamBuilder &getIpiBuilder();
   PDBStringTableBuilder &getStringTableBuilder();
   GSIStreamBuilder &getGsiBuilder();
 
-  Error commit(StringRef Filename);
+  // If HashPDBContentsToGUID is true on the InfoStreamBuilder, Guid is filled
+  // with the computed PDB GUID on return.
+  Error commit(StringRef Filename, codeview::GUID *Guid);
 
   Expected<uint32_t> getNamedStreamIndex(StringRef Name) const;
   Error addNamedStream(StringRef Name, StringRef Data);
   void addInjectedSource(StringRef Name, std::unique_ptr<MemoryBuffer> Buffer);
 
 private:
   struct InjectedSourceDescriptor {
     // The full name of the stream that contains the contents of this injected

llvm/include/llvm/Support/xxhash.h

 #define LLVM_SUPPORT_XXHASH_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
 
 namespace llvm {
 uint64_t xxHash64(llvm::StringRef Data);
 uint64_t xxHash64(llvm::ArrayRef<uint8_t> Data);
+
+// Streaming support.
+typedef enum { XXH_OK=0, XXH_ERROR } XXH_errorcode;
+typedef struct XXH64_state_s XXH64_state_t;   /* incomplete type */
+
+XXH64_state_t *XXH64_createState();
+XXH_errorcode XXH64_freeState(XXH64_state_t *statePtr);
+XXH_errorcode XXH64_reset(XXH64_state_t *statePtr, unsigned long long seed);
+XXH_errorcode XXH64_update(XXH64_state_t *statePtr, const void *input,
+                           size_t length);
+// FIXME: do i want this? do i want a 128 version?
+uint64_t XXH64_digest(const XXH64_state_t *statePtr);
 }
 
 #endif

llvm/lib/DebugInfo/MSF/MSFBuilder.cpp

 //===- MSFBuilder.cpp -----------------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/DebugInfo/MSF/MSFBuilder.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/DebugInfo/MSF/MSFError.h"
 #include "llvm/DebugInfo/MSF/MappedBlockStream.h"
-#include "llvm/Support/BinaryByteStream.h"
 #include "llvm/Support/BinaryStreamWriter.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileOutputBuffer.h"
 #include <algorithm>
 #include <cassert>
 #include <cstdint>
 #include <cstring>
for (uint32_t I = 0; I < 8; ++I) {
       ThisByte |= Mask;
       ++BI;
     }
     cantFail(FpmWriter.writeObject(ThisByte));
   }
   assert(FpmWriter.bytesRemaining() == 0);
 }
 
-Expected<FileBufferByteStream> MSFBuilder::commit(StringRef Path,
+Expected<HashingFileBufferByteStream> MSFBuilder::commit(StringRef Path,
                                                          MSFLayout &Layout) {
   Expected<MSFLayout> L = generateLayout();
   if (!L)
     return L.takeError();
 
   Layout = std::move(*L);
 
   uint64_t FileSize = Layout.SB->BlockSize * Layout.SB->NumBlocks;
   auto OutFileOrError = FileOutputBuffer::create(Path, FileSize);
   if (auto EC = OutFileOrError.takeError())
     return std::move(EC);
 
-  FileBufferByteStream Buffer(std::move(*OutFileOrError),
+  HashingFileBufferByteStream Buffer(std::move(*OutFileOrError),
                               llvm::support::little);
   BinaryStreamWriter Writer(Buffer);
 
   if (auto EC = Writer.writeObject(*Layout.SB))
     return std::move(EC);
 
   commitFpm(Buffer, Layout, Allocator);
 

llvm/lib/DebugInfo/PDB/Native/GSIStreamBuilder.cpp

 }
 
 Error GSIStreamBuilder::commitPublicsHashStream(
     WritableBinaryStreamRef Stream) {
   BinaryStreamWriter Writer(Stream);
   PublicsStreamHeader Header;
 
   // FIXME: Fill these in. They are for incremental linking.
+  Header.SymHash = PSH->calculateSerializedLength();
+  Header.AddrMap = PSH->Records.size() * 4;
   Header.NumThunks = 0;
   Header.SizeOfThunk = 0;
   Header.ISectThunkTable = 0;
+  memset(Header.Padding, 0, sizeof(Header.Padding));
   Header.OffThunkTable = 0;
   Header.NumSections = 0;
-  Header.SymHash = PSH->calculateSerializedLength();
-  Header.AddrMap = PSH->Records.size() * 4;
   if (auto EC = Writer.writeObject(Header))
     return EC;
 
   if (auto EC = PSH->commit(Writer))
     return EC;
 
   std::vector<ulittle32_t> AddrMap = computeAddrMap(PSH->Records);
   if (auto EC = Writer.writeArray(makeArrayRef(AddrMap)))

llvm/lib/DebugInfo/PDB/Native/InfoStreamBuilder.cpp

InfoStreamBuilder::InfoStreamBuilder(msf::MSFBuilder &Msf,
                                      NamedStreamMap &NamedStreams)
     : Msf(Msf), Ver(PdbRaw_ImplVer::PdbImplVC70), Age(0),
       NamedStreams(NamedStreams) {
   ::memset(&Guid, 0, sizeof(Guid));
 }
 
 void InfoStreamBuilder::setVersion(PdbRaw_ImplVer V) { Ver = V; }
 
+void InfoStreamBuilder::addFeature(PdbRaw_FeatureSig Sig) {
+  Features.push_back(Sig);
+}
+
+void InfoStreamBuilder::setHashPDBContentsToGUID(bool B) {
+  HashPDBContentsToGUID = B;
+}
+
 void InfoStreamBuilder::setAge(uint32_t A) { Age = A; }
 
 void InfoStreamBuilder::setSignature(uint32_t S) { Signature = S; }
 
 void InfoStreamBuilder::setGuid(GUID G) { Guid = G; }
 
-void InfoStreamBuilder::addFeature(PdbRaw_FeatureSig Sig) {
-  Features.push_back(Sig);
-}
 
 Error InfoStreamBuilder::finalizeMsfLayout() {
   uint32_t Length = sizeof(InfoStreamHeader) +
                     NamedStreams.calculateSerializedLength() +
                     (Features.size() + 1) * sizeof(uint32_t);
   if (auto EC = Msf.setStreamSize(StreamPDB, Length))
     return EC;
   return Error::success();

llvm/lib/DebugInfo/PDB/Native/PDBFileBuilder.cpp

auto SourceStream = WritableMappedBlockStream::createIndexedStream(
         Layout, MsfBuffer, SN, Allocator);
     BinaryStreamWriter SourceWriter(*SourceStream);
     assert(SourceWriter.bytesRemaining() == IS.Content->getBufferSize());
     cantFail(SourceWriter.writeBytes(
         arrayRefFromStringRef(IS.Content->getBuffer())));
   }
 }
 
-Error PDBFileBuilder::commit(StringRef Filename) {
+Error PDBFileBuilder::commit(StringRef Filename, codeview::GUID *Guid) {
   assert(!Filename.empty());
   if (auto EC = finalizeMsfLayout())
     return EC;
 
   MSFLayout Layout;
-  auto ExpectedMsfBuffer = Msf->commit(Filename, Layout);
+  Expected<HashingFileBufferByteStream> ExpectedMsfBuffer =
+      Msf->commit(Filename, Layout);
   if (!ExpectedMsfBuffer)
     return ExpectedMsfBuffer.takeError();
-  FileBufferByteStream Buffer = std::move(*ExpectedMsfBuffer);
+  HashingFileBufferByteStream Buffer = std::move(*ExpectedMsfBuffer);
 
   auto ExpectedSN = getNamedStreamIndex("/names");
   if (!ExpectedSN)
     return ExpectedSN.takeError();
 
   auto NS = WritableMappedBlockStream::createIndexedStream(
       Layout, Buffer, *ExpectedSN, Allocator);
   BinaryStreamWriter NSWriter(*NS);
uint64_t InfoStreamFileOffset =
       blockToOffset(InfoStreamBlocks.front(), Layout.SB->BlockSize);
   InfoStreamHeader *H = reinterpret_cast<InfoStreamHeader *>(
       Buffer.getBufferStart() + InfoStreamFileOffset);
 
   commitInjectedSources(Buffer, Layout);
 
   // Set the build id at the very end, after every other byte of the PDB
   // has been written.
-  // FIXME: Use a hash of the PDB rather than time(nullptr) for the signature.
+  if (Info->hashPDBContentsToGUID()) {
+    H->Age = 1;
+    uint64_t Digest = XXH64_digest(Buffer.HashState);
+    memcpy(H->Guid.Guid, &Digest, 8);
+    // xxhash only gives us 8 bytes, so put some fixed data in the other half.
+    memcpy(H->Guid.Guid + 8, "LLD PDB.", 8);
+
+    // Return GUID to caller.
+    memcpy(Guid, H->Guid.Guid, 16);
+  } else {
     H->Age = Info->getAge();
     H->Guid = Info->getGuid();
+  }
+
+  // FIXME: Use a hash of the PDB rather than time(nullptr) for the signature.
+  // XXX: change this too
   Optional<uint32_t> Sig = Info->getSignature();
   H->Signature = Sig.hasValue() ? *Sig : time(nullptr);
 
   return Buffer.commit();
 }

llvm/lib/Support/xxhash.cpp

uint64_t llvm::xxHash64(StringRef Data) {
   H64 ^= H64 >> 32;
 
   return H64;
 }
 
 uint64_t llvm::xxHash64(ArrayRef<uint8_t> Data) {
   return xxHash64({(const char *)Data.data(), Data.size()});
 }
+
+struct llvm::XXH64_state_s {
+  uint64_t total_len;
+  uint64_t V1;
+  uint64_t V2;
+  uint64_t V3;
+  uint64_t V4;
+  uint64_t mem64[4];
+  uint32_t memsize;
+}; /* typedef'd to XXH64_state_t */
+
+XXH64_state_t *llvm::XXH64_createState(void) {
+  return (XXH64_state_t *)malloc(sizeof(XXH64_state_t));
+}
+
+XXH_errorcode llvm::XXH64_freeState(XXH64_state_t *statePtr) {
+  free(statePtr);
+  return XXH_OK;
+}
+
+XXH_errorcode llvm::XXH64_reset(XXH64_state_t *statePtr, unsigned long long seed) {
+  XXH64_state_t state; /* using a local state to memcpy() in order to avoid
+                          strict-aliasing warnings */
+  memset(&state, 0, sizeof(state));
+  state.V1 = seed + PRIME64_1 + PRIME64_2;
+  state.V2 = seed + PRIME64_2;
+  state.V3 = seed + 0;
+  state.V4 = seed - PRIME64_1;
+  memcpy(statePtr, &state, sizeof(state));
+  return XXH_OK;
+}
+
+XXH_errorcode llvm::XXH64_update(XXH64_state_t *state, const void *input,
+                           size_t len) {
+  const unsigned char *P = (const unsigned char *)input;
+  const unsigned char *const BEnd = P + len;
+
+  if (input == NULL)
+    return XXH_ERROR;
+
+  state->total_len += len;
+
+  if (state->memsize + len < 32) { /* fill in tmp buffer */
+    memcpy(((unsigned char *)state->mem64) + state->memsize, input, len);
+    state->memsize += (uint32_t)len;
+    return XXH_OK;
+  }
+
+  if (state->memsize) { /* tmp buffer is full */
+    memcpy(((unsigned char *)state->mem64) + state->memsize, input,
+           32 - state->memsize);
+    state->V1 = round(state->V1, endian::read64le(state->mem64 + 0));
+    state->V2 = round(state->V2, endian::read64le(state->mem64 + 1));
+    state->V3 = round(state->V3, endian::read64le(state->mem64 + 2));
+    state->V4 = round(state->V4, endian::read64le(state->mem64 + 3));
+    P += 32 - state->memsize;
+    state->memsize = 0;
+  }
+
+  if (P + 32 <= BEnd) {
+    const unsigned char *const Limit = BEnd - 32;
+    uint64_t V1 = state->V1;
+    uint64_t V2 = state->V2;
+    uint64_t V3 = state->V3;
+    uint64_t V4 = state->V4;
+
+    do {
+      V1 = round(V1, endian::read64le(P));
+      P += 8;
+      V2 = round(V2, endian::read64le(P));
+      P += 8;
+      V3 = round(V3, endian::read64le(P));
+      P += 8;
+      V4 = round(V4, endian::read64le(P));
+      P += 8;
+    } while (P <= Limit);
+
+    state->V1 = V1;
+    state->V2 = V2;
+    state->V3 = V3;
+    state->V4 = V4;
+  }
+
+  if (P < BEnd) {
+    memcpy(state->mem64, P, (size_t)(BEnd - P));
+    state->memsize = (unsigned)(BEnd - P);
+  }
+
+  return XXH_OK;
+}
+
+uint64_t llvm::XXH64_digest(const XXH64_state_t *state) {
+  const unsigned char *P = (const unsigned char *)state->mem64;
+  const unsigned char *const BEnd =
+      (const unsigned char *)state->mem64 + state->memsize;
+  uint64_t h64;
+
+  if (state->total_len >= 32) {
+    uint64_t const V1 = state->V1;
+    uint64_t const V2 = state->V2;
+    uint64_t const V3 = state->V3;
+    uint64_t const V4 = state->V4;
+
+    h64 = rotl64(V1, 1) + rotl64(V2, 7) + rotl64(V3, 12) + rotl64(V4, 18);
+    h64 = mergeRound(h64, V1);
+    h64 = mergeRound(h64, V2);
+    h64 = mergeRound(h64, V3);
+    h64 = mergeRound(h64, V4);
+  } else {
+    h64 = state->V3 + PRIME64_5;
+  }
+
+  h64 += (uint64_t)state->total_len;
+
+  while (P + 8 <= BEnd) {
+    uint64_t const k1 = round(0, endian::read64le(P));
+    h64 ^= k1;
+    h64 = rotl64(h64, 27) * PRIME64_1 + PRIME64_4;
+    P += 8;
+  }
+
+  if (P + 4 <= BEnd) {
+    h64 ^= (uint64_t)(endian::read32le(P)) * PRIME64_1;
+    h64 = rotl64(h64, 23) * PRIME64_2 + PRIME64_3;
+    P += 4;
+  }
+
+  while (P < BEnd) {
+    h64 ^= (*P) * PRIME64_5;
+    h64 = rotl64(h64, 11) * PRIME64_1;
+    P++;
+  }
+
+  h64 ^= h64 >> 33;
+  h64 *= PRIME64_2;
+  h64 ^= h64 >> 29;
+  h64 *= PRIME64_3;
+  h64 ^= h64 >> 32;
+
+  return h64;
+}

llvm/tools/llvm-pdbutil/llvm-pdbutil.cpp

static void yamlToPdb(StringRef Path) {
   IpiBuilder.setVersionHeader(Ipi.Version);
   for (const auto &R : Ipi.Records) {
     CVType Type = R.toCodeViewRecord(TS);
     IpiBuilder.addTypeRecord(Type.RecordData, None);
   }
 
   Builder.getStringTableBuilder().setStrings(*Strings.strings());
 
-  ExitOnErr(Builder.commit(opts::yaml2pdb::YamlPdbOutputFile));
+  codeview::GUID IgnoredOutGuid;
+  ExitOnErr(Builder.commit(opts::yaml2pdb::YamlPdbOutputFile, &IgnoredOutGuid));
 }
 
 static PDBFile &loadPDB(StringRef Path, std::unique_ptr<IPDBSession> &Session) {
   ExitOnErr(loadDataForPDB(PDB_ReaderType::Native, Path, Session));
 
   NativeSession *NS = static_cast<NativeSession *>(Session.get());
   return NS->getPDBFile();
 }
static void mergePdbs() {
   });
   Builder.getInfoBuilder().addFeature(PdbRaw_FeatureSig::VC140);
 
   SmallString<64> OutFile(opts::merge::PdbOutputFile);
   if (OutFile.empty()) {
     OutFile = opts::merge::InputFilenames[0];
     llvm::sys::path::replace_extension(OutFile, "merged.pdb");
   }
-  ExitOnErr(Builder.commit(OutFile));
+
+  codeview::GUID IgnoredOutGuid;
+  ExitOnErr(Builder.commit(OutFile, &IgnoredOutGuid));
 }
 
 static void explain() {
   std::unique_ptr<IPDBSession> Session;
   InputFile IF =
       ExitOnErr(InputFile::open(opts::explain::InputFilename.front(), true));
 
   for (uint64_t Off : opts::explain::Offsets) {