[LLD][COFF] Survive empty and invalid PCH signature

[aganea]

Summary

While trying to use LLD with the Unreal Engine - which happens to heavily use PCH files - it was failing with "error: xxx.obj claims to be a PCH object, but does not have a valid signature". It seems recent versions of MSVC (or at least the one we use) do not generate the PCH signature in the S_OBJNAME record anymore. Instead we need to parse the entire .PCH.OBJ file until we reach the LF_ENDPRECOMP record, which contains a valid PCH signature. The code already was able to survive without that, by falling back to name lookup, but it's more reliable to use the precompSourceMappings map for matching the corresponding PCH file.

[rnk]

So, the signature moved from S_OBJNAME to LF_ENDPRECOMP, but everything else more or less stays the same.

Sounds good to me.

[thieta]

LGTM

[aganea]

Thanks for the review!

In D136762#3885821, @rnk wrote:

So, the signature moved from S_OBJNAME to LF_ENDPRECOMP, but everything else more or less stays the same.

Sounds good to me.

Actually the code supports both now, depending on the version of the MSVC compiler.

I'll hold on this for a bit, I'm still seeing weird warnings and the .PDB is only 450 MiB with LLD -- instead of 2 GiB with link.exe. I think the issues are unrelated, but i'd like to get to the bottom of it.

[aganea]

I finally found some time to look at this. The issue is more complicated than I thought, and the solution is not the same for GHASH than non-GHASH.

Another issue was that .PCH.OBJs are non-deterministic by default. Example:

>cat precomp.cpp
#include "precomp.h"

>cl /Yc /Z7 /c precomp.cpp /nologo
precomp.cpp

>llvm-pdbutil dump --all precomp.obj > dump.txt

>cl /Yc /Z7 /c precomp.cpp /nologo
precomp.cpp

>llvm-pdbutil dump --all precomp.obj > dump2.txt

>diff dump.txt dump2.txt
17991c17991
< 0x2382 | LF_ENDPRECOMP [size = 8] signature = 0x522700AA
---
> 0x2382 | LF_ENDPRECOMP [size = 8] signature = 0x2E1E4F09
18020c18020
<        0 | S_OBJNAME [size = 40] sig=1378287786, `C:\git\llvm-project\precomp.obj`
---
>        0 | S_OBJNAME [size = 40] sig=773738249, `C:\git\llvm-project\precomp.obj`

There was a situation where our .PCH.OBJs were always recompiled on a clean rebuild, but not the .OBJs (they were cached). MSVC link.exe was able to link without warnings or errors. When I first implemented the PCH support, I initially thought the PCH signature was a hard requirement, but it seems it's just a hint for the runtime. I relaxed that requirement in LLD, but now we'd still fail if the LF_ENDPRECOMP index doesn't line up with the data in LF_PRECOMP.

Also worth noting that using MSVC's cl /Yc /experimental:deterministic .. makes the PCH signature deterministic.

Would you mind taking a second look please?

[rnk]

Thanks, looks good.

In D136762#3892851, @aganea wrote:

There was a situation where our .PCH.OBJs were always recompiled on a clean rebuild, but not the .OBJs (they were cached).

I'm imagining some system outside the build system forcing the .pch.obj to be recompiled, but the build system (fastbuild?) has correct and complete knowledge that none of the PCH inputs changed.

lld/COFF/DebugTypes.cpp

319

Maintaining two codepaths for type merging has significant costs, but I think the ghash code path is too optimized, and too difficult to understand. Perhaps with more work and cleanups, it could be made straightforward enough that we could afford to delete the regular type merging codepath, but for now I guess we should continue maintaining both.

lld/COFF/DebugTypes.h

116–120

Part of me wants to say, make it a TypeIndex to eliminate this subtlety, but the new usage uses it as a count, so I don't think it makes sense.

[thieta]

LGTM

lld/COFF/DebugTypes.h

public:
   /// Is this a dependent file that needs to be processed first, before other
   /// OBJs?
   virtual bool isDependency() const { return false; }
 
   /// Returns true if this type record should be omitted from the PDB, even if
   /// it is unique. This prevents a record from being added to the input ghash
   /// table.
   bool shouldOmitFromPdb(uint32_t ghashIdx) {
-    return ghashIdx == endPrecompGHashIdx;
+    return ghashIdx == endPrecompIdx;
   }
 
   const TpiKind kind;
   bool ownedGHashes = true;
   uint32_t tpiSrcIdx = 0;
 
-protected:
-  /// The ghash index (zero based, not 0x1000-based) of the LF_ENDPRECOMP record
-  /// in this object, if one exists. This is the all ones value otherwise. It is
-  /// recorded here so that it can be omitted from the final ghash table.
-  uint32_t endPrecompGHashIdx = ~0U;
+  /// The index (zero based, not 0x1000-based) of the LF_ENDPRECOMP record in
+  /// this object, if one exists. This is the all ones value otherwise. It is
+  /// recorded here for validation, and so that it can be omitted from the final
+  /// ghash table.
+  uint32_t endPrecompIdx = ~0U;
 
 public:
   ObjFile *file;
 
   /// An error encountered during type merging, if any.
   Error typeMergingError = Error::success();
 
   // Storage for tpiMap or ipiMap, depending on the kind of source.

lld/COFF/DebugTypes.cpp

 
 // This class represents the debug type stream of a Microsoft precompiled
 // headers OBJ (PCH OBJ). This OBJ kind needs to be merged first in the output
 // PDB, before any other OBJs that depend on this. Note that only MSVC generate
 // such files, clang does not.
 class PrecompSource : public TpiSource {
 public:
   PrecompSource(COFFLinkerContext &ctx, ObjFile *f) : TpiSource(ctx, PCH, f) {
-    if (!f->pchSignature || !*f->pchSignature)
-      fatal(toString(f) +
-            " claims to be a PCH object, but does not have a valid signature");
-    auto it = ctx.precompSourceMappings.emplace(*f->pchSignature, this);
-    if (!it.second)
-      fatal("a PCH object with the same signature has already been provided (" +
-            toString(it.first->second->file) + " and " + toString(file) + ")");
+    // If the S_OBJNAME record contains the PCH signature, we'll register this
+    // source file right away.
+    registerMapping();
   }
 
+  Error mergeDebugT(TypeMerger *m) override;
+
   void loadGHashes() override;
 
   bool isDependency() const override { return true; }
+
+private:
+  void registerMapping();
+
+  // Whether this precomp OBJ was recorded in the precompSourceMappings map.
+  // Only happens if the file->pchSignature is valid.
+  bool registered = false;
 };
 
 // This class represents the debug type stream of an OBJ file that depends on a
 // Microsoft precompiled headers OBJ (see PrecompSource).
 class UsePrecompSource : public TpiSource {
 public:
   UsePrecompSource(COFFLinkerContext &ctx, ObjFile *f, PrecompRecord precomp)
       : TpiSource(ctx, UsingPCH, f), precompDependency(precomp) {}
Error TpiSource::mergeDebugT(TypeMerger *m) {
 
   CVTypeArray types;
   BinaryStreamReader reader(file->debugTypes, support::little);
   cantFail(reader.readArray(types, reader.getLength()));
 
   // When dealing with PCH.OBJ, some indices were already merged.
   unsigned nbHeadIndices = indexMapStorage.size();
 
-  if (auto err = mergeTypeAndIdRecords(
-          m->idTable, m->typeTable, indexMapStorage, types, file->pchSignature))
+  Optional<PCHMergerInfo> pchInfo;
+  if (auto err = mergeTypeAndIdRecords(m->idTable, m->typeTable,
+                                       indexMapStorage, types, pchInfo))
     fatal("codeview::mergeTypeAndIdRecords failed: " +
           toString(std::move(err)));
+  if (pchInfo) {
+    file->pchSignature = pchInfo->PCHSignature;
+    endPrecompIdx = pchInfo->EndPrecompIndex;
+  }
 
   // In an object, there is only one mapping for both types and items.
   tpiMap = indexMapStorage;
   ipiMap = indexMapStorage;
 
   if (config->showSummary) {
     nbTypeRecords = indexMapStorage.size() - nbHeadIndices;
     nbTypeRecordsBytes = reader.getLength();
Expected<PrecompSource *> UsePrecompSource::findPrecompMap(ObjFile *file,
                                                            PrecompRecord &pr) {
   PrecompSource *precomp = findPrecompSource(file, pr);
 
   if (!precomp)
     return createFileError(
         pr.getPrecompFilePath(),
         make_error<pdb::PDBError>(pdb::pdb_error_code::no_matching_pch));
 
-  if (pr.getSignature() != file->pchSignature)
+  // Don't rely on the PCH signature to validate the concordance between the PCH
+  // and the OBJ that uses it. However we do validate here that the
+  // LF_ENDPRECOMP record index lines up with the number of type records
+  // LF_PRECOMP is expecting.
+  if (precomp->endPrecompIdx != pr.getTypesCount())
     return createFileError(
         toString(file),
         make_error<pdb::PDBError>(pdb::pdb_error_code::no_matching_pch));
 
-  if (pr.getSignature() != *precomp->file->pchSignature)
-    return createFileError(
-        toString(precomp->file),
-        make_error<pdb::PDBError>(pdb::pdb_error_code::no_matching_pch));
-
   return precomp;
 }
 
 /// Merges a precompiled headers TPI map into the current TPI map. The
 /// precompiled headers object will also be loaded and remapped in the
 /// process.
 Error UsePrecompSource::mergeInPrecompHeaderObj() {
   auto e = findPrecompMap(file, precompDependency);
Error UsePrecompSource::mergeDebugT(TypeMerger *m) {
   // object are referencing data in the precompiled headers object, so we need
   // both to be loaded.
   if (Error e = mergeInPrecompHeaderObj())
     return e;
 
   return TpiSource::mergeDebugT(m);
 }
 
+Error PrecompSource::mergeDebugT(TypeMerger *m) {
+  // In some cases, the S_OBJNAME record doesn't contain the PCH signature.
+  // The signature comes later with the LF_ENDPRECOMP record, so we first need
+  // to merge in all the .PCH.OBJ file type records, before registering below.
+  if (Error e = TpiSource::mergeDebugT(m))
+    return e;
+
+  registerMapping();
+
+  return Error::success();
+}
+
+void PrecompSource::registerMapping() {
+  if (registered)
+    return;
+  if (file->pchSignature && *file->pchSignature) {
+    auto it = ctx.precompSourceMappings.emplace(*file->pchSignature, this);
+    if (!it.second)
+      fatal("a PCH object with the same signature has already been provided (" +
+            toString(it.first->second->file) + " and " + toString(file) + ")");
+    registered = true;
+  }
+}
+
 //===----------------------------------------------------------------------===//
 // Parellel GHash type merging implementation.
 //===----------------------------------------------------------------------===//
 
 void TpiSource::loadGHashes() {
   if (Optional<ArrayRef<uint8_t>> debugH = getDebugH(file)) {
     ghashes = getHashesFromDebugH(*debugH);
     ownedGHashes = false;
void PrecompSource::loadGHashes() {
   }
 
   uint32_t ghashIdx = 0;
   std::vector<GloballyHashedType> hashVec;
   forEachTypeChecked(file->debugTypes, [&](const CVType &ty) {
     // Remember the index of the LF_ENDPRECOMP record so it can be excluded from
     // the PDB. There must be an entry in the list of ghashes so that the type
     // indexes of the following records in the /Yc PCH object line up.
-    if (ty.kind() == LF_ENDPRECOMP)
-      endPrecompGHashIdx = ghashIdx;
+    if (ty.kind() == LF_ENDPRECOMP) {
+      EndPrecompRecord endPrecomp;
+      cantFail(TypeDeserializer::deserializeAs<EndPrecompRecord>(
+          const_cast<CVType &>(ty), endPrecomp));
+      file->pchSignature = endPrecomp.getSignature();
+      registerMapping();
+      endPrecompIdx = ghashIdx;
+    }
 
     hashVec.push_back(GloballyHashedType::hashType(ty, hashVec, hashVec));
     isItemIndex.push_back(isIdRecord(ty.kind()));
     ++ghashIdx;
   });
   assignGHashesFromVector(std::move(hashVec));
 }
 
 void UsePrecompSource::loadGHashes() {
-  PrecompSource *pchSrc = findPrecompSource(file, precompDependency);
-  if (!pchSrc)
+  auto e = findPrecompMap(file, precompDependency);
+  if (!e) {
+    warn(toString(e.takeError()));
     return;
+  }
+
+  PrecompSource *pchSrc = *e;
 
   // To compute ghashes of a /Yu object file, we need to build on the ghashes of
   // the /Yc PCH object. After we are done hashing, discard the ghashes from the
   // PCH source so we don't unnecessarily try to deduplicate them.
   std::vector<GloballyHashedType> hashVec =
       pchSrc->ghashes.take_front(precompDependency.getTypesCount());
   forEachTypeChecked(file->debugTypes, [&](const CVType &ty) {
     hashVec.push_back(GloballyHashedType::hashType(ty, hashVec, hashVec));

lld/COFF/InputFiles.cpp

for (unsigned i = 0; i < 2; ++i) {
         auto cs =
             cantFail(SymbolDeserializer::deserializeAs<Compile3Sym>(sym.get()));
         hotPatchable =
             (cs.Flags & CompileSym3Flags::HotPatch) != CompileSym3Flags::None;
       }
       if (sym->kind() == SymbolKind::S_OBJNAME) {
         auto objName = cantFail(SymbolDeserializer::deserializeAs<ObjNameSym>(
             sym.get()));
+        if (objName.Signature)
           pchSignature = objName.Signature;
       }
       offset += sym->length();
     }
   }
 }
 
 // Depending on the compilation flags, OBJs can refer to external files,
 // necessary to merge this OBJ into the final PDB. We currently support two
if (firstType->kind() == LF_TYPESERVER2) {
     return;
   }
 
   // This object was compiled with /Yu. It uses types from another object file
   // with a matching signature.
   if (firstType->kind() == LF_PRECOMP) {
     PrecompRecord precomp = cantFail(
         TypeDeserializer::deserializeAs<PrecompRecord>(firstType->data()));
+    // We're better off trusting the LF_PRECOMP signature. In some cases the
+    // S_OBJNAME record doesn't contain a valid PCH signature.
+    if (precomp.Signature)
+      pchSignature = precomp.Signature;
     debugTypesObj = makeUsePrecompSource(ctx, this, precomp);
     // Drop the LF_PRECOMP record from the input stream.
     debugTypes = debugTypes.drop_front(firstType->RecordData.size());
     return;
   }
 
   // This is a plain old object file.
   debugTypesObj = makeTpiSource(ctx, this);

lld/docs/ReleaseNotes.rst

 COFF Improvements
 -----------------
 
 * The linker command line entry in ``S_ENVBLOCK`` of the PDB is now stripped
   from input files, to align with MSVC behavior.
   (`D137723 <https://reviews.llvm.org/D137723>`_)
 * Switched from SHA1 to BLAKE3 for PDB type hashing / ``-gcodeview-ghash``
   (`D137101 <https://reviews.llvm.org/D137101>`_)
+* Improvements to the PCH.OBJ files handling. Now LLD behaves the same as MSVC
+  link.exe when merging PCH.OBJ files that don't have the same signature.
+  (`D136762 <https://reviews.llvm.org/D136762>`_)
 
 MinGW Improvements
 ------------------
 
 * The lld-specific options ``--guard-cf``, ``--no-guard-cf``,
   ``--guard-longjmp`` and ``--no-guard-longjmp`` has been added to allow
   enabling Control Flow Guard and long jump hardening. These options are
   disabled by default, but enabling ``--guard-cf`` will also enable

lld/test/COFF/precomp-link.test

-RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf /summary | FileCheck %s -check-prefix SUMMARY
+RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug:noghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf /summary | FileCheck %s -check-prefix SUMMARY
+RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
+RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf /summary | FileCheck %s -check-prefix SUMMARY
 RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
 
 RUN: lld-link %S/Inputs/precomp.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf
 RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
 
-RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-invalid.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1 | FileCheck %s -check-prefix FAILURE
-RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-invalid.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1 | FileCheck %s -check-prefix FAILURE
+RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-invalid.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug:noghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1
+RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-invalid.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1
 
 FIXME: The following RUN line should fail, regardless of whether debug info is
 enabled or not. Normally this would result in an error due to missing _PchSym_
 references, but SymbolTable.cpp suppresses such errors. MSVC seems to have a
 special case for those symbols and it emits the LNK2011 error.
 
 RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1 | FileCheck %s -check-prefix FAILURE-MISSING-PRECOMPOBJ
 
-FAILURE: warning: Cannot use debug info for '{{.*}}precomp-invalid.obj' [LNK4099]
-FAILURE-NEXT: failed to load reference '{{.*}}precomp.obj': No matching precompiled header could be located.
-
 FAILURE-MISSING-PRECOMPOBJ: warning: Cannot use debug info for '{{.*}}precomp-a.obj' [LNK4099]
 FAILURE-MISSING-PRECOMPOBJ-NEXT: failed to load reference '{{.*}}precomp.obj': No matching precompiled header could be located.
 
-Check that a PCH object file with a missing S_OBJNAME record results in an
-error. Edit out this record from the yaml-ified object:
+Check that a PCH object file with an empty S_OBJNAME record works fine.
+Edit out this record from the yaml-ified object:
   - Kind:            S_OBJNAME
     ObjNameSym:
       Signature:       545589255
       ObjectName:      'F:\svn\lld\test\COFF\precomp\precomp.obj'
 
 RUN: obj2yaml %S/Inputs/precomp.obj | grep -v 'SectionData: *04000000' > %t.precomp.yaml
 RUN: sed '/S_OBJNAME/,/ObjectName:/d' < %t.precomp.yaml > precomp-no-objname.yaml
-RUN: sed 's/Signature: *545589255/Signature: 0/' < %t.precomp.yaml > precomp-zero-sig.yaml
+RUN: sed '16,19s/Signature: *545589255/Signature: 0/' < %t.precomp.yaml > precomp-zero-sig.yaml
 RUN: yaml2obj precomp-no-objname.yaml -o %t.precomp-no-objname.obj
 RUN: yaml2obj precomp-zero-sig.yaml -o %t.precomp-zero-sig.obj
-
-RUN: env LLD_IN_TEST=1 not lld-link %t.precomp-no-objname.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix FAILURE-NO-SIGNATURE
-
-RUN: env LLD_IN_TEST=1 not lld-link %t.precomp-zero-sig.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix FAILURE-NO-SIGNATURE
-
-FAILURE-NO-SIGNATURE: error: {{.*}}.obj claims to be a PCH object, but does not have a valid signature
+RUN: env LLD_IN_TEST=1 lld-link %t.precomp-no-objname.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:noghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
+RUN: env LLD_IN_TEST=1 lld-link %t.precomp-no-objname.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
+RUN: env LLD_IN_TEST=1 lld-link %t.precomp-zero-sig.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:noghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
+RUN: env LLD_IN_TEST=1 lld-link %t.precomp-zero-sig.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
+
+WARNING-NOT: warning
+
+Check that a PCH with wrong signature, but with right LF_PRECOMP records count, works.
+
+RUN: sed '16,19s/Signature: *545589255/Signature: 123456789/' < %t.precomp.yaml > precomp-wrong-sig.yaml
+RUN: yaml2obj precomp-wrong-sig.yaml -o %T/precomp.obj
+RUN: env LLD_IN_TEST=1 lld-link %T/precomp.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:noghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
+RUN: env LLD_IN_TEST=1 lld-link %T/precomp.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug:ghash /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix WARNING --allow-empty
 
 Check that two PCH objs with duplicate signatures are an error.
 
 RUN: cp %S/Inputs/precomp.obj %t.precomp-dup.obj
-
 RUN: env LLD_IN_TEST=1 not lld-link %S/Inputs/precomp.obj %t.precomp-dup.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe 2>&1 | FileCheck %s -check-prefix FAILURE-DUP-SIGNATURE
-
 FAILURE-DUP-SIGNATURE: error: a PCH object with the same signature has already been provided ({{.*precomp.obj and .*precomp-dup.obj.*}})
 
 
 CHECK: Types (TPI Stream)
 CHECK-NOT: LF_PRECOMP
 CHECK-NOT: LF_ENDPRECOMP
 
-
-Re-run with ghash. Eventually, perhaps this will be the default.
-
-RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug /debug:ghash /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf /summary | FileCheck %s -check-prefix SUMMARY
-RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
-
-
 SUMMARY:                                     Summary
 SUMMARY-NEXT: --------------------------------------------------------------------------------
 SUMMARY-NEXT:               3 Input OBJ files (expanded from all cmd-line inputs)
 SUMMARY-NEXT:               0 PDB type server dependencies
 SUMMARY-NEXT:               1 Precomp OBJ dependencies
 SUMMARY-NEXT:            1066 Input type records
 SUMMARY-NEXT:           55968 Input type records bytes
 SUMMARY-NEXT:             874 Merged TPI records

llvm/include/llvm/DebugInfo/CodeView/TypeStreamMerger.h

 template <typename T> class SmallVectorImpl;
 namespace codeview {
 
 class TypeIndex;
 struct GloballyHashedType;
 class GlobalTypeTableBuilder;
 class MergingTypeTableBuilder;
 
+/// Used to forward information about PCH.OBJ (precompiled) files, when
+/// applicable.
+struct PCHMergerInfo {
+  uint32_t PCHSignature{};
+  uint32_t EndPrecompIndex = ~0U;
+};
+
 /// Merge one set of type records into another.  This method assumes
 /// that all records are type records, and there are no Id records present.
 ///
 /// \param Dest The table to store the re-written type records into.
 ///
 /// \param SourceToDest A vector, indexed by the TypeIndex in the source
 /// type stream, that contains the index of the corresponding type record
 /// in the destination stream.

 /// \param IdsAndTypes The collection of id records to merge in.
 ///
 /// \returns Error::success() if the operation succeeded, otherwise an
 /// appropriate error code.
 Error mergeTypeAndIdRecords(MergingTypeTableBuilder &DestIds,
                             MergingTypeTableBuilder &DestTypes,
                             SmallVectorImpl<TypeIndex> &SourceToDest,
                             const CVTypeArray &IdsAndTypes,
-                            Optional<uint32_t> &PCHSignature);
+                            Optional<PCHMergerInfo> &PCHInfo);
 
 Error mergeTypeAndIdRecords(GlobalTypeTableBuilder &DestIds,
                             GlobalTypeTableBuilder &DestTypes,
                             SmallVectorImpl<TypeIndex> &SourceToDest,
                             const CVTypeArray &IdsAndTypes,
                             ArrayRef<GloballyHashedType> Hashes,
-                            Optional<uint32_t> &PCHSignature);
+                            Optional<PCHMergerInfo> &PCHInfo);
 
 Error mergeTypeRecords(GlobalTypeTableBuilder &Dest,
                        SmallVectorImpl<TypeIndex> &SourceToDest,
                        const CVTypeArray &Types,
                        ArrayRef<GloballyHashedType> Hashes,
-                       Optional<uint32_t> &PCHSignature);
+                       Optional<PCHMergerInfo> &PCHInfo);
 
 Error mergeIdRecords(GlobalTypeTableBuilder &Dest, ArrayRef<TypeIndex> Types,
                      SmallVectorImpl<TypeIndex> &SourceToDest,
                      const CVTypeArray &Ids,
                      ArrayRef<GloballyHashedType> Hashes);
 
 } // end namespace codeview
 } // end namespace llvm
 
 #endif // LLVM_DEBUGINFO_CODEVIEW_TYPESTREAMMERGER_H

llvm/lib/DebugInfo/CodeView/TypeStreamMerger.cpp

explicit TypeStreamMerger(SmallVectorImpl<TypeIndex> &SourceToDest)
     CurIndex += SourceToDest.size();
   }
 
   static const TypeIndex Untranslated;
 
   // Local hashing entry points
   Error mergeTypesAndIds(MergingTypeTableBuilder &DestIds,
                          MergingTypeTableBuilder &DestTypes,
-                         const CVTypeArray &IdsAndTypes, Optional<uint32_t> &S);
+                         const CVTypeArray &IdsAndTypes,
+                         Optional<PCHMergerInfo> &PCHInfo);
   Error mergeIdRecords(MergingTypeTableBuilder &Dest,
                        ArrayRef<TypeIndex> TypeSourceToDest,
                        const CVTypeArray &Ids);
   Error mergeTypeRecords(MergingTypeTableBuilder &Dest,
                          const CVTypeArray &Types);
 
   // Global hashing entry points
   Error mergeTypesAndIds(GlobalTypeTableBuilder &DestIds,
                          GlobalTypeTableBuilder &DestTypes,
                          const CVTypeArray &IdsAndTypes,
                          ArrayRef<GloballyHashedType> Hashes,
-                         Optional<uint32_t> &S);
+                         Optional<PCHMergerInfo> &PCHInfo);
   Error mergeIdRecords(GlobalTypeTableBuilder &Dest,
                        ArrayRef<TypeIndex> TypeSourceToDest,
                        const CVTypeArray &Ids,
                        ArrayRef<GloballyHashedType> Hashes);
   Error mergeTypeRecords(GlobalTypeTableBuilder &Dest, const CVTypeArray &Types,
                          ArrayRef<GloballyHashedType> Hashes,
-                         Optional<uint32_t> &S);
+                         Optional<PCHMergerInfo> &PCHInfo);
 
 private:
   Error doit(const CVTypeArray &Types);
 
   Error remapAllTypes(const CVTypeArray &Types);
 
   Error remapType(const CVType &Type);
 
private:
   /// Map from source type index to destination type index. Indexed by source
   /// type index minus 0x1000.
   SmallVectorImpl<TypeIndex> &IndexMap;
 
   /// Temporary storage that we use to copy a record's data while re-writing
   /// its type indices.
   SmallVector<uint8_t, 256> RemapStorage;
 
-  Optional<uint32_t> PCHSignature;
+  Optional<PCHMergerInfo> PCHInfo;
 };
 
 } // end anonymous namespace
 
 const TypeIndex TypeStreamMerger::Untranslated(SimpleTypeKind::NotTranslated);
 
 void TypeStreamMerger::addMapping(TypeIndex Idx) {
   if (!IsSecondPass) {
Error TypeStreamMerger::mergeIdRecords(MergingTypeTableBuilder &Dest,
   UseGlobalHashes = false;
 
   return doit(Ids);
 }
 
 Error TypeStreamMerger::mergeTypesAndIds(MergingTypeTableBuilder &DestIds,
                                          MergingTypeTableBuilder &DestTypes,
                                          const CVTypeArray &IdsAndTypes,
-                                         Optional<uint32_t> &S) {
+                                         Optional<PCHMergerInfo> &PCHInfo) {
   DestIdStream = &DestIds;
   DestTypeStream = &DestTypes;
   UseGlobalHashes = false;
   auto Err = doit(IdsAndTypes);
-  S = PCHSignature;
+  PCHInfo = this->PCHInfo;
   return Err;
 }
 
 // Global hashing entry points
 Error TypeStreamMerger::mergeTypeRecords(GlobalTypeTableBuilder &Dest,
                                          const CVTypeArray &Types,
                                          ArrayRef<GloballyHashedType> Hashes,
-                                         Optional<uint32_t> &S) {
+                                         Optional<PCHMergerInfo> &PCHInfo) {
   DestGlobalTypeStream = &Dest;
   UseGlobalHashes = true;
   GlobalHashes = Hashes;
   auto Err = doit(Types);
-  S = PCHSignature;
+  PCHInfo = this->PCHInfo;
   return Err;
 }
 
 Error TypeStreamMerger::mergeIdRecords(GlobalTypeTableBuilder &Dest,
                                        ArrayRef<TypeIndex> TypeSourceToDest,
                                        const CVTypeArray &Ids,
                                        ArrayRef<GloballyHashedType> Hashes) {
   DestGlobalIdStream = &Dest;
   TypeLookup = TypeSourceToDest;
   UseGlobalHashes = true;
   GlobalHashes = Hashes;
 
   return doit(Ids);
 }
 
 Error TypeStreamMerger::mergeTypesAndIds(GlobalTypeTableBuilder &DestIds,
                                          GlobalTypeTableBuilder &DestTypes,
                                          const CVTypeArray &IdsAndTypes,
                                          ArrayRef<GloballyHashedType> Hashes,
-                                         Optional<uint32_t> &S) {
+                                         Optional<PCHMergerInfo> &PCHInfo) {
   DestGlobalIdStream = &DestIds;
   DestGlobalTypeStream = &DestTypes;
   UseGlobalHashes = true;
   GlobalHashes = Hashes;
   auto Err = doit(IdsAndTypes);
-  S = PCHSignature;
+  PCHInfo = this->PCHInfo;
   return Err;
 }
 
 Error TypeStreamMerger::doit(const CVTypeArray &Types) {
   if (auto EC = remapAllTypes(Types))
     return EC;
 
   // If we found bad indices but no other errors, try doing another pass and see
Error llvm::codeview::mergeIdRecords(MergingTypeTableBuilder &Dest,
                                      const CVTypeArray &Ids) {
   TypeStreamMerger M(SourceToDest);
   return M.mergeIdRecords(Dest, TypeSourceToDest, Ids);
 }
 
 Error llvm::codeview::mergeTypeAndIdRecords(
     MergingTypeTableBuilder &DestIds, MergingTypeTableBuilder &DestTypes,
     SmallVectorImpl<TypeIndex> &SourceToDest, const CVTypeArray &IdsAndTypes,
-    Optional<uint32_t> &PCHSignature) {
+    Optional<PCHMergerInfo> &PCHInfo) {
   TypeStreamMerger M(SourceToDest);
-  return M.mergeTypesAndIds(DestIds, DestTypes, IdsAndTypes, PCHSignature);
+  return M.mergeTypesAndIds(DestIds, DestTypes, IdsAndTypes, PCHInfo);
 }
 
 Error llvm::codeview::mergeTypeAndIdRecords(
     GlobalTypeTableBuilder &DestIds, GlobalTypeTableBuilder &DestTypes,
     SmallVectorImpl<TypeIndex> &SourceToDest, const CVTypeArray &IdsAndTypes,
-    ArrayRef<GloballyHashedType> Hashes, Optional<uint32_t> &PCHSignature) {
+    ArrayRef<GloballyHashedType> Hashes, Optional<PCHMergerInfo> &PCHInfo) {
   TypeStreamMerger M(SourceToDest);
-  return M.mergeTypesAndIds(DestIds, DestTypes, IdsAndTypes, Hashes,
-                            PCHSignature);
+  return M.mergeTypesAndIds(DestIds, DestTypes, IdsAndTypes, Hashes, PCHInfo);
 }
 
 Error llvm::codeview::mergeTypeRecords(GlobalTypeTableBuilder &Dest,
                                        SmallVectorImpl<TypeIndex> &SourceToDest,
                                        const CVTypeArray &Types,
                                        ArrayRef<GloballyHashedType> Hashes,
-                                       Optional<uint32_t> &PCHSignature) {
+                                       Optional<PCHMergerInfo> &PCHInfo) {
   TypeStreamMerger M(SourceToDest);
-  return M.mergeTypeRecords(Dest, Types, Hashes, PCHSignature);
+  return M.mergeTypeRecords(Dest, Types, Hashes, PCHInfo);
 }
 
 Error llvm::codeview::mergeIdRecords(GlobalTypeTableBuilder &Dest,
                                      ArrayRef<TypeIndex> Types,
                                      SmallVectorImpl<TypeIndex> &SourceToDest,
                                      const CVTypeArray &Ids,
                                      ArrayRef<GloballyHashedType> Hashes) {
   TypeStreamMerger M(SourceToDest);
   return M.mergeIdRecords(Dest, Types, Ids, Hashes);
 }
 
 Expected<bool> TypeStreamMerger::shouldRemapType(const CVType &Type) {
   // For object files containing precompiled types, we need to extract the
   // signature, through EndPrecompRecord. This is done here for performance
   // reasons, to avoid re-parsing the Types stream.
   if (Type.kind() == LF_ENDPRECOMP) {
     EndPrecompRecord EP;
     if (auto EC = TypeDeserializer::deserializeAs(const_cast<CVType &>(Type),
                                                   EP))
       return joinErrors(std::move(EC), errorCorruptRecord());
-    if (PCHSignature)
+    // Only one record of this kind can appear in a OBJ.
+    if (PCHInfo)
       return errorCorruptRecord();
-    PCHSignature.emplace(EP.getSignature());
+    PCHInfo.emplace(PCHMergerInfo{EP.getSignature(), CurIndex.toArrayIndex()});
     return false;
   }
   return true;
 }

llvm/tools/llvm-readobj/COFFDumper.cpp

if (SectionName == ".debug$T") {
       BinaryStreamReader Reader(Data, llvm::support::little);
       if (auto EC = Reader.readArray(Types, Reader.getLength())) {
         consumeError(std::move(EC));
         W.flush();
         reportError(errorCodeToError(object_error::parse_failed),
                     Obj->getFileName());
       }
       SmallVector<TypeIndex, 128> SourceToDest;
-      Optional<uint32_t> PCHSignature;
+      Optional<PCHMergerInfo> PCHInfo;
       if (GHash) {
         std::vector<GloballyHashedType> Hashes =
             GloballyHashedType::hashTypes(Types);
         if (Error E =
                 mergeTypeAndIdRecords(GlobalCVIDs, GlobalCVTypes, SourceToDest,
-                                      Types, Hashes, PCHSignature))
+                                      Types, Hashes, PCHInfo))
           return reportError(std::move(E), Obj->getFileName());
       } else {
         if (Error E = mergeTypeAndIdRecords(CVIDs, CVTypes, SourceToDest, Types,
-                                            PCHSignature))
+                                            PCHInfo))
           return reportError(std::move(E), Obj->getFileName());
       }
     }
   }
 }
 
 void COFFDumper::printCodeViewTypeSection(StringRef SectionName,
                                           const SectionRef &Section) {