[LLD] Fix precompiled headers build break on Linux

[aganea]

Summary

Currently, precompiled headers OBJs are only produced on Windows with cl.exe so any embedded paths are Windows-specific. Obviously manipulating these paths on Linux won't work.

Additionally, Microsoft link.exe requires a precompiled headers OBJ to be provided on the command-line (even if precomp.obj is there at the proper location, as specified internally by precomp-a.obj).
This fails:

link.exe precomp-a.obj /OUT:a.exe

This works:

link.exe precomp-a.obj precomp.obj /OUT:a.exe

I've consequently only kept the filename-based search. I'm not sure searching with the full embedded path makes sense.

---

The build break is here:

FAIL: lld :: COFF/precomp-link.test (171 of 1917)
******************** TEST 'lld :: COFF/precomp-link.test' FAILED ********************
Script:
--
: 'RUN: at line 1';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/lld-link /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-a.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-b.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.pdb /out:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.exe /opt:ref /opt:icf
: 'RUN: at line 2';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/llvm-pdbutil dump -types /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.pdb | /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/precomp-link.test
: 'RUN: at line 4';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/lld-link /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-a.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.pdb /out:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.exe /opt:ref /opt:icf
: 'RUN: at line 5';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/llvm-pdbutil dump -types /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.pdb | /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/precomp-link.test
: 'RUN: at line 7';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/lld-link /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-a.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp-invalid.obj /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.pdb /out:/b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/tools/lld/test/COFF/Output/precomp-link.test.tmp.exe /opt:ref /opt:icf 2>&1 | /b/sanitizer-x86_64-linux-fast/build/llvm_build_ubsan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm/tools/lld/test/COFF/precomp-link.test -check-prefix FAILURE
--
Exit Code: 1

Command Output (stderr):
--
lld-link: warning: Cannot use debug info for 'precomp-a.obj'
>>> failed to load reference 'f:\svn\lld\test\coff\precomp\precomp.obj': No such file or directory

lld-link: error: codeview::mergeTypeAndIdRecords failed: The CodeView record is corrupted.

[aganea]

COFF/SymbolTable.cpp

246 ↗

(On Diff #172653)

@neerajksingh: Out of curiosity, could you please explain how _PchSym_ symbols are mangled? If I don't add this check, I get:

lld-link: error: <root>: undefined symbol: __@@_PchSym_@00@UhemUoowUgvhgUxluuUkivxlnkUkivxlnkOlyq@4B2008FD98C1DD4

I find this confusing for users, I've consequently removed pch symbols, in order to emit a better message a bit later on.

[aganea]

Updated with better error message:

F:\svn\build> lld-link precomp-a.obj /OUT:a.exe
lld-link: warning: Cannot use debug info for 'precomp-a.obj'
>>> failed to load reference 'precomp.obj': The path to this file must be provided on the command-line.

[rnk]

lgtm

lld/trunk/COFF/PDB.cpp

677–678

Do we really need to make each object file path absolute before calling sys::path::filename? I would've expected we could just do StringRef CurrentFileName = sys::path::filename(F->getName());.

[aganea]

Thank you Reid! I've simplified the code as you suggested.

lld/trunk/COFF/PDB.cpp

PDBLinker::mergeInPrecompHeaderObj(ObjFile *File, const CVType &FirstType,
                                    CVIndexMap *ObjectIndexMap) {
   PrecompRecord Precomp;
   if (auto EC = TypeDeserializer::deserializeAs(const_cast<CVType &>(FirstType),
                                                 Precomp))
     fatal("error reading record: " + toString(std::move(EC)));
 
   auto E = aquirePrecompObj(File, Precomp);
   if (!E)
-    return createFileError(Precomp.getPrecompFilePath().str(), E.takeError());
+    return E.takeError();
 
   const CVIndexMap &PrecompIndexMap = *E;
   assert(PrecompIndexMap.IsPrecompiledTypeMap);
 
   if (PrecompIndexMap.TPIMap.empty())
     return PrecompIndexMap;
 
   assert(Precomp.getStartTypeIndex() == TypeIndex::FirstNonSimpleIndex);
   assert(Precomp.getTypesCount() <= PrecompIndexMap.TPIMap.size());
   // Use the previously remapped index map from the precompiled headers.
   ObjectIndexMap->TPIMap.append(PrecompIndexMap.TPIMap.begin(),
                                 PrecompIndexMap.TPIMap.begin() +
                                     Precomp.getTypesCount());
   return *ObjectIndexMap;
 }
 
 static bool equals_path(StringRef path1, StringRef path2) {
 #if defined(_WIN32)
   return path1.equals_lower(path2);
 #else
   return path1.equals(path2);
 #endif
 }
 
-// Find an OBJ provided on the command line, either by name or full path
-static Optional<std::pair<ObjFile *, std::string>>
-findObjByName(StringRef NameOrPath) {
+// Find by name an OBJ provided on the command line
+static ObjFile *findObjByName(StringRef FileNameOnly) {
   SmallString<128> CurrentPath;
 
-  StringRef FileNameOnly = sys::path::filename(NameOrPath);
-
   for (ObjFile *F : ObjFile::Instances) {
-    CurrentPath = F->getName();
-    sys::fs::make_absolute(CurrentPath);
-
-    // First compare with the full path name
-    if (equals_path(CurrentPath, NameOrPath))
-      return std::make_pair(F, CurrentPath.str().str());
+    StringRef CurrentFileName = sys::path::filename(F->getName());
 
-    StringRef CurrentFileName = sys::path::filename(CurrentPath);
-
-    // Otherwise compare based solely on the file name (link.exe behavior)
+    // Compare based solely on the file name (link.exe behavior)
     if (equals_path(CurrentFileName, FileNameOnly))
-      return std::make_pair(F, CurrentPath.str().str());
+      return F;
   }
-  return {};
+  return nullptr;
 }
 
 std::pair<CVIndexMap &, bool /*already there*/>
 PDBLinker::registerPrecompiledHeaders(uint32_t Signature) {
   auto Insertion = PrecompTypeIndexMappings.insert({Signature, CVIndexMap()});
   CVIndexMap &IndexMap = Insertion.first->second;
   if (!Insertion.second)
     return {IndexMap, true};
   // Mark this map as a precompiled types map.
   IndexMap.IsPrecompiledTypeMap = true;
   return {IndexMap, false};
 }
 
 Expected<const CVIndexMap &>
 PDBLinker::aquirePrecompObj(ObjFile *File, PrecompRecord Precomp) {
   // First, check if we already loaded the precompiled headers object with this
   // signature. Return the type index mapping if we've already seen it.
   auto R = registerPrecompiledHeaders(Precomp.getSignature());
   if (R.second)
     return R.first;
 
   CVIndexMap &IndexMap = R.first;
 
-  SmallString<128> PrecompPath = Precomp.getPrecompFilePath();
-  sys::fs::make_absolute(PrecompPath);
-
   // Cross-compile warning: given that Clang doesn't generate LF_PRECOMP
   // records, we assume the OBJ comes from a Windows build of cl.exe. Thusly,
   // the paths embedded in the OBJs are in the Windows format.
-  sys::path::native(PrecompPath, sys::path::Style::windows);
+  SmallString<128> PrecompFileName = sys::path::filename(
+      Precomp.getPrecompFilePath(), sys::path::Style::windows);
 
   // link.exe requires that a precompiled headers object must always be provided
   // on the command-line, even if that's not necessary.
-  auto PrecompFilePath = findObjByName(PrecompPath);
-  if (!PrecompFilePath)
-    return errorCodeToError(std::error_code(ENOENT, std::generic_category()));
-
-  ObjFile *CurrentFile = PrecompFilePath->first;
+  auto PrecompFile = findObjByName(PrecompFileName);
+  if (!PrecompFile)
+    return createFileError(
+        PrecompFileName.str(),
+        make_error<pdb::PDBError>(pdb::pdb_error_code::external_cmdline_ref));
 
-  addObjFile(CurrentFile, &IndexMap);
+  addObjFile(PrecompFile, &IndexMap);
 
-  if (!CurrentFile->EndPrecomp)
-    fatal(PrecompFilePath->second + " is not a precompiled headers object");
+  if (!PrecompFile->EndPrecomp)
+    fatal(PrecompFile->getName() + " is not a precompiled headers object");
 
-  if (Precomp.getSignature() != CurrentFile->EndPrecomp->getSignature())
-    return make_error<pdb::PDBError>(pdb::pdb_error_code::signature_out_of_date);
+  if (Precomp.getSignature() != PrecompFile->EndPrecomp->getSignature())
+    return createFileError(
+        Precomp.getPrecompFilePath().str(),
+        make_error<pdb::PDBError>(pdb::pdb_error_code::signature_out_of_date));
 
   return IndexMap;
 }
 
 static bool remapTypeIndex(TypeIndex &TI, ArrayRef<TypeIndex> TypeIndexMap) {
   if (TI.isSimple())
     return true;
   if (TI.toArrayIndex() >= TypeIndexMap.size())

lld/trunk/COFF/SymbolTable.cpp

if (Name.startswith("__imp_")) {
         auto *D = cast<Defined>(Imp);
         replaceSymbol<DefinedLocalImport>(Sym, Name, D);
         LocalImportChunks.push_back(cast<DefinedLocalImport>(Sym)->getChunk());
         LocalImports[Sym] = D;
         continue;
       }
     }
 
+    // We don't want to report missing Microsoft precompiled headers symbols.
+    // A proper message will be emitted instead in PDBLinker::aquirePrecompObj
+    if (Name.contains("_PchSym_"))
+      continue;
+
     if (Config->MinGW && handleMinGWAutomaticImport(Sym, Name))
       continue;
 
     // Remaining undefined symbols are not fatal if /force is specified.
     // They are replaced with dummy defined symbols.
     if (Config->ForceUnresolved)
       replaceSymbol<DefinedAbsolute>(Sym, Name, 0);
     Undefs.insert(Sym);

lld/trunk/test/COFF/precomp-link.test

-REQUIRES: system-windows
-
 RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf
 RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
 
 RUN: lld-link %S/Inputs/precomp.obj %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf
 RUN: llvm-pdbutil dump -types %t.pdb | FileCheck %s
 
 RUN: lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-invalid.obj %S/Inputs/precomp.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1 | FileCheck %s -check-prefix FAILURE
 
+RUN: not lld-link %S/Inputs/precomp-a.obj %S/Inputs/precomp-b.obj /nodefaultlib /entry:main /debug /pdb:%t.pdb /out:%t.exe /opt:ref /opt:icf 2>&1 | FileCheck %s -check-prefix FAILURE-MISSING-PRECOMPOBJ
+
 FAILURE: warning: Cannot use debug info for 'precomp-invalid.obj'
 FAILURE-NEXT: failed to load reference '{{.*}}precomp.obj': The signature does not match; the file(s) might be out of date.
 
+FAILURE-MISSING-PRECOMPOBJ: warning: Cannot use debug info for 'precomp-a.obj'
+FAILURE-MISSING-PRECOMPOBJ-NEXT: failed to load reference 'precomp.obj': The path to this file must be provided on the command-line
+
 CHECK: Types (TPI Stream)
 CHECK-NOT: LF_PRECOMP
 CHECK-NOT: LF_ENDPRECOMP
 
 // precomp.h
 #pragma once
 int Function(char A);
 

llvm/trunk/include/llvm/DebugInfo/PDB/GenericError.h

 namespace llvm {
 namespace pdb {
 
 enum class pdb_error_code {
   invalid_utf8_path = 1,
   dia_sdk_not_present,
   dia_failed_loading,
   signature_out_of_date,
+  external_cmdline_ref,
   unspecified,
 };
 } // namespace pdb
 } // namespace llvm
 
 namespace std {
 template <>
 struct is_error_code_enum<llvm::pdb::pdb_error_code> : std::true_type {};

llvm/trunk/lib/DebugInfo/PDB/GenericError.cpp

case pdb_error_code::dia_sdk_not_present:
              "that you are not using MSVC, or your Visual Studio "
              "installation is corrupt.";
     case pdb_error_code::dia_failed_loading:
       return "DIA is only supported when using MSVC.";
     case pdb_error_code::invalid_utf8_path:
       return "The PDB file path is an invalid UTF8 sequence.";
     case pdb_error_code::signature_out_of_date:
       return "The signature does not match; the file(s) might be out of date.";
+    case pdb_error_code::external_cmdline_ref:
+      return "The path to this file must be provided on the command-line.";
     }
     llvm_unreachable("Unrecognized generic_error_code");
   }
 };
 
 static llvm::ManagedStatic<PDBErrorCategory> PDBCategory;
 const std::error_category &llvm::pdb::PDBErrCategory() { return *PDBCategory; }
 
 char PDBError::ID;