Add some basic linker module symbols

[zturner]

Summary

To get this more correct we need to plumb a few additional values through, such as the argv0 and the command line. I need to get that done before I commit this, but throwing this up for early comments.

[ruiu]

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

Is this value the same on x86-64?

[zturner]

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

That's a good question, I should check.

[zturner]

Update the test to properly check for x64, and finish up the remaining missing pieces. In particular, we need the value of argv[0] in order to portably get the executable name, and we need the values of argv[1] - argv[argc-1] in order to write them to the S_ENVBLOCK symbol. To address this, I deleted the global Argv0 variable that LLD currently used, and instead stored the entire argv array in the global configuration object.

I also added a test. Unfortunately checking for paths is pretty obnoxious in tests, requiring some magic sed incantation, so I've just regex'ed them all out in this test. Perhaps someone else can do better here.

[smeenai]

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

Probably dumb question: what about ARM (and AArch64 in the future)?

[ruiu]

LGTM

lld/COFF/Config.h

95 ↗

(On Diff #105903)

This can be ArrayRef.

[zturner]

lld/COFF/Config.h

95 ↗

(On Diff #105903)

In order to make it ArrayRef I have to make it ArrayRef<const char *>. This is what I had in an earlier version of the patch, but I had problems on the call to llvm::join() (because you can't use operator+ on two const char*. So if I want to be able to use StringRef, it has to be std::vector

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

Whoever's interested in adding PDB support for ARM and AArch64` can update this code :) The CPUType enumeration has a bunch of ARM values in the enumeration:

ARM3 = 0x60,
ARM4 = 0x61,
ARM4T = 0x62,
ARM5 = 0x63,
ARM5T = 0x64,
ARM6 = 0x65,
ARM_XMAC = 0x66,
ARM_WMMX = 0x67,
ARM7 = 0x68,
ARMNT = 0xf4,

the differences between which are not obvious to me :) But the support is there (in theory) for whoever wants this.

[zturner]

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

Actually ARM64 is probably ARM8, which does not seem to be represented in this enumeration. So there will be some head scratching involved to figure out what to do there

[ruiu]

lld/COFF/Config.h

95 ↗

(On Diff #105903)

Makes sense.

lld/COFF/PDB.cpp

411 ↗

(On Diff #105706)

Agreed. We can change this later when someone wants PDB support for ARM/ARM64.

[ruiu]

lld/COFF/Error.h

21 ↗

(On Diff #105903)

Sorry for the last-minute comment, but can you keep this variable and just change it stype from llvm::StringRef to std::vector<llvm::StringRef> to make it consistent with ELF? I don't want to make them being diverged too much.

[zturner]

lld/COFF/Error.h

21 ↗

(On Diff #105903)

Is it ok to have a global std::vector? This might invoke an undesirable global constructor.

[ruiu]

lld/COFF/Error.h

21 ↗

(On Diff #105903)

Well, it might not be desirable, but we already have a few global std::vectors.

[zturner]

Should I still leave the variable in Error.h? It seems a little illogical to have the variable in Error.h given that it's now being used outside of Error related code. Config.h seems like the logical place for it. Maybe the best thing to do is add the vector in Configuration of ELF linker too?

[ruiu]

Updating the ELF side should work, too.

[zturner]

Updated the ELF side.

[ruiu]

LGTM

lld/COFF/Driver.cpp

59 ↗

(On Diff #105917)

I honestly didn't know that std::vector has assign member function. Does

Config->Argv = {Args.begin(), Args.end()};

work too? If so, I prefer this way.

lld/ELF/Config.h

107 ↗

(On Diff #105917)

Move this before AuxiliaryList.

[zturner]

lld/COFF/Driver.cpp

59 ↗

(On Diff #105917)

Yes, that works. In general assign is more efficient because it uses already reserved storage. In this case it doesn't matter since we're actually just initializing it. But say for example you have a vector with 1000 items in it, in that case assign() would be better since it wouldn't free the internal buffer. In any case, I've made the change you suggested and will submit like that.

lld/trunk/COFF/Config.h

struct Configuration {
   bool DoICF = true;
   uint64_t ErrorLimit = 20;
   bool Relocatable = true;
   bool Force = false;
   bool Debug = false;
   bool WriteSymtab = true;
   unsigned DebugTypes = static_cast<unsigned>(DebugType::None);
   llvm::SmallString<128> PDBPath;
+  std::vector<llvm::StringRef> Argv;
 
   // Symbols in this set are considered as live by the garbage collector.
   std::set<SymbolBody *> GCRoot;
 
   std::set<StringRef> NoDefaultLibs;
   bool NoDefaultLibAll = false;
 
   // True if we are creating a DLL.

lld/trunk/COFF/Driver.cpp

 
 BumpPtrAllocator BAlloc;
 StringSaver Saver{BAlloc};
 std::vector<SpecificAllocBase *> SpecificAllocBase::Instances;
 
 bool link(ArrayRef<const char *> Args, raw_ostream &Diag) {
   ErrorCount = 0;
   ErrorOS = &Diag;
-  Argv0 = Args[0];
   Config = make<Configuration>();
+  Config->Argv = {Args.begin(), Args.end()};
   Config->ColorDiagnostics =
       (ErrorOS == &llvm::errs() && Process::StandardErrHasColors());
   Driver = make<LinkerDriver>();
   Driver->link(Args);
   return !ErrorCount;
 }
 
 // Drop directory components and replace extension with ".exe" or ".dll".

lld/trunk/COFF/Error.h

 #include "lld/Core/LLVM.h"
 #include "llvm/Support/Error.h"
 
 namespace lld {
 namespace coff {
 
 extern uint64_t ErrorCount;
 extern llvm::raw_ostream *ErrorOS;
-extern llvm::StringRef Argv0;
 
 void log(const Twine &Msg);
 void message(const Twine &Msg);
 void warn(const Twine &Msg);
 void error(const Twine &Msg);
 LLVM_ATTRIBUTE_NORETURN void fatal(const Twine &Msg);
 LLVM_ATTRIBUTE_NORETURN void fatal(std::error_code EC, const Twine &Prefix);
 LLVM_ATTRIBUTE_NORETURN void fatal(llvm::Error &Err, const Twine &Prefix);

lld/trunk/COFF/Error.cpp

 using namespace llvm;
 
 namespace lld {
 // The functions defined in this file can be called from multiple threads,
 // but outs() or errs() are not thread-safe. We protect them using a mutex.
 static std::mutex Mu;
 
 namespace coff {
-StringRef Argv0;
 uint64_t ErrorCount;
 raw_ostream *ErrorOS;
 
 static LLVM_ATTRIBUTE_NORETURN void exitLld(int Val) {
   // Dealloc/destroy ManagedStatic variables before calling
   // _exit(). In a non-LTO build, this is a nop. In an LTO
   // build allows us to get the output of -time-passes.
   llvm_shutdown();
 
   outs().flush();
   errs().flush();
   _exit(Val);
 }
 
 static void print(StringRef S, raw_ostream::Colors C) {
-  *ErrorOS << Argv0 + ": ";
+  *ErrorOS << Config->Argv[0] << ": ";
   if (Config->ColorDiagnostics) {
     ErrorOS->changeColor(C, true);
     *ErrorOS << S;
     ErrorOS->resetColor();
   } else {
     *ErrorOS << S;
   }
 }
 
 void log(const Twine &Msg) {
   if (Config->Verbose) {
     std::lock_guard<std::mutex> Lock(Mu);
-    outs() << Argv0 << ": " << Msg << "\n";
+    outs() << Config->Argv[0] << ": " << Msg << "\n";
     outs().flush();
   }
 }
 
 void message(const Twine &Msg) {
   std::lock_guard<std::mutex> Lock(Mu);
   outs() << Msg << "\n";
   outs().flush();

lld/trunk/COFF/PDB.h

 union DebugInfo;
 }
 }
 
 namespace lld {
 namespace coff {
 class SymbolTable;
 
-void createPDB(llvm::StringRef Path, SymbolTable *Symtab,
-               llvm::ArrayRef<uint8_t> SectionTable,
+void createPDB(SymbolTable *Symtab, llvm::ArrayRef<uint8_t> SectionTable,
                const llvm::codeview::DebugInfo *DI);
 }
 }
 
 #endif

lld/trunk/COFF/PDB.cpp

 #include "Error.h"
 #include "SymbolTable.h"
 #include "Symbols.h"
 #include "llvm/DebugInfo/CodeView/CVDebugRecord.h"
 #include "llvm/DebugInfo/CodeView/CVTypeVisitor.h"
 #include "llvm/DebugInfo/CodeView/DebugSubsectionRecord.h"
 #include "llvm/DebugInfo/CodeView/DebugSubsectionVisitor.h"
 #include "llvm/DebugInfo/CodeView/LazyRandomTypeCollection.h"
+#include "llvm/DebugInfo/CodeView/SymbolSerializer.h"
 #include "llvm/DebugInfo/CodeView/TypeDumpVisitor.h"
 #include "llvm/DebugInfo/CodeView/TypeIndexDiscovery.h"
 #include "llvm/DebugInfo/CodeView/TypeStreamMerger.h"
 #include "llvm/DebugInfo/CodeView/TypeTableBuilder.h"
 #include "llvm/DebugInfo/MSF/MSFBuilder.h"
 #include "llvm/DebugInfo/MSF/MSFCommon.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/DbiStream.h"
 #include "llvm/DebugInfo/PDB/Native/DbiStreamBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStream.h"
 #include "llvm/DebugInfo/PDB/Native/InfoStreamBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFile.h"
 #include "llvm/DebugInfo/PDB/Native/PDBFileBuilder.h"
-#include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptorBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/PDBStringTableBuilder.h"
 #include "llvm/DebugInfo/PDB/Native/PDBTypeServerHandler.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStream.h"
 #include "llvm/DebugInfo/PDB/Native/TpiStreamBuilder.h"
 #include "llvm/Object/COFF.h"
 #include "llvm/Support/BinaryByteStream.h"
 #include "llvm/Support/Endian.h"
 #include "llvm/Support/FileOutputBuffer.h"
static void addObjectsToPDB(BumpPtrAllocator &Alloc, SymbolTable *Symtab,
 
   // Construct TPI stream contents.
   addTypeInfo(Builder.getTpiBuilder(), TypeTable);
 
   // Construct IPI stream contents.
   addTypeInfo(Builder.getIpiBuilder(), IDTable);
 }
 
+static void addLinkerModuleSymbols(StringRef Path,
+                                   pdb::DbiModuleDescriptorBuilder &Mod,
+                                   BumpPtrAllocator &Allocator) {
+  codeview::SymbolSerializer Serializer(Allocator, CodeViewContainer::Pdb);
+  codeview::ObjNameSym ONS(SymbolRecordKind::ObjNameSym);
+  codeview::Compile3Sym CS(SymbolRecordKind::Compile3Sym);
+  codeview::EnvBlockSym EBS(SymbolRecordKind::EnvBlockSym);
+
+  ONS.Name = "* Linker *";
+  ONS.Signature = 0;
+
+  CS.Machine = Config->is64() ? CPUType::X64 : CPUType::Intel80386;
+  CS.Flags = CompileSym3Flags::None;
+  CS.VersionBackendBuild = 0;
+  CS.VersionBackendMajor = 0;
+  CS.VersionBackendMinor = 0;
+  CS.VersionBackendQFE = 0;
+  CS.VersionFrontendBuild = 0;
+  CS.VersionFrontendMajor = 0;
+  CS.VersionFrontendMinor = 0;
+  CS.VersionFrontendQFE = 0;
+  CS.Version = "LLVM Linker";
+  CS.setLanguage(SourceLanguage::Link);
+
+  ArrayRef<StringRef> Args = makeArrayRef(Config->Argv).drop_front();
+  std::string ArgStr = llvm::join(Args, " ");
+  EBS.Fields.push_back("cwd");
+  SmallString<64> cwd;
+  llvm::sys::fs::current_path(cwd);
+  EBS.Fields.push_back(cwd);
+  EBS.Fields.push_back("exe");
+  std::string Exe =
+      llvm::sys::fs::getMainExecutable(Config->Argv[0].data(), nullptr);
+  EBS.Fields.push_back(Exe);
+  EBS.Fields.push_back("pdb");
+  EBS.Fields.push_back(Path);
+  EBS.Fields.push_back("cmd");
+  EBS.Fields.push_back(ArgStr);
+  Mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
+      ONS, Allocator, CodeViewContainer::Pdb));
+  Mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
+      CS, Allocator, CodeViewContainer::Pdb));
+  Mod.addSymbol(codeview::SymbolSerializer::writeOneSymbol(
+      EBS, Allocator, CodeViewContainer::Pdb));
+}
+
 // Creates a PDB file.
-void coff::createPDB(StringRef Path, SymbolTable *Symtab,
-                     ArrayRef<uint8_t> SectionTable,
+void coff::createPDB(SymbolTable *Symtab, ArrayRef<uint8_t> SectionTable,
                      const llvm::codeview::DebugInfo *DI) {
   BumpPtrAllocator Alloc;
   pdb::PDBFileBuilder Builder(Alloc);
   ExitOnErr(Builder.initialize(4096)); // 4096 is blocksize
 
   // Create streams in MSF for predefined streams, namely
   // PDB, TPI, DBI and IPI.
   for (int I = 0; I < (int)pdb::kSpecialStreamCount; ++I)
     ExitOnErr(Builder.getMsfBuilder().addStream(0));
 
   // Add an Info stream.
   auto &InfoBuilder = Builder.getInfoBuilder();
   InfoBuilder.setAge(DI ? DI->PDB70.Age : 0);
 
-  llvm::SmallString<128> NativePath(Path.begin(), Path.end());
+  llvm::SmallString<128> NativePath(Config->PDBPath.begin(),
+                                    Config->PDBPath.end());
   llvm::sys::fs::make_absolute(NativePath);
   llvm::sys::path::native(NativePath, llvm::sys::path::Style::windows);
 
   pdb::PDB_UniqueId uuid{};
   if (DI)
     memcpy(&uuid, &DI->PDB70.Signature, sizeof(uuid));
   InfoBuilder.setGuid(uuid);
   InfoBuilder.setSignature(time(nullptr));
ArrayRef<object::coff_section> Sections = {
       (const object::coff_section *)SectionTable.data(),
       SectionTable.size() / sizeof(object::coff_section)};
   std::vector<pdb::SecMapEntry> SectionMap =
       pdb::DbiStreamBuilder::createSectionMap(Sections);
   DbiBuilder.setSectionMap(SectionMap);
 
   auto &LinkerModule = ExitOnErr(DbiBuilder.addModuleInfo("* Linker *"));
   LinkerModule.setPdbFilePathNI(PdbFilePathNI);
+  addLinkerModuleSymbols(NativePath, LinkerModule, Alloc);
 
   // Add COFF section header stream.
   ExitOnErr(
       DbiBuilder.addDbgStream(pdb::DbgHeaderType::SectionHdr, SectionTable));
 
   // Write to a file.
-  ExitOnErr(Builder.commit(Path));
+  ExitOnErr(Builder.commit(Config->PDBPath));
 }

lld/trunk/COFF/Writer.cpp

void Writer::run() {
   writeSections();
   sortExceptionTable();
   writeBuildId();
 
   if (!Config->PDBPath.empty() && Config->Debug) {
     const llvm::codeview::DebugInfo *DI = nullptr;
     if (Config->DebugTypes & static_cast<unsigned>(coff::DebugType::CV))
       DI = BuildId->DI;
-    createPDB(Config->PDBPath, Symtab, SectionTable, DI);
+    createPDB(Symtab, SectionTable, DI);
   }
 
   writeMapFile(OutputSections);
 
   if (auto EC = Buffer->commit())
     fatal(EC, "failed to write the output file");
 }
 

lld/trunk/ELF/Config.h

struct Configuration {
   llvm::StringRef MapFile;
   llvm::StringRef OutputFile;
   llvm::StringRef OptRemarksFilename;
   llvm::StringRef SoName;
   llvm::StringRef Sysroot;
   llvm::StringRef ThinLTOCacheDir;
   std::string Rpath;
   std::vector<VersionDefinition> VersionDefinitions;
+  std::vector<llvm::StringRef> Argv;
   std::vector<llvm::StringRef> AuxiliaryList;
   std::vector<llvm::StringRef> SearchPaths;
   std::vector<llvm::StringRef> SymbolOrderingFile;
   std::vector<llvm::StringRef> Undefined;
   std::vector<SymbolVersion> VersionScriptGlobals;
   std::vector<SymbolVersion> VersionScriptLocals;
   std::vector<uint8_t> BuildIdVector;
   llvm::MapVector<Symbol *, RenamedSymbol> RenamedSymbols;

lld/trunk/ELF/Driver.cpp

 std::vector<SpecificAllocBase *> elf::SpecificAllocBase::Instances;
 
 static void setConfigs();
 
 bool elf::link(ArrayRef<const char *> Args, bool CanExitEarly,
                raw_ostream &Error) {
   ErrorCount = 0;
   ErrorOS = &Error;
-  Argv0 = Args[0];
   InputSections.clear();
   Tar = nullptr;
 
   Config = make<Configuration>();
   Driver = make<LinkerDriver>();
   Script = make<LinkerScript>();
+  Config->Argv = {Args.begin(), Args.end()};
 
   Driver->main(Args, CanExitEarly);
   freeArena();
   return !ErrorCount;
 }
 
 // Parses a linker -m option.
 static std::tuple<ELFKind, uint16_t, uint8_t> parseEmulation(StringRef Emul) {

lld/trunk/ELF/Error.h

 
 #include "llvm/Support/Error.h"
 
 namespace lld {
 namespace elf {
 
 extern uint64_t ErrorCount;
 extern llvm::raw_ostream *ErrorOS;
-extern llvm::StringRef Argv0;
 
 void log(const Twine &Msg);
 void message(const Twine &Msg);
 void warn(const Twine &Msg);
 void error(const Twine &Msg);
 LLVM_ATTRIBUTE_NORETURN void fatal(const Twine &Msg);
 
 LLVM_ATTRIBUTE_NORETURN void exitLld(int Val);

lld/trunk/ELF/Error.cpp

 
 using namespace llvm;
 
 using namespace lld;
 using namespace lld::elf;
 
 uint64_t elf::ErrorCount;
 raw_ostream *elf::ErrorOS;
-StringRef elf::Argv0;
 
 // The functions defined in this file can be called from multiple threads,
 // but outs() or errs() are not thread-safe. We protect them using a mutex.
 static std::mutex Mu;
 
 // Prints "\n" or does nothing, depending on Msg contents of
 // the previous call of this function.
 static void newline(const Twine &Msg) {
   // True if the previous error message contained "\n".
   // We want to separate multi-line error messages with a newline.
   static bool Flag;
 
   if (Flag)
     *ErrorOS << "\n";
   Flag = (StringRef(Msg.str()).find('\n') != StringRef::npos);
 }
 
 static void print(StringRef S, raw_ostream::Colors C) {
-  *ErrorOS << Argv0 + ": ";
+  *ErrorOS << Config->Argv[0] << ": ";
   if (Config->ColorDiagnostics) {
     ErrorOS->changeColor(C, true);
     *ErrorOS << S;
     ErrorOS->resetColor();
   } else {
     *ErrorOS << S;
   }
 }
 
 void elf::log(const Twine &Msg) {
   if (Config->Verbose) {
     std::lock_guard<std::mutex> Lock(Mu);
-    outs() << Argv0 << ": " << Msg << "\n";
+    outs() << Config->Argv[0] << ": " << Msg << "\n";
     outs().flush();
   }
 }
 
 void elf::message(const Twine &Msg) {
   std::lock_guard<std::mutex> Lock(Mu);
   outs() << Msg << "\n";
   outs().flush();

lld/trunk/test/COFF/pdb-linker-module.test

+RUN: lld-link /debug /pdb:%t.pdb /nodefaultlib /entry:main %S/Inputs/pdb-diff.obj
+RUN: llvm-pdbutil dump -modules -symbols %t.pdb | FileCheck %s
+
+CHECK:      Mod 0001 | `* Linker *`:
+CHECK-NEXT:  4 | S_OBJNAME [size = 20] sig=0, `* Linker *`
+CHECK-NEXT: 24 | S_COMPILE3 [size = 40]
+CHECK-NEXT:      machine = intel 80386, Ver = LLVM Linker, language = link
+CHECK-NEXT:      frontend = 0.0.0.0, backend = 0.0.0.0
+CHECK-NEXT:      flags = none
+CHECK-NEXT: 64 | S_ENVBLOCK
+CHECK-NEXT: - cwd
+CHECK-NEXT: -
+CHECK-NEXT: - exe
+CHECK-NEXT: - {{.*}}lld-link
+CHECK-NEXT: - pdb
+CHECK-NEXT: - {{.*}}pdb-linker-module{{.*}}pdb
+CHECK-NEXT: - cmd
+CHECK-NEXT: - /debug /pdb:{{.*}}pdb-linker-module{{.*}}pdb /nodefaultlib /entry:main {{.*}}pdb-diff.obj

llvm/trunk/include/llvm/DebugInfo/CodeView/SymbolRecord.h

public:
   uint16_t VersionFrontendBuild;
   uint16_t VersionFrontendQFE;
   uint16_t VersionBackendMajor;
   uint16_t VersionBackendMinor;
   uint16_t VersionBackendBuild;
   uint16_t VersionBackendQFE;
   StringRef Version;
 
+  void setLanguage(SourceLanguage Lang) {
+    Flags = CompileSym3Flags((uint32_t(Flags) & 0xFFFFFF00) | uint32_t(Lang));
+  }
+
   uint8_t getLanguage() const { return static_cast<uint32_t>(Flags) & 0xFF; }
   uint32_t getFlags() const { return static_cast<uint32_t>(Flags) & ~0xFF; }
 
   uint32_t RecordOffset;
 };
 
 // S_FRAMEPROC
 class FrameProcSym : public SymbolRecord {