This is an archive of the discontinued LLVM Phabricator instance.

[LLD][COFF] Early dependency detection
ClosedPublic

Authored by aganea on Mar 6 2019, 2:28 PM.

Details

Summary

This patch adds more fine grained information in ObjFile about PDB type server dependencies and precomp OBJ dependencies.
Dependencies are required to be loaded & merged first, before any dependent OBJ that relies on it.

Before, dependencies were discovered and loaded along the way, when the type stream for a given OBJ was being merged (in PDB.cpp).

This patch is part of ground work towards parallelizing the type merging. The objective is to split PDBLinker::addObjFile into several smaller parallelizable pieces. See D59226 for the overall intention.

Diff Detail

Repository
rLLD LLVM Linker

Event Timeline

aganea created this revision.Mar 6 2019, 2:28 PM
Herald added a project: Restricted Project. · View Herald TranscriptMar 6 2019, 2:28 PM
Herald added a subscriber: jdoerfert. · View Herald Transcript
rnk added inline comments.Mar 6 2019, 2:49 PM
COFF/InputFiles.h
168–182

I'm starting to feel like mergeDebugT should really be behind a virtual interface, something like this:

class TPISource { // TpiSource?
public:
  virtual CVIndexMap *mergeTPI(ObjFile *File, CVIndexMap *ObjectIndexMap) = 0;
};
...
class SectionTPISource { ... }; // ObjectTPISource? RegularTPISource? Z7TPISource? TpiSource?
class TypeServerTPISource { ... };
class PrecompTPISource { ... };
class UsePrecompTPISource { ... };

The class names I came up with are kind of crummy, though.

I feel like 4 public, optional members of ObjFile leaks a lot of PDB details into the rest of the linker. Instead, we could have a pointer to some interface. Do you think this, or something like it, is worth doing now, ever, or later?

The "regular" or .debug$T based type merger could just be some stateless global variable that all the regular objs point to, since we pass the object file in through the interface, and that's all the info it needs. Every file using a type server would point to the same TypeServerTPISource. I'm not sure how the PCH stuff would work, since I'm less familiar.

aganea marked an inline comment as done.Mar 7 2019, 5:00 AM
aganea added inline comments.
COFF/InputFiles.h
168–182

Yes you're completly right - somehow I started doing that at first, but there were too many changes, so I meant to do this incrementally. I'll lay out this new class hierarchy in this patch, and then I'll implement mergeTPI() in a subsequent patch.

aganea updated this revision to Diff 190154.Mar 11 2019, 2:34 PM

Introduce TpiSource and derived types as suggested by @rnk.
Please see D59226 for more information as for where this is going.

aganea marked an inline comment as done.Mar 11 2019, 2:34 PM
aganea edited the summary of this revision. (Show Details)Mar 11 2019, 2:39 PM
ruiu added inline comments.Mar 11 2019, 3:01 PM
COFF/DebugTypes.cpp
47

What is this? You are not using this. (If you need it in another patch, move this to the another patch.)

66

nit: Please add a blank line between function definitions.

COFF/InputFiles.cpp
688

If you do know the first argument for all function calls of makeTpiSource, you can define one function for each value of the first argument, instead of calling the same function and then dispatch according to the first argument.

693

Avoid auto.

COFF/PDB.cpp
406

Please do not use auto unless its type is obvious from the right-hand side (e.g. cast or new).

aganea updated this revision to Diff 190205.Mar 11 2019, 8:32 PM
aganea marked 7 inline comments as done.

Updated with changes as suggested by @ruiu

COFF/DebugTypes.cpp
47

Normally we should have std::unique_ptr<TpiSource> DebugTypesObj in COFF/InputFiles.h, L166, but I wanted to avoid #including DebugTypes.h just so we can use std::unique_ptr. I'm keeping hard references to TpiSource objects here instead.

COFF/InputFiles.cpp
688

I did like you said - however now there are details leaking out of DebugTypes.h, like TypeServer2Record parameter to makeUseTypeServerSource(), which now requires having #include "llvm/DebugInfo/CodeView/TypeRecord.h" directly in DebugTypes.h, which can slow down compilation - if DebugTypes.h is included elsewhere. It's a tradeoff between terseness and compilation speed, please let me know which you prefer.

693

The right side already specifies that it deserializes as a TypeServer2Record, but yes I made that explicit.

aganea updated this revision to Diff 190245.Mar 12 2019, 6:24 AM
aganea marked an inline comment as done.

Remove reference to llvm/DebugInfo/CodeView/TypeRecord.h from DebugTypes.h

COFF/InputFiles.cpp
688

Forget what I said, it was late last night. Changed the parameters to pointers and forward declarations.

ruiu added inline comments.Mar 12 2019, 1:48 PM
COFF/InputFiles.cpp
662

Could you write a function comment as to what this function is supposed to do so that it can be easily understood for a first-time reader?

688

I was about to say that you probably could make use of forward declaration, but you already did. :)

aganea updated this revision to Diff 190508.Mar 13 2019, 2:24 PM
aganea marked an inline comment as done.

Document ObjFile::initializeDependencies().

Ping! Any further comments?

rnk accepted this revision.Mar 22 2019, 1:52 PM

Not from me. :)

This revision is now accepted and ready to land.Mar 22 2019, 1:52 PM
This revision was automatically updated to reflect the committed changes.