This is an archive of the discontinued LLVM Phabricator instance.

[LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.
ClosedPublic

Authored by hoyFB on May 21 2020, 2:17 PM.

Details

Summary

This change introduces an LLD switch --thinlto-single-module to allow compiling only a part of the input modules. This is specifically enables:

  1. Fast investigating/debugging modules of interest without spending time on compiling unrelated modules.
  2. Compiler debug dump with -mllvm -debug-only= for specific modules.

It will be useful for large applications which has 1K+ input modules for thinLTO.

The switch can be combined with --lto-obj-path= or --lto-emit-asm to obtain intermediate object files or assembly files. So far the module name matching is implemented as a fuzzy name lookup where the modules with name containing the switch value are compiled.

E.g,
Command:

ld.lld main.o thin.a --thinlto-single-module=thin.a --lto-obj-path=single.o

log:

[ThinLTO] Selecting thin.a(thin1.o at 168) to compile
[ThinLTO] Selecting thin.a(thin2.o at 228) to compile

Command:

ld.lld main.o thin.a --thinlto-single-module=thin1.o --lto-obj-path=single.o

log:

[ThinLTO] Selecting thin.a(thin1.o at 168) to compile

Diff Detail

Event Timeline

hoyFB created this revision.May 21 2020, 2:17 PM
hoyFB edited the summary of this revision. (Show Details)May 21 2020, 2:19 PM
hoyFB added reviewers: tejohnson, wenlei.
hoyFB updated this revision to Diff 265603.May 21 2020, 2:25 PM
hoyFB added a comment.May 21 2020, 2:25 PM

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

MaskRay added inline comments.May 25 2020, 12:34 PM
lld/test/ELF/lto/thinlto-single-module.ll
9

llvm-readelf -S

See linkerscript/sizeof.s for an example

Check prefixes do not necessarily start with CHECK-

llvm/include/llvm/LTO/Config.h
134

If this can be extended to regular LTO, ThinLTO* is not appropriate.

std::vector<std::string> ModulesToCompile;

Needs a test specifying the option multiple times.

llvm/lib/LTO/LTO.cpp
858

The substring matching is not tested

860

This is an informational message. warning: is not suitable

hoyFB updated this revision to Diff 266238.May 26 2020, 8:58 AM
hoyFB marked 4 inline comments as done.

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

hoyFB marked an inline comment as done.May 26 2020, 8:59 AM
hoyFB added inline comments.
llvm/include/llvm/LTO/Config.h
134

That's a good point. So far it's unclear to me what compiling a single module means in mono LTO mode. E.g, should cross-module inlining be allowed which would end up generating object code for other modules? If so, figuring out what inlinees should be compiled need additional work. I'd like to see a strong need first before addressing that, since thinLTO is currently our primary scenario. On the other hand, single module is naturally well-defined in thinLTO mode, and with this new switch we'll end up having exactly same code generated for a module as we would get with a full compilation.

Allowing multiple modules to be compiled with this switch is an interesting idea!

llvm/lib/LTO/LTO.cpp
858

It's tested by the --thinlto-single-module=thin.a line in the added test. The modules with name thin.a(thin1.o at ...) and thin.a(thin2.o at ...) are matched.

hoyFB updated this revision to Diff 266566.May 27 2020, 9:03 AM
hoyFB marked an inline comment as done.

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

hoyFB added a comment.May 27 2020, 9:04 AM

Fixed clang-tidy warnings.

Thanks for adding this, it looks really useful.

Can you beef up the testing? I.e. ensure that the save temps output files for just the specified module get created. Also please add a test to make sure this interacts well with distributed thinlto build options, i.e. -thinlto-index-only (e.g. that it creates the expected output files just for the specified module).

lld/test/ELF/lto/thinlto-single-module.ll
2

Some comments in this file as to what and how it is testing would be good. I'm don't quite understand how it is testing that the single module option is working.

llvm/include/llvm/LTO/Config.h
134

@MaskRay I'm also not sure how this would work in regular LTO mode. In ThinLTO mode the compilation of the selected module will not be affected by this option - it will still import and optimize the exact same way it normally would for that module. For regularLTO how do you isolate the compilation of one module without changing the way it is optimized? This seems very ThinLTO specific to me.

llvm/include/llvm/LTO/LTO.h
341

Please add comment (I realize this structure is missing comments, but think we should start adding them for new fields at least).

hoyFB updated this revision to Diff 266604.May 27 2020, 10:58 AM
hoyFB marked 2 inline comments as done.

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

Address feedbacks.

tejohnson added inline comments.May 27 2020, 11:53 AM
llvm/include/llvm/LTO/LTO.h
343

This is the lld option, I wouldn't mention it here as the option name will be different for e.g. gold plugin if this is ported there. Just say if specified by the LTO Config.

hoyFB updated this revision to Diff 266618.May 27 2020, 12:08 PM
hoyFB marked 2 inline comments as done.

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

llvm/include/llvm/LTO/LTO.h
343

Done.

MaskRay added inline comments.May 27 2020, 10:08 PM
lld/test/ELF/lto/thinlto-single-module.ll
9

Use ;; for comments.

There are efforts making FileCheck detect potentially disabled check directives. It may have some false positives. ;; is a good way (and already used a lot (## ) in other lld tests) to suppress them.

llvm/lib/LTO/LTO.cpp
857

const std::string &

hoyFB updated this revision to Diff 266919.May 28 2020, 9:34 AM
hoyFB marked 2 inline comments as done.

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

hoyFB added a comment.May 28 2020, 9:35 AM

Addressed feedbacks.

Can you beef up the testing? I.e. ensure that the save temps output files for just the specified module get created. Also please add a test to make sure this interacts well with distributed thinlto build options, i.e. -thinlto-index-only (e.g. that it creates the expected output files just for the specified module).

Not sure if you saw this comment from my earlier reply - please add additional testing as suggested above.

Can you beef up the testing? I.e. ensure that the save temps output files for just the specified module get created. Also please add a test to make sure this interacts well with distributed thinlto build options, i.e. -thinlto-index-only (e.g. that it creates the expected output files just for the specified module).

Not sure if you saw this comment from my earlier reply - please add additional testing as suggested above.

It's added. E.g, the not ls single1.o2 line checks only two intermediate objects are generated, one is the dummy ld-temp.o and the other has the symbol specified in the command line. The IDX checks are for thinlto index testing. Do you want me to add more? Thanks.

Can you beef up the testing? I.e. ensure that the save temps output files for just the specified module get created. Also please add a test to make sure this interacts well with distributed thinlto build options, i.e. -thinlto-index-only (e.g. that it creates the expected output files just for the specified module).

Not sure if you saw this comment from my earlier reply - please add additional testing as suggested above.

It's added. E.g, the not ls single1.o2 line checks only two intermediate objects are generated, one is the dummy ld-temp.o and the other has the symbol specified in the command line. The IDX checks are for thinlto index testing. Do you want me to add more? Thanks.

Oh I see now that a -thinlto-index-only invocation was added. What you should be testing for is the output file specific to the specified object. E.g. typically there is a *.thinlto.bc file emitted for each input object. We should get only the one corresponding to the specified module in this case.

Also the other test suggestion I had was to make sure that the save-temps output files work as expected (still get the ones corresponding to the specified module, and not for the other modules). There are a number of .bc files emitted for each object with save-temps but it would be fine to check for one of them I think.

hoyFB added a comment.May 28 2020, 5:17 PM

Can you beef up the testing? I.e. ensure that the save temps output files for just the specified module get created. Also please add a test to make sure this interacts well with distributed thinlto build options, i.e. -thinlto-index-only (e.g. that it creates the expected output files just for the specified module).

Not sure if you saw this comment from my earlier reply - please add additional testing as suggested above.

It's added. E.g, the not ls single1.o2 line checks only two intermediate objects are generated, one is the dummy ld-temp.o and the other has the symbol specified in the command line. The IDX checks are for thinlto index testing. Do you want me to add more? Thanks.

Oh I see now that a -thinlto-index-only invocation was added. What you should be testing for is the output file specific to the specified object. E.g. typically there is a *.thinlto.bc file emitted for each input object. We should get only the one corresponding to the specified module in this case.

Also the other test suggestion I had was to make sure that the save-temps output files work as expected (still get the ones corresponding to the specified module, and not for the other modules). There are a number of .bc files emitted for each object with save-temps but it would be fine to check for one of them I think.

Gotcha, thanks for elaborating. Tests added correspondingly.

hoyFB updated this revision to Diff 267093.May 28 2020, 5:17 PM

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

hoyFB updated this revision to Diff 267680.Jun 1 2020, 11:34 AM

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

MaskRay added inline comments.Jun 1 2020, 9:38 PM
lld/ELF/LTO.cpp
142

const std::string & - Don't use auto unless it is obvious.

143

push_back. emplace_back makes people wonder whether you use a smart constructor

hoyFB marked 2 inline comments as done.Jun 3 2020, 9:21 AM

Addressed feedbacks.

lld/ELF/LTO.cpp
143

emplace_back is needed since name is not a std::string but a StringRef.

hoyFB updated this revision to Diff 268232.Jun 3 2020, 9:22 AM

Updating D80406: [LLD][ThinLTO] A switch to allow compilation of only one module.

Mostly looks good, and I think this feature will be useful. Some test suggestions.

The subject should be more specific. It can be like: Add --thinlto-single-module to allow compiling partial modules.

lld/test/ELF/lto/thinlto-single-module.ll
13

Prefer --check-prefix to -check-prefix

15

Add a not ls a.out to check the output is suppressed

23

No need to use ; on empty lines.

25

thin1.o/single3.o is a redundant case. It can be deleted.

42

Move IDX lines here.

; FileCheck %s --check-prefix=IDX < single5.idx
; count 1 < single5.idx

; IDX: main.o
44

typo: temporary

61

Some reviewers prefer CHECK lines closer to the associated RUN lines. (I am one of them:))

i.e.

RUN: ...

DEFAULT:
MAIN:

RUN: ...

FOO: ...
BLAH: ...
64

superfluous space on this line

72

superfluous space on this line

MaskRay added inline comments.Jun 9 2020, 2:41 PM
lld/test/ELF/lto/thinlto-single-module.ll
30

This line has trailing whitespace.

42

not ls thin.*.thinlto.bc

This does not work when nullglob is enabled for your shell. Consider RUN: ls | FileCheck --implicit-check='...' /dev/null

hoyFB retitled this revision from [LLD][ThinLTO] A switch to allow compilation of only one module. to [LLD][ThinLTO] Add a switch--thinlto-single-module to allow compiling partial modules..Jun 10 2020, 9:16 AM
hoyFB edited the summary of this revision. (Show Details)
hoyFB retitled this revision from [LLD][ThinLTO] Add a switch--thinlto-single-module to allow compiling partial modules. to [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules..
hoyFB marked 9 inline comments as done.
hoyFB updated this revision to Diff 269877.Jun 10 2020, 9:19 AM

Updating D80406: [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.

hoyFB added a comment.Jun 10 2020, 9:20 AM

Addressed feedbacks.

MaskRay added inline comments.Jun 10 2020, 9:26 AM
lld/test/ELF/lto/thinlto-single-module.ll
51

Omitted RUN:

hoyFB updated this revision to Diff 269888.Jun 10 2020, 10:03 AM

Updating D80406: [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.

hoyFB marked 3 inline comments as done.Jun 10 2020, 10:03 AM
MaskRay accepted this revision.Jun 10 2020, 10:40 AM

Hope @tejohnson can confirm this is good.

lld/test/ELF/lto/thinlto-single-module.ll
49

This line does not check anything.

You need FileCheck --implicit-check-not='thin.{{.*}}.thinlto.bc' /dev/null

This revision is now accepted and ready to land.Jun 10 2020, 10:40 AM
hoyFB updated this revision to Diff 269914.Jun 10 2020, 11:08 AM

Updating D80406: [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.

hoyFB marked an inline comment as done.Jun 10 2020, 11:10 AM

Hope @tejohnson can confirm this is good.

Thanks for signing off. @tejohnson Please let me know if the latest version looks good to you. Thanks.

This revision was automatically updated to reflect the committed changes.

Different but related, we also have other internal patches that help debugging/investigation. One example is a developer friendly IR printing mode (controls how IR printing works for -print-before/after*) that tweaks format and content of IR dump so it's more human readable. We found it helpful and we'd be happy send patch if folks think that can useful too.

Different but related, we also have other internal patches that help debugging/investigation. One example is a developer friendly IR printing mode (controls how IR printing works for -print-before/after*) that tweaks format and content of IR dump so it's more human readable. We found it helpful and we'd be happy send patch if folks think that can useful too.

This is probably generic and not specific to LTO. Having facility to help debugging will be useful.