This is an archive of the discontinued LLVM Phabricator instance.

[BOLT][DWARF] Change to process and write out TUs first then CUs in batches
ClosedPublic

Authored by ayermolo on Jun 1 2023, 10:34 AM.

Details

Summary

To reduce memory footprint changed so that we process and write out TUs first,
reset DIEBuilder and process CUs. CUs are processed in buckets. First bucket
contains all the CUs with cross CU references. Rest processd one at a time.

clang-17 build in debug mode, by clang-17.
before
8:25.81 real, 834.37 user, 86.03 sys, 0 amem, 79525064 mmem
8:02.20 real, 820.46 user, 81.81 sys, 0 amem, 79501616 mmem
7:52.69 real, 802.01 user, 83.99 sys, 0 amem, 79534392 mmem

after
7:49.35 real, 822.04 user, 66.19 sys, 0 amem, 34934260 mmem
7:42.16 real, 825.46 user, 63.52 sys, 0 amem, 34951660 mmem
7:46.71 real, 821.11 user, 63.14 sys, 0 amem, 34981164 mmem

Depends on D151908

Diff Detail

Event Timeline

ayermolo created this revision.Jun 1 2023, 10:34 AM
Herald added a reviewer: Amir. · View Herald Transcript
Herald added a reviewer: maksfb. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
ayermolo requested review of this revision.Jun 1 2023, 10:34 AM
Herald added a project: Restricted Project. · View Herald TranscriptJun 1 2023, 10:34 AM
ayermolo updated this revision to Diff 527987.Jun 2 2023, 2:35 PM

rebase + fixed cross-cu references when CUs don't share abbrev + tests.

Did you look into test failures?

Please update the summary and include runtime memory data.

bolt/include/bolt/Core/DIEBuilder.h
63

Maybe?

bolt/lib/Core/DIEBuilder.cpp
306
314

nit: braces

852

Use braces.

bolt/lib/Rewrite/DWARFRewriter.cpp
248

Not needed.

452

Add documentation.

471

Add braces.

654
1346

Secs?

1468
ayermolo marked 10 inline comments as done.Jun 23 2023, 1:09 PM
ayermolo updated this revision to Diff 534058.Jun 23 2023, 1:10 PM

addressed comments

ayermolo retitled this revision from [BOLT][DWARF] Change to process and write out TUs first, then CUs in batches. to [BOLT][DWARF] Change to process and write out TUs first then CUs in batches.Jun 30 2023, 2:18 PM
ayermolo edited the summary of this revision. (Show Details)Jul 5 2023, 4:39 PM
maksfb accepted this revision.Jul 6 2023, 12:00 PM

LGTM

This revision is now accepted and ready to land.Jul 6 2023, 12:00 PM
ayermolo closed this revision.Jul 6 2023, 5:00 PM
ayermolo reopened this revision.Jul 7 2023, 4:57 PM
This revision is now accepted and ready to land.Jul 7 2023, 4:57 PM
ayermolo updated this revision to Diff 538305.Jul 7 2023, 5:27 PM

fixed mac build + rebase

ayermolo updated this revision to Diff 538744.Jul 10 2023, 11:11 AM

Reduced the scope of variables

This revision was landed with ongoing or failed builds.Jul 10 2023, 2:42 PM
This revision was automatically updated to reflect the committed changes.