Page MenuHomePhabricator
Feed Advanced Search

Yesterday

tejohnson requested review of D90120: [sanitizer] Print errno for report file open failure.
Sun, Oct 25, 8:33 AM · Restricted Project

Sat, Oct 24

tejohnson committed rG13c62ce99aab: [MemProf] Temporarily disable part of test (authored by tejohnson).
[MemProf] Temporarily disable part of test
Sat, Oct 24, 11:09 PM

Fri, Oct 23

tejohnson committed rGeeba325b12c4: [MemProf] Attempt to debug avr bot failure (authored by tejohnson).
[MemProf] Attempt to debug avr bot failure
Fri, Oct 23, 4:00 PM
tejohnson committed rGb67a2aef8ac9: [MemProf] XFAIL test on avr until issue can be debugged (authored by tejohnson).
[MemProf] XFAIL test on avr until issue can be debugged
Fri, Oct 23, 11:32 AM

Thu, Oct 22

tejohnson committed rG5c20d7db9f27: [MemProf] Allow the binary to specify the profile output filename (authored by tejohnson).
[MemProf] Allow the binary to specify the profile output filename
Thu, Oct 22, 8:30 AM
tejohnson closed D89086: [MemProf] Allow the binary to specify the profile output filename.
Thu, Oct 22, 8:30 AM · Restricted Project
tejohnson added a comment to D88433: [IRMover] Fix up "CG Profile" MDNode after RAUW.

It seems a bit odd to have/need a custom fixup for one type of MDNode here. Can visitModuleFlagCGProfileEntry be fixed to handle the bitcast?

Thu, Oct 22, 8:28 AM · Restricted Project

Wed, Oct 21

tejohnson committed rG1bb68c9b189f: [sanitizer] Allow log_path to distinguish default from explicit stderr (authored by tejohnson).
[sanitizer] Allow log_path to distinguish default from explicit stderr
Wed, Oct 21, 7:33 PM
tejohnson closed D89629: [sanitizer] Allow log_path to distinguish default from explicit stderr.
Wed, Oct 21, 7:33 PM · Restricted Project
tejohnson committed rG31bc55d602a0: [sanitizer] Convert PrintModuleMap to DumpProcessMap (authored by tejohnson).
[sanitizer] Convert PrintModuleMap to DumpProcessMap
Wed, Oct 21, 12:47 PM
tejohnson closed D89630: [sanitizer] Convert PrintModuleMap to DumpProcessMap.
Wed, Oct 21, 12:47 PM · Restricted Project

Sat, Oct 17

tejohnson updated the diff for D89087: [MemProf] Pass down memory profile name with optional path from clang.

As discussed in review for D89086, move test changes to this patch since they are only required here.

Sat, Oct 17, 12:50 PM · Restricted Project, Restricted Project, Restricted Project
tejohnson updated the summary of D89086: [MemProf] Allow the binary to specify the profile output filename.
Sat, Oct 17, 12:34 PM · Restricted Project
tejohnson updated the diff for D89086: [MemProf] Allow the binary to specify the profile output filename.

Address comments.

Sat, Oct 17, 12:34 PM · Restricted Project
tejohnson added inline comments to D87120: [MemProf] Memory profiling runtime support.
Sat, Oct 17, 10:52 AM · Restricted Project
tejohnson requested review of D89630: [sanitizer] Convert PrintModuleMap to DumpProcessMap.
Sat, Oct 17, 10:52 AM · Restricted Project
tejohnson requested review of D89629: [sanitizer] Allow log_path to distinguish default from explicit stderr.
Sat, Oct 17, 10:46 AM · Restricted Project

Fri, Oct 16

tejohnson added inline comments to D89086: [MemProf] Allow the binary to specify the profile output filename.
Fri, Oct 16, 4:45 PM · Restricted Project
tejohnson added a comment to D89086: [MemProf] Allow the binary to specify the profile output filename.

Profile runtime has 4 ways of setting output: 1) default 2) compiler command line arg 3) environment variable at runtime; and 4) user invocation of runtime API at runtime. The order of the precedence is 4> 3 > 2 > 1. Is the behavior here consistent with that?

Fri, Oct 16, 4:23 PM · Restricted Project
tejohnson committed rG3ed77ecd0a5d: [MemProf] Don't build memprof if sanitizer not being built (authored by tejohnson).
[MemProf] Don't build memprof if sanitizer not being built
Fri, Oct 16, 10:52 AM
tejohnson committed rG3d4bba302d24: [MemProf] Memory profiling runtime support (authored by tejohnson).
[MemProf] Memory profiling runtime support
Fri, Oct 16, 9:47 AM
tejohnson closed D87120: [MemProf] Memory profiling runtime support.
Fri, Oct 16, 9:47 AM · Restricted Project

Thu, Oct 15

tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Use atomic_exchange as suggested

Thu, Oct 15, 9:41 PM · Restricted Project
tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

Thanks for the review!

Thu, Oct 15, 8:12 PM · Restricted Project
tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

New revision has not been uploaded yet?

Thu, Oct 15, 6:15 PM · Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Address comments and misc fixes.

Thu, Oct 15, 6:14 PM · Restricted Project
tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

Thanks, I've addressed your comments. I also have implemented a couple additional fixes I found through internal testing. Added implementations of some sanitizer_* functions that need to be implemented by sanitizer_common users (e.g. sanitizer_get_heap_size) and added a test to ensure they are all defined. I fixed a bug in the timestamp computation code. I also added a flag to ensure that we don't try to access the dynamically allocated CacheSet on the MemInfoBlockCache before it is constructed (e.g. before the Allocator object is initialized).

Thu, Oct 15, 6:00 PM · Restricted Project

Tue, Oct 13

tejohnson added a comment to D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

@tejohnson I added the test in LLD, since the gold tests only run on Linux, which is harder for me to test & debug. The test fails when the following block is removed: if (BackendProc->getThreadCount() == 1) { ... }.

Great, thanks! LGTM with request for comment as described below.

This test also implictly covers the "InProcessThinBackend" codepath with /opt:lldltojobs=1 which I don't see how to cover explicitly otherwise.

I don't see how this test covers that code path since it isn't using in process backends, but I'm not really sure how to test this case effectively - like the original patch, it is just a performance change on the in process backends case.

I meant if we have:
; RUN: lld-link /lldsavetemps /out:%t4.exe /entry:main /subsystem:console %t2.obj %t1.obj /opt:lldltojobs=1
We end up in if (BackendProc->getThreadCount() == 1) as if we did:
RUN: lld-link -thinlto-index-only:%t3 /entry:main %t1.obj %t2.obj

Tue, Oct 13, 7:05 PM · Restricted Project
tejohnson accepted D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

@tejohnson I added the test in LLD, since the gold tests only run on Linux, which is harder for me to test & debug. The test fails when the following block is removed: if (BackendProc->getThreadCount() == 1) { ... }.

Tue, Oct 13, 3:51 PM · Restricted Project
tejohnson added a comment to D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

It would be great to test that the ordering of the files in the object file list doesn't change with the write indexes backend. Unfortunately, currently llvm-lto2 doesn't have an option to trigger this output file, but you could use either the gold plugin or lld. There is an existing gold plugin test that does test this output file:
llvm/test/tools/gold/X86/thinlto_emit_linked_objects.ll

Tue, Oct 13, 9:03 AM · Restricted Project

Mon, Oct 12

tejohnson committed rGc27ab339ad8f: Restore "[ThinLTO] Avoid temporaries when loading global decl attachment… (authored by tejohnson).
Restore "[ThinLTO] Avoid temporaries when loading global decl attachment…
Mon, Oct 12, 10:12 AM
tejohnson closed D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Mon, Oct 12, 10:12 AM · Restricted Project

Sun, Oct 11

tejohnson added a comment to D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

ping, @evgeny777 can you take a quick look at the change. It's a pretty minor fix.

Sun, Oct 11, 7:42 PM · Restricted Project

Fri, Oct 9

tejohnson added a comment to D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

We had to revert due to unexpected side effects on distributed ThinLTO. See the comments on the revert commit about the type of failures. Essentially, for distributed ThinLTO, the backends is of type WriteIndexesThinBackend. It runs serially, not in parallel via threads, and writes all the files needed by the distributed backend processes. One of the files it writes is a list of files to include in the final native link, which is provided to the final link process. The order modules are written into this file is the order they will be linked. With this patch, the order is changed and no longer matches the original link order, and that can produce unexpected failures. Probably the best fix is to simply skip the reordering when BackendProc is a WriteIndexesThinBackend. It won't help anyway given that it is a serial backend.

Fri, Oct 9, 3:57 PM · Restricted Project

Thu, Oct 8

tejohnson requested review of D89087: [MemProf] Pass down memory profile name with optional path from clang.
Thu, Oct 8, 5:56 PM · Restricted Project, Restricted Project, Restricted Project
tejohnson requested review of D89086: [MemProf] Allow the binary to specify the profile output filename.
Thu, Oct 8, 5:52 PM · Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Address comments

Thu, Oct 8, 2:52 PM · Restricted Project
tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

Thanks for the detailed comments! I think I have addressed them all, except for a few as noted in the comments below.

Thu, Oct 8, 2:51 PM · Restricted Project
tejohnson added a comment to D88361: [sanitizer] Skip stack symbolization when not required for print format.

Hi! It seems that this change is leading to the undefined symbol errors we're seeing on our builders (https://luci-milo.appspot.com/p/fuchsia/builders/ci/clang-linux-x64/b8867057367989385504):

[716/753] Linking CXX shared library /b/s/w/ir/x/w/staging/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.asan.so
FAILED: /b/s/w/ir/x/w/staging/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.asan.so 
...
ld.lld: error: undefined symbol: __sanitizer::RenderNeedsSymbolization(char const*)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:29 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:29)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer::StackTrace::Print() const)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:118 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:118)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer_symbolize_pc)

ld.lld: error: undefined symbol: __sanitizer::RenderFrame(__sanitizer::InternalScopedString*, char const*, int, unsigned long, __sanitizer::AddressInfo const*, bool, char const*, char const*)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:43 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:43)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer::StackTrace::Print() const)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:135 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:135)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer_symbolize_pc)
>>> referenced by sanitizer_symbolizer_report.cpp:36 (compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_report.cpp:36)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_report.cpp.obj:(__sanitizer::ReportErrorSummary(char const*, __sanitizer::AddressInfo const&, char const*))

Could you take a look? Thanks.

I see the issue. Fuschia uses a different implementation of RenderFrame, I'll need to update that one too and add a dummy version of the new RenderNeedsSymbolization as well (looks like this version in sanitizer_symbolizer_markup.cpp is quite simple and doesn't attempt to symbolize at all, so RenderNeedsSymbolization can simply return false). Fix coming up.

Thu, Oct 8, 10:45 AM · Restricted Project
tejohnson committed rGf775cb8994cc: [sanitizer] Fix Fuchsia bot failure (authored by tejohnson).
[sanitizer] Fix Fuchsia bot failure
Thu, Oct 8, 10:45 AM
tejohnson added a comment to D88361: [sanitizer] Skip stack symbolization when not required for print format.

Hi! It seems that this change is leading to the undefined symbol errors we're seeing on our builders (https://luci-milo.appspot.com/p/fuchsia/builders/ci/clang-linux-x64/b8867057367989385504):

[716/753] Linking CXX shared library /b/s/w/ir/x/w/staging/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.asan.so
FAILED: /b/s/w/ir/x/w/staging/llvm_build/lib/clang/12.0.0/lib/x86_64-unknown-fuchsia/libclang_rt.asan.so 
...
ld.lld: error: undefined symbol: __sanitizer::RenderNeedsSymbolization(char const*)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:29 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:29)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer::StackTrace::Print() const)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:118 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:118)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer_symbolize_pc)

ld.lld: error: undefined symbol: __sanitizer::RenderFrame(__sanitizer::InternalScopedString*, char const*, int, unsigned long, __sanitizer::AddressInfo const*, bool, char const*, char const*)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:43 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:43)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer::StackTrace::Print() const)
>>> referenced by sanitizer_stacktrace_libcdep.cpp:135 (compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_libcdep.cpp:135)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stacktrace_libcdep.cpp.obj:(__sanitizer_symbolize_pc)
>>> referenced by sanitizer_symbolizer_report.cpp:36 (compiler-rt/lib/sanitizer_common/sanitizer_symbolizer_report.cpp:36)
>>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_symbolizer_report.cpp.obj:(__sanitizer::ReportErrorSummary(char const*, __sanitizer::AddressInfo const&, char const*))

Could you take a look? Thanks.

Thu, Oct 8, 10:17 AM · Restricted Project

Wed, Oct 7

tejohnson accepted D87001: [IRMover] Avoid materializing global value that belongs to not-yet-linked module.

LGTM with a minor comment nit and some clarification needed about stage 2 vs stage 3 in the description as noted below.

Wed, Oct 7, 4:48 PM · Restricted Project
tejohnson accepted D88349: Fix inconsistent flag for disabling Dead Virtual Function Elimination.

LGTM for test - thanks! Yeah, it looks like that cleanup patch inadvertently fixed this issue as a side effect.

Wed, Oct 7, 4:41 PM · Restricted Project
tejohnson committed rG4d5b1de40ecc: [sanitizer] Skip stack symbolization when not required for print format (authored by tejohnson).
[sanitizer] Skip stack symbolization when not required for print format
Wed, Oct 7, 3:39 PM
tejohnson closed D88361: [sanitizer] Skip stack symbolization when not required for print format.
Wed, Oct 7, 3:39 PM · Restricted Project
tejohnson added a comment to D87001: [IRMover] Avoid materializing global value that belongs to not-yet-linked module.

It sounds like even the test case that was added for D47898 passes when that fix is reverted?

Sorry, that's not what I meant. I meant to say that D47898 regressed the test case in this patch. (Ps, I've verified D47898 does fix its own test case).

Wed, Oct 7, 3:18 PM · Restricted Project
tejohnson updated the diff for D88361: [sanitizer] Skip stack symbolization when not required for print format.

Implement suggestion

Wed, Oct 7, 2:37 PM · Restricted Project
tejohnson accepted D88349: Fix inconsistent flag for disabling Dead Virtual Function Elimination.

LGTM with the test change suggested below.

Wed, Oct 7, 1:51 PM · Restricted Project
tejohnson added a comment to D87001: [IRMover] Avoid materializing global value that belongs to not-yet-linked module.

It sounds like even the test case that was added for D47898 passes when that fix is reverted? Can you also try the original test case from https://bugs.llvm.org/show_bug.cgi?id=37684, which is just a little bit different? Since this handling has been problematic, I'd like to understand what changed to make that older fix unnecessary.

Wed, Oct 7, 1:14 PM · Restricted Project
tejohnson added a comment to D88349: Fix inconsistent flag for disabling Dead Virtual Function Elimination.

Can you add a test? You can probably just beef up clang/test/CodeGenCXX/virtual-function-elimination.cpp to test this option as well

Wed, Oct 7, 11:03 AM · Restricted Project

Tue, Sep 29

tejohnson added a comment to D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

Ping - could someone take a quick look at the update and approve the recommit?

Tue, Sep 29, 10:41 AM · Restricted Project

Sat, Sep 26

tejohnson requested review of D88361: [sanitizer] Skip stack symbolization when not required for print format.
Sat, Sep 26, 8:30 AM · Restricted Project

Sep 25 2020

tejohnson requested review of D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Sep 25 2020, 10:09 AM · Restricted Project
tejohnson updated the diff for D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

Fix bug and add test case.

Sep 25 2020, 10:07 AM · Restricted Project
tejohnson reopened D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

I have a fix for the issue that required a revert. We shouldn't be using the lazy loading IndexCursor to load the global decl attachment metadata. That cursor is initialized with the necessary abbrev ids from the module level metadata block at the end of the lazy loading index setup, so that you can use those abbrev ids when jumping into random locations, as noted in the comment above its declaration:

Sep 25 2020, 10:07 AM · Restricted Project
tejohnson added a comment to D88309: [LTO][Legacy] Add API to set result type to assembly.

I'm not the right person to review this change, as we don't use the old LTO API at all. But I'll just add a quick note that there is a way to pass this through the new LTO API (lto::Config::CGFileType), and options to set it from lld/gold-plugin/llvm-lto2.

Sep 25 2020, 9:03 AM · Restricted Project

Sep 24 2020

tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

@vitalybuka Ping - let me know if you want me to split it up further before more review. I'm not sure the best way to split it up, since a lot of this is the basic infrastructure that works together. The main thing I can think of is to split it up onto 2 pieces: 1) with the infrastructure that just allows the heap profiler to intercept and handle the memory allocations/deallocations/intrinsics (which was largely cloned from the asan code); and 2) the part on top of that which does the actual heap tracking (the MemInfoBlock and MemInfoBlockCache stuff in memprof_allocator.cpp). If I split it I would keep most of the the chunk header part in the first patch, since it affects the memory allocations/alignment, but just not fill in the fields related to the heap tracking.

Sep 24 2020, 8:52 AM · Restricted Project

Sep 22 2020

tejohnson committed rGab1b4810b552: [ThinLTO] Avoid temporaries when loading global decl attachment metadata (authored by tejohnson).
[ThinLTO] Avoid temporaries when loading global decl attachment metadata
Sep 22 2020, 8:40 PM
tejohnson closed D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Sep 22 2020, 8:40 PM · Restricted Project
tejohnson updated the diff for D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

Address comments

Sep 22 2020, 8:31 PM · Restricted Project
tejohnson added a comment to D88041: [lld] Add a flag to enable split machine functions for LTO..

Is it expected that the user will want to specify this only at link time and not during compile time? If it is normally specified in the compile step, you should consider adding it to the IR in some fashion (e.g. function attributes). The advantage is that the user doesn't need to pass different options for the LTO and non-LTO cases.

Ideally we should only have to specify this once. However, using function attributes doesn't seem ideal since the pass will be scheduled and repeatedly invoked only to return without actually running the pass. It would be cleaner to marshal the codegen specific options from the compile invocation and restore them for the LTO step. There are a couple of other codegen options which would also benefit from this approach --lto-unique-basic-block-section-names, --lto-basic-block-sections=<value>. @mtrofin pointed out that -fembed-bitcode saves the invocation in .llvmcmd. A similar approach to stash codegen specific options always for LTO to pick up and enable might be less intrusive. However, this is a larger effort and for current LTO builds it would be nice to have a command line option to enable it. WDYT about this alternative?

Sep 22 2020, 6:20 PM · Restricted Project
tejohnson accepted D87949: [ThinLTO] Option to bypass function importing..

lgtm with comment typo fix

Sep 22 2020, 12:47 PM · Restricted Project, Restricted Project
tejohnson added a comment to D88041: [lld] Add a flag to enable split machine functions for LTO..

Is it expected that the user will want to specify this only at link time and not during compile time? If it is normally specified in the compile step, you should consider adding it to the IR in some fashion (e.g. function attributes). The advantage is that the user doesn't need to pass different options for the LTO and non-LTO cases.

Sep 22 2020, 8:26 AM · Restricted Project
tejohnson accepted D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

LGTM with comment suggestion.

Sep 22 2020, 8:03 AM · Restricted Project

Sep 21 2020

tejohnson added inline comments to D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Sep 21 2020, 7:44 AM · Restricted Project

Sep 19 2020

tejohnson updated the diff for D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

Add test options that exercise code, fix exposed bug.

Sep 19 2020, 4:11 PM · Restricted Project
tejohnson updated the summary of D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Sep 19 2020, 4:11 PM · Restricted Project
tejohnson added a comment to D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.

I realized after intentionally suppressing the new method that there was no regression test that broke. It turns out that the lazy loading index is only emitted if there is a certain number of metadatas, which the tests that exercise this code apparently don't have. Therefore, I added an option to drop the threshold for one of the ThinLTO WPD tests. This exposed a bug: Because parseGlobalObjectAttachment now resolves forward references via the index, the index cursor has moved when it returns, and we weren't parsing all of the global decl attachment metadatas. Fixed by saving and restoring the current position around that call.

Sep 19 2020, 4:11 PM · Restricted Project
tejohnson requested review of D87970: [ThinLTO] Avoid temporaries when loading global decl attachment metadata.
Sep 19 2020, 10:30 AM · Restricted Project
tejohnson added a comment to D87966: [ThinLTO] Re-order modules for optimal multi-threaded processing.

Thanks for doing this!

Sep 19 2020, 9:05 AM · Restricted Project

Sep 18 2020

tejohnson added a comment to D87949: [ThinLTO] Option to bypass function importing..

Minor comments.

Sep 18 2020, 4:35 PM · Restricted Project, Restricted Project
tejohnson accepted D87943: [clang] Remove profile available check for fsplit-machine-functions..

lgtm with a couple of minor suggestions

Sep 18 2020, 2:22 PM · Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Rebase

Sep 18 2020, 12:28 PM · Restricted Project
tejohnson added a comment to D87792: [sanitizer] Add facility to print the full StackDepot.

I lost connection to my Win box in the office. I will need Windows build
next week anyway. I will try to figure out why stderr is not captured there.
For now I relanded your patch and disabled the test on Windows.

Sep 18 2020, 8:06 AM · Restricted Project

Sep 17 2020

tejohnson added a comment to D87792: [sanitizer] Add facility to print the full StackDepot.

I had to revert due to 2 bot failures.

Sep 17 2020, 9:25 PM · Restricted Project
tejohnson added a reverting change for rG2ffaa9a1732c: [sanitizer] Add facility to print the full StackDepot: rG6e475e1288e3: Revert "[sanitizer] Add facility to print the full StackDepot".
Sep 17 2020, 9:20 PM
tejohnson committed rG6e475e1288e3: Revert "[sanitizer] Add facility to print the full StackDepot" (authored by tejohnson).
Revert "[sanitizer] Add facility to print the full StackDepot"
Sep 17 2020, 9:20 PM
tejohnson added a reverting change for D87792: [sanitizer] Add facility to print the full StackDepot: rG6e475e1288e3: Revert "[sanitizer] Add facility to print the full StackDepot".
Sep 17 2020, 9:20 PM · Restricted Project
tejohnson committed rG2ffaa9a1732c: [sanitizer] Add facility to print the full StackDepot (authored by tejohnson).
[sanitizer] Add facility to print the full StackDepot
Sep 17 2020, 8:25 PM
tejohnson closed D87792: [sanitizer] Add facility to print the full StackDepot.
Sep 17 2020, 8:25 PM · Restricted Project
tejohnson updated the diff for D87792: [sanitizer] Add facility to print the full StackDepot.

Implement test suggestion

Sep 17 2020, 5:18 PM · Restricted Project
tejohnson added inline comments to D87792: [sanitizer] Add facility to print the full StackDepot.
Sep 17 2020, 5:18 PM · Restricted Project
tejohnson updated the diff for D87792: [sanitizer] Add facility to print the full StackDepot.

Fix tmp file handling to work on all platforms

Sep 17 2020, 8:03 AM · Restricted Project
tejohnson added a comment to D87792: [sanitizer] Add facility to print the full StackDepot.

I realized I had to make a change, PTAL.

Sep 17 2020, 8:03 AM · Restricted Project

Sep 16 2020

tejohnson added a comment to D86847: [Bitcode] Add BITCODE_SIZE_BLOCK_ID to encode the size of the bitcode.

I am a bit worry that linker might concatenate bitcode file with padding to achieve alignment requirement, etc. I guess you can create a termination block to mark the end but it is hard to seek the next start.

We can set the section alignment to 1 to avoid padding: (I rushed a bit, sorry: 6ae7b403c3e1aebcb825d3dd4777d3c1149d6d67)

I didn't see that. I don't really have concerns then.

@bartell mentioned something about a bitcode wrapper that does provide the length - how does this patch's strategy compare to that approach? Are they redundant? Should the wrapper be replaced with this patch's approach? (or should the bitcode wrapper approach be used instead of adding this patch?)

I guess that is an implementation choice. This approach prefers a simple linker implementation, while the bitcode wrapper requires linker know to treat this section differently and write bitcode wrapper into section.
@MaskRay Do you have any specific use case in mind for this? If you are tied to a linker like lld, it might be better just teach lld to treat this section differently so we don't need worry about concatenated bitcode file.

It is not tied to a linker like lld. I believe in most binary formats, if you specify an alignment of 1, their linkers will concatenate input sections in the output without padding. Actually, the section content of .llvmbc is entirely opaque to lld/GNU ld/gold. ("dumb linker, smart format" design).

If the bitcode wrapper requires the linker to understand its format, I'd vote against that solution. (It will not work with GNU ld/gold.)

Sep 16 2020, 3:38 PM · Restricted Project
tejohnson added a comment to D85649: [AArch64] PAC/BTI code generation for LLVM generated functions.

The only question I have about module level attributes is what happens for LTO (which we use for Linux kernels used in Android)? Is there any conflict linking together objects from two different translation units with two different flags? Maybe this should fail hard; but it would be cool to alert developers at compile time, rather than have an awful bug to debug at runtime. Just food for thought; I don't know enough about LTO to know if it already has machinery to handle module level conflicts or not. Maybe @tejohnson or @pcc knows?

Sep 16 2020, 2:45 PM · Restricted Project, Restricted Project
tejohnson added a comment to D87120: [MemProf] Memory profiling runtime support.

I pulled out the sanitizer_common handling for printing the stack depot into another patch. I also removed the internal_getcpu handling as I couldn't reproduce why I needed it in the first place. Calling sched_getcpu seems to work fine without it.

Sep 16 2020, 2:33 PM · Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Remove sanitizer common handling from this patch

Sep 16 2020, 2:23 PM · Restricted Project
tejohnson requested review of D87792: [sanitizer] Add facility to print the full StackDepot.
Sep 16 2020, 1:49 PM · Restricted Project

Sep 15 2020

tejohnson accepted D87636: [ThinLTO] add post-thinlto-merge option to -lto-embed-bitcode.

LGTM, just needs a comment update as noted below and also an update to the patch title.

Sep 15 2020, 3:41 PM · Restricted Project, Restricted Project
tejohnson added inline comments to D87636: [ThinLTO] add post-thinlto-merge option to -lto-embed-bitcode.
Sep 15 2020, 3:06 PM · Restricted Project, Restricted Project
tejohnson retitled D87120: [MemProf] Memory profiling runtime support from [HeapProf] Heap profiling runtime support to [MemProf] Memory profiling runtime support.
Sep 15 2020, 11:12 AM · Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Remove a few extraneous files from commit. I will be splitting out the
sanitizer_common changes into another patch shortly, and will update the
patch with a note afterwards when this is ready to review again.

Sep 15 2020, 11:09 AM · Restricted Project
tejohnson added a comment to D86958: [Docs] Add/update release notes for D71913 (LTO WPD changes).

Please go ahead and commit to the branch.

Sep 15 2020, 8:54 AM · Restricted Project, Restricted Project

Sep 14 2020

tejohnson committed rG226d80ebe20e: [MemProf] Rename HeapProfiler to MemProfiler for consistency (authored by tejohnson).
[MemProf] Rename HeapProfiler to MemProfiler for consistency
Sep 14 2020, 1:16 PM
tejohnson closed D87622: [MemProf] Rename HeapProfiler to MemProfiler for consistency.
Sep 14 2020, 1:16 PM · Restricted Project, Restricted Project
tejohnson added a comment to D87622: [MemProf] Rename HeapProfiler to MemProfiler for consistency.

LGTM (if the option is documented, the documentation part also needs to be updated).

Sep 14 2020, 1:14 PM · Restricted Project, Restricted Project
tejohnson retitled D87622: [MemProf] Rename HeapProfiler to MemProfiler for consistency from [MemProf] Rename HeapProfiler to MemProfiler for consistency (NFC) to [MemProf] Rename HeapProfiler to MemProfiler for consistency.
Sep 14 2020, 12:59 PM · Restricted Project, Restricted Project
tejohnson added a comment to D87622: [MemProf] Rename HeapProfiler to MemProfiler for consistency.

LG. (Nit: technically this is not NFC: it affects command line options and runtime function names (ABI changes).)

Sep 14 2020, 12:59 PM · Restricted Project, Restricted Project
tejohnson updated the diff for D87120: [MemProf] Memory profiling runtime support.

Rename from heapprof to memprof.

Sep 14 2020, 12:28 PM · Restricted Project