Page MenuHomePhabricator

stephan.yichao.zhao
User

Projects

User does not belong to any projects.

User Details

User Since
Aug 1 2020, 11:38 PM (11 w, 3 d)

Recent Activity

Yesterday

stephan.yichao.zhao closed D89662: Set Huge Page mode on shadow regions based on no_huge_pages_for_shadow.

submitted by https://github.com/llvm/llvm-project/commit/91dc545bf24daa60c21c93039408061194dd0ab3

Tue, Oct 20, 9:53 AM · Restricted Project

Mon, Oct 19

stephan.yichao.zhao updated the diff for D89199: Release pages to OS when setting 0 label.

moved constant definition.

Mon, Oct 19, 3:33 PM · Restricted Project
stephan.yichao.zhao updated the diff for D89662: Set Huge Page mode on shadow regions based on no_huge_pages_for_shadow.

update

Mon, Oct 19, 1:09 PM · Restricted Project
stephan.yichao.zhao updated the diff for D89662: Set Huge Page mode on shadow regions based on no_huge_pages_for_shadow.

addressed comments

Mon, Oct 19, 1:08 PM · Restricted Project

Sun, Oct 18

stephan.yichao.zhao requested review of D89662: Set Huge Page mode on shadow regions based on no_huge_pages_for_shadow.
Sun, Oct 18, 6:27 PM · Restricted Project
stephan.yichao.zhao updated the diff for D89199: Release pages to OS when setting 0 label.

addressed comments

Sun, Oct 18, 10:19 AM · Restricted Project
stephan.yichao.zhao added inline comments to D89199: Release pages to OS when setting 0 label.
Sun, Oct 18, 10:18 AM · Restricted Project

Sat, Oct 10

stephan.yichao.zhao requested review of D89199: Release pages to OS when setting 0 label.
Sat, Oct 10, 6:36 PM · Restricted Project

Tue, Oct 6

stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Tue, Oct 6, 1:39 PM · Restricted Project
stephan.yichao.zhao retitled D88755: Replace shadow space zero-out by madvise at mmap from Remove shadow space zero-out at mmap to Replace shadow space zero-out by madvise at mmap.
Tue, Oct 6, 1:35 PM · Restricted Project
stephan.yichao.zhao added inline comments to D88755: Replace shadow space zero-out by madvise at mmap.
Tue, Oct 6, 1:17 PM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Tue, Oct 6, 1:17 PM · Restricted Project
stephan.yichao.zhao added inline comments to D88755: Replace shadow space zero-out by madvise at mmap.
Tue, Oct 6, 10:43 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

changed from statm to smap

Tue, Oct 6, 10:42 AM · Restricted Project

Mon, Oct 5

stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Mon, Oct 5, 8:58 PM · Restricted Project
stephan.yichao.zhao added a comment to D88755: Replace shadow space zero-out by madvise at mmap.

I agree with @eugenis that the madvise call is more beneficial as a special case for large sizes in dfsan_set_label.

dfsan_set_label is only called from wrappers (which already have lots of branches and shadow propagation), so the overhead from the branch is likely insignificant.

That said, I'm fine with making that change in follow-up patch if you prefer.

Mon, Oct 5, 8:31 PM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

addressed comments

Mon, Oct 5, 8:29 PM · Restricted Project
stephan.yichao.zhao added a comment to D88755: Replace shadow space zero-out by madvise at mmap.

Sure, madvise has overhead. Check SetShadow in MSan and FastPoisonShadow in ASan.

Mon, Oct 5, 5:10 PM · Restricted Project
stephan.yichao.zhao added a comment to D88755: Replace shadow space zero-out by madvise at mmap.

I would move this under dfsan_set_label for consistency with asan & msan, and also to optimize large memset calls outside of the mmap interceptor (I assume they result in large dfsan_set_label calls?).

Mon, Oct 5, 4:00 PM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Mon, Oct 5, 10:03 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Mon, Oct 5, 10:01 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

update

Mon, Oct 5, 10:00 AM · Restricted Project
stephan.yichao.zhao added inline comments to D88755: Replace shadow space zero-out by madvise at mmap.
Mon, Oct 5, 9:58 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

addressed comments

Mon, Oct 5, 9:58 AM · Restricted Project
stephan.yichao.zhao added a comment to D88686: Release the shadow memory used by the mmap range at munmap.

I might be missing something, but if you release one of the shadow pages to the OS, couldn't it then accidentally use that page for one of the user anonymous mappings, thus breaking DFSan logic?

Mon, Oct 5, 8:53 AM · Restricted Project

Fri, Oct 2

stephan.yichao.zhao updated the diff for D88755: Replace shadow space zero-out by madvise at mmap.

addressed comments

Fri, Oct 2, 6:05 PM · Restricted Project
stephan.yichao.zhao requested review of D88755: Replace shadow space zero-out by madvise at mmap.
Fri, Oct 2, 2:03 PM · Restricted Project
stephan.yichao.zhao added inline comments to D88686: Release the shadow memory used by the mmap range at munmap.
Fri, Oct 2, 1:15 PM · Restricted Project
stephan.yichao.zhao updated the diff for D88686: Release the shadow memory used by the mmap range at munmap.

addressed comment

Fri, Oct 2, 1:15 PM · Restricted Project
stephan.yichao.zhao added inline comments to D88686: Release the shadow memory used by the mmap range at munmap.
Fri, Oct 2, 11:51 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88686: Release the shadow memory used by the mmap range at munmap.

updated

Fri, Oct 2, 11:49 AM · Restricted Project
stephan.yichao.zhao updated the diff for D88686: Release the shadow memory used by the mmap range at munmap.

addressed comments

Fri, Oct 2, 11:40 AM · Restricted Project
stephan.yichao.zhao retitled D88686: Release the shadow memory used by the mmap range at munmap from Release memory at munmap to Release the shadow memory used by the mmap range at munmap.
Fri, Oct 2, 11:40 AM · Restricted Project
stephan.yichao.zhao added inline comments to D88686: Release the shadow memory used by the mmap range at munmap.
Fri, Oct 2, 8:41 AM · Restricted Project

Thu, Oct 1

stephan.yichao.zhao updated the diff for D88686: Release the shadow memory used by the mmap range at munmap.

addressed comments

Thu, Oct 1, 6:11 PM · Restricted Project
stephan.yichao.zhao updated the diff for D88686: Release the shadow memory used by the mmap range at munmap.

addressed comments
added a test case

Thu, Oct 1, 2:19 PM · Restricted Project
stephan.yichao.zhao requested review of D88686: Release the shadow memory used by the mmap range at munmap.
Thu, Oct 1, 11:06 AM · Restricted Project

Sep 18 2020

stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 18 2020, 9:44 AM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 18 2020, 9:38 AM · Restricted Project

Sep 12 2020

stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

updated

Sep 12 2020, 11:46 PM · Restricted Project
stephan.yichao.zhao added a comment to D86905: Flush bitcode incrementally for LTO output.

fixed the if and for clauses style issue.

Sep 12 2020, 11:44 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

updated

Sep 12 2020, 11:44 PM · Restricted Project
stephan.yichao.zhao updated the summary of D86905: Flush bitcode incrementally for LTO output.
Sep 12 2020, 6:27 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

update

Sep 12 2020, 6:14 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 12 2020, 1:37 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 12 2020, 1:34 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

addressed comments

Sep 12 2020, 1:33 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 12 2020, 1:08 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

addressed comments

Sep 12 2020, 1:07 AM · Restricted Project
stephan.yichao.zhao closed D86913: Add raw_fd_stream that supports reading/seeking/writing.

submitted by https://github.com/llvm/llvm-project/commit/0ece51c60c51f0d4c285dbda3b6cff794041bdd7 and https://github.com/llvm/llvm-project/commit/19531a81f1de8ef8ee219765c74c32c6fcd3323f

Sep 12 2020, 12:43 AM · Restricted Project

Sep 10 2020

stephan.yichao.zhao abandoned D87399: Revert "[InstCombine] erase instructions leading up to unreachable".

Abandoned this one after @nikic applied https://github.com/llvm/llvm-project/commit/4e413e16216d0c94ada2171f3c59e0a85f4fa4b6
Thank you for everyone's help.

Sep 10 2020, 9:27 AM · Restricted Project

Sep 9 2020

stephan.yichao.zhao added a comment to D86905: Flush bitcode incrementally for LTO output.

ping

Sep 9 2020, 3:32 PM · Restricted Project
stephan.yichao.zhao added a comment to D87149: [InstCombine] erase instructions leading up to unreachable.

The requirements on volatile operations have been clarified in D53184 after a llvm-dev discussion. Of course we can re-evaluate this decision, in which case a LangRef patch needs to be proposed and a new RFC on llvm-dev started.

Sep 9 2020, 3:27 PM · Restricted Project
stephan.yichao.zhao reclaimed D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 12:08 PM · Restricted Project
stephan.yichao.zhao abandoned D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 11:37 AM · Restricted Project
stephan.yichao.zhao updated the summary of D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 10:54 AM · Restricted Project
stephan.yichao.zhao updated the summary of D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 10:52 AM · Restricted Project
stephan.yichao.zhao added reviewers for D87399: Revert "[InstCombine] erase instructions leading up to unreachable": lebedev.ri, spatel, chandlerc.
Sep 9 2020, 10:52 AM · Restricted Project
stephan.yichao.zhao added a reverting change for rGb22910daab95: [InstCombine] erase instructions leading up to unreachable: D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 10:51 AM
stephan.yichao.zhao requested review of D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 10:51 AM · Restricted Project
stephan.yichao.zhao added a reverting change for D87149: [InstCombine] erase instructions leading up to unreachable: D87399: Revert "[InstCombine] erase instructions leading up to unreachable".
Sep 9 2020, 10:51 AM · Restricted Project
stephan.yichao.zhao added a comment to D87149: [InstCombine] erase instructions leading up to unreachable.

Checked llvm::isGuaranteedToTransferExecutionToSuccessor from https://llvm.org/doxygen/ValueTracking_8cpp_source.html.
Yes, non-throw and return calls are considered a through instruction. But it does not check volatile specially about whether such an access can abort a program.

Sep 9 2020, 10:37 AM · Restricted Project
stephan.yichao.zhao added a comment to D87149: [InstCombine] erase instructions leading up to unreachable.

Although executing __builtin_unreachable is undefined, removing the code before it deletes their side effects.

I believe this behavior is correct. LangRef explicitly states that execution continues past a volatile store. As such, unreachable must be reached, which is undefined behavior. As such, we are free to optimize as we wish, including removing a preceding volatile store.

Sep 9 2020, 10:04 AM · Restricted Project
stephan.yichao.zhao added a comment to D87149: [InstCombine] erase instructions leading up to unreachable.

This change affected a case like this:

Sep 9 2020, 8:57 AM · Restricted Project

Sep 4 2020

stephan.yichao.zhao removed a reviewer for D86913: Add raw_fd_stream that supports reading/seeking/writing: espindola.
Sep 4 2020, 10:26 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

addressed comments

Sep 4 2020, 10:25 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

addressed comments

Sep 4 2020, 10:24 PM · Restricted Project

Sep 3 2020

stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

rebased from D86913

Sep 3 2020, 10:23 AM · Restricted Project

Sep 2 2020

stephan.yichao.zhao added inline comments to D86913: Add raw_fd_stream that supports reading/seeking/writing.
Sep 2 2020, 2:07 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

update

Sep 2 2020, 2:07 PM · Restricted Project
stephan.yichao.zhao added a comment to D86905: Flush bitcode incrementally for LTO output.

Our case is a bit different. Given a 512M incremental flush threshold, I tested an LTO built that outputs a 5G bitcode file. The BackpatchWord is called 16,613,927 times, among which only 12 needs disk seek. Plus, each access visits 4-8 bytes on a page, and all visited pages are far away from each other. It is likely that the pages are not cached, and need to load anyway, and after a load, our code does not access enough data on a page to 'cancel' the page fault cost. So its cost could be very similar to seek.

It seems that you're trying to implement your own I/O caching. I don't understand why you're not letting OS to do this for you. For instance on systems with larger amount of memory (I have 64 GB on my home PC, typical build server may have even more) mmap will buffer all your 5G bc file in memoy and then write it back to disk without any seek operations (which are costly on traditional HDD).

Sep 2 2020, 1:10 PM · Restricted Project

Sep 1 2020

stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

addressed comments

Sep 1 2020, 5:24 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86913: Add raw_fd_stream that supports reading/seeking/writing.
Sep 1 2020, 5:24 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 1 2020, 4:11 PM · Restricted Project
stephan.yichao.zhao added inline comments to D86905: Flush bitcode incrementally for LTO output.
Sep 1 2020, 2:53 PM · Restricted Project
stephan.yichao.zhao added a comment to D86957: [Bitstream] Use alignTo to make code more readable. NFC.

So this does both 64bit cast and alignment. Thank you.

Hi Craig, in your case, will NumElts be actually larger than 2^32? NumElts is read from https://llvm.org/docs/BitCodeFormat.html#enter-subblock-encoding that defines the blocklen to be 32bit. So when it gets larger, the overflow can happen at the writer side (https://llvm.org/doxygen/BitstreamWriter_8h_source.html#l00384).

My specific case was a blob for metadata strings that was ~1GB in size. The multipy by 8 to convert its size to bits was overflowing. I do worry that it might break again if the blob of metadata strings exceed 4GB.

The case I fixed is similar. One way to address is to extend that blocklen field to 64bit. imo this does not introduce any back-compatibility issue because 32 is not a fixed width, but VBR.

  1. when an old reader reads a bitcode written by a new writer, it works if blocklen is <= 2^32. Although it gets broken if blocklen is > 2^32, this case it does not work anyway.
  2. when a new reader reads a bitcode written by an old writer, it works fine since blocklen is <= 2^32.

So it is possible to extend it to 64bit.

The blocklen field in ENTER_SUBBLOCK isn't a VBR from what I could see. Its just a 32 bit value allowing a maximum block size of 16GB. There is a VBR6 to store the size of the blob. That one we could change to use uint64_t to allow blobs larger than 4GB, but we'd still be limited by the 16GB limit.

Sep 1 2020, 2:17 PM · Restricted Project
stephan.yichao.zhao added a comment to D86957: [Bitstream] Use alignTo to make code more readable. NFC.

So this does both 64bit cast and alignment. Thank you.

Hi Craig, in your case, will NumElts be actually larger than 2^32? NumElts is read from https://llvm.org/docs/BitCodeFormat.html#enter-subblock-encoding that defines the blocklen to be 32bit. So when it gets larger, the overflow can happen at the writer side (https://llvm.org/doxygen/BitstreamWriter_8h_source.html#l00384).

My specific case was a blob for metadata strings that was ~1GB in size. The multipy by 8 to convert its size to bits was overflowing. I do worry that it might break again if the blob of metadata strings exceed 4GB.

Sep 1 2020, 11:32 AM · Restricted Project
stephan.yichao.zhao added a comment to D86957: [Bitstream] Use alignTo to make code more readable. NFC.

So this does both 64bit cast and alignment. Thank you.

Sep 1 2020, 11:09 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

update

Sep 1 2020, 12:42 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86913: Add raw_fd_stream that supports reading/seeking/writing.

update

Sep 1 2020, 12:40 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

update

Sep 1 2020, 12:37 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

update

Sep 1 2020, 12:32 AM · Restricted Project
stephan.yichao.zhao added a comment to D86905: Flush bitcode incrementally for LTO output.

I can understand the read-write stream requirement, but the changes to lib/Support may require an additional set of reviewers. You will need some unittests (see llvm/unittests/Support/raw_ostream_test.cpp for example) Probably consider splitting the patch into two.

Sep 1 2020, 12:28 AM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

addressed comments

Sep 1 2020, 12:28 AM · Restricted Project
stephan.yichao.zhao requested review of D86913: Add raw_fd_stream that supports reading/seeking/writing.
Sep 1 2020, 12:25 AM · Restricted Project

Aug 31 2020

stephan.yichao.zhao removed a reviewer for D86905: Flush bitcode incrementally for LTO output: espindola.
Aug 31 2020, 8:56 PM · Restricted Project
stephan.yichao.zhao edited reviewers for D86905: Flush bitcode incrementally for LTO output, added: tejohnson; removed: espindola.
Aug 31 2020, 8:55 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86905: Flush bitcode incrementally for LTO output.

updated

Aug 31 2020, 8:51 PM · Restricted Project
stephan.yichao.zhao requested review of D86905: Flush bitcode incrementally for LTO output.
Aug 31 2020, 8:48 PM · Restricted Project
stephan.yichao.zhao abandoned D86901: Change comments of openFileForWrite.
Aug 31 2020, 5:34 PM · Restricted Project
stephan.yichao.zhao updated the diff for D86901: Change comments of openFileForWrite.

updated

Aug 31 2020, 5:33 PM · Restricted Project
stephan.yichao.zhao requested review of D86901: Change comments of openFileForWrite.
Aug 31 2020, 5:26 PM · Restricted Project

Aug 26 2020

stephan.yichao.zhao updated stephan.yichao.zhao.
Aug 26 2020, 11:03 AM
stephan.yichao.zhao requested review of D86645: Fix an overflow issue at BackpatchWord.
Aug 26 2020, 10:54 AM · Restricted Project

Aug 25 2020

stephan.yichao.zhao updated stephan.yichao.zhao.
Aug 25 2020, 9:44 PM
stephan.yichao.zhao updated stephan.yichao.zhao.
Aug 25 2020, 1:56 PM

Aug 24 2020

stephan.yichao.zhao updated the diff for D86500: Fix a 32-bit overflow issue when reading LTO-generated bitcode files whose strtab are of size > 2^29.

addressed comments

Aug 24 2020, 7:13 PM · Restricted Project
stephan.yichao.zhao requested review of D86500: Fix a 32-bit overflow issue when reading LTO-generated bitcode files whose strtab are of size > 2^29.
Aug 24 2020, 5:47 PM · Restricted Project

Aug 6 2020

stephan.yichao.zhao added inline comments to D85379: Reduce dropTriviallyDeadConstantArrays cumulative time percentage from 17% to 4%.
Aug 6 2020, 11:35 AM · Restricted Project
stephan.yichao.zhao retitled D85379: Reduce dropTriviallyDeadConstantArrays cumulative time percentage from 17% to 4% from Improve dropTriviallyDeadConstantArrays cumulative time percentage from 17% to 4% to Reduce dropTriviallyDeadConstantArrays cumulative time percentage from 17% to 4%.
Aug 6 2020, 12:48 AM · Restricted Project