Page MenuHomePhabricator

victoryang (Vic Yang)
User

Projects

User does not belong to any projects.

User Details

User Since
Jan 4 2019, 12:51 PM (27 w, 4 d)

Recent Activity

Feb 4 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

George,

0.5% is not a micro-optimization if it works for many applications. If you find a heuristic that can generally reduce memory usage by 0.5%, that's not negligible. Imagine for example 0.5% memory usage reduction in a huge cloud application. You might be able to save multi million dollars.

Vic,

That said, the numbers you've shown so far don't honestly seem very convincing. The number of data points is too few that I can't see if it generally works. I believe I can test your patch myself and see how it works for Linux applications by parsing /proc/self/smaps (which contains information about the number of dirty pages). Let me try to do that for a few applications like clang.

Rui,

Feb 4 2019, 7:52 PM

Jan 18 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

Another data point: Sorting the data section for Chrome on Android, data section dirty pages went down from 6308KB to 6280KB (arm32).

Or 0.5%..

Jan 18 2019, 11:34 AM

Jan 17 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

Another data point: Sorting the data section for Chrome on Android, data section dirty pages went down from 6308KB to 6280KB (arm32).

Jan 17 2019, 12:09 PM

Jan 16 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..
In D56325#1360770, @pcc wrote:

I was able to build and test Chrome on Android. Without this, the dirty pages from all libraries included in Chrome APK sum up to 364KB. With the symbols in those libraries sorted, this comes down to 360KB.

How exactly did you get those figures? For measuring anything memory related in Chrome I'd recommend using the system_health.memory_mobile benchmarks (I'd be happy to help you run them).

Jan 16 2019, 4:27 PM
victoryang added a comment to D56325: Sort symbols in .bss by size..

4KB is just one page. Honestly I think it is hard to draw a conclusion from only that data as the difference is too small. Could you give us more data points so that we are convinced that the patch actually makes a difference?

Jan 16 2019, 4:22 PM
victoryang added a comment to D56325: Sort symbols in .bss by size..

I was able to build and test Chrome on Android. Without this, the dirty pages from all libraries included in Chrome APK sum up to 364KB. With the symbols in those libraries sorted, this comes down to 360KB.

Jan 16 2019, 3:48 PM

Jan 13 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

I don't think we need a detailed benchmark for other targets, as how programs use .bss (and in general other parts of data sections) doesn't depend too much on targets.

Hard to say. I remember we had a patch that fixed inconsistency of handling .bss symbols (we had an issue in a case when application wanted to have the same order of symbols it would have if we would create them in a command line order, it had a relocation overflow because we create the .bss symbols first now). The patch was declined (I can find it if you want). So if this patch solves a local issue of low-end Android handsets only I really see no reason of why we want to change the behavior of LLD? It is the almost the same case that patch had. The difference is that now it is needed for Android and not for an unknown application. Until we have benchmarks showing it has a benefit for other platforms I am not convinced we should change the linker. It is up to you anyways, I am just saying my opinion about that change.

For the record, the reason why I'm in favor of this patch not because I'm working for Google; it is not correct to say that this patch is to fix a local issue. I've been trying to handle all contributors equally. As to this patch, I think it doesn't only improve Android but improve every target. Android Go team is working hard to reduce memory usage, and that's why they found this heuristics, and I added that to the comment because a concrete example is more reader-friendly than saying the same thing abstractly. But nothing special about Android Go.

That said, I can ask Vic to build other regular Linux applications with/without this patch to see if the same improvement (or at least no regression) can be observed. Vic, do you mind if I ask you to do that?

Jan 13 2019, 9:14 PM

Jan 9 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

I didn't test the descending order, but assuming there's no special alignment requirement, you'd likely end up with similar result because you are essentially shifting all page boundaries and flipping individual symbols at the same time.

Jan 9 2019, 10:04 AM

Jan 4 2019

victoryang added a comment to D56325: Sort symbols in .bss by size..

What is the entire size of your program?

The numbers are measured across all processes running, so there are multiple programs. If you are looking for numbers from a single binary, I saw private dirty from .bss in libc went down from 36KB to 16KB. Unfortunately I didn't record the saving from sorting .bss+.data in libc. I can do the test again if that's necessary.

I'm tempted to sort all SHF_WRITE sections by size by default. Unlike some other sections such as .init_array, there should not be really any program that have an assumption on how .data sections are laid out, but I'm not sure if that wouldn't surprise users. But maybe, we should do that?

Unless there's a certain logic in how things in SHF_WRITE sections are ordered right now, I don't think anyone can realistically depend on the ordering, so this should be safe.

Jan 4 2019, 1:20 PM
victoryang added a comment to D56325: Sort symbols in .bss by size..

For .data section specifically, the logic does apply, and I saw more memory saving when I sorted .bss+.data vs only .bss on Android. Applying this to all SHF_WRITE sections should be fine. It is possible that sorting results in no memory save at all in the worst case, but that shouldn't be a problem.

Do you have a number on how much memory you could save on Android by sorting .data section? I'm pretty interested in that number
.

Jan 4 2019, 1:05 PM
victoryang added inline comments to D56325: Sort symbols in .bss by size..
Jan 4 2019, 12:56 PM