This is an archive of the discontinued LLVM Phabricator instance.

[ESan][MIPS] Adds support for MIPS64
ClosedPublic

Authored by slthakur on Aug 23 2016, 6:05 AM.

Details

Summary

With this patch 12 out of 13 tests are passing.

Diff Detail

Repository
rL LLVM

Event Timeline

slthakur updated this revision to Diff 68976.Aug 23 2016, 6:05 AM
slthakur retitled this revision from to [ESan][MIPS] Adds support for MIPS64.
slthakur updated this object.
slthakur added reviewers: aizatsky, bruening.
slthakur set the repository for this revision to rL LLVM.
slthakur added a project: Restricted Project.
slthakur updated this revision to Diff 69085.Aug 24 2016, 12:33 AM
slthakur added subscribers: kcc, eugenis.
  • vmaSize for x86_64 case was not correct in previous diff.
  • Corrected expected output for test verbose-simple.c
bruening added inline comments.Aug 25 2016, 8:41 PM
lib/esan/esan.cpp
144

Style

150

This is making assumptions about where the initial stack is allocated. Please comment those assumptions (and why they would hold on all supported platforms X ulimit or other stack parameters).

lib/esan/esan.h
37

Please follow LLVM style in these new files: variables should start with upper-case

lib/esan/esan_shadow.h
118

Providing the formula would help clarify

127

The final 2 aren't adjacent, so I don't understand why they would be merged?

136

It may be worth keeping this comment, as the shadow translation is getting more expensive with extra variable loads that we could turn back into constants.

185

Style: h (

186

Style: { on prior line

188

Style: { on prior line

203

Seems best to have a default raising an error

lib/sanitizer_common/sanitizer_linux_mips64.S
19

Suggest: a comment identifying this magic # 5211 as SYS_rt_sigreturn

21

Why a nop?

slthakur updated this revision to Diff 69364.Aug 26 2016, 6:40 AM
slthakur marked 11 inline comments as done.

Addressed review comments

lib/esan/esan_shadow.h
127

Yes, the last two don't need to be merged.

lib/sanitizer_common/sanitizer_linux_mips64.S
21

Sorry a nop is not required there. I felt that a syscall would require a delay slot but thats not the case.

bruening added inline comments.Aug 26 2016, 4:05 PM
lib/esan/esan.cpp
150

nit: for x86-64 the vsyscall region is a higher address than the stack

lib/esan/esan_shadow.h
136

This was overlooked?

slthakur marked 3 inline comments as done.Aug 29 2016, 2:16 AM
slthakur added inline comments.
lib/esan/esan.cpp
150

Okay. The vsyscall region is placed in the upper half of the canonical address space which is used by the kernel. So would it be correct to say that "the initial stack is always allocated
in the topmost segment of the user address space" ?

lib/esan/esan_shadow.h
136

The comment was preserved in the updated diff. I forgot to mark this as "done".

bruening added inline comments.Aug 29 2016, 11:42 AM
lib/esan/esan.cpp
150

Works for me

lib/esan/esan_shadow.h
136

The comment as-is is not really relevant though b/c even more platforms beyond these 2 are unlikely to change the picture: adding the 2nd one here is what moved several constants to variables and that's what we might want to templatize away.

slthakur updated this revision to Diff 69827.Aug 31 2016, 4:57 AM
slthakur marked 2 inline comments as done.

Used templatized functions over ShadowMapping.

slthakur updated this revision to Diff 69835.Aug 31 2016, 5:09 AM
slthakur marked 2 inline comments as done.

Changed comment in esan.cpp as per suggestion.

bruening accepted this revision.Aug 31 2016, 10:25 PM
bruening edited edge metadata.
bruening added inline comments.
lib/esan/esan_shadow.h
243

I think the prevailing LLVM style is to not indent the cases, though an if..else is always faster than a switch for so few cases. However, given that the associated compiler instrumentation is hardcoded for one target, do we need to pay a runtime cost when a different library is going to picked for MIPS vs x86? I.e., is there a reason not to hardcode the library in the same way as the compiler instru?

I don't want to drag this out, and maybe this should be measured before over-thinking it, though with all of the memsets and memcpys coming here intuition says that appToShadow is going to be a bottleneck. I'm fine with either the prior diff or sthg like this one with a comment about making the mask static. The offset and scale are already loaded and not constants unfortunately.

This revision is now accepted and ready to land.Aug 31 2016, 10:25 PM
slthakur added inline comments.Sep 7 2016, 2:25 AM
lib/esan/esan_shadow.h
243

Yes, we can hard code the mask just like we did in the compiler instrumentation to avoid runtime cost in the library. I have added a TODO in the current diff and committed it to the repository (rL280795.). In my next patch I will try to make the mask static and replace switch cases with if...else construct where cases are few,

slthakur closed this revision.Sep 7 2016, 2:25 AM
slthakur reopened this revision.Sep 13 2016, 11:11 PM
This revision is now accepted and ready to land.Sep 13 2016, 11:11 PM
slthakur updated this revision to Diff 71299.Sep 13 2016, 11:18 PM
slthakur edited edge metadata.

This patch was reverted due to failures on x86. The reason of failures was that the shadow offsets were zero on x86 because the offset array was defined as class variable. In this diff I have defined the offset array in the initialize() function which resolves the issue. Tested this patch along with its dependents on x86 and the test results are clean.

slthakur requested a review of this revision.Sep 13 2016, 11:18 PM
slthakur edited edge metadata.
zhaoqin requested changes to this revision.Sep 14 2016, 12:30 PM
zhaoqin edited edge metadata.

I am not sure this CL can be build on non-mips64.
Could you please test on a non-mips64 (e.g. x86_64) so we can avoid unnecessary bot failure?

test/esan/TestCases/mmap-shadow-conflict.c
16

Are you sure the code can be compiled on non-mips64?
Please at least build and test on one non-mips64 (e.g., x86_64) machine

This revision now requires changes to proceed.Sep 14 2016, 12:30 PM
slthakur updated this revision to Diff 71597.Sep 15 2016, 11:47 PM
slthakur edited edge metadata.
slthakur added inline comments.Sep 15 2016, 11:57 PM
test/esan/TestCases/mmap-shadow-conflict.c
16

Sorry, there was a mistake in uploading of the diff. This test case was tested on x86_64 with a different diff which included this line of mmap and all tests were passing. I just uploaded the wrong diff here.

A few questions.
Nothing major, but better to have another round.

lib/esan/esan_shadow.h
193–212

We are using virtual address in the user space to determine the shadow memory translation.
Are we sure it would work for all the archs that have the same VmaSize?
I guess it would work for now. so just curious how likely it would work for any possible future arch/os.

lib/sanitizer_common/sanitizer_platform_limits_posix.h
631

Why multiple #if separate from each other?
Can we move them together?

test/esan/TestCases/mmap-shadow-conflict.c
30–37

hmm feel better if we can add CHECK, something like CHECK-x86_64 and CHECK-NEXT-x86_64.
forget about it if it is too hard to do so.

test/esan/TestCases/verbose-simple.c
15

Are we sure CHECK still works if we use --check-prefix option?
It probably is true, just be cautious.

test/lit.common.cfg
91

can we do similar thing as CHECK-%os by adding CHECK-%arch, or there will be a conflict.

slthakur added inline comments.Sep 22 2016, 1:51 AM
lib/esan/esan_shadow.h
193–212

Yes, it would work on any arch with same VmaSize provided that proper application memory ranges for that arch are defined.

lib/sanitizer_common/sanitizer_platform_limits_posix.h
631

Yes we can move them together.

test/esan/TestCases/mmap-shadow-conflict.c
30–37

We can do a similar thing for CHECK-%acrh as done for CHECK-%os but we will not be able to check the order of output lines with CHECK-NEXT-%arch. Thats why I switched to --check-prefix=%arch only.

test/esan/TestCases/verbose-simple.c
15

No it does not work. The CHECK lines are not checked by FileCheck. I will add --check-prefix=CHECK to the FileCheck command to make it work. Thanks for catching this.

test/lit.common.cfg
91

We can do a similar thing for CHECK-%acrh as done for CHECK-%os but we will not be able to check the order of output lines with CHECK-NEXT-%arch. Thats why I switched to --check-prefix=%arch only.

slthakur updated this revision to Diff 72236.Sep 22 2016, 11:53 PM
slthakur updated this object.
slthakur edited edge metadata.

Addressed review comments.

zhaoqin accepted this revision.Sep 23 2016, 8:09 AM
zhaoqin edited edge metadata.

LGTM

This revision is now accepted and ready to land.Sep 23 2016, 8:09 AM

Committed in rL280795, but reverted in rL280954.

slthakur closed this revision.Oct 6 2016, 3:08 AM

Committed revision 283435