This is an archive of the discontinued LLVM Phabricator instance.

[esan] EfficiencySanitizer instrumentation pass
ClosedPublic

Authored by bruening on Apr 15 2016, 11:06 AM.

Details

Summary

Adds an instrumentation pass for the new EfficiencySanitizer ("esan")
performance tuning family of tools. Multiple tools will be supported
within the same framework. Preliminary support for a cache fragmentation
tool is included here.

The shared instrumentation includes:
+ Turn mem{set,cpy,move} instrinsics into library calls.
+ Slowpath instrumentation of loads and stores via callouts to

the runtime library.

+ Fastpath instrumentation will be per-tool.
+ Which memory accesses to ignore will be per-tool.

Diff Detail

Event Timeline

bruening updated this revision to Diff 53917.Apr 15 2016, 11:06 AM
bruening retitled this revision from to [esan] EfficiencySanitizer instrumentation pass.
bruening updated this object.
bruening added a reviewer: eugenis.
bruening added subscribers: kcc, zhaoqin, llvm-commits.
aizatsky added inline comments.Apr 15 2016, 3:29 PM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
22

wrong import order. Please run clang-tidy & clang-format on the file.

49

do you imagine these used for other esan sanitizers as well? Maybe esan-cache-frag-*?

60

this don't follow llvm naming convention.

65

Why do you need this at all, rather than using Options directly?

157

why not __esan_memmove?

168

Ctx

183

this function always returns false.

197

Is it necessary to call it for each function? There could be quite a lot of them.

290

maybe have 3 cases here? Will be more future-proof.

295

llvm_unreachable?

296

return true?

test/Instrumentation/EfficiencySanitizer/cache_frag_basic.ll
53

please add unaligned read & odd size tests.

bruening added inline comments.Apr 15 2016, 4:42 PM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
22

Will do. Note that AddressSanitizer.cpp, MemorySanitizer, ThreadSanitizer.cpp, etc. on which this file is based all have this same (incorrect) order.

49

Yes, most of the esan tools instrument all memory references and these options will apply to all of them.

60

Please clarify: the kEsanModuleCtorName, or the esan.module.ctor? Both are matching what the other sanitizers use. Neither matches the official LLVM guide, though lower-case k seems to be used enough it feels like an informal convention, and the runtime interface routines seem to all be C-style to match the extern C interface.

65

There are two methods of invocation: from the clang front-end where -fsanitize=efficiency-foo will become EfficiencySanitizerOptions::ESAN_Foo, or when opt is directly invoked and -esan-foo wlil become ClToolFoo. One needs to be converted to the other. If you have a solution that is simpler, please elaborate on what you mean?

157

That would be more consistent. This current code is modeled after what ThreadSanitizer.cpp and AddressSanitizer.cpp do: they have xsan_* for the others but just "mem*" for the intrinsics. I am happy to add esan_.

168

You're saying current LLVM code prefers "Ctx" as an abbrevation for Context over "Cxt"? OK, sure.

183

Yes, but not in a future CL. We have built some prototypes and some return true for some cases here. We are trying to submit in small pieces for easier reviews. I will add a comment explaining that this will not always be a nop.

197

Hmm, this is modeled after what AddressSanitizer.cpp and ThreadSanitizer.cpp do. They are all function passes, so I believe they do need to re-acquire these functions every time and can't store them in a simple manner. Are you proposing changing them all (esan, asan, tsan, maybe others) to be module passes? How about tackling that in a separate CL?

290

Sure. Again, this was shamelessly modeled after the tsan and asan code.

295

OK.

296

Agreed, this is modifying the function. It seems to be a bug in ThreadSanitizer.cpp as well.

test/Instrumentation/EfficiencySanitizer/cache_frag_basic.ll
53

WIll do.

silvas added a subscriber: silvas.Apr 15 2016, 4:43 PM

Is there an RFC for this or something? I'm having a hard time figuring out exactly what it is trying to accomplish or what the ultimate "performance tuning family of tools" is going to be.

I mean, I think it is interesting, but I think llvm-dev would like to see the big picture first.

Is the mem*/str* instrumentation for a shadow-based version of http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40679.pdf ?

pcc added a subscriber: pcc.Apr 15 2016, 4:55 PM
pcc added inline comments.
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
22
aizatsky added inline comments.Apr 15 2016, 4:59 PM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
22

Those files do not conform to LLVM coding style right now, but we'd like them to. There's no reason why new code shouldn't.

60

kEsanModuleCtorName

There has been couple complaints about this kind of naming. AFAIK the consensus is that clang-tidy should be happy.

65

I'd rather you convert in the different direction then. From flag to options. Globals are worse than locals :)

157

I'm not sure if there was a reason for that. Maybe history. If nothing prevents it, I suggest you use __esan_ prefix everywhere. Would be much easier to spot/understand the assemby dump.

168

I actually thought this is a typo :) Seems to be 20x usages for Ctx than Cxt :)

183

Yep. A simple TBD will do.

197

No, I don't propose that. I guess if this becomes a problem, you'd be able to change it to ModulePass. Let's ignore this for a while.

Is there an RFC for this or something? I'm having a hard time figuring out exactly what it is trying to accomplish or what the ultimate "performance tuning family of tools" is going to be.

I mean, I think it is interesting, but I think llvm-dev would like to see the big picture first.

Is the mem*/str* instrumentation for a shadow-based version of http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40679.pdf ?

I probed a bit on IRC and there didn't seem to be any general awareness of this EfficiencySanitizer. Would you mind starting an RFC on llvm-dev?

Is there an RFC for this or something? I'm having a hard time figuring out exactly what it is trying to accomplish or what the ultimate "performance tuning family of tools" is going to be.

I will send an RFC to llvm-dev.

bruening added inline comments.Apr 16 2016, 7:05 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
157

These are calling the libc functions, which are then intercepted, rather than calling custom routines in the runtime: thus they must remain the exported libc names.

bruening marked 11 inline comments as done.Apr 18 2016, 8:42 PM
bruening added inline comments.
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
22

So the current code is correct. Note that clang-format does not think so.

bruening updated this revision to Diff 54154.Apr 18 2016, 8:43 PM
bruening marked an inline comment as done.

Addresses reviewer comments + clang-tidy and clang-format

aizatsky added inline comments.Apr 19 2016, 1:00 PM
include/llvm/Transforms/Instrumentation.h
128

Update comment.

lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
318

for simplicity/consistency let's make TypeSize in bytes?

bruening marked 2 inline comments as done.Apr 19 2016, 3:43 PM
bruening updated this revision to Diff 54286.Apr 19 2016, 3:43 PM

Address reviewer comments: typo and size in bytes.

aizatsky accepted this revision.Apr 19 2016, 4:00 PM
aizatsky edited edge metadata.

The code LGTM.

This revision is now accepted and ready to land.Apr 19 2016, 4:00 PM
filcab added a subscriber: filcab.Apr 20 2016, 8:13 AM

Thanks a lot for working on this. I'm interested in it. :-)

lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
51

Would "loads and stores" be better than a general "memory accesses"?

94

shouldIgnoreMemoryAccess?

138

Why not load/store, like the llvm instructions, instead of read and write?

146

Why the different SmallString size?

167

Any reason for the extra parens?

306

The three cases above can be written more succintly (and made easier to follow/spot (lack of) differences).
Check out AddressSanitizer::instrumentMemIntrinsic.

322

Do we want to warn once (per run of whatever tool), though?

bruening added inline comments.Apr 20 2016, 10:01 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
306

Reviewer ping-pong :) Please see the original version of this code in the first patchset, which matched what asan had, and the reviewer comment above asking to change it to the current form.

filcab added inline comments.Apr 20 2016, 10:32 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
306

I have a very strong preference over the simpler (to read and (even more important: ) to spot differences) code.
Mike: Do you have a strong preference over this idiom?
MemTransferIntrin has been there, with only these two subclasses, since 2009.
I think it's better to have the simpler version, and eventually change it if we need.

btw, Derek: You can reduce the noise by using isa<Mem...>, and getting the arguments from a MemIntrinsic, like ASan does.

aizatsky added inline comments.Apr 20 2016, 11:12 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
306

No I do not have any strong preference.

bruening marked 6 inline comments as done.Apr 20 2016, 2:55 PM
bruening added inline comments.
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
51

OK. The flag was modeled after tsan's flag which has this name.

138

This was modeled after tsan, which calls them read and write. It looks like asan uses load and store. I am fine changing it.

146

That's what tsan has, presumably being paranoid about the extra length of "unaligned".

167

This is what AddressSanitizer.cpp and MemorySanitizer.cpp have. Will remove from here.

306

btw, Derek: You can reduce the noise by using isa<Mem...>, and getting the arguments from a MemIntrinsic, like ASan does.

This code was originally modeled after tsan. Yes, asan is more succinct than the tsan code, once it does the cast.

322

OK. I don't see any DO_ONCE macro or std::call_once in the code base so I suppose a direct static bool is the method.

How do I test for this warning in the .ll file? Also, it seems to pollute the lit output a bit.

bruening updated this revision to Diff 54429.Apr 20 2016, 2:56 PM
bruening marked 4 inline comments as done.
bruening edited edge metadata.

Rename read to load; add warning; other misc requested changes.

filcab added inline comments.Apr 21 2016, 5:43 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
324

Polluting output too much is annoying. Thinking about it, I guess we might hit this on most runs, not just rarely, which makes it less useful.
How about doing something like ASan does in instrumentUnusualSizeOrAlignment?

filcab added inline comments.Apr 21 2016, 5:47 AM
lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
324

Or just add an __esan_{un,}aligned_load_n function which we call with addr and size, and then we can figure out the fastpath later (which depends on the tool anyway).

bruening updated this revision to Diff 54548.Apr 21 2016, 11:12 AM
bruening marked an inline comment as done.

Add callouts for unusually-sized loads and stores.

filcab accepted this revision.Apr 21 2016, 11:36 AM
filcab added a reviewer: filcab.

My bigger nits are name related, so I'll give you the LGTM now.

lib/Transforms/Instrumentation/EfficiencySanitizer.cpp
60

We probably want to change "Bad" to something else. Non-power-of-two-below-or-equal-to-16 doesn't roll off the tongue, though.
How about "irregular", or something like that? Or even "unusual". My problem here is that it's not explanatory.

113

I'd prefer a better explained name, like EsanStoreN (probably with a comment saying we assume it's unaligned, for now?)

158

Same thing, I'd prefer a __esan_unaligned_store_n name (or __esan_unaligned_store_range, if you're following tsan's names, instead of asan's), like the other sanitizers.

This revision was automatically updated to reflect the committed changes.
bruening marked 3 inline comments as done.