This is an archive of the discontinued LLVM Phabricator instance.

[clang-tidy] introduce legacy resource functions to 'cppcoreguidelines-owning-memory'
ClosedPublic

Authored by JonasToth on Sep 29 2017, 3:59 AM.

Details

Summary

This patch introduces support for legacy C-style resource functions that must obey
the 'owner<>' semantics.

  • added legacy creators like malloc,fopen,...
  • added legacy consumers like free,fclose,...

This helps codes that mostly benefit from owner:
Legacy, C-Style code that isn't feasable to port directly to RAII but needs a step in between
to identify actual resource management and just using the resources.

Diff Detail

Repository
rL LLVM

Event Timeline

JonasToth created this revision.Sep 29 2017, 3:59 AM
JonasToth updated this revision to Diff 117120.Sep 29 2017, 4:06 AM
  • Clean up stuff that slipped through
aaron.ballman added inline comments.Sep 29 2017, 6:46 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.cpp
53 ↗(On Diff #117120)

can not -> cannot

62 ↗(On Diff #117120)

can not -> cannot

86 ↗(On Diff #117120)

Remove the backticks from the comment. Also, this seems like a pretty major false positive -- any way to suppress it?

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

Missing ::aligned_alloc

There's a comment below about C-style functions that create resources -- do you care about things from thread.h, mutex.h, or other resource-creating C functions as well?

docs/clang-tidy/checks/cppcoreguidelines-owning-memory.rst
89 ↗(On Diff #117120)

can not -> cannot

95 ↗(On Diff #117120)

can not -> cannot

test/clang-tidy/cppcoreguidelines-owning-memory-legacy-functions.cpp
17 ↗(On Diff #117120)

Missing a test for calloc() and aligned_alloc().

134 ↗(On Diff #117120)

can not -> cannot

JonasToth marked 7 inline comments as done.Sep 29 2017, 8:50 AM
JonasToth added inline comments.
clang-tidy/cppcoreguidelines/OwningMemoryCheck.cpp
86 ↗(On Diff #117120)

The comment is misleading!
The tests actually do have this case and it is caught correctly. (see line 126-130 in testfile).

The false positive that does exist is std::free(std::malloc(100)), because the argument to free is a non-owner beeing initialized with a newly created resource.

I adjust the comment.

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

In general, i care about everything beeing a resource! I just didn't think long enough to come up with everything from the standard (and i don't know enough as well :D)

So this list should contain everything and if you point to me to the right corners i will search the references.

The docs could also mention other common functions (posix, Windows, maybe even CUDA and the like). But i am not so sure about that.

JonasToth updated this revision to Diff 117158.Sep 29 2017, 8:50 AM
JonasToth marked 2 inline comments as done.
  • fix typos and wrong comment about false positive
JonasToth added inline comments.Sep 29 2017, 9:02 AM
test/clang-tidy/cppcoreguidelines-owning-memory-legacy-functions.cpp
17 ↗(On Diff #117120)

Would it be enough, to check if
ìnt* Test = calloc(100) works?
Since the matcher either works for everything or nothing, that would kind of work.

Or do you require more testing code?

aaron.ballman added inline comments.Oct 2 2017, 8:13 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
50 ↗(On Diff #117158)

Comment is out of date.

29 ↗(On Diff #117120)

We usually only add the APIs that are most common to users of the check, but when in doubt we cover just the standards-based functions, since everyone (generally) has access to those.

I think aligned_alloc() was the only one that's missing from C, but that's going from my (faulty) memory.

docs/clang-tidy/checks/cppcoreguidelines-owning-memory.rst
90 ↗(On Diff #117158)

This comment is now out of date.

test/clang-tidy/cppcoreguidelines-owning-memory-legacy-functions.cpp
17 ↗(On Diff #117120)

It's good to test all of the functions when possible to easily do it so that we don't accidentally introduce regressions with just one of them.

JonasToth added inline comments.Oct 2 2017, 11:19 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

Should i ask the guideline people about facilities from C that shall be treated as resources?

IMHO everything from the C/C++ Standards that is a resource/has resource semantics and is not RAII should be catched here.

Adding more OS stuff, that might be a resource (sockets, dunno what else) could be good as well.
Maybe suggestions for these things in the docs would be nice. But i actually don't know enough about these issues to create them myself. But that is optional and maybe the guidelines will even add guidence on this.

aaron.ballman added inline comments.Oct 4 2017, 5:58 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

Should i ask the guideline people about facilities from C that shall be treated as resources?

I think that's a good idea. There are mutex, thread, thread storage creation functions, etc -- should those be treated the same as malloc() and friends?

Adding more OS stuff, that might be a resource (sockets, dunno what else) could be good as well.

Maybe suggestions for these things in the docs would be nice. But i actually don't know enough about these issues to create them myself. But that is optional and maybe the guidelines will even add guidence on this.

This is much harder because it's pretty open-ended. I would say it's fine to leave OS and POSIX functionality out for right now.

JonasToth added inline comments.Oct 4 2017, 9:39 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

I opened an issue here https://github.com/isocpp/CppCoreGuidelines/issues/1050
Lets see what comes out of it. :)

I will add a comment in the docs that functionality from the operating system can qualify as resources, but that everybody needs to modify then configuration to fit his/her needs.

JonasToth added inline comments.Oct 6 2017, 1:03 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

The Coreguideline guys understood it as workoffloading :(, so the list iam currently thinking of is:

But getline does not return a value, it expects a char** where the allocation will happen, so its not very nice for the check.

  • fopen,fopen_s,freopen,freopen_s,fclose,tmpfile,tmpfile_s for file IO, all _s functions are again not nice with the check.

For the threading functions:
They all simulate classes with RAII, so in a sense they could be considered as owners, but the whole owner<> thing does not seem to play nice with these utilities. Since the check requires C++11 to have a templated typedef and the C++ standard has all of the funcitonality this check could ignore these functions. For completeness:

  • cnd_init, cnd_destroy, tss_create, tss_delete, mtx_init, mtx_destroy

My proposed (for now) list would be the classical memory allocations, strdup and the classical fileio-functions.

aaron.ballman added inline comments.Oct 8 2017, 7:22 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

Rather than an incomplete list of POSIX functions like strdup(), I'd say the first round of the patch would make sense to hit all the C11 standard functions (you can elect to support Annex K or not as you see fit) and then a follow-up patch for getting a mostly comprehensive list of POSIX functions if that's desirable. Otherwise, your list looks good.

JonasToth marked 10 inline comments as done.Oct 10 2017, 11:16 AM

work on the list of considered functions for legacy resource functions.

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

Posix and nonfitting functions could be done later. But i honestly don't want to put a lot of effort into C11 stuff and more about actual legacy code that.

IMHO: if people write C11 in C++, they don't want tooling support and the gsl.

JonasToth marked an inline comment as done.
  • list of legacy functions updated, test come later
JonasToth marked an inline comment as done.Oct 10 2017, 11:18 AM
JonasToth updated this revision to Diff 118760.EditedOct 12 2017, 3:09 AM
  • add unittesting both for legacy function and demonstrate bad result with std::vector

I added basic testing for legacy functions. Please request more if necessary, but for now
i think the tests actually cover all possible cases, since the legacy functions either
all work on none of them work. That's why did not add all possible usecases for every single function.

There is a false positive for freopen, since it takes 3 pointer arguments but only one must be an owner<>. The check heuristically assumes that all pointer arguments must be an owner<>, since I don't know a way to encode the position information into the list. This false positive can be fixed with a special cases for freopen, but not in general for now.

Additionally I added a testcase that demonstrates how 'owner<>' could work with
'std::vector<>' as the container to manage resources. Unfortunately this is not
possible right now. Typededuction in templates removes 'owner<>' and uses the underlying
type. The underlying problem is already present in the default testcases.
This additional test could be the base for further work to circumvent the limitation
and tries to make it more obvious.

JonasToth updated this revision to Diff 118889.Oct 13 2017, 1:37 AM
  • update doc to explicitly mention, that std::vector<owner<>> does not work properly
JonasToth marked 3 inline comments as done.Oct 13 2017, 1:38 AM
aaron.ballman added inline comments.Oct 15 2017, 7:49 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.cpp
82 ↗(On Diff #118889)

since -> because

227 ↗(On Diff #118889)

s/seperat, since/separately because

228 ↗(On Diff #118889)

as pointer -> as a pointer

be a owner -> be an owner

not certain what you mean by "and is known"?

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

IMHO: if people write C11 in C++, they don't want tooling support and the gsl.

I don't think that's a correct assumption to make, but I'm also fine with the more complex bits being done in a follow-up patch.

docs/clang-tidy/checks/cppcoreguidelines-owning-memory.rst
24 ↗(On Diff #118889)

s/as generated/to produce

86 ↗(On Diff #118889)

The more I look at this option name, the more I think it should be named LegacyResourceProducers to match with LegacyResourceConsumers. What do you think?

(If you decide to make this change, be sure to update the member variables we use as well.)

95 ↗(On Diff #118889)

s/expection/expecting

114 ↗(On Diff #118889)

s/beeing/being

JonasToth marked 7 inline comments as done.Oct 18 2017, 12:34 AM
JonasToth added inline comments.
clang-tidy/cppcoreguidelines/OwningMemoryCheck.cpp
82 ↗(On Diff #118889)

i dropped the comma, is it correct?
sorry for my bad english grammar :/

228 ↗(On Diff #118889)

reformulated. Reopen takes 3 pointer arguments, but only one must be an owner. This property is known, hopefully clearer now.

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
29 ↗(On Diff #117120)

it was stated wrong and too judging. But if c++11 is available, why use C11 threading?

Non-memory related issues might be suitable for another check as well, so it's probably counterproductive.

docs/clang-tidy/checks/cppcoreguidelines-owning-memory.rst
86 ↗(On Diff #118889)

agreed, changed.

JonasToth marked 6 inline comments as done.Oct 18 2017, 12:35 AM
  • address review comments
aaron.ballman added inline comments.Oct 18 2017, 8:37 AM
clang-tidy/cppcoreguidelines/OwningMemoryCheck.cpp
36 ↗(On Diff #119437)

Producers (plural) here as well as the documentation.

82 ↗(On Diff #118889)

Yes, this is correct -- and English grammar is not easy, no need to apologize!

227 ↗(On Diff #118889)

seperatly -> separately

clang-tidy/cppcoreguidelines/OwningMemoryCheck.h
53 ↗(On Diff #119437)

Should be Producers (plural) here and elsewhere.

29 ↗(On Diff #117120)

Seems reasonable enough. Can you re-flow this so that :: and calloc are not on different lines?

JonasToth marked 7 inline comments as done.Oct 18 2017, 8:50 AM
JonasToth updated this revision to Diff 119493.Oct 18 2017, 8:53 AM
  • addressed review comments
aaron.ballman added inline comments.Oct 18 2017, 8:56 AM
test/clang-tidy/cppcoreguidelines-owning-memory-legacy-functions.cpp
3 ↗(On Diff #119493)

I think the RUN line needs to be updated to Producers (plural).

Does this test pass as-is? (That would be something worth looking into if it does.)

JonasToth updated this revision to Diff 119494.Oct 18 2017, 9:00 AM
  • fix test line
JonasToth marked an inline comment as done.Oct 18 2017, 9:02 AM
aaron.ballman accepted this revision.Oct 18 2017, 9:05 AM

LGTM, thank you!

This revision is now accepted and ready to land.Oct 18 2017, 9:05 AM
This revision was automatically updated to reflect the committed changes.