This is an archive of the discontinued LLVM Phabricator instance.

[LibTooling] Add Stencil library for format-string style codegen.
ClosedPublic

Authored by ymandel on Mar 14 2019, 10:09 AM.

Details

Summary

This file defines the *Stencil* abstraction: a code-generating object, parameterized by named references to (bound) AST nodes. Given a match result, a stencil can be evaluated to a string of source code.

A stencil is similar in spirit to a format string: it is composed of a series of raw text strings, references to nodes (the parameters) and helper code-generation operations.

See thread on cfe-dev list with subject "[RFC] Easier source-to-source transformations with clang tooling" for background.

Diff Detail

Repository
rC Clang

Event Timeline

ymandel created this revision.Mar 14 2019, 10:09 AM
ymandel updated this revision to Diff 190683.Mar 14 2019, 11:11 AM

Build fixes.

gentle ping...

ymandel updated this revision to Diff 193496.Apr 3 2019, 7:43 AM

Sever dependency on NodeId and some general cleanup.

ymandel updated this revision to Diff 193498.Apr 3 2019, 7:50 AM

Remove noisy default-defined constructors/operators

ymandel updated this revision to Diff 193906.Apr 5 2019, 10:00 AM

Adjust to use SourceCode instead of FixIt.

ymandel updated this revision to Diff 194157.Apr 8 2019, 8:49 AM

Add operator() to Stencil for compatability with TextGenerator.

Stencil is technically independent of Transformer. However, Transformers
replacements and explanations take a generic std::function (there abbreviated as
TextGenerator). This addition let's Stencils act as TextGenerators.

sbenza added inline comments.Apr 10 2019, 9:13 AM
clang/include/clang/Tooling/Refactoring/Stencil.h
45 ↗(On Diff #194157)

Why not use llvm::Expected<std::string> as the return type?

75 ↗(On Diff #194157)

The interface API is all const. Why not keep a shared_ptr instead?
That way we don't have to have users implement a clone function.

94 ↗(On Diff #194157)

Parens around Other.Impl are not necessary.

110 ↗(On Diff #194157)

I don't like naming the variable the same as the type.
You could just use S as the variable name. That is ok with llvm style for small functions.

127 ↗(On Diff #194157)

"asserts" as it only fails in debug mode?

130 ↗(On Diff #194157)

I usually prefer free functions for binary operators (friends is fine).
It makes them symmetric.

155 ↗(On Diff #194157)

sNode ?
ie camelCase

clang/lib/Tooling/Refactoring/Stencil.cpp
45 ↗(On Diff #194157)

maybe this anon namespace should cover the functions above?

76 ↗(On Diff #194157)

If you define ==, imo you should define != too.
Otherwise just call it isEqual

94 ↗(On Diff #194157)

alignment.
(below too)

ymandel updated this revision to Diff 194549.Apr 10 2019, 11:13 AM
ymandel marked 13 inline comments as done.

Responded to comments and added wrapper for Stencil::cat in stencil namespace.

ymandel updated this revision to Diff 194550.Apr 10 2019, 11:14 AM

deleted some stray comments.

Thanks for the review. I've changed most of the things with only a few open questions. PTAL.

clang/include/clang/Tooling/Refactoring/Stencil.h
45 ↗(On Diff #194157)

so that each stencil part can append to a single string that we construct when evaluating the whole stencil. this was an (attempted?) optimization. if concatenating is about as efficient, I'd prefer that approach. What do you advise?

75 ↗(On Diff #194157)

Excellent! I guess I'm allergic to shared_ptr because of its atomics but its clearly a win here. Thanks!

127 ↗(On Diff #194157)

I thought assert() also fails in normal mode, based on my reading of the llvm docs, but it's not explicit about this: http://llvm.org/docs/ProgrammersManual.html#programmatic-errors

FWIW, I'm on the fence whether eval() should actually have the same signature and similarly just crash the program on errors. Its not clear there's any value to propogating the errors up the stack via Expected.

clang/lib/Tooling/Refactoring/Stencil.cpp
45 ↗(On Diff #194157)

the style guide advises against. I've been told to only put type definitions inside anon namespaces.

76 ↗(On Diff #194157)

Since I don't have a general need for these as operators, just went w/ the latter. Used the same style as evalData().

sbenza added inline comments.Apr 17 2019, 9:07 AM
clang/include/clang/Tooling/Refactoring/Stencil.h
45 ↗(On Diff #194157)

I think its fine as it is then.
I don't really think the performance would matter that much, but I don't see a reason to add all those copies unnecessarily.
Mention that Result is in an unspecified state in case of error.

127 ↗(On Diff #194157)

assert follows NDEBUG, which is not explicitly related to -Ox, but they usually go together.

Don't make it crash, please.
Otherwise it would be hard to have a refactoring service or alike.

101 ↗(On Diff #194551)

We could simplify this code if you change void push(T) to instead StencilPart wrap(T).
Then this could be:

Stencil S;
S.Parts = {wrap(std::forward<Ts>(Parts))...};
return S;
clang/lib/Tooling/Refactoring/Stencil.cpp
78 ↗(On Diff #194551)

Any particular reason to use function template specialization instead of function overloading?
The former is rare and thus harder to reason about.

(same for evalData below)

ymandel updated this revision to Diff 195602.Apr 17 2019, 11:08 AM
ymandel marked 9 inline comments as done.

Responded to comments, including changing call operator to return Expected<string> instead of string.

ymandel added inline comments.Apr 17 2019, 11:09 AM
clang/include/clang/Tooling/Refactoring/Stencil.h
127 ↗(On Diff #194157)

Right, you've convinced me. For some background: TextGenerator (in Transformer) originally return an Expected<std::string>, but Ilya suggested that I simplify to just a string and use assertions. The logic being that these are compiler-style tools. However, I agree that this simply doesn't work for running these in servers, which is a very real concern. It's also trivial to imagine a user mistyping a stencil (say, in web UI) which results in a failure in stencil eval. We want to return a nice error to the user, not crash the server.

So, I've adjusted the signature. I will (separately) change the definition of TextGenerator in Transformer.

101 ↗(On Diff #194551)

Much better. I believe the current design came about when StencilPart was move only. Thanks!

clang/lib/Tooling/Refactoring/Stencil.cpp
78 ↗(On Diff #194551)

Stylistic -- it forces you to explicitly declare the "overload set" so to speak. However, i'm not particularly attached -- if you think this a bad idea given its rarity, I'm happy to use overload sets. WDYT?

ymandel updated this revision to Diff 195606.Apr 17 2019, 11:31 AM

Convert template specialization to overload sets.

ymandel marked an inline comment as done.Apr 17 2019, 11:32 AM
ymandel added inline comments.
clang/lib/Tooling/Refactoring/Stencil.cpp
78 ↗(On Diff #194551)

Converted to normal overload sets (as per off-thread chat).

sbenza accepted this revision.Apr 18 2019, 7:37 AM
This revision is now accepted and ready to land.Apr 18 2019, 7:37 AM
This revision was automatically updated to reflect the committed changes.