Change the contract of GetStyle so that it returns an error when an error occurs (i.e. when it writes to stderr), and only returns the fallback style when it can't find a configuration file.
klimek hans djasper ioeric
- rC292174: clang-format: Make GetStyle return Expected<FormatStyle> instead of FormatStyle
rG3adfb6a3eed2: clang-format: Make GetStyle return Expected<FormatStyle> instead of FormatStyle
rL292174: clang-format: Make GetStyle return Expected<FormatStyle> instead of FormatStyle
This change works, and passes all tests; however, I have a bunch of questions, which you'll find in the diffs. I look forward to your feedback. Thanks!
Since we won't always return a FormatStyle, we no longer construct one here. Rather, instances are created in local scopes below when required.
I am unsure whether I should be returning these error strings at all. I say this because some of the functions called in this one will output to errs(), which means the caller doesn't get the full picture of what went wrong.
Maybe it's better to keep the original code that outputs to errs(), and just return an error code instead. Thoughts?
I must remember to set the Language each time. I could do this right after declaring the local variable (line 1907). (NOTE: will fix formatting - ah the irony!).
Better variable name IMHO. Condition checks both that it exists and is a regular file.
See the TODO comment above (which will be removed obviously). Is it an error if we find no suitable config for the input language? If that happens, should the fallback style be returned?
As I wrote above, by returning the string, the caller has the burden of outputting the error. More importantly, getStyle may already have written other errors to errs(), so I'm not sure this makes sense.
If we go with only returning error code, I suppose I'd use llvm::ErrorOr<FormatStyle> as the return type.
Good way to validate Expected<T>?
Below, I added tests for all the error cases. Only thing is that they may output to errs(); is that okay? Will that break test bots?
Test 7 below: there's an error case in getStyle where it fails to open the file. Not sure how to make it fail or whether it's even worth testing. (See Format.cpp line 1961)
One more thing I forgot to mention is that this change comes from a discussion we had on this other change: https://reviews.llvm.org/D27440 where @djasper agreed that "fallback-style should only be used when there is no .clang-format file. If we find one, and it doesn't parse correctly, we should neither use the fallback style nor scan in higher-level directories (not sure whether we currently do that).".
Is this still true?
Why do you need a template if you only use this function with one argument?
I'd probably keep the default style so that we don't need to set Language repeatedly below.
Nit: prefer no curly brace around one liners. Same below.
I think returning an Expected is the right approach. I think we should consider using customized format-relayed error codes (like that in the tooling::Replacements library) to provide richer information. For example, you could use customized error codes instead of llvm::inconvertibleErrorCode when constructing a StringError to provide additional information besides the error message. Other interfaces in the format library can potentially benefit from codes as well (e.g. ParseError can be merged).
Is this indentation intended?
I don't think getStyle should write to errs() if it returns an Error.
Thank you, @ioeric for your feedback!
No, it's not true, and I intend to update this comment. Thanks for pointing it out :)
In the end, I didn't need it. But since make_error accepts variable args, I figured this was more flexible. I can simplify it in the end, I only pass a single param.
The thing is, it made a lot of sense to have a default constructed FormatStyle before when we were sure to always return it. Since now we only return a Style under 2 specific conditions (found a config file, or did not return the FallbackStyle), otherwise return an error, I feel like it's more clear this way. But if you firmly disagree, I can go back.
To make returning the Style with current Language more clear, I could use a function that ties the two together and returns it (perhaps a local lambda), like StyleWithLanguage(Style, Language).
I agree with you, returning more specific error codes would be useful. However, I'm thinking we go at it incrementally. The point of this change is to have getStyle return when an error occurs so that we return non-zero from main, and more importantly, not use the fallback style when this happens.
In that respect, what I'm wondering is whether I should just leave the errs() output in getStyle, and simply return an error code, or keep what I've done (returning StringError) as a stepping stone in the right direction - that is, eventually returning customized format-relayed error codes, as you say.
No. I will definitely do a formatting pass. I'm using Visual Studio with the clang-format extension, which works alright, but Visual Studio fights against it with its own formatting. Indeed, the reason I even started contributing to clang-format was because I wanted to improve the VS extension. I've already pushed a few changes to it, and intend to do a little more to make the experience simpler. In any case, my next diff upload will have proper formatting everywhere.
I'd go with the old way with some comments, which should be clear enough.
I think the interface should no longer write to errs() if we want to introduce error handling (either with error codes or llvm::Error).
StringError does not really collide with error codes AFAICT - it can also carry error codes which you would have return directly. And even if you just return error codes, you would need some sort of customized error codes right?
@ioeric Do you have any thoughts on my question here? Say the user specified "-file" and a fallback style, and we find files but they are not suitable (for a different language), do we use the fallback style, since it's as if we found no config file. If so, then we wouldn't consider this an error, and therefore would not print nor return the message that we see here ("Configuration file(s) do(es) not support...").
The fact that the original code output to errs() here leads me to believe that this should be considered an error condition, in which case I should keep my change - that is, return an error here and _not_ return the fallback style.
Hello everyone, so after a few more tests, I've uncovered a bug and perhaps a different meaning for fallback style. First, the bug: if you set fallback style to "none", clang-format will perform no replacements. This happens because getStyle will first initialize its local Style variable to LLVM style, and then because a fallback style is set, will then set it to the "none" style, will ends up setting Style.DisableFormatting to true. After that, when we parse YAML (either from Style arg or a config file), we use the Style variable as the "template" for fields that haven't been set. In this case, the "none" fallback style causes DisableFormatting to remain true, so no formatting will take place.
As it happens, my first diff patch uploaded here fixed this issue by accident. Instead of reusing the same local Style variable, I declared one for each case where we'd need to parse. The fallback style case would use its own variable, FallbackStyle, which would not be used as the template style when parsing the YAML config.
What's interesting is that the way the code is originally written allows you to use fallback style as a way to set the "base" configuration for which the subsequently parsed YAML overlays. For example, if I don't set fallback style, the assumed base style is "LLVM", and any YAML parsed modifies this LLVM base style. But if I pass a fallback style of "Mozilla", then this becomes the base style over which the YAML overlays.
So to my mind, we have 2 approaches to fix the "none" style bug:
- Go with a similar approach to what I did originally; that is, we always assume LLVM as the base style, and make sure that the fallback style is not used as the base style, but rather only as the style to return if none is found. I think this is what FallbackStyle was originally intended for.
- Allow fallback style to maintain its current meaning - that is, as a way to set the base style when "style" is "file" or YAML. In this case, I believe the right thing is to treat FallbackStyle set to "none" as though no fallback style were passed in at all. Concretely, we might want t to modify getPredefinedStyle to return LLVM style when "none" is passed in, instead of what it does now. I personally think this is more confusing, and also introduces more risk.
Let me know what you think. If we go with option 1, I could fold the fix into this change.
Going over my diff, I realize that I didn't revert this local FormatStyle instance to make use of the top-level declared "FormatStyle Style" local variable. I will revert this one too.
I wrote the above comment before realizing there was a bug when fallback style is "none". As I explained in my long comment, we may actually want to keep this as a separate variable to fix the bug and ensure fallback style is only ever returned when no config is found, which is exactly what my change currently does.
This is a good YAQ, which IMO should be tackled in a separate patch. In this patch though, it might be easier to proceed by keeping the original behavior and leaving a FIXME. In general, reviewers like smaller patches with single purpose :)
Some nits. Some is almost good :)
BTW, do you have clang-tools-extra in your source tree? There are also some references in the subtree to the changed interface. It would be nice if you could also fix them in a separate patch and commit these two patches together (I mean, within a short period of time) so that you wouldn't break build bots.
References should be found in these files:
extra/change-namespace/ChangeNamespace.cpp extra/clang-move/ClangMove.cpp extra/include-fixer/tool/ClangIncludeFixer.cpp extra/clang-apply-replacements/tool/ClangApplyReplacementsMain.cpp extra/clang-tidy/ClangTidy.cpp
Maybe make this inline?
I'd state the problem with a specific solution only if I am sure it is the best solution.
Redundant braces. Same below.
There is a NewReplacements below which is also llvm::Expected<T>. FormatStyleOrError is a fine name, but to be we have been naming Expected types without OrError postfix, so I'd go without OrError postfix for consistency append OrError postfix to other expected variables. Personally, I find expected variables without OrError postfix easier to understand, especially in error checking. For example, if (!FormatStyleOrError) is a bit awkward to read while if (!FormatStyle) is more straight-forward IMO.
Same for other changes.
This is a bit strange... Just llvm::consumeError(Style.takeError()) if you don't bother to check the error message, e.g. with llvm::StringRef::starswith or RE match.
Please add proper checking as above for returned values.
I'll grab clang-tools-extras and make the second patch as you suggest. Btw, can you explain how I would avoid breaking build bots? I assume you mean that clang-tools-extras gets built separately against some version of clang, which gets auto-updated. When would I know the right time to push the second patch through?
Also, I assume I'd have to get this second patch approved before ever pushing the first, right?
Good point, will remove the solution.
Agreed. For consistency, would you prefer I also use 'auto' for the return type rather than llvm::Expected<format::FormatStyle> as is done for NewReplacements?
Yeah, I'll go with the consumeError.
Hmm, so I could replace the Style member of the fixture class with Expected<FormatStyle>, and then change all "Style." with "Style->" in the rest of the test file, or only in this specific test, I could store the result in a local Expected<FormatStyle>, check that it's valid, and then assign to Style. The latter is simpler; only question I have is how to name the local variable - can I go with StyleOrError? Style2?
The patch LGTM now. I'll accept both this and the one for clang-tool-extra when it is ready.
Regarding builbots, we have bots that continually run builds/tests (http://lab.llvm.org:8011/). Many buildbots test llvm and clang as well as clang-tools-extra (e.g. with ninja check-all) at some revision. Also note that although llvm/clang and clang-tools-extra are different repos, they do share the same revision sequence. So if clang-tools-extra is in a inconsistent state, many buildbots can fail and affect llvm/clang builds. Unfortunately, there is no atomic way to commit two revisions to two repositories, so we just commit them quickly one after another so that we do less damage. Do you have commit access to LLVM btw?
maybe "format is disabled" is clearer?
auto is fine.
Yeah, making this non-fixture sounds like a better idea. StyleOrError is fine here.
regarding this function and some other helper functions in this file:
(if this stuff is not used outside of this .cpp - but yeah, i assume i'm missing smth)
"Function names should be verb phrases (as they represent actions), and command-like function should be imperative. The name should be camel case, and start with a lower case letter (e.g. openFile() or isFoo())"