This is an archive of the discontinued LLVM Phabricator instance.

[clang-format] Add possibility to be based on parent directory
ClosedPublic

Authored by HazardyKnusperkeks on Dec 27 2020, 1:38 PM.

Details

Summary

This allows the define BasedOnStyle: InheritParentConfig and then clang-format looks into the parent directories for their .clang-format and takes that as a basis.

My use case is to remove the column limit for my tests.

Diff Detail

Event Timeline

HazardyKnusperkeks requested review of this revision.Dec 27 2020, 1:38 PM
HazardyKnusperkeks created this revision.
Herald added a project: Restricted Project. · View Herald TranscriptDec 27 2020, 1:38 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript

I'm a little confused why this is needed as clang-format always read up the directory tree until it see a .clang-format file, perhaps I don't quite understand the use case. Can't you just have a different .clang-format in the subdirectory?

I'm a little confused why this is needed as clang-format always read up the directory tree until it see a .clang-format file, perhaps I don't quite understand the use case. Can't you just have a different .clang-format in the subdirectory?

Yes of course I can, but I don't want to copy everything except the one thing I want to change.

I have one .clang-format in my main directory and since I only want to change the column limit for my tests I want to write a .clang-format in the test directory with the content:

BasedOnStyle: File
ColumnLimit: 0

So when I change (or better add) a value in my main .clang-format it will also apply to my tests, without touching their file.

njames93 added a comment.EditedDec 27 2020, 3:12 PM

I'm a little confused why this is needed as clang-format always read up the directory tree until it see a .clang-format file, perhaps I don't quite understand the use case. Can't you just have a different .clang-format in the subdirectory?

It's my understanding this is akin to the InheritParentConfig option found in .clang-tidy.
The idea is you can put a .clang-format file in your project root, then for specific directories you can add another .clang-format file that modifies that root .clang-format
The example here is a test folder that needs extended lines but want the rest of the formatting to match the root folder without having to copy the contents of the root and make the small edit.

We have it quite lucky here at LLVM where for tests we can just use:

BasedOnStyle: LLVM
ColumnLimit: 0

However for other projects with their own style they could then use:

# Copy the root directory clang.format config
BasedOnStyle: file
ColumnLimit: 0

This would also make modifying the formatting style of a project much easier as you would only need to touch the root directory .clang-format file in most cases.

With this implementation I'm not too sure on the specifics though. .clang-format files are more complex to parse than .clang-tidy files so there's more edge cases to consider.
Also it appears this wouldn't handle the case of invoking clang-format with the style set inline on the command line.
clang-format <file> --style='{BasedOnStyle: File, ColumnLimit: 80}'
Should this then check the directory for .clang-format files?

clang/lib/Format/Format.cpp
3014

This seems a very inefficient way of doing this, getStyle does a lot of things we simple don't need to worry about. Maybe the file loading part should be extracted out into its own function, that can then recursively call itself. Or use the clang-tidy approach of storing configurations in a stack (will have to be unparsed cause how clang-format works) and then parse them on top of each other.

I'm a little confused why this is needed as clang-format always read up the directory tree until it see a .clang-format file, perhaps I don't quite understand the use case. Can't you just have a different .clang-format in the subdirectory?

It's my understanding this is akin to the InheritParentConfig option found in .clang-tidy.
The idea is you can put a .clang-format file in your project root, then for specific directories you can add another .clang-format file that modifies that root .clang-format
The example here is a test folder that needs extended lines but want the rest of the formatting to match the root folder without having to copy the contents of the root and make the small edit.

We have it quite lucky here at LLVM where for tests we can just use:

BasedOnStyle: LLVM
ColumnLimit: 0

However for other projects with their own style they could then use:

# Copy the root directory clang.format config
BasedOnStyle: file
ColumnLimit: 0

This would also make modifying the formatting style of a project much easier as you would only need to touch the root directory .clang-format file in most cases.

Exactly.

With this implementation I'm not too sure on the specifics though. .clang-format files are more complex to parse than .clang-tidy files so there's more edge cases to consider.
Also it appears this wouldn't handle the case of invoking clang-format with the style set inline on the command line.
clang-format <file> --style='{BasedOnStyle: File, ColumnLimit: 80}'
Should this then check the directory for .clang-format files?

Initially I thought it shouldn't. But now I think maybe it should.

clang/lib/Format/Format.cpp
3014

Yeah I know that this is double work, but I thought it only affects the case BasedOnStyle: File and keeps the "normal" way untouched.

If I do build a stack I would actually parse all .clang-format files up to the root directory and in the not BasedOnStyle: File case throw away the results, wouldn't I? The question is how often someone has a .clang-fomrat in one of his parent directories which is not used because it got overwritten.

But I can do that.

Or is there an easy way to just check the BasedOnStyle value? As far as I understood the yaml code it parses the whole file into an internal structure and we only query this structure, right?

MyDeveloperDay added a comment.EditedDec 29 2020, 3:42 AM

I like what you are suggesting, do you think BasedOnStyle: File is the best terminology as the term File is used elsewhere to mean read from the .clang_format file, how about

BasedOnStyle: Inherit
BasedOnStyle: InheritFromParent
BasedOnStyle: InheritFromFile

or why not just

BasedOnStyle: InheritParentConfig

to match clang-tidy?

I like what you are suggesting, do you think BasedOnStyle: File is the best terminology as the term File is used elsewhere to mean read from the .clang_format file, how about

BasedOnStyle: Inherit
BasedOnStyle: InheritFromParent
BasedOnStyle: InheritFromFile

or why not just

BasedOnStyle: InheritParentConfig

to match clang-tidy?

I will update it to InheritParentConfig.

njames93 added inline comments.Dec 30 2020, 6:56 AM
clang/lib/Format/Format.cpp
3014

Yeah I know that this is double work, but I thought it only affects the case BasedOnStyle: File and keeps the "normal" way untouched.

If I do build a stack I would actually parse all .clang-format files up to the root directory and in the not BasedOnStyle: File case throw away the results, wouldn't I? The question is how often someone has a .clang-fomrat in one of his parent directories which is not used because it got overwritten.

I'm not saying start at the route directory and work down to the child. Still start at the child, but when you parse a configuration that has needs the parent, put that unparsed config into a stack, then carry on. At the end then you can join all those configurations

But I can do that.

Or is there an easy way to just check the BasedOnStyle value? As far as I understood the yaml code it parses the whole file into an internal structure and we only query this structure, right?

It should be possible, not an expert but I'm not sure it like being traversed multiple times normally, though a workaround should be feasible.

HazardyKnusperkeks marked 2 inline comments as done.
HazardyKnusperkeks edited the summary of this revision. (Show Details)
  • Rebased
  • Renamed to InhteritParentConfig
  • Reworked to remove the recursion of getStyle
  • Now supports the command line style options
  • Test cases extended for the latter

What I can't easily tell from the tests is if you are overriding any styles defined in the parent with a local style.

clang/include/clang/Format/Format.h
57

I don't quite understand this last sentence

What I can't easily tell from the tests is if you are overriding any styles defined in the parent with a local style.

Yes I do, but I will try to make it clearer in the tests.

clang/include/clang/Format/Format.h
57

If I would use /// it would end up in the ClangFormatStyleOptions.rst. I only want to make clear it is no mistake that it only got // while all other members are documented with ///.

What I can't easily tell from the tests is if you are overriding any styles defined in the parent with a local style.

Yes I do, but I will try to make it clearer in the tests.

Okay after rechecking the code (it is a bit ago that I wrote it), what more do you need? I compare the parsed style against a base style (Google, to show that it’s not the default LLVM) modified with the values from the overwrite. Only for the last checks where I basically check the same over and over again, just trough different means I use a stored style.

Found and fixed the case when no BasedOnStyle apart from InheritParentConfig is used.

Rebased and Ping.

curdeius added inline comments.Feb 1 2021, 9:35 AM
clang/docs/ClangFormatStyleOptions.rst
160

Typo: fount -> found.

160

Why should it fallback to LLVM and not the fallback style?
It seems to me that it's a documentation problem.

HazardyKnusperkeks marked 3 inline comments as done.Feb 1 2021, 1:38 PM
HazardyKnusperkeks added inline comments.
clang/docs/ClangFormatStyleOptions.rst
160

Yap, fixed that.

HazardyKnusperkeks marked an inline comment as done.

Fixed documentation (and rebased).

MyDeveloperDay accepted this revision.Feb 2 2021, 1:32 AM
This revision is now accepted and ready to land.Feb 2 2021, 1:32 AM

As a follow up it may be wise to pass a diag handler to parseConfiguration as when we parse it a second time, we probably want to disregard any warnings (like unknown key) detected as they will have been printed on the first pass.

clang/lib/Format/Format.cpp
3011–3017

Nit: just use a for loop over a lambda.

3051

Nit: Add a void cast to silence unused warnings in release builds.

HazardyKnusperkeks marked 2 inline comments as done.Feb 2 2021, 12:40 PM

As a follow up it may be wise to pass a diag handler to parseConfiguration as when we parse it a second time, we probably want to disregard any warnings (like unknown key) detected as they will have been printed on the first pass.

Will try to look into it.

clang/lib/Format/Format.cpp
3011–3017

Did not know about llvm::reverse.

HazardyKnusperkeks marked an inline comment as done.

Silence unused variable warnings in release and using llvm::reverse with range based for.

This revision was landed with ongoing or failed builds.Feb 14 2021, 10:56 AM
This revision was automatically updated to reflect the committed changes.
zwliew added a subscriber: zwliew.Dec 26 2021, 7:50 PM
zwliew added inline comments.
clang/lib/Format/Format.cpp
3043

Is there a reason behind limiting the number of children configs to 1 in this case? When the fallback is not used, more than 1 children configs are allowed (line 3036).

Sorry for digging this up, I came across this seemingly arbitrary decision when working on some changes to https://reviews.llvm.org/D72326

HazardyKnusperkeks marked an inline comment as done.Dec 27 2021, 4:02 AM
HazardyKnusperkeks added inline comments.
clang/lib/Format/Format.cpp
3043

Yeah but it doesn't happen, there is at most only one .clang-format in the parent directory path which is found. So we assert on that and then don't need to loop over what is exactly one element big.

zwliew added inline comments.Dec 27 2021, 4:25 AM
clang/lib/Format/Format.cpp
3043

Thanks for the reply. However, I don't think that's true. Yes, it's only possible to find one .clang-format file in the first parent directory. But if it has BasedOnStyle: InheritParentConfig set, then we could find another .clang-format file in the "grandparent" directory. In this case, we'll have 2 elements in ChildFormatTextToApply, but only the very first one will actually be applied.

To illustrate, suppose we have the following file structure:

- .clang-format
- foo
  |- .clang-format
  |- input.cpp

Both .clang-format files have BasedOnStyle: InheritParentConfig set. When running clang-format --style=file foo/input.cpp, only the inner config is applied on the fallback style, while the outer config is ignored. When testing on a debug build, I encountered a crash due to the failed assert. When removing the assert, and using a loop to apply the configs, both the inner and outer configs are applied, which I believe is the expected behaviour.

HazardyKnusperkeks marked 2 inline comments as done.Dec 28 2021, 11:38 AM
HazardyKnusperkeks added inline comments.
clang/lib/Format/Format.cpp
3043

I can not explain it to you anymore, I would have to dig into it again. But if my tests are correct everything works. You can see that for Test 9.4 I create a .clang-format in /e/sub/sub/ which is based on /e/sub/.clang-format which again is based on /e/.clang-format.

The tests do not fail, the parsed style is as the three files suggest, and the assert holds. I'm pretty sure I have thought about that case, it happens in some kind of recursion.

zwliew added inline comments.Dec 28 2021, 3:52 PM
clang/lib/Format/Format.cpp
3043

I took a look through Test 9.4, and it doesn't test the case I'm thinking of, as it doesn't execute the fallback case.

In Test 9.4 the file structure is as follows:

- e/
  |- .clang-format (BasedOnStyle: Google) <-- outermost config
  |- sub/
    |- .clang-format (BasedOnStyle: `InheritParentConfig)
    |- sub/
      |- .clang-format (BasedOnStyle: InheritParentConfig)
      |- code.cpp

The reason it doesn't execute the fallback case is that the outermost config file doesn't have BasedOnStyle: InheritParentConfig set.

On the other hand, in the following directory structure, the fallback case would execute, the assert would fail (on debug builds), and only the innermost config would be applied (on release builds):

- e/
  |- .clang-format (BasedOnStyle: InheritParentConfig) <-- outermost config
  |- sub/
    |- .clang-format (BasedOnStyle: `InheritParentConfig)
    |- sub/
      |- .clang-format (BasedOnStyle: InheritParentConfig)
      |- code.cpp

In order to verify my claims, I think I should write a new test case for this. However, I do not know how to run test cases for only clang-format. Is there a way to do so? Thanks!

zwliew added inline comments.Dec 28 2021, 7:54 PM
clang/lib/Format/Format.cpp
3043

I added a new test case for the latter scenario and ran all the regression test cases with make check-clang. The test case does indeed fail due to the assertion failure. I've updated https://reviews.llvm.org/D72326 with the test case and the fix for it. Please have a look, thanks!

HazardyKnusperkeks marked an inline comment as done.Dec 29 2021, 7:07 AM
HazardyKnusperkeks added inline comments.
clang/lib/Format/Format.cpp
3043

In that case thanks for catching that.

But I think it should be a different patch for the fix. Do not put a bug fix with a new feature in one commit.
There should be a target FormatTests, which builds and runs only the tests for clang-format.