Page MenuHomePhabricator

[clang-format] Added new option IndentExternBlock
ClosedPublic

Authored by MarcusJohnson91 on Mar 6 2020, 6:05 PM.

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I agree that changing formatting randomly isn't a good idea, and I think converting AfterExternBlock to an enum is the way to go, but I'm just not sure on how it should be implemented.

Ok, I've got an idea to deprecate the AfterExternBlock option and map it to a new option, I'm gonna start implementing it right now.

MyDeveloperDay added a comment.EditedApr 3 2020, 3:37 AM

Ok, I've got an idea to deprecate the AfterExternBlock option and map it to a new option

Really? can't you just model IndentExternBlock as an enum? then say

bool shouldIndent = Style.IndentExternBlock==Indented ||
                                  ( AfterExternBlock && Style.IndentExternBlock==NotSpecified)

parseBlock(/*MustBeDeclaration=*/true,
        /*AddLevel=*/shouldIndent);

You can even keep "true" and "false" the the enumeration marshaller and you don't even have to change the documentation.

@MyDeveloperDay

but I'm also constantly surprised by how many of the enumeration cases started out as booleans only later to have to be converted to enums. The more I think about this the more I think the problem can probably be dealt with better by making it an enumeration. (even if you support true and false to mean "indent" and "don't indent"

I FULLY support all new options being required to be enums from now on, bools cause a whole lotta trouble when they have to be changed.

I've rewritten my patch, it works when manually testing it, now I'm just working on the automated tests.

A brand new patch should be up by either tonight or tomorrow.

MarcusJohnson91 edited the summary of this revision. (Show Details)

You need to regenerate the ClangFormatStyleOption.rst file using docs/tools/dump_format_style.py

clang/include/clang/Format/Format.h
1032

I'm a little confused here by the use of this enum, can't it be removed now?

clang/lib/Format/UnwrappedLineParser.cpp
1116–1121

can't you just use?

parseBlock(/*MustBeDeclaration=*/true, /*AddLevel=*/Style.BraceWrapping.AfterExternBlock);
1123–1125
parseBlock(/*MustBeDeclaration=*/true, /*AddLevel=*/Style.IndentExternBlock == FormatStyle::IEBS_Indent)
MyDeveloperDay requested changes to this revision.May 5 2020, 3:04 AM
This revision now requires changes to proceed.May 5 2020, 3:04 AM
MarcusJohnson91 marked 3 inline comments as done.Sat, May 16, 7:32 AM

I've fixed all of your comments as well as fixed the tests.

MarcusJohnson91 edited the summary of this revision. (Show Details)

Did everything you asked and did a littl bit of my own cleanup as well.

Removed forgotten comment from control logic of UnwrappedLineParser

MyDeveloperDay accepted this revision.Sat, May 16, 10:08 AM

LGTM just drop the Noindent it will encourage inconsistency people can read the manual (plus you don't check it in the tests)

clang/lib/Format/Format.cpp
215

I don't think that's necessary

This revision is now accepted and ready to land.Sat, May 16, 10:08 AM
MarcusJohnson91 marked an inline comment as done.Sat, May 16, 5:58 PM

Removed the lowercase Noindent case, that was a last minute addition I thought might make it a tad easier to work with, but you're right I didn't even test it, and honestly adding that complexity is just pointless at best.

Removed.

LGTM

So what's the next step? I've never committed to LLVM before.

LGTM

So what's the next step? I've never committed to LLVM before.

In my mind you have two choices

  1. require commit access
  2. get someone else to land it

If you think you'd like to continue to be involved and maybe help out from time to time (especially with reviews which is where we REALLY lack people) then I would recommend 1)

but I'm happy to do 2) for you if not, but you have to be prepared to support the patch a little afterwards. (no fire and forget please ;-) )

MyDeveloperDay added a comment.EditedSun, May 17, 8:50 AM

Please clang-format the patch, I'm also getting a crash when running the tests, please make sure they pass.

Please add yourself to the pre-merge testing project so your reviews get checked before updating the patch

https://reviews.llvm.org/project/view/78/

clang/include/clang/Format/Format.h
1619

Something is not quie right here, this text isn't ending up in the ClangFormatStyleOptions.rst

Something is not quite right here, this text isn't ending up in the ClangFormatStyleOptions.rst

You're right, I didn't catch that before, turns out having a comment before the variable is required for dump_format_style.py to work.

I've fixed this, I'm still working on the tests, and I'll clang-format the files when it's all done.

Please clang-format the patch, I'm also getting a crash when running the tests, please make sure they pass.

I'm not sure why the tests crash, I know that when I manually test all the options for IndentExternBlock , and when testing IndentExternBlock: AfterExternBlock and setting BraceWrapping.AfterExternBlock, everything works.

i just get gibberish about loading the default LLVM style failed, and a nonsensical hex dump (0xFF 0xFE then a bunch of NULLs)

I honestly thought these crashes were unrelated.


Please add yourself to the pre-merge testing project so your reviews get checked before updating the patch

https://reviews.llvm.org/project/view/78/

When I go there and click Watch it says:

You Shall Not Pass: #pre-merge_beta_testing
You do not have permission to edit this object.
Users with the "Can Edit" capability:
Administrators can take this action.


As for my testing, I'm doing both manual and autmoated testing, automated with ninja check all

and manual testing with main.c:

#ifdef __cplusplus

extern "C" {
#endif

void blah1(void);

#ifdef __cplusplus
}
#endif

extern "C++" {

void blah2(void) {
    int one = 1;
}

}

and here's the command line:

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: true}" /Users/Marcus/Desktop/Test_Clang-Format.c

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: false}" /Users/Marcus/Desktop/Test_Clang-Format.c

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: Indent}" /Users/Marcus/Desktop/Test_Clang-Format.c

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: NoIndent}" /Users/Marcus/Desktop/Test_Clang-Format.c

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: AfterExternBlock, BraceWrapping: {AfterExternBlock: false}}" /Users/Marcus/Desktop/Test_Clang-Format.c

~/Source/External/LLVM_BUILD/bin/clang-format -i -style="{IndentWidth: 4, IndentExternBlock: AfterExternBlock, BraceWrapping: {AfterExternBlock: true}}" /Users/Marcus/Desktop/Test_Clang-Format.c

tho now that I'm manually testing it again (I really only used manual testing to make sure the options were accepted, to iterate more quickly), it looks like the AfterExternBlock: true option isn't working, but false is.

if thats true how didn't the automated tests catch it?

Fixed the generation of ReleaseNotes.rst

clang/include/clang/Format/Format.h
1619

you need to add a comment before the actual variable (you can use your own choice of text)

/// Option to control indenting of code inside an extern block
IndentExternBlockStyle IndentExternBlock;

Doing so will pull in the enum values into the ClangFormatStyleOptions.rst when running the docs/tools/dump_format_style.py and you'll get a different diff

-  * ``bool AfterExternBlock`` Wrap extern blocks.
+  * ``bool AfterExternBlock`` Wrap extern blocks; Partially superseded by IndentExternBlock

     .. code-block:: c++

@@ -1680,6 +1680,49 @@ the configuration (without a prefix: ``Auto``).
        plop();                                  plop();
      }                                      }

+**IndentExternBlock** (``IndentExternBlockStyle``)
+  Option to control indenting of code inside an extern block
+
+  Possible values:
+
+  * ``IEBS_NoIndent`` (in configuration: ``NoIndent``)
+    Does not indent extern blocks.
+
+    .. code-block:: c++
+
+     extern "C" {
+     void foo();
+     }
+
+  * ``IEBS_Indent`` (in configuration: ``Indent``)
+    Indents extern blocks.
+
+    .. code-block:: c++
+
+     extern "C" {
+       void foo();
+     }
+
+  * ``IEBS_AfterExternBlock`` (in configuration: ``AfterExternBlock``)
+    Backwards compatible with AfterExternBlock's indenting.
+    AfterExternBlock: true
+
+    .. code-block:: c++
+
+    extern "C"
+    {
+      void foo();
+    }
+    AfterExternBlock: false
+
+    .. code-block:: c++
+
+    extern "C" {
+    void foo();
+    }
+
+
+
 **IndentGotoLabels** (``bool``)
   Indent goto labels.

Sorry forgot to submit the comment issue, seems like you worked that bit out already

if thats true how didn't the automated tests catch it?

If you are not in the premerge testing group the unit tests won't be run. (which is why I added you)

When I run the gtest it fails, I cannot see why.. but one suggestion I have is to drop the whole true/false its not like you have any backwards compatibility.

ok it crashes because you are not initializing IndentExternBlock in the getLLVMStyle() function

LLVMStyle.IndentExternBlock = FormatStyle::IEBS_NoIndent;

You cannot leave it uninitialized, now what the correct value is in my view may be an issue

I think it needs to be:

LLVMStyle.IndentExternBlock = FormatStyle::IEBS_NoIndent;

I'm wondering if the GNU style also needs

Expanded.IndentExternBlock = FormatStyle::IEBS_Indent;

I've got the indenting to work manually now as well, the issue was you need to have BreakBeforeBraces: Custom in the inline style for it to pick up BraceWrapping.AfterExternBlock's value.

Now I'm working on the automated tests, thanks for the tip about initializing, I'll look into that.

I've initialized all styles to either AfterExternBlock, if there was a BraceWrapping block, or NoIndent if there wasn't.

re-running my tests locally.

Added the style initializers, moved IEBS_AfterExternBlock to be the first enum value so that it's zero, that way the bool logic works.

Regenerated the docs as well, and also clang-formatting the files I've touched.

I reran the tests before creating this diff and everything worked.

MarcusJohnson91 added a comment.EditedMon, May 18, 6:49 PM

As for crashes, none of them seem relevant; I'm on MacOS, the windows ABI crash seems especially irrelevent.

opt crashed, there were no arguments, and abort() was called.

llvm-lto2 crashed, not-prevailing.ll.tmp1-3.bc was the cause

llvm-dwarfdump crashed, arguments: -debug-line /Users/Marcus/Source/External/LLVM_BUILD/test/Object/Output/invalid.test.tmp3

llvm-as crashed, arguments: /Users/Marcus/Source/External/LLVM/llvm/test/Assembler/datalayout-invalid-stack-natural-alignment.ll

llvm-mc crashed, arguments: -triple i386-pc-win32 -filetype=obj

llvm-readobj crashed, arguments: -r /Users/Marcus/Source/External/LLVM/llvm/test/Object/Inputs/invalid-bad-section-address.coff

llc crashed, arguments: -stop-before=nonexistent -o /dev/null

This is totally fine, now I'm just concerned by the choice of defaults, I really don't know if we want to change the defaults for all the styles, I don't want to break all those people using it

One way might be if we canvas opinion from those developers who work on some of those proejcts for example @sylvestre.ledru, @Abpostelnicu what impact might this have on the Mozilla sources (if any?)

clang/lib/Format/Format.cpp
728

I'm sorry but I feel these are changing the previous default which as I understood would indent ONLY if Style.BraceWrapping.AfterExternBlock == true

I think in all cases other than GNU this was false, isn't that correct?

739

I think you can remove this to avoid confusion that you are changing from the default LLVM style

744

Isn't this changing the default?

759

Isn't this changing the default?

774

Ok this one feel correct.

970

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

1105

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

1154

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

1176

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

1194

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

1235

everyone inherits from LLVM so no need for this it only makes people think its different from the base style

MarcusJohnson91 marked 4 inline comments as done.Tue, May 19, 3:24 AM
MarcusJohnson91 added inline comments.
clang/lib/Format/Format.cpp
728

I chose IEBS_AfterExternBlock here, because it already uses the BraceWrapping.AfterExternBlock style, that way it will still use AfterExternBlock: true value a few lines lower.

739

k, that makes sense; I figured nothing was specified so it should be set to no, but I can remove it also.

744

Expanded.BraceWrapping.AfterExternBlock = true; is a few lines lower, so it will use that value.

Maybe I should move this option to right below Expanded.BraceWrapping.AfterExternBlock = true;?

970

ok, I can remove the inherited ones too.

MarcusJohnson91 updated this revision to Diff 264839.EditedTue, May 19, 3:37 AM

Ok, I've removed the inherited ones, and also removed the times I was setting a style when there wasn't one before.

also I moved the IEBS_AfterExternBlock line to right underneath the BraceWrapping.AfterExternBlock = true/false; line so it's easier to see.

and reformatted ofc.


btw, in .BraceWrapping = {true, false); blocks, AfterExternBlock is the 9th option, I checked the BraceWrapping enum.

All this confusion over the defaults is because we don't have the unit tests to check the default of each style. If we had that we could have a before/after conversation more easily

Nit: also please mark comments done once you have addressed them.

clang/lib/Format/Format.cpp
734

didn't see that either yep thats correct

744

totally didn't see that.. yep thats correct

754

thats also correct, sorry for not seeing that

768

ok thats correct too

836

is this one correct? AfterExternBLock is false in this case correct? should this be NoIndent?

btw, in .BraceWrapping = {true, false); blocks, AfterExternBlock is the 9th option, I checked the BraceWrapping enum.

This will become clearer I hope when I land D79325: [clang-format] [PR42164] Add Option to Break before While

Sorry to "go around the houses" but we'll get there in the end...I think we are close

MarcusJohnson91 marked 10 inline comments as done.Tue, May 19, 6:10 AM
MarcusJohnson91 marked 5 inline comments as done.Tue, May 19, 6:16 AM
MarcusJohnson91 added inline comments.
clang/lib/Format/Format.cpp
836

Yes AfterExternBlock is set to false here, but IndentExternBlock is being set to IEBS_AfterExternBlock, so the code falls back to parsing it as if IndentExternBlock wasn't set, and AfterExternBlock was set.

so IEBS_AfterExternBlock works for both true and false values.

Setting it to NoIndent would change the codepath to the new one and it would lose the newline between extern "C" and { in the process.

MarcusJohnson91 marked an inline comment as done.Tue, May 19, 6:17 AM

Sorry to "go around the houses" but we'll get there in the end...I think we are close

I think we're close too.

Your other comment was interesting, about testing the styles to make sure they haven't changed with these new options, that didn't occur to me.

I think it might be worth looking into.

I'm just not really sure how we could do such a thing, or if after the option is submitted if it'd really be worth it to have run because I don't really think this option will be further modified, but the original AfterExternBlock guy didn't think anyone would want to expand that either and as a result we had to work around some of the issues with his design so who can really tell what the future holds?

MyDeveloperDay accepted this revision.Tue, May 19, 7:23 AM

I think this LGTM now...

Just fixed the formatting of the ReleaseNotes.rst file, the extern blocks were slightly askew, and it might've made it a bit confusing

Made the IndentExternBlockStyle enum comments a bit clearer, and regenerated the .rst file

Format.h: indented the `AfterExternBlock: true` example code snippet with 4 spaces like the Indent option so it's more visible and matches.

I think it's perfect now.

Let's first see we don't break anything on mozilla.

Abpostelnicu accepted this revision.Tue, May 19, 8:12 AM

If you want me to land this for you, I'd feel more comfortable landing it if:

a) We can land D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults first
b) The Mozilla team have tested the impact (they clang-format their entire code base I think)

If you want me to land this for you, I'd feel more comfortable landing it if:

a) We can land D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults first
b) The Mozilla team have tested the impact (they clang-format their entire code base I think)

I'm ok with accepting commit access, and I agree lets get D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults in, and see if Mozilla, Microsoft, Google, etc has any comments; I'm just not sure of who to ping.

Is there anything else that D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults needs? it looked pretty well fleshed out.

If you want me to land this for you, I'd feel more comfortable landing it if:

a) We can land D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults first
b) The Mozilla team have tested the impact (they clang-format their entire code base I think)

I'm ok with accepting commit access, and I agree lets get D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults in, and see if Mozilla, Microsoft, Google, etc has any comments; I'm just not sure of who to ping.

Is there anything else that D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults needs? it looked pretty well fleshed out.

I need an Accept on D80214: [clang-format] Set of unit test to begin to validate that we don't change defaults and I think @Abpostelnicu would have let us know if it failed.

This revision was automatically updated to reflect the committed changes.
MyDeveloperDay added inline comments.Wed, May 20, 1:49 PM
clang/include/clang/Format/Format.h
1531

@MarcusJohnson91 I had to make a couple of minor changes before committing

  1. it needed rebasing (because of changes from this morning)
  2. these \code statements need to be indented by 3 spaces otherwise it can't see the beginning and end of the code segment and would have failed the doc build
  3. There were a couple of extra blank lines in the release notes (I actually think that was my mistake this morning) but it failed the rest lint check that I use

The tests ran ok especially our new one, so I don't think this has broken the default styles, we'll see if anyone else complains

Thank you for the patch