This is an archive of the discontinued LLVM Phabricator instance.

FileCheck [9/12]: Add support for matching formats
ClosedPublic

Authored by thopre on Apr 7 2019, 4:22 PM.

Details

Summary

This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for selecting a
matching format to match a numeric value against (ie. decimal, hex lower
case letters or hex upper case letters).

This commit allows to select what format a numeric value should be
matched against. The following formats are supported: decimal value,
lower case hex value and upper case hex value. Matching formats impact
both the format of numeric value to be matched as well as the format of
accepted numbers in a definition with empty numeric expression
constraint.

Default for absence of format is decimal value unless the numeric
expression constraint is non null and use a variable in which case the
format is the one used to define that variable. Conclict of format in
case of several variable being used is diagnosed and forces the user to
select a matching format explicitely.

This commit also enables immediates in numeric expressions to be in any
radix known to StringRef's GetAsInteger method, except for legacy
numeric expressions (ie [[@LINE+<offset>]] which only support decimal
immediates.

Copyright:

  • Linaro (changes up to diff 183612 of revision D55940)
  • GraphCore (changes in later versions of revision D55940 and in new revision created off D55940)

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
MaskRay added inline comments.Jul 14 2019, 3:49 AM
llvm/lib/Support/FileCheck.cpp
55

I think if (StrVal.getAsInteger(Hex ? 16 : 10, Value)) is just as clear. (Or (Hex ? 16 : 10). The empty line above can also be deleted.

100

LeftFormat.Valid ? LeftFormat : RightFormat;

163

Delete the helper and inline it into the call sites.

265–266

I think if (!Expr.consumeInteger(AO == Literal ? 10 : 0, LiteralValue)) is just as clear as the current one.

arichardson added inline comments.Jul 24 2019, 2:13 PM
llvm/lib/Support/FileCheck.cpp
36

Maybe [0-9]+ instead of "[[:digit:]]+"?

80–81

if (Expression && Expression->getAST()) is a bit shorter and just as readable.

jhenderson added inline comments.Jul 25 2019, 6:29 AM
llvm/docs/CommandGuide/FileCheck.rst
609

Perhaps worth stating "61680" in hex, rather than decimal.

611

I'm getting confused by the syntax here not matching the syntax above for defining a numeric variable. Aren't the two essentially the same syntax, just with different parameters missing? If I'm not mistaken they can be unified to [[#%<fmtspc>,<NUMVAR>:<expr>]] where <expr> says what this matches against (if specified), <NUMVAR> says what numeric variable to store the result in (if specified) and fmtspec defines the format of the expression (if any) and that stored in the variable (if any).

Aside: does #%X match a hex number, but not store it?

615

"constraint, if any, and defaults to". The later sentence handles the "all the same" bit by talking of conflicts.

llvm/include/llvm/Support/FileCheck.h
51 ↗(On Diff #205764)

I would call this UpperCase or Capitalize. If it is only ever used for hex values (I'm not sure if there are plans for e.g. uppercasing string values later on) storing an enumeration with HexUpper and HexLower, Unsigned values could also make sense.

I agree that an enum seems like the right way forward here, possibly with Conflict and Unspecified as other values in that enum.

67 ↗(On Diff #205764)

"regexp"? I assume that should be "regular expression pattern"?

84 ↗(On Diff #205764)

Is there going to be an ODR violation here, due to these being instantiated in the header?

152 ↗(On Diff #205764)

Why shared_ptr? Shouldn't an expression be the sole owner of its AST?

154 ↗(On Diff #205764)

i.e -> i.e.

161 ↗(On Diff #205764)

ie. -> i.e.

Perhaps Format should take a default argument for an unsigned decimal?

The last clause ("set it to default one"...) then becomes unnecessary.

170 ↗(On Diff #205764)

ie -> i.e.

(i) its explicit format, if any, otherwise (ii) its implicit format, if any, otherwise (iii) the default format.

186 ↗(On Diff #205764)

If the expression is empty, Expression points to a FileCheckExpression instance...

264 ↗(On Diff #205764)

if the implicit

265 ↗(On Diff #205764)

conflict, and FormatNone

llvm/lib/Support/FileCheck.cpp
66

I believe you could do this in the initializer list:

: AST(AST)
, Format(Format.Valid ? Format : FormatUnsigned) {}
80–81

Readability is in the eye of the beholder. I personally find it more readable to compare against nullptr, because it is then clear that Expression is a pointer, not a boolean.

llvm/test/FileCheck/numeric-defines.txt
3–5

I feel like you're losing coverage here: where are command-line defined numeric variables tested with an implicit format specifier? I'd have a separate test for format specifiers, that explicitly focus on testing those and nothing else.

llvm/test/FileCheck/numeric-expression.txt
45–49

Do you really need every one of these test cases? It feels like the last one would be enough (and one with no spaces, which you have earlier on). Same goes for other instances like this elsewhere.

53

Should this be USE EXPL FMT IMPL MATCH?

139

Nit: missing trailing full stop.

165

Nit: missing trailing full stop.

The grammar of this statement needs significant improvement. How about "Explicitly specified format can override conflicting implicit formats."

This test case should be after the one that shows that explicit format specifiers override non-conflicting implicit ones, and probably also after a test case showing what happens when they conflict.

172

Nit: missing trailing full stop.

What is a "conversion matching format"?

184

Nit: missing trailing full stop.

198

In one of the other changes, you went to some effort to improve the readability of these using --strict-whitespace, so that the '^' line up with the correct thing. Could you replicate these improvements in this patch too, please?

332

Missing trailing full stop.

333–334

You're being inconsistent here with your '-' versus '--' switch usage. Please standardise to one or the other throughout the test.

340

This message is incomplete - which variables? What were their format specifiers?

llvm/test/FileCheck/string-defines-diagnostics.txt
30–31

Please re-add the second dash, to remain consistent with other tests in this file (why did you change it?)

llvm/unittests/Support/FileCheckTest.cpp
247–248

FileCheckExpression NumVarExpr(nullptr, FormatUnsigned);

Same kind of comment goes throughout changes in this file.

248–249

Why make_shared and not make_unique? In fact, why is this a smart pointer at all? Why not just create this on the stack, i.e. FileCheckNumericVariable FooVar (1, "FOO", &NumVarExpr);

Same comment goes in all sorts of places elsewhere.

thopre updated this revision to Diff 233319.Dec 11 2019, 4:48 AM
thopre marked 44 inline comments as done.

Address most comments, rebase.

thopre planned changes to this revision.Dec 11 2019, 4:48 AM

Uploading the current status of this patch. I know there are still issues (in particular add more unittests), and I'll address them.

llvm/docs/CommandGuide/FileCheck.rst
611

That's a mistake, well spotted. Yes #%X matches an hex number in capital letters but does not store it.

llvm/test/FileCheck/string-defines-diagnostics.txt
30–31

The patch initially only changed ERRCLIFMT into ERRCLINAME. Later when rebasing it after the updated previous patch with the -- syntax I missed the - to -- change when resolving conflicts in this file. Same for all other issues you mention in an earlier patch and end up in a later patch: either incorrect merging (happens a lot when the conflict spans many lines because it's easy to miss something) or it's new code and I forgot to apply some fix on there as well (for example the eg -> e.g. fixes).

In other word, my apologies for making you repeat your point, it's due to being absent-minded at times.

I've not looked at the tests in the latest version, but I've made a number of style and comment comments for you.

llvm/docs/CommandGuide/FileCheck.rst
615

default -> defaults

llvm/lib/Support/FileCheck.cpp
30–32

Probably the more canonical way of writing these sort of switch statements is:

switch (Value) {
case 1:
...
case 2:
...
default:
  llvm_unreachable("unknown format value");
}

(If the code isn't truly unreachable on the other hand, it should be an error, not an assertion)

99

I don't know about anybody else, but LeftFormat != NoFormat ? LeftFormat : RightFormat is more readable to me (and removes the need to hard-code the enum value).

llvm/lib/Support/FileCheckImpl.h
33–34

The comments in this class and its enum need updating following the recent changes to it.

Also, "printed into for matching" doesn't really make sense. Probably something like "converted into" would make more sense.

42

Perhaps change this to "Used when there are..."

68

explicit?

104–105

either implicit format of this AST ... or NoFormat if the AST has...

106

made -> made up

178

"without an explicit"

I don't think you need the rest of the sentence after that: the concept of a conflict isn't important to the variable.

Some minor comments, but generally looking good to me.
We have a few downstream tests that will really benefit from matching Hex upper/lower so looking forward to this landing.

llvm/lib/Support/FileCheck.cpp
99

I agree, checking != NoFormat seems clearer to me rather than relying on NoFormat being zero.
Alternatively, we could use Optional<ExpressionFormat> but that seems like unnecessary overhead.

232

Not sure about the FormatValue name for the enum. Maybe something like Kind or Type?
Then this would be ExpressionFormat::Kind::Unsigned which I think reads slightly better.

333

Instead of checking for a comma (which be allowed to appear after the : in the future, I would check if the next non-whitespace character is a %. Or to simplify this we could require the % to immediately follow the # character?

373

Where does the following code modify Expr? Is it inside the call to parseNumericOperand?

llvm/lib/Support/FileCheckImpl.h
45

The hex comments don't match the order of the enum.
Since you are documenting all other members, maybe add Value should be printed an unsigned decimal number. to Unsigned?

56

This should be explicit. Although we could also remove it and always compare to NoFormat.

156

I would avoid the i.e+e.g sequence:
The format to use (e.g. hex upper case letters) when matching the value.

llvm/test/FileCheck/numeric-defines.txt
20–23

I find this slightly surprising. If I define a numeric variable on the command line with hex format, I would expect the value to be parsed as a hex number, i.e. 0x12 and not 0xc.

thopre updated this revision to Diff 236043.Jan 3 2020, 6:13 AM
thopre marked 21 inline comments as done.

Address most review comments

thopre added inline comments.Jan 3 2020, 6:35 AM
llvm/lib/Support/FileCheck.cpp
99

Changed. Note that ExpressionFormat is a structure so this calls the overloaded boolean operator and does not rely on any hard-coded value.

333

I'm not sure I follow the intent of your message, are you against the syntax (i.e. there should be no comma at all) or the way it is parsed? Anyway, since the format comes first, I'm not sure how allowing a comma in the later part of the syntax would be a problem. If I remember well its use in the syntax was suggested to match the API of printf/scanf. Otherwise we'd have:
#%d FOOBAR:N+1 which I find less easy to read.

373

Yes and parseBinop.

llvm/test/FileCheck/numeric-defines.txt
20–23

I didn't think of that. What would be your expectation for implicit format though?

E.g. -D#%X,NUMVAL1=7 -D#NUMVAL2=12+NUMVAL1. Should NUMVAL2 be 0x17 or decimal 17?

arichardson added inline comments.Jan 3 2020, 7:12 AM
llvm/lib/Support/FileCheck.cpp
333

Sorry about the ambiguity. The current syntax is perfectly fine.

I was suggesting a change to the parsing code to check that the first non-whitespace char is a % first instead of searching for a comma. But this would only change the which error messages for certain invalid input so it shouldn't really matter.

llvm/test/FileCheck/numeric-defines.txt
20–23

I'm not entirely sure what the best solution there is.
I think NUMVAL2 should always evaluate to 19.
However, I don't have a strong preference whether it should capture decimal 19 or hex 0x13 (i.e. parse the number in the expresssion as decimal, but inherit format from the other variable).

Alternatively we could require an explicit format for those (probably rare?) cases.

jhenderson added inline comments.Jan 3 2020, 8:58 AM
llvm/lib/Support/FileCheckImpl.h
40

Am I right in thinking you can get rid of the = 0 now?

56

This comment isn't clear to me. I assume what it means by "if it can be matched" is "if it is a format that can be used in a match" ot something similar?

62

if neither is NoFormat and their kinds are the same.

llvm/test/FileCheck/numeric-defines.txt
4–5

Nit: trailing full stop.

You're also mixing your comment characters. Here you use '#', but two lines above you use ';'.

50

Nit: too many blank lines (1 is enough)

54

I'm not familiar with this "%ProtectFileCheckOutput". What is it for, and why do only some cases seem to use it?

73

You should probably also have this case where the format is on the second value, and not the first.

llvm/test/FileCheck/numeric-expression.txt
173

Aside from being fewer in number, how is this set different from the "USE DEF FMT IMPL MATCH" set?

llvm/unittests/Support/FileCheckTest.cpp
46

It might be interesting to show what happens when you pass an upper-case hex digit to a lower-case format and vice versa.

63

What's going on here?

thopre updated this revision to Diff 236098.Jan 3 2020, 11:47 AM
thopre marked 6 inline comments as done.

Add example showing literals are always parsed as decimal

llvm/lib/Support/FileCheck.cpp
333

Yes, in one case one could diagnose a missing comma while in the other a missing percentage. This approach allows to reorder the code to parse each element of a numeric substitution (format specifier, variable definition, expression) when internal API change, as was done in earlier version of this patch. If you don't mind I'll keep it this way.

llvm/test/FileCheck/numeric-defines.txt
20–23

Is the way you expect 12 to be interpreted in '-D#%X,NUMVAL=12' and in '-D#%X,NUMVAL1=7 -D#NUMVAL2=12+NUMVAL1' different because of the explicit Vs implicit format or because you see the former as similar to a value in an input text (and thus 12 as 0x12) and the latter as a literal in a numeric expression (and thus 12 as decimal 12)?

I find it even more confusing to distinguish the behavior of literals between implicit and explicit format (e.g. '-D#%X,NUMVAL1=7 -D#%X,NUMVAL2=12+NUMVAL1' would interpret 12 as 0x12 then).

I also see several reasons to avoid the distinction between the 2 examples based on whether a variable is present after the equal sign:

  • complexity: (i) more text needs to be added in the documentation to describe this distinction thus making the feature harder to comprehend IMHO and (ii) the parsing code needs to distinguish between these 2 cases
  • consistency: the same syntax is used in both cases but the behaviour is different due to the use of a numeric variable after the equal sign
  • unnecessary: the format specifier is needed because (i) the input text can contain a mixture of text and numeric values and (ii) hex numeric values can come with or without prefix, in lowercase or uppercase. It is thus necessary to indicate when something like "dead" is to be interpreted as a numeric value and when it shouldn't. There is not problem in a numeric substitution (whether one defined on the command line or in a check file) since there is no text in it and it is under control of the test writer which can add a prefix.

Therefore I strongly think anything after the equal sign is to be interpreted as a numeric expression and not an input. The distinction might appear clearer in the provision made by one of the later patch to support numeric subsitutions such as #%X,NUMVAL:<12 where the numeric expression value would be 12 and the input value could be 11 (input being B) and thus match.

I've added an example in the documentation to document that a literal is always intepreted as decimal in the absence of prefix (0x12 is allowed and interpreted as hex). Hopefully that'll alleviate some of your concern.

thopre updated this revision to Diff 236109.Jan 3 2020, 12:33 PM
thopre marked 12 inline comments as done.

Address latest round of review comments

llvm/test/FileCheck/numeric-defines.txt
54

This is used when parsing the output of FileCheck to avoid things such as localization, see https://reviews.llvm.org/D65121 for more details

llvm/unittests/Support/FileCheckTest.cpp
46

valueFromStringRepr does not give an error in that case as it expects the value to match the format. This is because getAsInteger does not allow to check for the casing and I don't want to make the function more complex by checking it.

jhenderson added inline comments.Jan 6 2020, 2:33 AM
llvm/test/FileCheck/numeric-defines.txt
20–23

FWIW, I would find treating any number without 0x as hex to be ambiguous. I don't think the format specifier really should make a difference to that. After all, if you think of it in printf terms: printf("0x%x", 12) prints "0xc", not "0x12".

jhenderson added inline comments.Jan 6 2020, 2:33 AM
llvm/test/FileCheck/numeric-defines.txt
54

Got it, thanks!

llvm/unittests/Support/FileCheckTest.cpp
70

Have you considered using ASSERT_THAT_ERROR and EXPECT_THAT_ERROR in these tests?

thopre marked 4 inline comments as done.Jan 6 2020, 5:41 AM
thopre added inline comments.
llvm/test/FileCheck/numeric-defines.txt
20–23

@jhenderson I presume you were answering to Paul? Because FileCheck will indeed check the input file for 0xC when encountering CHECK: 0x[[#%X, 12]] so it's all good. As to the prefix, I supposed you meant for the literals inside the # syntax since prefix is not needed for printf/scanf: scanf("%X", x) will happily scan CAFE without expecting a prefix.

@probinson Since we are talking about parallels between variable definition and scanf on one side and numeric substitution and printf on the other side, we can look at the following examples:

  • 0x[[#%X, VAR2:]] is equivalent to: scanf("%X", VAR2).
  • 0x[[#%X, VAR1+12]] when VAR1 was defined with -D#VAR1=3 will be equivalent to checking the output of the following against the input file:
VAR1=3;
printf("0x%X", VAR1+12);

Note that C interprets 12 in decimal despite the %X format specifier of the printf which only influences the conversion of the resulting numeric value (3+12=15 is converted to F and thus the input file is checked for "0xF").

#SOMEVAR:<EXPR> is a shorthand for both, and thus a combination of printf+scanf with some input file check inbetween, e.g. 0x[[#%X, VAR2:VAR1+12]] when VAR1 is defined with -D#VAR1=3 is equivalent to checking the output of the following against the input file:

VAR1=3;
printf("0x%X", VAR1+12);

and scanning the matched input with:

scanf("0x%X", VAR2);

The case of #%X,VAR:12 is then just a special case, i.e. matching the input file against:

printf("%X", 12);

followed by scanning the matched text with:

scanf("%X", VAR);
thopre updated this revision to Diff 236336.Jan 6 2020, 5:41 AM
thopre marked an inline comment as done.

Use EXPECT_THAT_ERROR when testing for an error in unittest

jhenderson accepted this revision.Jan 6 2020, 5:49 AM

LGTM from me on this. I can't think of anything else I'd like addressing, but please wait for @arichardson and anybody else who wants to comment to be happy.

llvm/test/FileCheck/numeric-defines.txt
20–23

@thopre - I assume you meant @arichardson here, not @probinson...

I was responding to the general conversation, and giving my thoughts, so not addressing anybody specifically. Yes, the as-it-stands behaviour is what I would prefer, precisely for the reasons you outlined with the printf semantics.

This revision is now accepted and ready to land.Jan 6 2020, 5:49 AM
arichardson accepted this revision.Jan 6 2020, 6:59 AM

LGTM. Thanks for working on this feature!

llvm/lib/Support/FileCheck.cpp
333

Yes that seems perfectly fine.

llvm/test/FileCheck/numeric-defines.txt
20–23

@thopre Thanks for the detailed explanation. I think the current behaviour in the tests makes sense. Having the match format not affect the parsing of whats to the right of the = seems simpler and avoids problems.

thopre planned changes to this revision.Jan 6 2020, 7:10 AM

My apologies but I still think there's changes needed on this patch. I had changed the status earlier but I presume updating the patch did reset the status to "needs review". I haven't covered all the API changes in the unittests and I'd like to address the comment from Paul about the lack of clarity of which variables have implicit format conflict. I'm happy to deal with the latter in a separate patch if people are eager to have this change (I'm personally looking forward to it being committed as well) but I want to at least finish the unit testing.

thopre updated this revision to Diff 236817.Jan 8 2020, 6:58 AM

Complete unit testing of changes in the patch

This revision is now accepted and ready to land.Jan 8 2020, 6:58 AM
thopre requested review of this revision.Jan 8 2020, 6:59 AM

Is everybody happy with the constness changes to the ExpressionFormat operators and the new unit tests?

thopre marked 4 inline comments as done.Jan 8 2020, 7:01 AM
thopre added inline comments.
llvm/test/FileCheck/numeric-expression.txt
340

Since this is already a sizable diff, I'd like to address this in a separate patch if you don't mind. It'll require recording more parsing state and I'm thinking about using a parser object for that and thus simplifying the interface of parsing functions (lots of info would be kept as internal state of the object).

arichardson accepted this revision.Jan 12 2020, 2:50 AM

Is everybody happy with the constness changes to the ExpressionFormat operators and the new unit tests?

Looks fine to me. I would say say this can be committed if @jhenderson is happy with it and all tests pass. If there are any remaining issues those can always be fixed in follow-up commit.

llvm/lib/Support/FileCheckImpl.h
71

Since Kind is just an enum, the const is unnecessary for by-value arguments but I don't think it does any harm either.

This revision is now accepted and ready to land.Jan 12 2020, 2:50 AM

No problem with the constness changes.

llvm/lib/Support/FileCheckImpl.h
71

It won't do any harm, I think, but it's inconsistent with style elsewhere, so I think it should be deleted (the const on the method is fine though of course).

llvm/unittests/Support/FileCheckTest.cpp
34

Nit: this comment isn't quite right. You probably wanted "methods' output", since its the output of the methods. However, I think a clearer phrase this and the similar comments to "Check unsgined decimal format methods", or possibly even "Check unsigned decimal format properties".

In fact, now that I think about it, perhaps it would make more sense for this test to be split into a separate TEST for each different format. The test name would then document what the test is for, and you wouldn't need the comment. You might even want to consider a parameterised test (see the TEST_P function), to avoid code duplication, for the Unsigned, HexLower and HexUpper cases.

37

ASSERT_THAT_EXPECTED(Wildcard, Succeeded());

Assert because otherwise the next line will crash if it fails, and the _THAT_EXPECTED macro for readability.

Same comment below.

Perhaps worth naming this UnsignedWildcard (HexLowerWildcard etc), and avoid reusing the variable in the other cases.

47–48

I've given this a bit more thought, and I think it would be better here and in similar situations with failing Expecteds to actually check the error contents. You can see examples of this in the DWARFDebugLineTest.cpp unit tests (look for the checkError function for a rough guide).

takeError on an Expected seems weird without first having checked that the Expected actually failed. You probably want EXPECT_THAT_EXPECTED(someFunctionThatReturnsAnExpected(), Failed()); if you aren't going to actually check the properties of the Error within the Expected.

82

Test "NoFormat" explicitly by passing it into the constructor and then consider a separate test to show that the default constructor generates a NoFormat kind.

Also NoneFormat -> NoFormat.

92

What about the other formats? Why is this here ratehr than in the individual test cases?

95

Self-comparison feels like a special case. You probably want to use a second format instance, I reckon.

104

Vs -> versus

291–292

This comment seems stale compared to what is being done below (i.e. it's not just about the eval() function it appears on the surface).

422

This '++' change, and the corresponding changes below, seem unrelated?

574–576

I've lost track. What's the difference between parsePatternExpect and parseSubstExpect? Why is it pattern, not subst here? Finally, what do these test cases have to do with formats?

684

does a check against "C" here make sense?

688

Does a check against "b" here make sense?

735

Debug printing left in by mistake?

898

ASSERT_THAT_EXPECTED(ExpressionPointer, Succeeded());
Same all over the place.

913–915

I know this isn't something directly related to your change, so it should be a later one, but you should also remove all uses of errorToBool in favour of checking the actual Error.

Sorry I didn't pick up on those in earlier reviews.

thopre updated this revision to Diff 238737.Jan 17 2020, 3:52 AM
thopre marked 20 inline comments as done.
  • Address unit tests review comments
  • fix casing of first letter in an error message
thopre marked an inline comment as done.Jan 17 2020, 3:54 AM
thopre added inline comments.
llvm/unittests/Support/FileCheckTest.cpp
422

Indeed, made it into a separate patch: https://reviews.llvm.org/D72913

574–576

parsePatternExpect will call parsePattern which parse the rhs of a CHECK directive. parseSubstExpect calls parseNumericSubstitutionBlock which parses what's inside a # block. The use of parsePatternExpect is because I'm testing that legacy @LINE expression only accept what the old [[@LINE]] (without #) accepted before this patch set. The restriction for legacy @LINE expression is in parseNumericSubstitutionBlock and private functions it calls but the detection of a legacy @LINE expression is in parsePattern hence the use of parsePatternExpect.

The last of the 3 tests is format related because this patch also enables hex literals (I didn't want to make a separate patch for a few line diff). The other 2 should have been done in earlier patch. I've split these 2 into a separate patch: https://reviews.llvm.org/D72912

913–915
jhenderson added inline comments.Jan 20 2020, 2:28 AM
llvm/unittests/Support/FileCheckTest.cpp
68–76

How about:

StringRef HexDigits = AllowUpperHex ? "ABCDEF" : "abcdef";
EXPECT_TRUE(WildcardRegex.match(HexDigits, &Matches));
EXPECT_EQ(Matches[0], HexDigits);

Remind me why we can't check the opposite cases are rejected here? That needs commenting at the very least.

Also, why is the check at line 70 ASSERT_TRUE, rather than EXPECT_TRUE?

94–105

I'd recommend the following, to reduce duplication of checks.

StringRef Ten;
StringRef Fifteen;
if (AllowHex) {
  if (AllowUpperHex) {
    Ten = "A";
    Fifteen = "F";
  } else {
    Ten = "a";
    Fifteen = "f";
  }
} else {
  Ten = "10";
  Fifteen = "15;
}
EXPECT_EQ(*TenMatchingString, Ten);
EXPECT_EQ(*FifteenMatchingString, Fifteen);
156

Formats -> Format

163

Probably worth a blank line to separate out the Conflict and NoFormat cases. Or just factor them into separate tests.

171

Formats -> Format

172

This comment is unnecessary - the test code is self-documenting.

177

Put a line break above the start of each case, i.e. at the start of each (set of) declaration(s). This will show which the cases more distinctly.

189

On further thought, consider replacing this comment with a new TEST, with the test naem documenting the test purpose.

291

, -> ;

618

format -> formats

622

format -> formats

663

What's the diference between this test case and the one above?

682

See comments in another review. Are this and the similar patterns below safe, if the Expected is in a success state?

914–915

Should this test the actual Error contents, like you do elsewhere?

930

Ditto.

thopre updated this revision to Diff 239204.Jan 20 2020, 3:33 PM
thopre marked 17 inline comments as done.

Address all review comments

thopre added inline comments.Jan 21 2020, 1:58 AM
llvm/unittests/Support/FileCheckTest.cpp
68–76

It's an ASSERT because if the match fails it does not make sense checking Matches[0]. I've added negative testing, it's only for valueFromStringRepr that it is not possible (I've added a comment for that there).

682

With the latest previous patch yes.

Nearly there, from my point of view.

llvm/unittests/Support/FileCheckTest.cpp
94–105

This doesn't appear to have been done here, only in the bufferized bit below.

146–147

Spurious "to allow"?

thopre updated this revision to Diff 239393.Jan 21 2020, 11:35 AM
thopre marked 3 inline comments as done.

Address unit tests review comments

llvm/unittests/Support/FileCheckTest.cpp
94–105

My bad, I don't know how I missed that.

jhenderson accepted this revision.Jan 23 2020, 1:29 AM

LGTM.

@arichardson, do you have any more comments?

arichardson accepted this revision.Jan 23 2020, 5:01 PM

Thanks! Looks good, just one minor typo in the documentation.

llvm/docs/CommandGuide/FileCheck.rst
596

accepted format specifiers

thopre updated this revision to Diff 240173.Jan 24 2020, 6:15 AM
thopre marked an inline comment as done.
  • Fix typo in doc
  • remove sneaky literal for LineNumber in unit test
This revision was automatically updated to reflect the committed changes.