This is an archive of the discontinued LLVM Phabricator instance.

FileCheck [5/12]: Introduce regular numeric variables
ClosedPublic

Authored by thopre on Apr 7 2019, 4:20 PM.

Details

Summary

This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch introduces regular numeric
variables which can be set on the command-line.

This commit introduces regular numeric variable that can be set on the
command-line with the -D option to a numeric value. They can then be
used in CHECK patterns in numeric expression with the same shape as
@LINE numeric expression, ie. VAR, VAR+offset or VAR-offset where offset
is an integer literal.

The commit also enable strict whitespace in the verbose.txt testcase to
check that the position or the location diagnostics. It fixes one of the
existing CHECK in the process which was not accurately testing a
location diagnostic (ie. the diagnostic was correct, not the CHECK).

Copyright:

  • Linaro (changes up to diff 183612 of revision D55940)
  • GraphCore (changes in later versions of revision D55940 and in new revision created off D55940)

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
thopre added inline comments.May 2 2019, 1:35 AM
llvm/lib/Support/FileCheck.cpp
27–51

Sorry I don't follow. An Optional variable as the member of the binop? How does that help? I still have to check whether that variable has a value or not.

arichardson added inline comments.May 2 2019, 2:36 AM
llvm/include/llvm/Support/FileCheck.h
54

Since the FileCheckNumExpr takes ownership of the pointer, I think the argument should be std::unique_ptr

123

I would name this OpL/OpLeft or LHS since the lowercase l in Opl doesn't make the "left" bit obvious.

Same for Opr below

llvm/lib/Support/FileCheck.cpp
234–235

make_unique?

702–703

I think this should take a unique_ptr for AST and use std::move() to pass it to the constructor.

(sorry for some duplicated comments, @arichardson posted his whilst I was writing mine).

Not looked at the tests in detail yet, but there's plenty to get on with.

llvm/docs/CommandGuide/FileCheck.rst
622

evaluate -> evaluates

llvm/include/llvm/Support/FileCheck.h
54

It's not particularly clear here that the FileCheckNumExpr gets ownership of the AST pointer. As a result, it could very easily lead to multiple frees or similar issues. An obvious example would be:

{
  FileCheckASTBinOp Op(...);
  FileCheckNumExpr(&Op);
} // Op gets destroyed twice.

Assuming that it does need ownership (as implied by the use of unique_ptr), perhaps it would be better to construct the AST instance within the FileCheckNumExpr constructor or some helper method that returns a FileCheckNumExpr instance?

87

NumVar (and to a lesser extent NumExpr) is hard to interpret, so probably shouldn't be overly abbreviated. NumericVar or NumericVariable (I'm happy with either) would be better, I think.

87

You should explain under what situations None can be returned here.

92

... if defined ... -> ..., if defined, ...

100

Here and elsewhere, you should probably use the doxygen style for "Return".

123

Why not call this Left, and the other Right (or LeftOp and RightOp if you want Op to be in the name)? The meaning is then implicit and the comment unnecessary.

243

numeric variables parsed -> parsed numeric variables

305

tables -> table

llvm/lib/Support/FileCheck.cpp
43

"Variable is undefined". (?) It doesn't really matter whether it was upfront undefined or later undefined.

148–149

You can probably just name these "add" and "sub".

177

Therefore, the

187

LeftOp

220–221

RightOp

255

This should probably not be auto (is it a pointer? What is the type etc).

1727

Therefore, we

llvm/test/FileCheck/defines.txt
15

The test is getting quite dense. It might be worth either a) splitting it up, or b) labelling each sub-test with a comment.

llvm/test/FileCheck/numeric-expression.txt
3

It may be worth splitting this (and the other cases) over multiple lines like this:

; RUN: not FileCheck -check-prefix UNDEF-USE -input-file %s %s 2>&1 \
; RUN:   | FileCheck --strict-whitespace -check-prefix UNDEF-USE-MSG %s

I also would find it easier to read if you grouped the runs with their corresponding checks:

; RUN: FileCheck... --check-prefix=FOO

; FOO: blah
; FOO-NEXT: blahblah

; RUN: FileCheck... --check-prefix=BAR

; BAR: bar
...
8

No need for double blank lines here and below.

10

attemt -> attempt

11

Doesn't the second "ensure" just repeat what you said in the first part of the sentence?

thopre updated this revision to Diff 197796.May 2 2019, 8:51 AM
thopre marked 26 inline comments as done.
  • Rename FileCheckASTBinop into FileCheckNumExpr and delete FileCheckNumExpr
  • Address outstanding review comments
thopre added inline comments.
llvm/include/llvm/Support/FileCheck.h
100

I've only done it for those added in this patch. I've created a separate patch for the existing uses, https://reviews.llvm.org/D61445

305

There's 2 tables: one for pattern variables and one for numeric variables. I can split into 2 items if you prefer, I didn't because the comment would be mostly redundant.

thopre updated this revision to Diff 197825.May 2 2019, 10:34 AM
  • Remove references to AST
  • Exchange content of the *-defines.txt files to match their name
thopre updated this revision to Diff 197975.May 3 2019, 6:03 AM

Only refer to numeric value since there is only one at this point

thopre updated this revision to Diff 197984.May 3 2019, 6:17 AM

Fix some codestyle issues

thopre updated this revision to Diff 197985.May 3 2019, 6:22 AM

Undo mistaken merge of #6 into #5

Nearly there! One final grammar nit and some test comments.

For defines.txt/pattern-defines.txt, please do the reorganization of the RUN lines as a separate NFC commit first, so then we can better see the functional difference that goes with this patch. Right now it's very hard to follow.

llvm/include/llvm/Support/FileCheck.h
175

expression -> expressions
value -> values

(to be grammatically consistent with the next sentence)

llvm/test/FileCheck/var-scope.txt
9

Is there a different test that validates the diagnostic for using an undefined local? All this test does is verify that FileCheck returns nonzero.

thopre updated this revision to Diff 198424.May 7 2019, 3:20 AM
thopre marked 2 inline comments as done.
  • Test error message when using local variable undefined by --enable-var-scope
  • Fix grammar in getResult heading comment
thopre updated this revision to Diff 198434.May 7 2019, 4:12 AM

Rename test of NumericVariable class to NumericVariable

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

I've still not reviewed the tests in depth yet, but I'm out of time for the day, so here's some more comments from me.

llvm/include/llvm/Support/FileCheck.h
86–124

Is it legal to have two "\returns" in a doxygen comment? Perhaps it's better to fold the two into one e.g:

Evaluates the value of this numeric expression, using EvalBinop to perform the binary operation it consists of. \returns None i the numeric variable used is undefined, or the expression value.
250

definition -> definitions

251

from -> by

305

Okay, perhaps worth changing it slightly to:

- separate tables with the values of live pattern and numeric variables respectively at ...
llvm/lib/Support/FileCheck.cpp
706–707

Use push_back and make_unique here and below.

llvm/test/FileCheck/defines.txt
1–2

Similar to elsewhere, it would be nice for some comments explaining the purpose of this test.

llvm/test/FileCheck/numeric-defines.txt
1 ↗(On Diff #198434)

I know this isn't necessarily a new issue, but it grates slightly seeing a mixture of single and double dash long options in the same test. Would you mind rationalising it to one or the other (I prefer double, but don't really care). Same goes for the other tests.

24 ↗(On Diff #198434)

Thanks for splitting things up. I'd go one step further and split the bad format tests into a separate test from the good format ones, to make the test easier to follow. I'd also add comments to the start of the test files to clearly describe what the test is intended to actually test.

probinson added inline comments.May 7 2019, 11:01 AM
llvm/test/FileCheck/numeric-defines.txt
1 ↗(On Diff #198434)

Actually there's work in progress (I believe) to stop supporting single-dash long options, and it would be best to stay ahead of the curve there.

thopre marked 10 inline comments as done.May 7 2019, 1:31 PM

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

llvm/lib/Support/FileCheck.cpp
706–707

Being quite inexperienced with C++, would you mind teaching me why push_back is better in this case? Note that I cannot use make_unique because it needs C++14 or later so I've only changed emplace_back to push_back and kept the existing new as follows:

auto NewNumExpr = new FileCheckNumExpr(EvalBinop, OperandLeft, OperandRight);
NumExprs.push_back(std::unique_ptr<FileCheckNumExpr>(NewNumExpr));

llvm/test/FileCheck/numeric-defines.txt
1 ↗(On Diff #198434)

I'm confused, the documentation for FileCheck does not mention options having a single and double dash version for each options. AFAICT it's either double dash or single. Did I miss something?

thopre updated this revision to Diff 198521.May 7 2019, 1:32 PM
thopre marked an inline comment as done.

Address review comments

jhenderson added inline comments.May 8 2019, 2:19 AM
llvm/lib/Support/FileCheck.cpp
706–707

LLVM has a hand-rolled make_unique in STLExtras.h, so you can use that. There are various reasons for that, but one of the main reasons is to avoid potential memory leaks resulting from a subsequent allocation failure. I'm not sure that your code necessarily has an issue here, but it's good to get into the habit of doing things right due to other situations. You can find various discussions on this topic on the Internet, e.g. here: https://stackoverflow.com/questions/37514509/advantages-of-using-stdmake-unique-over-new-operator.

One issue with using emplace_back and a raw pointer is that the vector size increase may cause an allocation failure. The raw pointer then hasn't been captured by the unique_ptr and so is a leak. I don't think the push_back has the same issue though. It gets worse when multiple pointers are in play with the first allocation working but a later one failing.

llvm/test/FileCheck/numeric-defines-diagnostics.txt
4 ↗(On Diff #198521)

You don't actually need the semi-colons in this file. Lit will work just fine without them. You might want to keep them for the comments though to make them clearer.

llvm/test/FileCheck/numeric-defines.txt
1 ↗(On Diff #198434)

I'm confused, the documentation for FileCheck does not mention options having a single and double dash version for each options. AFAICT it's either double dash or single. Did I miss something?

At the moment, tools that use cl::opt directly (i.e. not via tablegen) support both single and double dashes before every option (it might allow more than that too or have specific limitations for single-letter options etc, but I haven't checked). FileCheck is one such tool. The documentation and help text usually don't make this clear.

Actually there's work in progress (I believe) to stop supporting single-dash long options, and it would be best to stay ahead of the curve there.

This discussion on the mailing list was specifically aimed at the LLVM binutils which are intended to be drop-in replacements for GNU equivalents, where using single-dash on long options can lead to ambiguity with grouped short options (e.g. -abc could mean -a -b -c or --abc). I don't think there's a plan to force it wider than this (specifically the consensus appears to be to do it on a tool-by-tool basis), although we might make the decision at some point to normalise it I guess on other tools.

2 ↗(On Diff #198521)

using it -> using them

llvm/test/FileCheck/pattern-defines.txt
2 ↗(On Diff #198521)

using it -> using them

thopre updated this revision to Diff 198616.May 8 2019, 3:31 AM
thopre marked 7 inline comments as done.
  • use llvm_make_unique in makeNumExpr and makeNumericVariable
  • remove semicolon prefix in tests except for comments
  • Use -- for FileCheck long options

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

If there's no functional difference, it should be separated into its own NFC patch.

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

If there's no functional difference, it should be separated into its own NFC patch.

Done in https://reviews.llvm.org/D61679.

thopre updated this revision to Diff 198643.May 8 2019, 6:51 AM

Split changes to defines.txt into a separate patch

thopre updated this revision to Diff 198684.May 8 2019, 9:27 AM

Include update to pattern-defines-diagnostics.txt

thopre marked an inline comment as done.May 9 2019, 6:24 AM
thopre added inline comments.
llvm/lib/Support/FileCheck.cpp
27–51

Ping?

arsenm added inline comments.May 9 2019, 6:35 AM
llvm/lib/Support/FileCheck.cpp
27–51

I'm confused because the code here doesn't match what I see in the Phabricator email anymore.

If you are separately tracking Defined and returning None or Value, this is the same as just returning an Optional member

I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again.

llvm/lib/Support/FileCheck.cpp
27–51

I think the comment was originally on FileCheckNumericVariable, which used to have a separate bool, but @thopre changed this to an Optional a while ago. So this is done.

395

collisions ... variables

probinson added inline comments.May 9 2019, 7:16 AM
llvm/lib/Support/FileCheck.cpp
1657

collisions ... variables

1697

collisions ... variables

llvm/test/FileCheck/numeric-defines-diagnostics.txt
30 ↗(On Diff #198684)

Maybe also -D#VALUE3=VALUE1+2 (i.e. without quotes or whitespace).

jhenderson added inline comments.May 9 2019, 7:29 AM
llvm/test/FileCheck/pattern-defines-diagnostics.txt
15 ↗(On Diff #198684)

Nit: Missin -> missing (sorry, didn't see this before)

llvm/test/FileCheck/var-scope.txt
5–7

What's the purpose of having these 3 as separate RUN lines, rather than a single one with check-prefixes=CHECK,GLOBAL,LOCAL3?

llvm/test/FileCheck/verbose.txt
135

Why has this changed?

llvm/unittests/Support/FileCheckTest.cpp
23 ↗(On Diff #198684)

What's the purpose of the cast here and elsewhere in this test?

42 ↗(On Diff #198684)

Why is this a pointer here?

43 ↗(On Diff #198684)

Why is this a pointer?

147 ↗(On Diff #198684)

Why struct?

229 ↗(On Diff #198684)

Why is this a pointer?

269 ↗(On Diff #198684)

Why is this a pointer?

295 ↗(On Diff #198684)

"Empty variable name"?

303 ↗(On Diff #198684)

"Invalid variable name"?

352 ↗(On Diff #198684)

Do you need to be explicit about the unsigned-ness here?

361–366 ↗(On Diff #198684)

I'm not sure I follow the purpose of this test case addition.

386 ↗(On Diff #198684)

Is it necessary to be explicit about the unsigned-ness here?

398 ↗(On Diff #198684)

Is it necessary to be explicit about the unsigned-ness here?

thopre updated this revision to Diff 198866.May 9 2019, 10:08 AM
thopre marked 32 inline comments as done.

Address all outstanding comments

thopre added inline comments.May 9 2019, 10:09 AM
llvm/lib/Support/FileCheck.cpp
27–51

As Paul mentioned, I've addressed your other comment regarding using llvm::Optional instead of having an uint64_t and a bool. I've also adapted functions accordingly and some of them became trivial. This one was not much change since I need to call EvalBinop when the value is defined so I still need an if and separate return depending on whether the Optional value is true or not. Marking the issue as done since it seems I addressed the issue you were raising.

llvm/test/FileCheck/var-scope.txt
5–7

None, inherited from the original regex-scope.txt file. I've combined them now.

llvm/test/FileCheck/verbose.txt
135

As per the patch description, I've enabled --strict-whitespace for these tests to check that the location information is correct in the CHECK I was adding but this check was failing. I've checked that the number of space I'm changing it to matches the column information from the diagnostic (18 spaces, caret on the 19th column).

llvm/unittests/Support/FileCheckTest.cpp
23 ↗(On Diff #198684)

To avoid a warning about comparison between sined (42) and unsigned (*Value which is uint64_t) integers. Elsewhere I've suffixed the literal with U, I've now normalized and that approach (ie. 42U in this case)

42 ↗(On Diff #198684)

My bad, must have been a copy/paste from somewhere else. FileCheckNumericVariable are referenced by a std::unique_ptr in one of the member of FileCheckPatternContext so need to be a pointer to avoid freeing something from the stack in some of the unit test.

43 ↗(On Diff #198684)

Likewise, bad copy/paste. Fixed.

352 ↗(On Diff #198684)

Yes to avoid a compiler warning, see earlier comment.

361–366 ↗(On Diff #198684)

Numeric expressions are linked to numeric variable structure at parse time but clearing of local variable happens at runtime. It is therefore important that evaluating a cleared variable gives an error even if we have a pointer from before it was cleared. I've updated the comment to explain it.

386 ↗(On Diff #198684)

Yes, see earlier comment.

398 ↗(On Diff #198684)

Ditto.

thopre updated this revision to Diff 198946.May 9 2019, 4:49 PM

Fix codestyle

I may have missed it, but I'm not sure I saw any unit tests for parsing a numeric expression involve an addition or subtraction?

llvm/test/FileCheck/var-scope.txt
2

This test has the same mixed -/-- as mentioned in other test cases. Please fix!

llvm/unittests/Support/FileCheckTest.cpp
18 ↗(On Diff #198946)

Maybe worth adding a check of getName here.

24 ↗(On Diff #198946)

Check getValue() here after setting, to show that the new value is returned for a defined variable.

41 ↗(On Diff #198946)

Undef? Maybe this test should be just called NumExpr, since it tests the entire class.

45 ↗(On Diff #198946)

undef -> undefined here and below

58 ↗(On Diff #198946)

I might be worth showing somewhere that getUndefVarName can be called without an eval call.

183 ↗(On Diff #198946)

Nit: full stop here and next comment.

228–229 ↗(On Diff #198946)

I've lost track a little. Is @LINE the only allowed numeric variable currently, or can we have others now (I thought we could)? If we can, we should have a non-pseudo variable test case here (possibly in addition to a pseudo one).

255 ↗(On Diff #198946)

Explicitly say that "getUndefVarName returns ..." instead of "Undef var ..." here and below.

275 ↗(On Diff #198946)

What is this comment line referring to?

361–363 ↗(On Diff #198946)

variable -> variables

Based on your comment, I think this can be further improved:

"Check a numeric expression's eval fails if called after clearing of local variables, if it was created before. This is important because local variable clearing because of --enable-var-scope happens after numeric expressions are linked to the numeric variables they use."

23 ↗(On Diff #198684)

Okay. I feel like it's a bug if that warning comes out and the signed number is known to be a positive literal, but that's a separate issue.

thopre updated this revision to Diff 199131.May 11 2019, 12:07 AM
thopre marked 14 inline comments as done.

Address outstanding comments

thopre added inline comments.May 11 2019, 12:32 AM
llvm/unittests/Support/FileCheckTest.cpp
24 ↗(On Diff #198946)

I suppose you mean the old value. As mentioned in the comment, setValue is supposed to fail when the variable is already defined and the value is then unchanged.

23 ↗(On Diff #198684)

Yes but not a bug in FileCheck :-)

jhenderson added inline comments.May 13 2019, 3:26 AM
llvm/unittests/Support/FileCheckTest.cpp
239 ↗(On Diff #199131)

Shouldn't this use something other than "@LINE"? How does this even work given its used in a substitution "N"?

282 ↗(On Diff #199131)

is empty -> returns an empty string.

288 ↗(On Diff #199131)

does not return anything (empty StringRef) -> returns an empty string.

thopre updated this revision to Diff 199234.May 13 2019, 3:43 AM
thopre marked 4 inline comments as done.

Address outstanding review comments

llvm/unittests/Support/FileCheckTest.cpp
239 ↗(On Diff #199131)

The substitution only cares about what variable object it points to. The name in the variable object is used only to populate the name of undefined variables in a substitution. Anyway, fixed now.

jhenderson accepted this revision.May 13 2019, 4:23 AM

Okay, LGTM, but make sure @probinson is happy at least.

This revision is now accepted and ready to land.May 13 2019, 4:23 AM

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

This revision was automatically updated to reflect the committed changes.

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

I'm afraid I cannot reproduce this error. I'm using sphinx-build 1.8.5 from the python-sphinx Ubuntu package version 1.6.7-1ubuntu1. Can anyone with a different version try to reproduce? Thanks in advance.

jhenderson added a comment.EditedMay 14 2019, 1:23 AM

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

I'm afraid I cannot reproduce this error. I'm using sphinx-build 1.8.5 from the python-sphinx Ubuntu package version 1.6.7-1ubuntu1. Can anyone with a different version try to reproduce? Thanks in advance.

I don't have Sphinx setup on my machine. I'd suggest a couple of things: 1) make sure that the problem was directly caused by your change according to the build bots, and isn't a case of flakiness. 2) Ask for help on the llvm-dev mailing list, CC'ing the build bot maintainer who may be able to better reproduce. You should (unfortunately) probably also revert if you haven't so that the build bots aren't bad for everyone else's changes.

thopre reopened this revision.May 14 2019, 2:30 AM

Reopen to easily review changes to fix FileCheck.rst build failure

This revision is now accepted and ready to land.May 14 2019, 2:30 AM
thopre updated this revision to Diff 199383.May 14 2019, 2:30 AM

Fix list in FileCheck.rst

thopre updated this revision to Diff 199388.May 14 2019, 3:07 AM

Really submit fix for list in FileCheck.rst

thopre requested review of this revision.May 14 2019, 3:08 AM
thopre updated this revision to Diff 199389.May 14 2019, 3:21 AM
  • Address Paul's review comments
  • Rename numeric expression subsitution for numeric substitution to clearly distinguish numeric expression (the part that gets evaluated) from the whole #<something> that gets substituted (which might include a numeric variable definition).
  • Rename ParsePattern into parsePattern since it is modified
thopre updated this revision to Diff 199391.May 14 2019, 3:33 AM

Undo latest submission

Unfortunately, the /new link for me isn't working for this. Could you point out what has changed since your previous version?

Unfortunately, the /new link for me isn't working for this. Could you point out what has changed since your previous version?

Of course: the list introduced by "The syntax to check a numeric expression constraint is" in FileCheck.rst has now one space between the asterisks and the beginning of the items. When an item span several lines (as is the case for the second and third items in the list), the follow-on lines start by 2 spaces as required by sphinx which also allow them to nicely align with the first line. This allows sphinx to build without warning.

This revision is now accepted and ready to land.May 14 2019, 4:42 AM
This revision was automatically updated to reflect the committed changes.
rnk removed a reviewer: rnk.May 14 2019, 11:29 AM
rnk removed a subscriber: rnk.
ychen added a subscriber: ychen.Jul 1 2019, 11:41 PM
llvm/test/FileCheck/numeric-expression.txt