Page MenuHomePhabricator

[docs] Document pattern of using CHECK-SAME to skip irrelevant lines
Needs ReviewPublic

Authored by rupprecht on Sep 25 2019, 4:05 PM.

Details

Summary

This came up during the review for D67656. It's nice but also subtle, so documenting it as an idiom will make tests easier to understand.

Event Timeline

rupprecht created this revision.Sep 25 2019, 4:05 PM
Herald added a project: Restricted Project. · View Herald TranscriptSep 25 2019, 4:05 PM
MaskRay added inline comments.Sep 25 2019, 10:23 PM
llvm/docs/CommandGuide/FileCheck.rst
333

Maybe align 1 to the position 2 columns right of Value: above.

thopre added inline comments.Sep 26 2019, 1:15 AM
llvm/docs/CommandGuide/FileCheck.rst
290

Nit: rather than say that it is common, it might be better to say that CHECK-SAME is useful in dealing with X. Whether it is common or not does not seem very relevant.

jhenderson added inline comments.
llvm/docs/CommandGuide/FileCheck.rst
290

Yeah, I agree with this. I wrote it for the first time a few months ago, and as far as I know, it was the first time it was used for this purpose (certainly I came up with it independently!), so "common" seems like a stretch at the moment.

jhenderson added inline comments.Sep 26 2019, 1:51 AM
llvm/docs/CommandGuide/FileCheck.rst
335–336

Maybe this would be better phrased as "This verifies that the next time "`Value:`" appears in the output, it has a value of 1.

It might also be worth a note saying to use {{$}} immediately after too, to avoid a spurious match on Value: 16 or similar. Not sure if it really belongs here, or somewhere else though.

MaskRay added inline comments.Sep 26 2019, 2:02 AM
llvm/docs/CommandGuide/FileCheck.rst
335–336

The suggestion looks good. So the example should be

CHECK:      Value:
CHECK-SAME:        1{{$}}
rupprecht marked 8 inline comments as done.
  • Remove mention of "common" case
  • Make the bit about the next line containing Value: more clear
  • Fix some indentation
llvm/docs/CommandGuide/FileCheck.rst
290

Fair enough, I just figured everyone besides me was familiar with this trick :)

333

Did you mean 1 column (from the colon)?

335–336

I added {{$}} to the original matcher, it should be there too

greened added inline comments.
llvm/docs/CommandGuide/FileCheck.rst
324

This is what CHECK-LABEL is for. This isn't a very motivating example. Is there a better one?

rupprecht added inline comments.Sep 26 2019, 11:56 AM
llvm/docs/CommandGuide/FileCheck.rst
324

I don't think CHECK-LABEL works in this case. CHECK-LABEL requires unique lines, and the only thing unique here would be "Name: <...>" which is not the thing we have a problem checking.

e.g. something like:

CHECK: Name: foo
CHECK-LABEL: Value: 1

would fail on the example text above because foo and baz both have a value of one. Similarly,

CHECK-LABEL: Name: foo
CHECK: Value: 1

has the same false-passing problem as described in these added docs.

Of course, you could add tests for every name/value pair (and using check-label on the names to make error messages clearer), but then the test overall becomes not a unit test: you really just want to write a unit test that verifies foo has a value of 1; you don't want to assert the universe to do that.

I don't use check-label very often, I'm mostly relying on docs for my understanding. Could you clarify your comment if I'm missing something? Also: if I'm not missing something, let me know how I can make the wording here more clear.

thopre added inline comments.Sep 27 2019, 12:59 AM
llvm/docs/CommandGuide/FileCheck.rst
324

CHECK-LABEL will constraints the CHECK directive between them to match between the two posititions where the CHECK-LABEL match. In other word, if the name in your example are uniques then

CHECK-LABEL: Name: foo
CHECK: Value: 1
CHECK-LABEL: Name: bar

will guarantee that the Value: 1 will match something in the block of text between "Name: foo" and "Name: bar" or fail to match. Now if you only care about foo, your solution avoids adding a CHECK-LABEL for bar which might not be easy if you do not know the relative ordering of the blocks for foo and bar.

So I do think your use case is useful, but perhaps you should remove the parts about bar and baz (my preference) or mention that the order of blocks cannot be relied upon (which might be solved in some future time by having something like CHECK-LABEL-DAG/CHECK-REGION-DAG which would force to update this text to my first suggestion).

jhenderson added inline comments.Sep 27 2019, 1:21 AM
llvm/docs/CommandGuide/FileCheck.rst
324

Perhaps replacing the reference to baz with something like "a later symbol in the output" would be sufficient? It might also be worth noting why CHECK-LABEL isn't appropriate in this case (e.g. unknown ordering).