FileCheck [5/12]: Introduce regular numeric variables

[thopre]

Summary

This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch introduces regular numeric
variables which can be set on the command-line.

This commit introduces regular numeric variable that can be set on the
command-line with the -D option to a numeric value. They can then be
used in CHECK patterns in numeric expression with the same shape as
@LINE numeric expression, ie. VAR, VAR+offset or VAR-offset where offset
is an integer literal.

The commit also enable strict whitespace in the verbose.txt testcase to
check that the position or the location diagnostics. It fixes one of the
existing CHECK in the process which was not accurately testing a
location diagnostic (ie. the diagnostic was correct, not the CHECK).

Copyright:

• Linaro (changes up to diff 183612 of revision D55940)
• GraphCore (changes in later versions of revision D55940 and in new revision created off D55940)

[thopre]

llvm/lib/Support/FileCheck.cpp

27–51

Sorry I don't follow. An Optional variable as the member of the binop? How does that help? I still have to check whether that variable has a value or not.

[arichardson]

llvm/include/llvm/Support/FileCheck.h

87

Since the FileCheckNumExpr takes ownership of the pointer, I think the argument should be std::unique_ptr

133

I would name this OpL/OpLeft or LHS since the lowercase l in Opl doesn't make the "left" bit obvious.

Same for Opr below

llvm/lib/Support/FileCheck.cpp

200

make_unique?

643–648

I think this should take a unique_ptr for AST and use std::move() to pass it to the constructor.

[jhenderson]

(sorry for some duplicated comments, @arichardson posted his whilst I was writing mine).

Not looked at the tests in detail yet, but there's plenty to get on with.

llvm/docs/CommandGuide/FileCheck.rst

621

evaluate -> evaluates

llvm/include/llvm/Support/FileCheck.h

87

It's not particularly clear here that the FileCheckNumExpr gets ownership of the AST pointer. As a result, it could very easily lead to multiple frees or similar issues. An obvious example would be:

{
  FileCheckASTBinOp Op(...);
  FileCheckNumExpr(&Op);
} // Op gets destroyed twice.

Assuming that it does need ownership (as implied by the use of unique_ptr), perhaps it would be better to construct the AST instance within the FileCheckNumExpr constructor or some helper method that returns a FileCheckNumExpr instance?

97

NumVar (and to a lesser extent NumExpr) is hard to interpret, so probably shouldn't be overly abbreviated. NumericVar or NumericVariable (I'm happy with either) would be better, I think.

97

You should explain under what situations None can be returned here.

102

... if defined ... -> ..., if defined, ...

110

Here and elsewhere, you should probably use the doxygen style for "Return".

133

Why not call this Left, and the other Right (or LeftOp and RightOp if you want Op to be in the name)? The meaning is then implicit and the comment unnecessary.

241

numeric variables parsed -> parsed numeric variables

307

tables -> table

llvm/lib/Support/FileCheck.cpp

43

"Variable is undefined". (?) It doesn't really matter whether it was upfront undefined or later undefined.

124–125

You can probably just name these "add" and "sub".

144

Therefore, the

154

LeftOp

186–187

RightOp

214

This should probably not be auto (is it a pointer? What is the type etc).

1659

Therefore, we

llvm/test/FileCheck/defines.txt

15 ↗

(On Diff #197687)

The test is getting quite dense. It might be worth either a) splitting it up, or b) labelling each sub-test with a comment.

llvm/test/FileCheck/numeric-expression.txt

3

It may be worth splitting this (and the other cases) over multiple lines like this:

; RUN: not FileCheck -check-prefix UNDEF-USE -input-file %s %s 2>&1 \
; RUN:   | FileCheck --strict-whitespace -check-prefix UNDEF-USE-MSG %s

I also would find it easier to read if you grouped the runs with their corresponding checks:

; RUN: FileCheck... --check-prefix=FOO

; FOO: blah
; FOO-NEXT: blahblah

; RUN: FileCheck... --check-prefix=BAR

; BAR: bar
...

8

No need for double blank lines here and below.

10

attemt -> attempt

11

Doesn't the second "ensure" just repeat what you said in the first part of the sentence?

[thopre]

• Rename FileCheckASTBinop into FileCheckNumExpr and delete FileCheckNumExpr
• Address outstanding review comments

[thopre]

llvm/include/llvm/Support/FileCheck.h

110

I've only done it for those added in this patch. I've created a separate patch for the existing uses, https://reviews.llvm.org/D61445

307

There's 2 tables: one for pattern variables and one for numeric variables. I can split into 2 items if you prefer, I didn't because the comment would be mostly redundant.

[thopre]

• Remove references to AST
• Exchange content of the *-defines.txt files to match their name

[thopre]

Only refer to numeric value since there is only one at this point

[thopre]

Fix some codestyle issues

[thopre]

Undo mistaken merge of #6 into #5

[probinson]

Nearly there! One final grammar nit and some test comments.

For defines.txt/pattern-defines.txt, please do the reorganization of the RUN lines as a separate NFC commit first, so then we can better see the functional difference that goes with this patch. Right now it's very hard to follow.

llvm/include/llvm/Support/FileCheck.h

156

expression -> expressions
value -> values

(to be grammatically consistent with the next sentence)

llvm/test/FileCheck/var-scope.txt

9

Is there a different test that validates the diagnostic for using an undefined local? All this test does is verify that FileCheck returns nonzero.

[thopre]

• Test error message when using local variable undefined by --enable-var-scope
• Fix grammar in getResult heading comment

[thopre]

Rename test of NumericVariable class to NumericVariable

[probinson]

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

[jhenderson]

I've still not reviewed the tests in depth yet, but I'm out of time for the day, so here's some more comments from me.

llvm/include/llvm/Support/FileCheck.h

96–104

Is it legal to have two "\returns" in a doxygen comment? Perhaps it's better to fold the two into one e.g:

Evaluates the value of this numeric expression, using EvalBinop to perform the binary operation it consists of. \returns None i the numeric variable used is undefined, or the expression value.

232

definition -> definitions

233

from -> by

307

Okay, perhaps worth changing it slightly to:

- separate tables with the values of live pattern and numeric variables respectively at ...

llvm/lib/Support/FileCheck.cpp

647–648

Use push_back and make_unique here and below.

llvm/test/FileCheck/numeric-defines.txt

2

I know this isn't necessarily a new issue, but it grates slightly seeing a mixture of single and double dash long options in the same test. Would you mind rationalising it to one or the other (I prefer double, but don't really care). Same goes for the other tests.

25

Thanks for splitting things up. I'd go one step further and split the bad format tests into a separate test from the good format ones, to make the test easier to follow. I'd also add comments to the start of the test files to clearly describe what the test is intended to actually test.

llvm/test/FileCheck/pattern-defines.txt

1 ↗

(On Diff #198434)

Similar to elsewhere, it would be nice for some comments explaining the purpose of this test.

[probinson]

llvm/test/FileCheck/numeric-defines.txt

2

Actually there's work in progress (I believe) to stop supporting single-dash long options, and it would be best to stay ahead of the curve there.

[thopre]

In D60385#1493635, @probinson wrote:

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

llvm/lib/Support/FileCheck.cpp

647–648

Being quite inexperienced with C++, would you mind teaching me why push_back is better in this case? Note that I cannot use make_unique because it needs C++14 or later so I've only changed emplace_back to push_back and kept the existing new as follows:

auto NewNumExpr = new FileCheckNumExpr(EvalBinop, OperandLeft, OperandRight);
NumExprs.push_back(std::unique_ptr<FileCheckNumExpr>(NewNumExpr));

llvm/test/FileCheck/numeric-defines.txt

2

I'm confused, the documentation for FileCheck does not mention options having a single and double dash version for each options. AFAICT it's either double dash or single. Did I miss something?

[thopre]

Address review comments

[jhenderson]

llvm/lib/Support/FileCheck.cpp

647–648

LLVM has a hand-rolled make_unique in STLExtras.h, so you can use that. There are various reasons for that, but one of the main reasons is to avoid potential memory leaks resulting from a subsequent allocation failure. I'm not sure that your code necessarily has an issue here, but it's good to get into the habit of doing things right due to other situations. You can find various discussions on this topic on the Internet, e.g. here: https://stackoverflow.com/questions/37514509/advantages-of-using-stdmake-unique-over-new-operator.

One issue with using emplace_back and a raw pointer is that the vector size increase may cause an allocation failure. The raw pointer then hasn't been captured by the unique_ptr and so is a leak. I don't think the push_back has the same issue though. It gets worse when multiple pointers are in play with the first allocation working but a later one failing.

llvm/test/FileCheck/numeric-defines-diagnostics.txt

5

You don't actually need the semi-colons in this file. Lit will work just fine without them. You might want to keep them for the comments though to make them clearer.

llvm/test/FileCheck/numeric-defines.txt

2

I'm confused, the documentation for FileCheck does not mention options having a single and double dash version for each options. AFAICT it's either double dash or single. Did I miss something?

At the moment, tools that use cl::opt directly (i.e. not via tablegen) support both single and double dashes before every option (it might allow more than that too or have specific limitations for single-letter options etc, but I haven't checked). FileCheck is one such tool. The documentation and help text usually don't make this clear.

Actually there's work in progress (I believe) to stop supporting single-dash long options, and it would be best to stay ahead of the curve there.

This discussion on the mailing list was specifically aimed at the LLVM binutils which are intended to be drop-in replacements for GNU equivalents, where using single-dash on long options can lead to ambiguity with grouped short options (e.g. -abc could mean -a -b -c or --abc). I don't think there's a plan to force it wider than this (specifically the consensus appears to be to do it on a tool-by-tool basis), although we might make the decision at some point to normalise it I guess on other tools.

3

using it -> using them

llvm/test/FileCheck/pattern-defines.txt

2 ↗

(On Diff #198521)

using it -> using them

[thopre]

• use llvm_make_unique in makeNumExpr and makeNumericVariable
• remove semicolon prefix in tests except for comments
• Use -- for FileCheck long options

[probinson]

In D60385#1494062, @thopre wrote:

In D60385#1493635, @probinson wrote:

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

If there's no functional difference, it should be separated into its own NFC patch.

[thopre]

In D60385#1494841, @probinson wrote:

In D60385#1494062, @thopre wrote:

In D60385#1493635, @probinson wrote:

I still can't tell what the functional difference is, in pattern-defines.txt. Everything else looks fine.

There is none, pattern-defines + pattern-defines-diagnostics = old defines.txt + some new comments, as per James' comments.

If there's no functional difference, it should be separated into its own NFC patch.

Done in https://reviews.llvm.org/D61679.

[thopre]

Split changes to defines.txt into a separate patch

[thopre]

Include update to pattern-defines-diagnostics.txt

[thopre]

llvm/lib/Support/FileCheck.cpp

27–51

Ping?

[arsenm]

llvm/lib/Support/FileCheck.cpp

27–51

I'm confused because the code here doesn't match what I see in the Phabricator email anymore.

If you are separately tracking Defined and returning None or Value, this is the same as just returning an Optional member

[probinson]

I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again.

llvm/lib/Support/FileCheck.cpp

27–51

I think the comment was originally on FileCheckNumericVariable, which used to have a separate bool, but @thopre changed this to an Optional a while ago. So this is done.

356

collisions ... variables

[probinson]

llvm/lib/Support/FileCheck.cpp

1588

collisions ... variables

1628

collisions ... variables

llvm/test/FileCheck/numeric-defines-diagnostics.txt

31

Maybe also -D#VALUE3=VALUE1+2 (i.e. without quotes or whitespace).

[jhenderson]

llvm/test/FileCheck/pattern-defines-diagnostics.txt

15

Nit: Missin -> missing (sorry, didn't see this before)

llvm/test/FileCheck/var-scope.txt

5–7

What's the purpose of having these 3 as separate RUN lines, rather than a single one with check-prefixes=CHECK,GLOBAL,LOCAL3?

llvm/test/FileCheck/verbose.txt

135

Why has this changed?

llvm/unittests/Support/FileCheckTest.cpp

23

What's the purpose of the cast here and elsewhere in this test?

42

Why is this a pointer here?

43

Why is this a pointer?

155

Why struct?

234

Why is this a pointer?

289

Why is this a pointer?

314

"Empty variable name"?

322

"Invalid variable name"?

371

Do you need to be explicit about the unsigned-ness here?

380–385

I'm not sure I follow the purpose of this test case addition.

408

Is it necessary to be explicit about the unsigned-ness here?

420

Is it necessary to be explicit about the unsigned-ness here?

[thopre]

Address all outstanding comments

[thopre]

llvm/lib/Support/FileCheck.cpp

27–51

As Paul mentioned, I've addressed your other comment regarding using llvm::Optional instead of having an uint64_t and a bool. I've also adapted functions accordingly and some of them became trivial. This one was not much change since I need to call EvalBinop when the value is defined so I still need an if and separate return depending on whether the Optional value is true or not. Marking the issue as done since it seems I addressed the issue you were raising.

llvm/test/FileCheck/var-scope.txt

5–7

None, inherited from the original regex-scope.txt file. I've combined them now.

llvm/test/FileCheck/verbose.txt

135

As per the patch description, I've enabled --strict-whitespace for these tests to check that the location information is correct in the CHECK I was adding but this check was failing. I've checked that the number of space I'm changing it to matches the column information from the diagnostic (18 spaces, caret on the 19th column).

llvm/unittests/Support/FileCheckTest.cpp

23

To avoid a warning about comparison between sined (42) and unsigned (*Value which is uint64_t) integers. Elsewhere I've suffixed the literal with U, I've now normalized and that approach (ie. 42U in this case)

42

My bad, must have been a copy/paste from somewhere else. FileCheckNumericVariable are referenced by a std::unique_ptr in one of the member of FileCheckPatternContext so need to be a pointer to avoid freeing something from the stack in some of the unit test.

43

Likewise, bad copy/paste. Fixed.

371

Yes to avoid a compiler warning, see earlier comment.

380–385

Numeric expressions are linked to numeric variable structure at parse time but clearing of local variable happens at runtime. It is therefore important that evaluating a cleared variable gives an error even if we have a pointer from before it was cleared. I've updated the comment to explain it.

408

Yes, see earlier comment.

420

Ditto.

[thopre]

Fix codestyle

[jhenderson]

I may have missed it, but I'm not sure I saw any unit tests for parsing a numeric expression involve an addition or subtraction?

llvm/test/FileCheck/var-scope.txt

2

This test has the same mixed -/-- as mentioned in other test cases. Please fix!

llvm/unittests/Support/FileCheckTest.cpp

18

Maybe worth adding a check of getName here.

23

Okay. I feel like it's a bug if that warning comes out and the signed number is known to be a positive literal, but that's a separate issue.

24

Check getValue() here after setting, to show that the new value is returned for a defined variable.

41

Undef? Maybe this test should be just called NumExpr, since it tests the entire class.

45

undef -> undefined here and below

58

I might be worth showing somewhere that getUndefVarName can be called without an eval call.

191

Nit: full stop here and next comment.

233–234

I've lost track a little. Is @LINE the only allowed numeric variable currently, or can we have others now (I thought we could)? If we can, we should have a non-pseudo variable test case here (possibly in addition to a pseudo one).

274

Explicitly say that "getUndefVarName returns ..." instead of "Undef var ..." here and below.

295

What is this comment line referring to?

380–382

variable -> variables

Based on your comment, I think this can be further improved:

"Check a numeric expression's eval fails if called after clearing of local variables, if it was created before. This is important because local variable clearing because of --enable-var-scope happens after numeric expressions are linked to the numeric variables they use."

[thopre]

Address outstanding comments

[thopre]

llvm/unittests/Support/FileCheckTest.cpp

23

Yes but not a bug in FileCheck :-)

24

I suppose you mean the old value. As mentioned in the comment, setValue is supposed to fail when the variable is already defined and the value is then unchanged.

[jhenderson]

llvm/unittests/Support/FileCheckTest.cpp

239

Shouldn't this use something other than "@LINE"? How does this even work given its used in a substitution "N"?

282

is empty -> returns an empty string.

288

does not return anything (empty StringRef) -> returns an empty string.

[thopre]

Address outstanding review comments

llvm/unittests/Support/FileCheckTest.cpp

239

The substitution only cares about what variable object it points to. The name in the variable object is used only to populate the name of undefined variables in a substitution. Anyway, fixed now.

[jhenderson]

Okay, LGTM, but make sure @probinson is happy at least.

[thopre]

In D60385#1499812, @jhenderson wrote:

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

[thopre]

In D60385#1499819, @thopre wrote:

In D60385#1499812, @jhenderson wrote:

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

[thopre]

In D60385#1499921, @thopre wrote:

In D60385#1499819, @thopre wrote:

In D60385#1499812, @jhenderson wrote:

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

I'm afraid I cannot reproduce this error. I'm using sphinx-build 1.8.5 from the python-sphinx Ubuntu package version 1.6.7-1ubuntu1. Can anyone with a different version try to reproduce? Thanks in advance.

[jhenderson]

In D60385#1500037, @thopre wrote:

In D60385#1499921, @thopre wrote:

In D60385#1499819, @thopre wrote:

In D60385#1499812, @jhenderson wrote:

Okay, LGTM, but make sure @probinson is happy at least.

He wrote "I am pretty happy with this now, maybe give the other people who have been participating a day or so to chime in again" earlier so will commit this now. Thanks everyone who has tirelessly reviewed this patch! It's good to get the first big patch adding support for numeric expression landing.

FYI I've noticed the sphinx error in [1] and am investigated it. I changed laptop recently and sphinx build was not enabled on the new one which is why I missed it. Am doing a build now with sphinx enabled and will send a patch as soon as I can reproduce the issue.

[1] http://lab.llvm.org:8011/builders/llvm-sphinx-docs/builds/31133/steps/docs-llvm-html/logs/stdio

I'm afraid I cannot reproduce this error. I'm using sphinx-build 1.8.5 from the python-sphinx Ubuntu package version 1.6.7-1ubuntu1. Can anyone with a different version try to reproduce? Thanks in advance.

I don't have Sphinx setup on my machine. I'd suggest a couple of things: 1) make sure that the problem was directly caused by your change according to the build bots, and isn't a case of flakiness. 2) Ask for help on the llvm-dev mailing list, CC'ing the build bot maintainer who may be able to better reproduce. You should (unfortunately) probably also revert if you haven't so that the build bots aren't bad for everyone else's changes.

[thopre]

Reopen to easily review changes to fix FileCheck.rst build failure

[thopre]

Fix list in FileCheck.rst

[thopre]

Really submit fix for list in FileCheck.rst

[thopre]

• Address Paul's review comments
• Rename numeric expression subsitution for numeric substitution to clearly distinguish numeric expression (the part that gets evaluated) from the whole #<something> that gets substituted (which might include a numeric variable definition).
• Rename ParsePattern into parsePattern since it is modified

[thopre]

Undo latest submission

[jhenderson]

Unfortunately, the /new link for me isn't working for this. Could you point out what has changed since your previous version?

[thopre]

In D60385#1501063, @jhenderson wrote:

Unfortunately, the /new link for me isn't working for this. Could you point out what has changed since your previous version?

Of course: the list introduced by "The syntax to check a numeric expression constraint is" in FileCheck.rst has now one space between the asterisks and the beginning of the items. When an item span several lines (as is the case for the second and third items in the list), the follow-on lines start by 2 spaces as required by sphinx which also allow them to nicely align with the first line. This allows sphinx to build without warning.

[jhenderson]

LGTM.

llvm/docs/CommandGuide/FileCheck.rst

.. option:: --enable-var-scope
 
   All other variables get undefined after each encountered ``CHECK-LABEL``.
 
 .. option:: -D<VAR=VALUE>
 
   Sets a filecheck pattern variable ``VAR`` with value ``VALUE`` that can be
   used in ``CHECK:`` lines.
 
+.. option:: -D#<NUMVAR>=<VALUE>
+
+  Sets a filecheck numeric variable ``NUMVAR`` to ``<VALUE>`` that can be used
+  in ``CHECK:`` lines.
+
 .. option:: -version
 
  Show the version number of this program.
 
 .. option:: -v
 
   Print good directive pattern matches.  However, if ``-input-dump=fail`` or
   ``-input-dump=always``, add those matches as input annotations instead.

 
 If ``--enable-var-scope`` is in effect, variables with names that
 start with ``$`` are considered to be global. All others variables are
 local.  All local variables get undefined at the beginning of each
 CHECK-LABEL block. Global variables are not affected by CHECK-LABEL.
 This makes it easier to ensure that individual tests are not affected
 by variables set in preceding tests.
 
-FileCheck Numeric Expressions
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+FileCheck Numeric Variables and Expressions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:program:`FileCheck` also allows checking for numeric values that satisfy a
+numeric expression constraint based on numeric variables. This allows
+``CHECK:`` directives to verify a numeric relation between two numbers, such as
+the need for consecutive registers to be used.
+
+The syntax to check a numeric expression constraint is
+``[[#<NUMVAR><op><offset>]]`` where:
+
+*``<NUMVAR>`` is the name of a numeric variable defined on the command line.
+
+*``<op>`` is an optional numeric operation to perform on the value of
+``<NUMVAR>``. Currently supported numeric operations are ``+`` and ``-``.
+
+*``<offset>`` is the immediate value that constitutes the second operand of
+the numeric operation <op>. It must be present if ``<op>`` is present,
+absent otherwise.
+
+For example:
+
+.. code-block:: llvm
+
+    ; CHECK: add r[[#REG]], r[[#REG]], r[[#REG+1]]
+
+The above example would match the line:
+
+.. code-block:: llvm
+
+    add r5, r5, r6
+
+but would not match the line:
+
+.. code-block:: llvm
+
+    add r5, r5, r7
+
+due to ``7`` being unequal to ``5 + 1``.
+
+The ``--enable-var-scope`` option has the same effect on numeric variables as
+on pattern variables.
+
+FileCheck Pseudo Numeric Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes there's a need to verify output that contains line numbers of the
 match file, e.g. when testing compiler diagnostics.  This introduces a certain
 fragility of the match file structure, as "``CHECK:``" lines contain absolute
 line numbers in the same file, which have to be updated whenever line numbers
 change due to text addition or deletion.
 
-To support this case, FileCheck allows using ``[[#@LINE]]``,
-``[[#@LINE+<offset>]]`` and ``[[#@LINE-<offset>]]`` numeric expressions in
-patterns, with an arbitrary number of spaces between each element of the
-expression. These expressions expand to the number of the line where a pattern
-is located (with an optional integer offset).
+To support this case, FileCheck understands the ``@LINE`` pseudo numeric
+variable which evaluates to the line number of the CHECK pattern where it is
+found.
 
 This way match patterns can be put near the relevant test lines and include
 relative line number references, for example:
 
 .. code-block:: c++
 
    // CHECK: test.cpp:[[# @LINE + 4]]:6: error: expected ';' after top level declarator
    // CHECK-NEXT: {{^int a}}

llvm/include/llvm/Support/FileCheck.h

struct FileCheckRequest {
   bool Verbose = false;
   bool VerboseVerbose = false;
 };
 
 //===----------------------------------------------------------------------===//
 // Numeric expression handling code.
 //===----------------------------------------------------------------------===//
 
-/// Class representing a numeric expression.
+/// Class representing a numeric variable with a given value in a numeric
+/// expression.
+class FileCheckNumericVariable {
+private:
+  /// Name of the numeric variable.
+  StringRef Name;
+
+  /// Value of numeric variable, if defined, or None otherwise.
+  llvm::Optional<uint64_t> Value;
+
+public:
+  /// Constructor for numeric variable \p Name with a known \p Value at parse
+  /// time (e.g. the @LINE numeric variable).
+  FileCheckNumericVariable(StringRef Name, uint64_t Value)
+      : Name(Name), Value(Value) {}
+
+  /// \returns name of that numeric variable.
+  StringRef getName() const { return Name; }
+
+  /// \returns value of this numeric variable.
+  llvm::Optional<uint64_t> getValue() const { return Value; }
+
+  /// Sets value of this numeric variable if not defined. \returns whether the
+  /// variable was already defined.
+  bool setValue(uint64_t Value);
+
+  /// Clears value of this numeric variable. \returns whether the variable was
+  /// already undefined.
+  bool clearValue();
+};
+
+/// Type of functions evaluating a given binary operation.
+using binop_eval_t = uint64_t (*)(uint64_t, uint64_t);
+
+/// Class representing a numeric expression consisting of either a single
+/// numeric variable or a binary operation between a numeric variable and an
+/// immediate.
 class FileCheckNumExpr {
 private:
-  /// Value of the numeric expression.
-  uint64_t Value;
+  /// Left operand.
+  FileCheckNumericVariable *LeftOp;
+
+  /// Right operand.
+  uint64_t RightOp;
+
+  /// Pointer to function that can evaluate this binary operation.
+  binop_eval_t EvalBinop;
 
 public:
-  /// Constructor for a numeric expression with a known value at parse time,
-  /// e.g. the implicit numeric expression defining the @LINE numeric pseudo
-  /// variable.
-  explicit FileCheckNumExpr(uint64_t Value) : Value(Value) {}
+  FileCheckNumExpr(binop_eval_t EvalBinop,
+                   FileCheckNumericVariable *OperandLeft, uint64_t OperandRight)
+      : LeftOp(OperandLeft), RightOp(OperandRight), EvalBinop(EvalBinop) {}
+
+  /// Evaluates the value of this numeric expression, using EvalBinop to
+  /// perform the binary operation it consists of. \returns None if the numeric
+  /// variable used is undefined, or the expression value otherwise.
+  llvm::Optional<uint64_t> eval() const;
 
-  /// \returns the value being matched against.
-  uint64_t getValue() const { return Value; }
+  /// \returns the name of the undefined variable used in this expression if
+  /// any or an empty string otherwise.
+  StringRef getUndefVarName() const;
 };
 
 class FileCheckPatternContext;
 
 /// Class representing a substitution to perform in the string to match.
 class FileCheckPatternSubstitution {
 private:
   /// Pointer to a class instance holding the table with the values of live
   /// pattern variables at the start of any given CHECK line. Used for
   /// substituting pattern variables (numeric variables have their value in the
   /// FileCheckNumExpr class instance pointed to by NumExpr).
   FileCheckPatternContext *Context;
 
   /// Whether this represents a numeric expression substitution.
   bool IsNumExpr;
 
   /// The string that needs to be substituted for something else. For a
   /// pattern variable this is its name, otherwise this is the whole numeric
   /// expression.
   StringRef FromStr;
 
   /// If this is a numeric expression substitution, this is the pointer to the
   /// class representing that numeric expression.
   FileCheckNumExpr *NumExpr = nullptr;
 
   // Index in RegExStr of where to do the substitution.
   size_t InsertIdx;
 
 public:
   /// Constructor for a pattern variable substitution.
   FileCheckPatternSubstitution(FileCheckPatternContext *Context,
                                StringRef VarName, size_t InsertIdx)
       : Context(Context), IsNumExpr(false), FromStr(VarName),
         InsertIdx(InsertIdx) {}
 
   /// Constructor for a numeric expression substitution.
   FileCheckPatternSubstitution(FileCheckPatternContext *Context, StringRef Expr,
                                FileCheckNumExpr *NumExpr, size_t InsertIdx)
       : Context(Context), IsNumExpr(true), FromStr(Expr), NumExpr(NumExpr),
         InsertIdx(InsertIdx) {}
 
   /// \returns whether this is a numeric expression substitution.
   bool isNumExpr() const { return IsNumExpr; }
 
   /// \returns the string to be substituted.
   StringRef getFromString() const { return FromStr; }
 
   /// \returns the index where the substitution is to be performed.
   size_t getIndex() const { return InsertIdx; }
 
   /// \returns the result of the substitution represented by this class
-  /// instance or None if substitution failed. For a numeric expression we
-  /// substitute it by its value. For a pattern variable we simply replace it
-  /// by the text its definition matched.
+  /// instance or None if substitution failed. Numeric expressions are
+  /// substituted by their values. Pattern variables are simply replaced by the
+  /// text their definition matched.
   llvm::Optional<std::string> getResult() const;
 
   /// \returns the name of the undefined variable used in this substitution, if
   /// any, or an empty string otherwise.
   StringRef getUndefVarName() const;
 };
 
 //===----------------------------------------------------------------------===//

 
 private:
   /// When matching a given pattern, this holds the value of all the FileCheck
   /// pattern variables defined in previous patterns. In a pattern, only the
   /// last definition for a given variable is recorded in this table.
   /// Back-references are used for uses after any the other definition.
   StringMap<StringRef> GlobalVariableTable;
 
+  /// Map of all pattern variables defined so far. Used at parse time to detect
+  /// a name conflict between a numeric variable and a pattern variable when
+  /// the former is defined on a later line than the latter.
+  StringMap<bool> DefinedVariableTable;
+
+  /// When matching a given pattern, this holds the pointers to the classes
+  /// representing the last definitions of numeric variables defined in
+  /// previous patterns. Earlier definition of the variables, if any, have
+  /// their own class instance not referenced by this table.
+  StringMap<FileCheckNumericVariable *> GlobalNumericVariableTable;
+
   /// Vector holding pointers to all parsed numeric expressions. Used to
   /// automatically free the numeric expressions once they are guaranteed to no
   /// longer be used.
   std::vector<std::unique_ptr<FileCheckNumExpr>> NumExprs;
 
+  /// Vector holding pointers to all parsed numeric variables. Used to
+  /// automatically free them once they are guaranteed to no longer be used.
+  std::vector<std::unique_ptr<FileCheckNumericVariable>> NumericVariables;
+
 public:
   /// \returns the value of pattern variable \p VarName or None if no such
   /// variable has been defined.
   llvm::Optional<StringRef> getPatternVarValue(StringRef VarName);
 
-  /// Defines pattern variables from definitions given on the command line,
-  /// passed as a vector of VAR=VAL strings in \p CmdlineDefines. Reports any
-  /// error to \p SM and \returns whether an error occured.
+  /// Defines pattern and numeric variables from definitions given on the
+  /// command line, passed as a vector of [#]VAR=VAL strings in
+  /// \p CmdlineDefines. Reports any error to \p SM and \returns whether an
+  /// error occured.
   bool defineCmdlineVariables(std::vector<std::string> &CmdlineDefines,
                               SourceMgr &SM);
 
   /// Undefines local variables (variables whose name does not start with a '$'
   /// sign), i.e. removes them from GlobalVariableTable.
   void clearLocalVars();
 
 private:
   /// Makes a new numeric expression instance and registers it for destruction
   /// when the context is destroyed.
-  template <class... Types> FileCheckNumExpr *makeNumExpr(Types... Args);
+  FileCheckNumExpr *makeNumExpr(binop_eval_t EvalBinop,
+                                FileCheckNumericVariable *OperandLeft,
+                                uint64_t OperandRight);
+
+  /// Makes a new numeric variable and registers it for destruction when the
+  /// context is destroyed.
+  FileCheckNumericVariable *makeNumericVariable(StringRef Name, uint64_t Value);
 };
 
 class FileCheckPattern {
   SMLoc PatternLoc;
 
   /// A fixed string to match as the pattern or empty if this pattern requires
   /// a regex match.
   StringRef FixedStr;
 
   /// A regex string to match as the pattern or empty if this pattern requires
   /// a fixed string to match.
   std::string RegExStr;
 
   /// Entries in this vector represent uses of a pattern variable or a numeric
   /// expression in the pattern that need to be substituted in the regexp
-  /// pattern at match time, e.g. "foo[[bar]]baz[[#@LINE+1]]". In this case,
-  /// the RegExStr will contain "foobaz" and we'll get two entries in this
-  /// vector that tells us to insert the value of pattern variable "bar" at
-  /// offset 3 and the value of numeric expression "@LINE+1" at offset 6. Uses
-  /// are represented by a FileCheckPatternSubstitution class to abstract
-  /// whether it is a pattern variable or a numeric expression.
+  /// pattern at match time, e.g. "foo[[bar]]baz[[#N+1]]". In this case, the
+  /// RegExStr will contain "foobaz" and we'll get two entries in this vector
+  /// that tells us to insert the value of pattern variable "bar" at offset 3
+  /// and the value of numeric expression "N+1" at offset 6. Uses are
+  /// represented by a FileCheckPatternSubstitution class to abstract whether
+  /// it is a pattern variable or a numeric expression.
   std::vector<FileCheckPatternSubstitution> Substitutions;
 
   /// Maps names of pattern variables defined in a pattern to the parenthesized
   /// capture numbers of their last definition.
   ///
   /// E.g. for the pattern "foo[[bar:.*]]baz[[bar]]quux[[bar:.*]]",
   /// VariableDefs will map "bar" to 2 corresponding to the second definition
   /// of "bar".
   ///
   /// Note: uses std::map rather than StringMap to be able to get the key when
   /// iterating over values.
   std::map<StringRef, unsigned> VariableDefs;
 
-  /// Pointer to the class instance shared by all patterns holding a table with
-  /// the values of live variables at the start of any given CHECK line.
+  /// Pointer to a class instance holding the global state shared by all
+  /// patterns:
+  /// - separate tables with the values of live pattern and numeric variables
+  ///   respectively at the start of any given CHECK line;
+  /// - table holding whether a pattern variable has been defined at any given
+  ///   point during the parsing phase.
   FileCheckPatternContext *Context;
 
   Check::FileCheckType CheckTy;
 
   /// Contains the number of line this pattern is in.
   unsigned LineNumber;
 
 public:
public:
   /// \returns whether \p C is a valid first character for a variable name.
   static bool isValidVarNameStart(char C);
   /// Verifies that the string at the start of \p Str is a well formed
   /// variable. \returns false if it is and sets \p IsPseudo to indicate if it
   /// is a pseudo variable and \p TrailIdx to the position of the last
   /// character that is part of the variable name. Otherwise, only
   /// \returns true.
   static bool parseVariable(StringRef Str, bool &IsPseudo, unsigned &TrailIdx);
-  /// Parses a numeric expression involving pseudo variable \p Name with the
-  /// string corresponding to the operation being performed in \p Trailer.
-  /// \returns the class representing the numeric expression or nullptr if
-  /// parsing fails in which case errors are reported on \p SM.
-  FileCheckNumExpr *parseNumericExpression(StringRef Name, StringRef Trailer,
+  /// Parses a numeric expression involving (pseudo if \p IsPseudo is true)
+  /// variable \p Name with the string corresponding to the operation being
+  /// performed in \p Trailer. \returns the class representing the numeric
+  /// expression or nullptr if parsing fails in which case errors are reported
+  /// on \p SM.
+  FileCheckNumExpr *parseNumericExpression(StringRef Name, bool IsPseudo,
+                                           StringRef Trailer,
                                            const SourceMgr &SM) const;
   /// Parses the pattern in \p PatternStr and initializes this FileCheckPattern
   /// instance accordingly.
   ///
   /// \p Prefix provides which prefix is being matched, \p Req describes the
   /// global options that influence the parsing such as whitespace
   /// canonicalization, \p SM provides the SourceMgr used for error reports,
   /// and \p LineNumber is the line number in the input file from which the

llvm/lib/Support/FileCheck.cpp

 #include <cstdint>
 #include <list>
 #include <map>
 #include <tuple>
 #include <utility>
 
 using namespace llvm;
 
+bool FileCheckNumericVariable::setValue(uint64_t NewValue) {
+  if (Value)
+    return true;
+  Value = NewValue;
+  return false;
+}
+
+bool FileCheckNumericVariable::clearValue() {
+  if (!Value)
+    return true;
+  Value = llvm::None;
+  return false;
+}
+
+llvm::Optional<uint64_t> FileCheckNumExpr::eval() const {
+  llvm::Optional<uint64_t> LeftOp = this->LeftOp->getValue();
+  // Variable is undefined.
+  if (!LeftOp)
+    return llvm::None;
+  return EvalBinop(*LeftOp, RightOp);
+}
+
+StringRef FileCheckNumExpr::getUndefVarName() const {
+  if (!LeftOp->getValue())
+    return LeftOp->getName();
+  return StringRef();
+}
+
 llvm::Optional<std::string> FileCheckPatternSubstitution::getResult() const {
   if (IsNumExpr) {
-    return utostr(NumExpr->getValue());
-  } else {
-    // Look up the value and escape it so that we can put it into the
-    // regex.
-    llvm::Optional<StringRef> VarVal = Context->getPatternVarValue(FromStr);
-    if (!VarVal)
-      return llvm::None;
-    return Regex::escape(*VarVal);
-  }
-}
+    llvm::Optional<uint64_t> EvaluatedValue = NumExpr->eval();
+    if (!EvaluatedValue)
+      return llvm::None;
+    return utostr(*EvaluatedValue);
+  }
+
+  // Look up the value and escape it so that we can put it into the regex.
 
 StringRef FileCheckPatternSubstitution::getUndefVarName() const {
-  // Parsing guarantees only @LINE is ever referenced and it is not undefined
-  // by ClearLocalVars.
   if (IsNumExpr)
-    return StringRef();
+    // Although a use of an undefined numeric variable is detected at parse
+    // time, a numeric variable can be undefined later by ClearLocalVariables.
+    return NumExpr->getUndefVarName();
 
   if (!Context->getPatternVarValue(FromStr))
     return FromStr;
 
   return StringRef();
 }
 
 bool FileCheckPattern::isValidVarNameStart(char C) {
   return C == '_' || isalpha(C);
 }
 
 bool FileCheckPattern::parseVariable(StringRef Str, bool &IsPseudo,
                                      unsigned &TrailIdx) {
   if (Str.empty())

 
 // Parsing helper function that strips the first character in S and returns it.
 static char popFront(StringRef &S) {
   char C = S.front();
   S = S.drop_front();
   return C;
 }
 
+static uint64_t add(uint64_t LeftOp, uint64_t RightOp) {
+  return LeftOp + RightOp;
+}
+static uint64_t sub(uint64_t LeftOp, uint64_t RightOp) {
+  return LeftOp - RightOp;
+}
+
 FileCheckNumExpr *
-FileCheckPattern::parseNumericExpression(StringRef Name, StringRef Trailer,
+FileCheckPattern::parseNumericExpression(StringRef Name, bool IsPseudo,
+                                         StringRef Trailer,
                                          const SourceMgr &SM) const {
-  if (!Name.equals("@LINE")) {
+  if (IsPseudo && !Name.equals("@LINE")) {
     SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
                     "invalid pseudo numeric variable '" + Name + "'");
     return nullptr;
   }
 
-  // Check if this is a supported operation and select function to perform it.
+  // This method is indirectly called from ParsePattern for all numeric
+  // variable definitions and uses in the order in which they appear in the
+  // CHECK pattern. For each definition, the pointer to the class instance of
+  // the corresponding numeric variable definition is stored in
+  // GlobalNumericVariableTable. Therefore, the pointer we get below is for the
+  // class instance corresponding to the last definition of this variable use.
+  auto VarTableIter = Context->GlobalNumericVariableTable.find(Name);
+  if (VarTableIter == Context->GlobalNumericVariableTable.end()) {
+    SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+                    "using undefined numeric variable '" + Name + "'");
+    return nullptr;
+  }
+
+  FileCheckNumericVariable *LeftOp = VarTableIter->second;
+
+  // Check if this is a supported operation and select a function to perform
+  // it.
   Trailer = Trailer.ltrim(SpaceChars);
-  if (Trailer.empty())
-    return Context->makeNumExpr(LineNumber);
+  if (Trailer.empty()) {
+    return Context->makeNumExpr(add, LeftOp, 0);
+  }
   SMLoc OpLoc = SMLoc::getFromPointer(Trailer.data());
   char Operator = popFront(Trailer);
+  binop_eval_t EvalBinop;
+  switch (Operator) {
+  case '+':
+    EvalBinop = add;
+    break;
+  case '-':
+    EvalBinop = sub;
+    break;
+  default:
+    SM.PrintMessage(OpLoc, SourceMgr::DK_Error,
+                    Twine("unsupported numeric operation '") + Twine(Operator) +
+                        "'");
+    return nullptr;
+  }
 
   // Parse right operand.
   Trailer = Trailer.ltrim(SpaceChars);
   if (Trailer.empty()) {
     SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()), SourceMgr::DK_Error,
-                    "missing operand in numeric expression '" + Trailer + "'");
+                    "missing operand in numeric expression");
     return nullptr;
   }
-  uint64_t Offset;
-  if (Trailer.consumeInteger(10, Offset)) {
+  uint64_t RightOp;
+  if (Trailer.consumeInteger(10, RightOp)) {
     SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()), SourceMgr::DK_Error,
                     "invalid offset in numeric expression '" + Trailer + "'");
     return nullptr;
   }
   Trailer = Trailer.ltrim(SpaceChars);
   if (!Trailer.empty()) {
     SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()), SourceMgr::DK_Error,
                     "unexpected characters at end of numeric expression '" +
                         Trailer + "'");
     return nullptr;
   }
 
-  uint64_t Value;
-  switch (Operator) {
-  case '+':
-    Value = LineNumber + Offset;
-    break;
-  case '-':
-    Value = LineNumber - Offset;
-    break;
-  default:
-    SM.PrintMessage(OpLoc, SourceMgr::DK_Error,
-                    Twine("unsupported numeric operation '") + Twine(Operator) +
-                        "'");
-    return nullptr;
-  }
-  return Context->makeNumExpr(Value);
+  return Context->makeNumExpr(EvalBinop, LeftOp, RightOp);
 }
 
 bool FileCheckPattern::ParsePattern(StringRef PatternStr, StringRef Prefix,
                                     SourceMgr &SM, unsigned LineNumber,
                                     const FileCheckRequest &Req) {
   bool MatchFullLinesHere = Req.MatchFullLines && CheckTy != Check::CheckNot;
 
   this->LineNumber = LineNumber;
   PatternLoc = SMLoc::getFromPointer(PatternStr.data());
 
+  // Create fake @LINE pseudo variable definition.
+  StringRef LinePseudo = "@LINE";
+  uint64_t LineNumber64 = LineNumber;
+  FileCheckNumericVariable *LinePseudoVar =
+      Context->makeNumericVariable(LinePseudo, LineNumber64);
+  Context->GlobalNumericVariableTable[LinePseudo] = LinePseudoVar;
+
   if (!(Req.NoCanonicalizeWhiteSpace && Req.MatchFullLines))
     // Ignore trailing whitespace.
     while (!PatternStr.empty() &&
            (PatternStr.back() == ' ' || PatternStr.back() == '\t'))
       PatternStr = PatternStr.substr(0, PatternStr.size() - 1);
 
   // Check that there is something on the line.
   if (PatternStr.empty() && CheckTy != Check::CheckEmpty) {
if (PatternStr.startswith("{{")) {
       PatternStr = PatternStr.substr(End + 2);
       continue;
     }
 
     // Pattern and numeric expression matches. Pattern expressions come in two
     // forms: [[foo:.*]] and [[foo]]. The former matches .* (or some other
     // regex) and assigns it to the FileCheck variable 'foo'. The latter
     // substitutes foo's value. Numeric expressions start with a '#' sign after
-    // the double brackets and only have the substitution form. Pattern
-    // variables must satisfy the regular expression "[a-zA-Z_][0-9a-zA-Z_]*"
-    // to be valid, as this helps catch some common errors. Numeric expressions
-    // only support the @LINE pseudo numeric variable.
+    // the double brackets and only have the substitution form. Both pattern
+    // and numeric variables must satisfy the regular expression
+    // "[a-zA-Z_][0-9a-zA-Z_]*" to be valid, as this helps catch some common
+    // errors.
     if (PatternStr.startswith("[[")) {
       StringRef UnparsedPatternStr = PatternStr.substr(2);
       // Find the closing bracket pair ending the match.  End is going to be an
       // offset relative to the beginning of the match string.
       size_t End = FindRegexVarEnd(UnparsedPatternStr, SM);
       StringRef MatchStr = UnparsedPatternStr.substr(0, End);
       bool IsNumExpr = MatchStr.consume_front("#");
       const char *RefTypeStr =
if (PatternStr.startswith("[[")) {
 
       size_t SubstInsertIdx = RegExStr.size();
       FileCheckNumExpr *NumExpr;
 
       StringRef Name = MatchStr.substr(0, TrailIdx);
       StringRef Trailer = MatchStr.substr(TrailIdx);
       bool IsVarDef = (VarEndIdx != StringRef::npos);
 
-      if (IsVarDef && (IsPseudo || !Trailer.consume_front(":"))) {
+      if (IsVarDef) {
+        if (IsPseudo || !Trailer.consume_front(":")) {
           SM.PrintMessage(SMLoc::getFromPointer(MatchStr.data()),
                           SourceMgr::DK_Error,
                           "invalid name in pattern variable definition");
           return true;
         }
 
-      if (!IsVarDef && IsPseudo) {
-        NumExpr = parseNumericExpression(Name, Trailer, SM);
+        // Detect collisions between pattern and numeric variables when the
+        // former is created later than the latter.
+        if (Context->GlobalNumericVariableTable.find(Name) !=
+            Context->GlobalNumericVariableTable.end()) {
+          SM.PrintMessage(
+              SMLoc::getFromPointer(MatchStr.data()), SourceMgr::DK_Error,
+              "numeric variable with name '" + Name + "' already exists");
+          return true;
+        }
+      }
+
+      if (IsNumExpr || (!IsVarDef && IsPseudo)) {
+        NumExpr = parseNumericExpression(Name, IsPseudo, Trailer, SM);
         if (NumExpr == nullptr)
           return true;
         IsNumExpr = true;
       }
 
-      // Handle [[foo]].
+      // Handle variable use: [[foo]] and [[#<foo expr>]].
       if (!IsVarDef) {
         // Handle use of pattern variables that were defined earlier on the
         // same line by emitting a backreference.
         if (!IsNumExpr && VariableDefs.find(Name) != VariableDefs.end()) {
           unsigned CaptureParen = VariableDefs[Name];
           if (CaptureParen < 1 || CaptureParen > 9) {
             SM.PrintMessage(SMLoc::getFromPointer(Name.data()),
                             SourceMgr::DK_Error,

 FileCheckPatternContext::getPatternVarValue(StringRef VarName) {
   auto VarIter = GlobalVariableTable.find(VarName);
   if (VarIter == GlobalVariableTable.end())
     return llvm::None;
 
   return VarIter->second;
 }
 
-template <class... Types>
-FileCheckNumExpr *FileCheckPatternContext::makeNumExpr(Types... Args) {
-  NumExprs.emplace_back(new FileCheckNumExpr(Args...));
+FileCheckNumExpr *
+FileCheckPatternContext::makeNumExpr(binop_eval_t EvalBinop,
+                                     FileCheckNumericVariable *OperandLeft,
+                                     uint64_t OperandRight) {
+  NumExprs.push_back(llvm::make_unique<FileCheckNumExpr>(EvalBinop, OperandLeft,
+                                                         OperandRight));
   return NumExprs.back().get();
 }
 
+FileCheckNumericVariable *
+FileCheckPatternContext::makeNumericVariable(StringRef Name, uint64_t Value) {
+  NumericVariables.push_back(
+      llvm::make_unique<FileCheckNumericVariable>(Name, Value));
+  return NumericVariables.back().get();
+}
+
 size_t FileCheckPattern::FindRegexVarEnd(StringRef Str, SourceMgr &SM) {
   // Offset keeps track of the current offset within the input Str
   size_t Offset = 0;
   // [...] Nesting depth
   size_t BracketDepth = 0;
 
   while (!Str.empty()) {
     if (Str.startswith("]]") && BracketDepth == 0)
for (StringRef Prefix : Req.CheckPrefixes) {
     PrefixRegexStr.append(Prefix);
   }
 
   return Regex(PrefixRegexStr);
 }
 
 bool FileCheckPatternContext::defineCmdlineVariables(
     std::vector<std::string> &CmdlineDefines, SourceMgr &SM) {
-  assert(GlobalVariableTable.empty() &&
+  assert(GlobalVariableTable.empty() && GlobalNumericVariableTable.empty() &&
          "Overriding defined variable with command-line variable definitions");
 
   if (CmdlineDefines.empty())
     return false;
 
   // Create a string representing the vector of command-line definitions. Each
   // definition is on its own line and prefixed with a definition number to
   // clarify which definition a given diagnostic corresponds to.
   unsigned I = 0;
   bool ErrorFound = false;
   std::string CmdlineDefsDiag;
   StringRef Prefix1 = "Global define #";
   StringRef Prefix2 = ": ";
   for (StringRef CmdlineDef : CmdlineDefines)
     CmdlineDefsDiag +=
         (Prefix1 + Twine(++I) + Prefix2 + CmdlineDef + "\n").str();
 
+  // Create a buffer with fake command line content in order to display
+  // parsing diagnostic with location information and point to the
+  // global definition with invalid syntax.
   std::unique_ptr<MemoryBuffer> CmdLineDefsDiagBuffer =
       MemoryBuffer::getMemBufferCopy(CmdlineDefsDiag, "Global defines");
   StringRef CmdlineDefsDiagRef = CmdLineDefsDiagBuffer->getBuffer();
   SM.AddNewSourceBuffer(std::move(CmdLineDefsDiagBuffer), SMLoc());
 
   SmallVector<StringRef, 4> CmdlineDefsDiagVec;
   CmdlineDefsDiagRef.split(CmdlineDefsDiagVec, '\n', -1 /*MaxSplit*/,
                            false /*KeepEmpty*/);
   for (StringRef CmdlineDefDiag : CmdlineDefsDiagVec) {
-    unsigned NameStart = CmdlineDefDiag.find(Prefix2) + Prefix2.size();
-    if (CmdlineDefDiag.substr(NameStart).find('=') == StringRef::npos) {
-      SM.PrintMessage(SMLoc::getFromPointer(CmdlineDefDiag.data()),
+    unsigned DefStart = CmdlineDefDiag.find(Prefix2) + Prefix2.size();
+    StringRef CmdlineDef = CmdlineDefDiag.substr(DefStart);
+    if (CmdlineDef.find('=') == StringRef::npos) {
+      SM.PrintMessage(SMLoc::getFromPointer(CmdlineDef.data()),
                       SourceMgr::DK_Error,
                       "Missing equal sign in global definition");
       ErrorFound = true;
       continue;
     }
-    std::pair<StringRef, StringRef> CmdlineNameVal =
-        CmdlineDefDiag.substr(NameStart).split('=');
+
+    // Numeric variable definition.
+    if (CmdlineDef[0] == '#') {
+      bool IsPseudo;
+      unsigned TrailIdx;
+      size_t EqIdx = CmdlineDef.find('=');
+      StringRef CmdlineName = CmdlineDef.substr(1, EqIdx - 1);
+      if (FileCheckPattern::parseVariable(CmdlineName, IsPseudo, TrailIdx) ||
+          IsPseudo || TrailIdx != CmdlineName.size() || CmdlineName.empty()) {
+        SM.PrintMessage(SMLoc::getFromPointer(CmdlineName.data()),
+                        SourceMgr::DK_Error,
+                        "invalid name in numeric variable definition '" +
+                            CmdlineName + "'");
+        ErrorFound = true;
+        continue;
+      }
+
+      // Detect collisions between pattern and numeric variables when the
+      // latter is created later than the former.
+      if (DefinedVariableTable.find(CmdlineName) !=
+          DefinedVariableTable.end()) {
+        SM.PrintMessage(
+            SMLoc::getFromPointer(CmdlineName.data()), SourceMgr::DK_Error,
+            "pattern variable with name '" + CmdlineName + "' already exists");
+        ErrorFound = true;
+        continue;
+      }
+
+      StringRef CmdlineVal = CmdlineDef.substr(EqIdx + 1);
+      uint64_t Val;
+      if (CmdlineVal.getAsInteger(10, Val)) {
+        SM.PrintMessage(SMLoc::getFromPointer(CmdlineVal.data()),
+                        SourceMgr::DK_Error,
+                        "invalid value in numeric variable definition '" +
+                            CmdlineVal + "'");
+        ErrorFound = true;
+        continue;
+      }
+      auto DefinedNumericVariable = makeNumericVariable(CmdlineName, Val);
+
+      // Record this variable definition.
+      GlobalNumericVariableTable[CmdlineName] = DefinedNumericVariable;
+    } else {
+      // Pattern variable definition.
+      std::pair<StringRef, StringRef> CmdlineNameVal = CmdlineDef.split('=');
       StringRef Name = CmdlineNameVal.first;
       bool IsPseudo;
       unsigned TrailIdx;
-    if (FileCheckPattern::parseVariable(Name, IsPseudo, TrailIdx) || IsPseudo ||
-        TrailIdx != Name.size() || Name.empty()) {
+      if (FileCheckPattern::parseVariable(Name, IsPseudo, TrailIdx) ||
+          IsPseudo || TrailIdx != Name.size() || Name.empty()) {
         SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
-                      "invalid name for variable definition '" + Name + "'");
+                        "invalid name in pattern variable definition '" + Name +
+                            "'");
+        ErrorFound = true;
+        continue;
+      }
+
+      // Detect collisions between pattern and numeric variables when the
+      // former is created later than the latter.
+      if (GlobalNumericVariableTable.find(Name) !=
+          GlobalNumericVariableTable.end()) {
+        SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+                        "numeric variable with name '" + Name +
+                            "' already exists");
         ErrorFound = true;
         continue;
       }
       GlobalVariableTable.insert(CmdlineNameVal);
+      // Mark the pattern variable as defined to detect collisions between
+      // pattern and numeric variables in DefineCmdlineVariables when the
+      // latter is created later than the former. We cannot reuse
+      // GlobalVariableTable for that by populating it with an empty string
+      // since we would then lose the ability to detect the use of an undefined
+      // variable in Match().
+      DefinedVariableTable[Name] = true;
+    }
   }
 
   return ErrorFound;
 }
 
 void FileCheckPatternContext::clearLocalVars() {
   SmallVector<StringRef, 16> LocalPatternVars, LocalNumericVars;
   for (const StringMapEntry<StringRef> &Var : GlobalVariableTable)
     if (Var.first()[0] != '$')
       LocalPatternVars.push_back(Var.first());
 
+  // Numeric expression substitution reads the value of a variable directly,
+  // not via GlobalNumericVariableTable. Therefore, we clear local variables by
+  // clearing their value which will lead to a numeric expression substitution
+  // failure. We also mark the variable for removal from
+  // GlobalNumericVariableTable since this is what defineCmdlineVariables
+  // checks to decide that no global variable has been defined.
+  for (const auto &Var : GlobalNumericVariableTable)
+    if (Var.first()[0] != '$') {
+      Var.getValue()->clearValue();
+      LocalNumericVars.push_back(Var.first());
+    }
+
   for (const auto &Var : LocalPatternVars)
     GlobalVariableTable.erase(Var);
+  for (const auto &Var : LocalNumericVars)
+    GlobalNumericVariableTable.erase(Var);
 }
 
 bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
                                  ArrayRef<FileCheckString> CheckStrings,
                                  std::vector<FileCheckDiag> *Diags) {
   bool ChecksFailed = false;
 
   unsigned i = 0, j = 0, e = CheckStrings.size();
if (j == e) {
         // Immediately bail if CHECK-LABEL fails, nothing else we can do.
         return false;
 
       CheckRegion = Buffer.substr(0, MatchLabelPos + MatchLabelLen);
       Buffer = Buffer.substr(MatchLabelPos + MatchLabelLen);
       ++j;
     }
 
-    if (Req.EnableVarScope)
+    // Do not clear the first region as it's the one before the first
+    // CHECK-LABEL and it would clear variables defined on the command-line
+    // before they get used.
+    if (i != 0 && Req.EnableVarScope)
       PatternContext.clearLocalVars();
 
     for (; i != j; ++i) {
       const FileCheckString &CheckStr = CheckStrings[i];
 
       // Check each string within the scanned region, including a second check
       // of any final CHECK-LABEL (to verify CHECK-NOT and CHECK-DAG)
       size_t MatchLen = 0;

llvm/test/FileCheck/numeric-defines-diagnostics.txt

+; Test incorrect syntax for -D# option is correctly diagnosed.
+
+; Invalid variable name: starts with a digit.
+RUN: not FileCheck -D#10VALUE=10 --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NUMERRCLIFMT
+
+NUMERRCLIFMT: Global defines:1:20: error: invalid name in numeric variable definition '10VALUE'
+NUMERRCLIFMT-NEXT: Global define #1: #10VALUE=10
+NUMERRCLIFMT-NEXT: {{^                   \^$}}
+
+; Invalid definition of pseudo variable.
+RUN: not FileCheck -D#@VALUE=10 --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NUMERRCLIPSEUDO
+
+NUMERRCLIPSEUDO: Global defines:1:20: error: invalid name in numeric variable definition '@VALUE'
+NUMERRCLIPSEUDO-NEXT: Global define #1: #@VALUE=10
+NUMERRCLIPSEUDO-NEXT: {{^                   \^$}}
+
+; Invalid definition of an expression.
+RUN: not FileCheck -D#VALUE+2=10 --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NUMERRCLITRAIL
+
+NUMERRCLITRAIL: Global defines:1:20: error: invalid name in numeric variable definition 'VALUE+2'
+NUMERRCLITRAIL-NEXT: Global define #1: #VALUE+2=10
+NUMERRCLITRAIL-NEXT: {{^                   \^$}}
+
+; Invalid value: numeric expression instead of literal.
+RUN: not FileCheck -D#VALUE1=3 -D#VALUE2='VALUE1 + 2' --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NUMERRCLIEXPR
+
+NUMERRCLIEXPR: Global defines:2:27: error: invalid value in numeric variable definition 'VALUE1 + 2'
+NUMERRCLIEXPR-NEXT: Global define #2: #VALUE2=VALUE1 + 2
+NUMERRCLIEXPR-NEXT: {{^                          \^$}}

llvm/test/FileCheck/numeric-defines.txt

+; Test functionality of -D# option: numeric variables are defined to the right
+; value and CHECK directives using them match as expected given the value set.
+
+RUN: FileCheck -D#NUMVAL=12 --check-prefix CHECKNUM --input-file %s %s
+RUN: not FileCheck -D#NUMVAL=8 --check-prefix CHECKNUM --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NUMERRMSG
+RUN: not FileCheck -D#NUMVAL=12 --check-prefix NUMNOT --input-file %s %s 2>&1 \
+RUN:   | FileCheck %s --strict-whitespace --check-prefix NOT-NUMERRMSG
+RUN: FileCheck -D#NUMVAL=8 --check-prefixes NUMNOT --input-file %s %s
+
+Numeric value = 12
+CHECKNUM: Numeric value = [[#NUMVAL]]
+NUMNOT-NOT: Numeric value = [[#NUMVAL]]
+
+NUMERRMSG: defines.txt:[[#@LINE-3]]:11: error: CHECKNUM: expected string not found in input
+NUMERRMSG: defines.txt:1:1: note: scanning from here
+NUMERRMSG: defines.txt:1:1: note: with numeric expression "NUMVAL" equal to "8"
+NUMERRMSG: defines.txt:[[#@LINE-7]]:1: note: possible intended match here
+
+NOT-NUMERRMSG: defines.txt:[[#@LINE-7]]:13: error: {{NUMNOT}}-NOT: excluded string found in input
+NOT-NUMERRMSG: defines.txt:[[#@LINE-10]]:1: note: found here
+NOT-NUMERRMSG: defines.txt:[[#@LINE-11]]:1: note: with numeric expression "NUMVAL" equal to "12"

llvm/test/FileCheck/numeric-expression.txt

+RUN: FileCheck -D#VAR1=11 --input-file %s %s
+
+; We use CHECK-NEXT directives to force a match on all lines with digits.
+
+; Numeric expressions using variables defined on the command-line without
+; spaces
+USE NO SPC
+11
+12
+10
+CHECK-LABEL: USE NO SPC
+CHECK-NEXT: [[#VAR1]]
+CHECK-NEXT: [[#VAR1+1]]
+CHECK-NEXT: [[#VAR1-1]]
+
+; Numeric expressions using variables defined on the command-line in alternate
+; spacing
+USE ALT SPC
+11
+11
+12
+12
+12
+12
+10
+10
+10
+10
+CHECK-LABEL: USE ALT SPC
+CHECK-NEXT: [[# VAR1]]
+CHECK-NEXT: [[# VAR1 ]]
+CHECK-NEXT: [[# VAR1+1]]
+CHECK-NEXT: [[# VAR1 +1]]
+CHECK-NEXT: [[# VAR1 + 1]]
+CHECK-NEXT: [[# VAR1 + 1 ]]
+CHECK-NEXT: [[# VAR1-1]]
+CHECK-NEXT: [[# VAR1 -1]]
+CHECK-NEXT: [[# VAR1 - 1]]
+CHECK-NEXT: [[# VAR1 - 1 ]]
+
+; Numeric expressions using variables defined on the command-line and an
+; immediate interpreted as an unsigned value
+; Note: 9223372036854775819 = 0x8000000000000000 + 11
+;       9223372036854775808 = 0x8000000000000000
+USE UNSIGNED IMM
+9223372036854775819
+CHECK-LABEL: USE UNSIGNED IMM
+CHECK-NEXT: [[#VAR1+9223372036854775808]]
+
+; Numeric expression using undefined variable
+RUN: not FileCheck --check-prefix UNDEF-USE --input-file %s %s 2>&1 \
+RUN:   | FileCheck --strict-whitespace --check-prefix UNDEF-USE-MSG %s
+
+UNDEF VAR USE
+UNDEFVAR: 11
+UNDEF-USE-LABEL: UNDEF VAR USE
+UNDEF-USE-NEXT: UNDEFVAR: [[#UNDEFVAR]]
+UNDEF-USE-MSG: numeric-expression.txt:[[#@LINE-1]]:30: error: using undefined numeric variable 'UNDEFVAR'
+UNDEF-USE-MSG-NEXT: {{U}}NDEF-USE-NEXT: UNDEFVAR: {{\[\[#UNDEFVAR\]\]}}
+UNDEF-USE-MSG-NEXT: {{^                             \^$}}
+
+; Numeric expression with unsupported operator
+RUN: not FileCheck -D#VAR1=11 --check-prefixes CHECK,INVAL-OP --input-file %s %s 2>&1 \
+RUN:   | FileCheck --strict-whitespace --check-prefix INVAL-OP-MSG %s
+
+INVALID OPERATOR
+VAR1*2: 22
+INVAL-OP-LABEL: INVALID OPERATOR
+INVAL-OP-NEXT: VAR1*2: [[#VAR1*2]]
+INVAL-OP-MSG: numeric-expression.txt:[[#@LINE-1]]:31: error: unsupported numeric operation '*'
+INVAL-OP-MSG-NEXT: {{I}}NVAL-OP-NEXT: VAR1*2: {{\[\[#VAR1\*2\]\]}}
+INVAL-OP-MSG-NEXT: {{^                              \^$}}
+
+; Name conflict between Numeric variable definition and pattern variable
+; definition
+RUN: not FileCheck -D#VAR1=11 -D#NUMVAR=42 --check-prefixes CONFLICT,CONFLICT1 --input-file %s %s 2>&1 \
+RUN:   | FileCheck --strict-whitespace --check-prefix CLI-INPUT-PAT-CONFLICT %s
+RUN: not FileCheck -D#VAR1=11 -D#NUMVAR=42 -DNUMVAR=foobar --check-prefix CONFLICT --input-file %s %s 2>&1 \
+RUN:   | FileCheck --strict-whitespace --check-prefix CLI-CLI-PAT-CONFLICT %s
+RUN: not FileCheck -D#VAR1=11 -DPATVAR=foobar -D#PATVAR=42 --check-prefix CONFLICT --input-file %s %s 2>&1 \
+RUN:   | FileCheck --strict-whitespace --check-prefix CLI-CLI-NUM-CONFLICT %s
+
+PATVAR NUMVAR CONFLICT
+foobar
+CONFLICT-LABEL: PATVAR NUMVAR CONFLICT
+CONFLICT1-NEXT: [[NUMVAR:foo.*]]
+CLI-INPUT-PAT-CONFLICT: numeric-expression.txt:[[#@LINE-1]]:19: error: numeric variable with name 'NUMVAR' already exists
+CLI-INPUT-PAT-CONFLICT-NEXT: {{C}}ONFLICT1-NEXT: {{\[\[NUMVAR:foo\.\*\]\]}}
+CLI-INPUT-PAT-CONFLICT-NEXT: {{^                  \^$}}
+CLI-CLI-PAT-CONFLICT: Global defines:3:19: error: numeric variable with name 'NUMVAR' already exists
+CLI-CLI-PAT-CONFLICT-NEXT: Global define #3: NUMVAR=foobar
+CLI-CLI-PAT-CONFLICT-NEXT: {{^                  \^$}}
+CLI-CLI-NUM-CONFLICT: Global defines:3:20: error: pattern variable with name 'PATVAR' already exists
+CLI-CLI-NUM-CONFLICT-NEXT: Global define #3: #PATVAR=42
+CLI-CLI-NUM-CONFLICT-NEXT: {{^                   \^$}}

llvm/test/FileCheck/pattern-defines-diagnostics.txt

 ; Test incorrect syntax for -D option is correctly diagnosed.
 
 ; Missing equal sign.
 RUN: not FileCheck -DVALUE10 --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --check-prefix ERRCLIEQ1
 
 ERRCLIEQ1: Missing equal sign in command-line definition '-DVALUE10'
 
 ; Missing definition expression (variable name, equal sign and value).
 RUN: not FileCheck -D --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --check-prefix ERRCLIEQ2
 
 ERRCLIEQ2: {{F|f}}ile{{C|c}}heck{{[^:]*}}: for the -D option: requires a value!
 
 ; Missing variable name.
 RUN: not FileCheck -D=10 --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --check-prefix ERRCLIVAR1
 
 ERRCLIVAR1: Missing pattern variable name in command-line definition '-D=10'
 
 ; Missing variable name.
 RUN: not FileCheck -D= --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --check-prefix ERRCLIVAR2
 
 ERRCLIVAR2: Missing pattern variable name in command-line definition '-D='
 
 ; Invalid variable name: starts with a digit.
 RUN: not FileCheck -D10VALUE=10 --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --strict-whitespace --check-prefix ERRCLIFMT
 
-ERRCLIFMT: Global defines:1:19: error: invalid name for variable definition '10VALUE'
+ERRCLIFMT: Global defines:1:19: error: invalid name in pattern variable definition '10VALUE'
 ERRCLIFMT-NEXT: Global define #1: 10VALUE=10
 ERRCLIFMT-NEXT: {{^                  \^$}}
 
 ; Invalid definition of pseudo variable.
 RUN: not FileCheck -D@VALUE=10 --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --strict-whitespace --check-prefix ERRCLIPSEUDO
 
-ERRCLIPSEUDO: Global defines:1:19: error: invalid name for variable definition '@VALUE'
+ERRCLIPSEUDO: Global defines:1:19: error: invalid name in pattern variable definition '@VALUE'
 ERRCLIPSEUDO-NEXT: Global define #1: @VALUE=10
 ERRCLIPSEUDO-NEXT: {{^                  \^$}}
 
 ; Invalid definition of an expression.
 RUN: not FileCheck -D'VALUE + 2=10' --input-file %s %s 2>&1 \
 RUN:   | FileCheck %s --strict-whitespace --check-prefix ERRCLITRAIL
 
-ERRCLITRAIL: Global defines:1:19: error: invalid name for variable definition 'VALUE + 2'
+ERRCLITRAIL: Global defines:1:19: error: invalid name in pattern variable definition 'VALUE + 2'
 ERRCLITRAIL-NEXT: Global define #1: VALUE + 2=10
 ERRCLITRAIL-NEXT: {{^                  \^$}}

llvm/test/FileCheck/regex-scope.txt

-// RUN: FileCheck -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,GLOBAL -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,LOCAL -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,GLOBAL --enable-var-scope -input-file %s %s
-// RUN: not FileCheck -check-prefixes CHECK,LOCAL --enable-var-scope -input-file %s %s
-
-local
-global
-; CHECK: [[LOCAL:loc.*]]
-; CHECK: [[$GLOBAL:glo.*]]
-
-local2
-global2
-; CHECK: [[LOCAL]]2
-; CHECK: [[$GLOBAL]]2
-
-barrier:
-; CHECK-LABEL: barrier
-
-local3
-global3
-; LOCAL: [[LOCAL]]3
-; GLOBAL: [[$GLOBAL]]3

llvm/test/FileCheck/var-scope.txt

+; Test that variables not starting with dollar sign get undefined after a
+; CHECK-LABEL directive iff --enable-var-scope is used.
+
+RUN: FileCheck -D#LOCNUM=1 -D'#$GLOBNUM=1' --check-prefixes CHECK,LOCAL3,GLOBAL --input-file %s %s
+RUN: FileCheck -D#LOCNUM=1 -D'#$GLOBNUM=1' --check-prefixes CHECK,GLOBAL --enable-var-scope --input-file %s %s
+RUN: not FileCheck -D#LOCNUM=1 -D#'$GLOBNUM=1' --check-prefixes CHECK,LOCAL1 --enable-var-scope --input-file %s %s 2>&1 \
+RUN:   | FileCheck --check-prefixes ERRUNDEF,ERRUNDEFLOCAL %s
+RUN: not FileCheck -D#LOCNUM=1 -D#'$GLOBNUM=1' --check-prefixes CHECK,LOCAL2 --enable-var-scope --input-file %s %s 2>&1 \
+RUN:   | FileCheck --check-prefixes ERRUNDEF,ERRUNDEFLOCNUM %s
+RUN: not FileCheck -D#LOCNUM=1 -D#'$GLOBNUM=1' --check-prefixes CHECK,LOCAL3 --enable-var-scope --input-file %s %s 2>&1 \
+RUN:   | FileCheck --check-prefixes ERRUNDEF,ERRUNDEFLOCAL,ERRUNDEFLOCNUM %s
+
+local1
+global1
+CHECK: [[LOCAL:loc[^[:digit:]]*]][[#LOCNUM]]
+CHECK: [[$GLOBAL:glo[^[:digit:]]*]][[#$GLOBNUM]]
+
+local2
+global2
+CHECK: [[LOCAL]][[#LOCNUM+1]]
+CHECK: [[$GLOBAL]][[#$GLOBNUM+1]]
+
+barrier:
+CHECK-LABEL: barrier
+
+local3
+global3
+LOCAL1: [[LOCAL]]3
+LOCAL2: local[[#LOCNUM+2]]
+LOCAL3: [[LOCAL]][[#LOCNUM+2]]
+GLOBAL: [[$GLOBAL]][[#$GLOBNUM+2]]
+
+ERRUNDEF: expected string not found in input
+ERRUNDEFLOCAL: uses undefined variable "LOCAL"
+ERRUNDEFLOCNUM: uses undefined variable "LOCNUM"

llvm/test/FileCheck/verbose.txt

-; RUN: FileCheck -input-file %s %s 2>&1 | FileCheck -check-prefix QUIET --allow-empty %s
-; RUN: FileCheck -v -input-file %s %s 2>&1 | FileCheck -check-prefix V %s
-; RUN: FileCheck -vv -input-file %s %s 2>&1 | FileCheck -check-prefixes V,VV %s
+; RUN: FileCheck -D#NUMVAR=42 -input-file %s %s 2>&1 | FileCheck -check-prefix QUIET --allow-empty %s
+; RUN: FileCheck -v -D#NUMVAR=42 -input-file %s %s 2>&1 | FileCheck --strict-whitespace -check-prefix V %s
+; RUN: FileCheck -vv -D#NUMVAR=42 -input-file %s %s 2>&1 | FileCheck --strict-whitespace -check-prefixes V,VV %s
 
 foo
 bar
 CHECK: foo
 CHECK-NOT: raboof
 CHECK-NEXT: bar
 
 V:      verbose.txt:[[@LINE-4]]:8: remark: {{C}}HECK: expected string found in input

 
 VV-NEXT: verbose.txt:[[@LINE-17]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT: {{C}}HECK-NOT: raboof{{$}}
 VV-NEXT:  {{^           \^$}}
 VV-NEXT: verbose.txt:[[@LINE-22]]:1: note: scanning from here
 VV-NEXT: {{^}}bar{{$}}
 VV-NEXT: {{^}}^{{$}}
 
+NUMVAR:42
+NUMVAR - 1:41
+CHECK: NUMVAR:[[#NUMVAR]]
+CHECK-NOT: [[#NUMVAR + 1]]
+CHECK-NEXT: NUMVAR - 1:[[#NUMVAR - 1]]
+
+V:      verbose.txt:[[#@LINE-4]]:8: remark: {{C}}HECK: expected string found in input
+V-NEXT: {{C}}HECK: {{NUMVAR:[[][[]#NUMVAR[]][]]$}}
+V-NEXT: {{^       \^$}}
+V-NEXT: verbose.txt:[[#@LINE-9]]:1: note: found here
+V-NEXT: {{^}}NUMVAR:42{{$}}
+V-NEXT: {{^}}^~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#@LINE-12]]:1: note: with numeric expression "NUMVAR" equal to "42"
+V-NEXT: {{^}}NUMVAR:42{{$}}
+V-NEXT: {{^}}^~~~~~~~~{{$}}
+
+V-NEXT: verbose.txt:[[#@LINE-12]]:13: remark: {{C}}HECK-NEXT: expected string found in input
+V-NEXT: {{C}}HECK-NEXT: {{NUMVAR - 1:[[][[]#NUMVAR - 1[]][]]$}}
+V-NEXT: {{^            \^$}}
+V-NEXT: verbose.txt:[[#@LINE-18]]:1: note: found here
+V-NEXT: {{^}}NUMVAR - 1:41{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#@LINE-21]]:1: note: with numeric expression "NUMVAR - 1" equal to "41"
+V-NEXT: {{^}}NUMVAR - 1:41{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
+
+VV-NEXT: verbose.txt:[[#@LINE-23]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
+VV-NEXT: {{C}}HECK-NOT: {{[[][[]#NUMVAR [+] 1[]][]]$}}
+VV-NEXT: {{^           \^$}}
+VV-NEXT: verbose.txt:[[#@LINE-28]]:1: note: scanning from here
+VV-NEXT: {{^}}NUMVAR - 1:41{{$}}
+VV-NEXT: {{^}}^{{$}}
+
 before empty
 
 after empty
 CHECK: before empty
 CHECK-EMPTY:
 CHECK-NEXT: after empty
 
 V:      verbose.txt:[[@LINE-4]]:8: remark: {{C}}HECK: expected string found in input

 V-NEXT: {{C}}HECK: xyz{{$}}
 V-NEXT:  {{^       \^$}}
 V-NEXT: verbose.txt:[[@LINE-7]]:1: note: found here
 V-NEXT: {{^}}xyz{{$}}
 V-NEXT: {{^}}^~~{{$}}
 
 VV-NEXT: verbose.txt:[[@LINE-9]]:19: remark: implicit EOF: expected string found in input
 VV-NEXT: {{C}}HECK-NOT: {{[{][{]z[}][}]yx$}}
-VV-NEXT:   {{^                 \^$}}
+VV-NEXT:   {{^                  \^$}}
 VV-NEXT: verbose.txt:[[@LINE+13]]:1: note: found here
 VV-NOT:  {{.}}
 VV:      {{^\^$}}
 
 VV-NEXT: verbose.txt:[[@LINE-16]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT: {{C}}HECK-NOT: {{[{][{]z[}][}]yx$}}
 VV-NEXT:  {{^           \^$}}
 VV-NEXT: verbose.txt:[[@LINE-20]]:1: note: scanning from here
 VV-NEXT: {{^C}}HECK: xyz{{$}}
 VV-NEXT: {{^\^$}}
 
 QUIET-NOT: {{.}}
 V-NOT: {{.}}

llvm/unittests/Support/FileCheckTest.cpp

 //===- llvm/unittest/Support/FileCheckTest.cpp - FileCheck tests --===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Support/FileCheck.h"
 #include "gtest/gtest.h"
 
 using namespace llvm;
 namespace {
 
 class FileCheckTest : public ::testing::Test {};
 
+TEST_F(FileCheckTest, NumericVariable) {
+  FileCheckNumericVariable FooVar = FileCheckNumericVariable("FOO", 42);
+  EXPECT_EQ("FOO", FooVar.getName());
+
+  // Defined variable: getValue returns a value, setValue fails and value
+  // remains unchanged.
+  llvm::Optional<uint64_t> Value = FooVar.getValue();
+  EXPECT_TRUE(Value);
+  EXPECT_EQ(42U, *Value);
+  EXPECT_TRUE(FooVar.setValue(43));
+  Value = FooVar.getValue();
+  EXPECT_TRUE(Value);
+  EXPECT_EQ(42U, *Value);
+
+  // Clearing variable: getValue fails, clearValue again fails.
+  EXPECT_FALSE(FooVar.clearValue());
+  Value = FooVar.getValue();
+  EXPECT_FALSE(Value);
+  EXPECT_TRUE(FooVar.clearValue());
+
+  // Undefined variable: setValue works, getValue returns value set.
+  EXPECT_FALSE(FooVar.setValue(43));
+  Value = FooVar.getValue();
+  EXPECT_TRUE(Value);
+  EXPECT_EQ(43U, *Value);
+}
+
+uint64_t doAdd(uint64_t OpL, uint64_t OpR) { return OpL + OpR; }
+
+TEST_F(FileCheckTest, NumExpr) {
+  FileCheckNumericVariable FooVar = FileCheckNumericVariable("FOO", 42);
+  FileCheckNumExpr NumExpr = FileCheckNumExpr(doAdd, &FooVar, 18);
+
+  // Defined variable: eval returns right value, no undefined variable
+  // returned.
+  llvm::Optional<uint64_t> Value = NumExpr.eval();
+  EXPECT_TRUE(Value);
+  EXPECT_EQ(60U, *Value);
+  StringRef UndefVar = NumExpr.getUndefVarName();
+  EXPECT_EQ("", UndefVar);
+
+  // Undefined variable: eval fails, undefined variable returned. We call
+  // getUndefVarName first to check that it can be called without calling
+  // eval() first.
+  FooVar.clearValue();
+  UndefVar = NumExpr.getUndefVarName();
+  EXPECT_EQ("FOO", UndefVar);
+  Value = NumExpr.eval();
+  EXPECT_FALSE(Value);
+}
+
 TEST_F(FileCheckTest, ValidVarNameStart) {
   EXPECT_TRUE(FileCheckPattern::isValidVarNameStart('a'));
   EXPECT_TRUE(FileCheckPattern::isValidVarNameStart('G'));
   EXPECT_TRUE(FileCheckPattern::isValidVarNameStart('_'));
   EXPECT_FALSE(FileCheckPattern::isValidVarNameStart('2'));
   EXPECT_FALSE(FileCheckPattern::isValidVarNameStart('$'));
   EXPECT_FALSE(FileCheckPattern::isValidVarNameStart('@'));
   EXPECT_FALSE(FileCheckPattern::isValidVarNameStart('+'));
TEST_F(FileCheckTest, ParseVar) {
   VarName = "BadVar:";
   IsPseudo = true;
   TrailIdx = 0;
   EXPECT_FALSE(FileCheckPattern::parseVariable(VarName, IsPseudo, TrailIdx));
   EXPECT_FALSE(IsPseudo);
   EXPECT_EQ(TrailIdx, VarName.size() - 1);
 }
 
+static StringRef bufferize(SourceMgr &SM, StringRef Str) {
+  std::unique_ptr<MemoryBuffer> Buffer =
+      MemoryBuffer::getMemBufferCopy(Str, "TestBuffer");
+  StringRef StrBufferRef = Buffer->getBuffer();
+  SM.AddNewSourceBuffer(std::move(Buffer), SMLoc());
+  return StrBufferRef;
+}
+
 class ExprTester {
 private:
   SourceMgr SM;
+  FileCheckRequest Req;
   FileCheckPatternContext Context;
   FileCheckPattern P = FileCheckPattern(Check::CheckPlain, &Context);
 
 public:
+  ExprTester() {
+    std::vector<std::string> GlobalDefines;
+    GlobalDefines.emplace_back(std::string("#FOO=42"));
+    Context.defineCmdlineVariables(GlobalDefines, SM);
+    // Call ParsePattern to have @LINE defined.
+    P.ParsePattern("N/A", "CHECK", SM, 1, Req);
+  }
+
   bool parseExpect(std::string &VarName, std::string &Trailer) {
+    bool IsPseudo = VarName[0] == '@';
     std::string NameTrailer = VarName + Trailer;
-    std::unique_ptr<MemoryBuffer> Buffer =
-        MemoryBuffer::getMemBufferCopy(NameTrailer, "TestBuffer");
-    StringRef NameTrailerRef = Buffer->getBuffer();
-    SM.AddNewSourceBuffer(std::move(Buffer), SMLoc());
+    StringRef NameTrailerRef = bufferize(SM, NameTrailer);
     StringRef VarNameRef = NameTrailerRef.substr(0, VarName.size());
     StringRef TrailerRef = NameTrailerRef.substr(VarName.size());
-    return P.parseNumericExpression(VarNameRef, TrailerRef, SM) == nullptr;
+    return P.parseNumericExpression(VarNameRef, IsPseudo, TrailerRef, SM) ==
+           nullptr;
   }
 };
 
 TEST_F(FileCheckTest, ParseExpr) {
   ExprTester Tester;
 
   // @LINE with offset.
   std::string VarName = "@LINE";
   std::string Trailer = "+3";
   EXPECT_FALSE(Tester.parseExpect(VarName, Trailer));
 
   // @LINE only.
   Trailer = "";
   EXPECT_FALSE(Tester.parseExpect(VarName, Trailer));
 
+  // Defined variable.
+  VarName = "FOO";
+  EXPECT_FALSE(Tester.parseExpect(VarName, Trailer));
+
+  // Undefined variable.
+  VarName = "UNDEF";
+  EXPECT_TRUE(Tester.parseExpect(VarName, Trailer));
+
   // Wrong Pseudovar.
   VarName = "@FOO";
   EXPECT_TRUE(Tester.parseExpect(VarName, Trailer));
 
   // Unsupported operator.
   VarName = "@LINE";
   Trailer = "/2";
   EXPECT_TRUE(Tester.parseExpect(VarName, Trailer));

 
 TEST_F(FileCheckTest, Substitution) {
   SourceMgr SM;
   FileCheckPatternContext Context;
   std::vector<std::string> GlobalDefines;
   GlobalDefines.emplace_back(std::string("FOO=BAR"));
   Context.defineCmdlineVariables(GlobalDefines, SM);
 
-  FileCheckPatternSubstitution Substitution =
+  // Substitution of undefined pattern variable fails.
+  FileCheckPatternSubstitution PatternSubstitution =
       FileCheckPatternSubstitution(&Context, "VAR404", 42);
-  EXPECT_FALSE(Substitution.getResult());
+  EXPECT_FALSE(PatternSubstitution.getResult());
 
-  FileCheckNumExpr NumExpr = FileCheckNumExpr(42);
-  Substitution = FileCheckPatternSubstitution(&Context, "@LINE", &NumExpr, 12);
-  llvm::Optional<std::string> Value = Substitution.getResult();
+  // Substitutions of defined pseudo and non-pseudo numeric variables return
+  // the right value.
+  FileCheckNumericVariable LineVar = FileCheckNumericVariable("@LINE", 42);
+  FileCheckNumericVariable NVar = FileCheckNumericVariable("@N", 10);
+  FileCheckNumExpr NumExprLine = FileCheckNumExpr(doAdd, &LineVar, 0);
+  FileCheckNumExpr NumExprN = FileCheckNumExpr(doAdd, &NVar, 3);
+  FileCheckPatternSubstitution SubstitutionLine =
+      FileCheckPatternSubstitution(&Context, "@LINE", &NumExprLine, 12);
+  FileCheckPatternSubstitution SubstitutionN =
+      FileCheckPatternSubstitution(&Context, "N", &NumExprN, 30);
+  llvm::Optional<std::string> Value = SubstitutionLine.getResult();
   EXPECT_TRUE(Value);
   EXPECT_EQ("42", *Value);
+  Value = SubstitutionN.getResult();
+  EXPECT_TRUE(Value);
+  EXPECT_EQ("13", *Value);
+
+  // Substitution of undefined numeric variable fails.
+  LineVar.clearValue();
+  EXPECT_FALSE(SubstitutionLine.getResult());
+  NVar.clearValue();
+  EXPECT_FALSE(SubstitutionN.getResult());
 
+  // Substitution of defined pattern variable returns the right value.
   FileCheckPattern P = FileCheckPattern(Check::CheckPlain, &Context);
-  Substitution = FileCheckPatternSubstitution(&Context, "FOO", 42);
-  Value = Substitution.getResult();
+  PatternSubstitution = FileCheckPatternSubstitution(&Context, "FOO", 42);
+  Value = PatternSubstitution.getResult();
   EXPECT_TRUE(Value);
   EXPECT_EQ("BAR", *Value);
 }
 
 TEST_F(FileCheckTest, UndefVars) {
   SourceMgr SM;
   FileCheckPatternContext Context;
   std::vector<std::string> GlobalDefines;
   GlobalDefines.emplace_back(std::string("FOO=BAR"));
   Context.defineCmdlineVariables(GlobalDefines, SM);
 
+  // getUndefVarName() on a pattern variable substitution with an undefined
+  // variable returns that variable.
   FileCheckPatternSubstitution Substitution =
       FileCheckPatternSubstitution(&Context, "VAR404", 42);
   StringRef UndefVar = Substitution.getUndefVarName();
   EXPECT_EQ("VAR404", UndefVar);
 
-  FileCheckNumExpr NumExpr = FileCheckNumExpr(42);
-  Substitution = FileCheckPatternSubstitution(&Context, "@LINE", &NumExpr, 12);
+  // getUndefVarName() on a pattern variable substitution with a defined
+  // variable returns an empty string.
+  Substitution = FileCheckPatternSubstitution(&Context, "FOO", 42);
   UndefVar = Substitution.getUndefVarName();
   EXPECT_EQ("", UndefVar);
 
-  Substitution = FileCheckPatternSubstitution(&Context, "FOO", 42);
+  // getUndefVarName() on a numeric expression substitution with a defined
+  // variable returns an empty string.
+  FileCheckNumericVariable LineVar = FileCheckNumericVariable("@LINE", 42);
+  FileCheckNumExpr NumExpr = FileCheckNumExpr(doAdd, &LineVar, 0);
+  Substitution = FileCheckPatternSubstitution(&Context, "@LINE", &NumExpr, 12);
   UndefVar = Substitution.getUndefVarName();
   EXPECT_EQ("", UndefVar);
+
+  // getUndefVarName() on a numeric expression substitution with an undefined
+  // variable returns that variable.
+  LineVar.clearValue();
+  UndefVar = Substitution.getUndefVarName();
+  EXPECT_EQ("@LINE", UndefVar);
 }
 
 TEST_F(FileCheckTest, FileCheckContext) {
   FileCheckPatternContext Cxt = FileCheckPatternContext();
   std::vector<std::string> GlobalDefines;
   SourceMgr SM;
 
-  // Missing equal sign
+  // Missing equal sign.
   GlobalDefines.emplace_back(std::string("LocalVar"));
   EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("#LocalNumVar"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
 
-  // Empty variable
+  // Empty variable name.
   GlobalDefines.clear();
   GlobalDefines.emplace_back(std::string("=18"));
   EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("#=18"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
 
-  // Invalid variable
+  // Invalid variable name.
   GlobalDefines.clear();
   GlobalDefines.emplace_back(std::string("18LocalVar=18"));
   EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("#18LocalNumVar=18"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+
+  // Name conflict between pattern and numeric variable.
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("LocalVar=18"));
+  GlobalDefines.emplace_back(std::string("#LocalVar=36"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+  Cxt = FileCheckPatternContext();
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("#LocalNumVar=18"));
+  GlobalDefines.emplace_back(std::string("LocalNumVar=36"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
+  Cxt = FileCheckPatternContext();
+
+  // Invalid numeric value for numeric variable.
+  GlobalDefines.clear();
+  GlobalDefines.emplace_back(std::string("#LocalNumVar=x"));
+  EXPECT_TRUE(Cxt.defineCmdlineVariables(GlobalDefines, SM));
 
   // Define local variables from command-line.
   GlobalDefines.clear();
   GlobalDefines.emplace_back(std::string("LocalVar=FOO"));
   GlobalDefines.emplace_back(std::string("EmptyVar="));
+  GlobalDefines.emplace_back(std::string("#LocalNumVar=18"));
   bool GotError = Cxt.defineCmdlineVariables(GlobalDefines, SM);
   EXPECT_FALSE(GotError);
 
   // Check defined variables are present and undefined is absent.
   StringRef LocalVarStr = "LocalVar";
+  StringRef LocalNumVarRef = bufferize(SM, "LocalNumVar");
   StringRef EmptyVarStr = "EmptyVar";
   StringRef UnknownVarStr = "UnknownVar";
   llvm::Optional<StringRef> LocalVar = Cxt.getPatternVarValue(LocalVarStr);
+  FileCheckPattern P = FileCheckPattern(Check::CheckPlain, &Cxt);
+  FileCheckNumExpr *NumExpr =
+      P.parseNumericExpression(LocalNumVarRef, false /*IsPseudo*/, "", SM);
   llvm::Optional<StringRef> EmptyVar = Cxt.getPatternVarValue(EmptyVarStr);
   llvm::Optional<StringRef> UnknownVar = Cxt.getPatternVarValue(UnknownVarStr);
   EXPECT_TRUE(LocalVar);
   EXPECT_EQ(*LocalVar, "FOO");
+  EXPECT_TRUE(NumExpr);
+  llvm::Optional<uint64_t> NumExprVal = NumExpr->eval();
+  EXPECT_TRUE(NumExprVal);
+  EXPECT_EQ(*NumExprVal, 18U);
   EXPECT_TRUE(EmptyVar);
   EXPECT_EQ(*EmptyVar, "");
   EXPECT_FALSE(UnknownVar);
 
   // Clear local variables and check they become absent.
   Cxt.clearLocalVars();
   LocalVar = Cxt.getPatternVarValue(LocalVarStr);
   EXPECT_FALSE(LocalVar);
+  // Check a numeric expression's evaluation fails if called after clearing of
+  // local variables, if it was created before. This is important because local
+  // variable clearing due to --enable-var-scope happens after numeric
+  // expressions are linked to the numeric variables they use.
+  EXPECT_FALSE(NumExpr->eval());
+  P = FileCheckPattern(Check::CheckPlain, &Cxt);
+  NumExpr =
+      P.parseNumericExpression(LocalNumVarRef, false /*IsPseudo*/, "", SM);
+  EXPECT_FALSE(NumExpr);
   EmptyVar = Cxt.getPatternVarValue(EmptyVarStr);
   EXPECT_FALSE(EmptyVar);
 
   // Redefine global variables and check variables are defined again.
   GlobalDefines.emplace_back(std::string("$GlobalVar=BAR"));
+  GlobalDefines.emplace_back(std::string("#$GlobalNumVar=36"));
   GotError = Cxt.defineCmdlineVariables(GlobalDefines, SM);
   EXPECT_FALSE(GotError);
   StringRef GlobalVarStr = "$GlobalVar";
+  StringRef GlobalNumVarRef = bufferize(SM, "$GlobalNumVar");
   llvm::Optional<StringRef> GlobalVar = Cxt.getPatternVarValue(GlobalVarStr);
   EXPECT_TRUE(GlobalVar);
   EXPECT_EQ(*GlobalVar, "BAR");
+  P = FileCheckPattern(Check::CheckPlain, &Cxt);
+  NumExpr =
+      P.parseNumericExpression(GlobalNumVarRef, false /*IsPseudo*/, "", SM);
+  EXPECT_TRUE(NumExpr);
+  NumExprVal = NumExpr->eval();
+  EXPECT_TRUE(NumExprVal);
+  EXPECT_EQ(*NumExprVal, 36U);
 
   // Clear local variables and check global variables remain defined.
   Cxt.clearLocalVars();
   GlobalVar = Cxt.getPatternVarValue(GlobalVarStr);
   EXPECT_TRUE(GlobalVar);
+  P = FileCheckPattern(Check::CheckPlain, &Cxt);
+  NumExpr =
+      P.parseNumericExpression(GlobalNumVarRef, false /*IsPseudo*/, "", SM);
+  EXPECT_TRUE(NumExpr);
+  NumExprVal = NumExpr->eval();
+  EXPECT_TRUE(NumExprVal);
+  EXPECT_EQ(*NumExprVal, 36U);
 }
 } // namespace