FileCheck [12/12]: Support use of var defined on same line

[thopre]

Summary

This patch is part of a patch series to add support for FileCheck
numeric expressions. This specific patch adds support for using a
numeric variable with empty numeric expression defined on the same line
either directly or indirectly.

The regex language used to represent a constraint corresponding to a
given CHECK directive does not allow to express a numeric relation. It
is therefore necessary to initially match any number in the appropriate
matching format and check the constraints are verified in a follow-up
step. Such a process must then be repeated as long as the constraints
fail because of cases with a check directive like #N: #N+1
where the input contains lines such as:
10 12
10 11

The first line would match the regex "[0-9]+ [0-9]+" generated for the
CHECK directive but the constraint would fail. The second line would
then need to be tried since it both matches successfully and verifies
the constraints of the CHECK directive. Note that this process would
still fail to match if the input was:
10 12 13
because the regex would match 10 and 12 which fail to verify the
constraints and the process would then skip to the next line since there
is no way to ask the regex engine if the line had other possible
matches. This caveat is documented.

Copyright:

• Linaro (changes up to diff 183612 of revision D55940)
• GraphCore (changes in later versions of revision D55940 and in new revision created off D55940)

[thopre]

Use camel casing for new functions

[thopre]

Rebase

[thopre]

Make numeric value constructor explicit

[thopre]

Rebase on latest changes

[thopre]

Rebase on latest changes

[thopre]

Rebase on latest unittest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

Rebase on top of latest changes

[thopre]

rebase on top of latest changes

[thopre]

rebase on top of latest changes

[thopre]

Rebase on top of latest changes in patch series

[thopre]

Rebase on top of latest changes in the series

[thopre]

Rebase on top of latest changes in patch series

[thopre]

Codebase has changed considerably since last time this review was updated. Patch will need some reworking and before that I want to build consensus on whether the feature should be done given the caveats it has.

[jhenderson]

I don't have a clear solution to the problems you're describing, but I have already run into one or two places where I wanted the behaviour of being able to use a numeric variable on the same line it was defined. CHECK-SAME is a valid workaround, but has its own issues (for example, it doesn't inherently ensure the next match starts immediately after the previous one, and is just simply more verbose). My instinct is that the right approach is to ignore the constraint initially, match a number, and then verify that it is the right number, failing if not, without further search attempts. I think it's relatively easy to workaround that issue in most cases, with additional checks elsewhere, although I can't be positive that is the case.

[thopre]

In D60392#2033357, @jhenderson wrote:

I don't have a clear solution to the problems you're describing, but I have already run into one or two places where I wanted the behaviour of being able to use a numeric variable on the same line it was defined. CHECK-SAME is a valid workaround, but has its own issues (for example, it doesn't inherently ensure the next match starts immediately after the previous one, and is just simply more verbose). My instinct is that the right approach is to ignore the constraint initially, match a number, and then verify that it is the right number, failing if not, without further search attempts. I think it's relatively easy to workaround that issue in most cases, with additional checks elsewhere, although I can't be positive that is the case.

What worries me is the effect on a CHECK-NOT directive since it could lead to the directive being fulfilled in a case where the intent was to forbid it. Consider the following:

CHECK-NOT: [[#VAR:]] [[#VAR+1]]

Say the author of the test was looking a a bug where the output being checked was 10 11 14. The test would pass. Now if the bug wasn't fully fixed, perhaps this doesn't happen anymore but we can have cases like 10 13 14. The above directive will turn into ([[:digit:]]+) ([[:digit:]]+) which will match 10 and 13 and will then check if they are consecutive numbers. They are not and thus the CHECK-NOT is satisfied even though there was 13 14 later on the line. Of course here we could retry after the first number matched but that doesn't work in the general case where regex can appear anywhere (e.g. #VAR: {{.*}} #VAR+1 where the input is now 10 8 11).

Perhaps the solution is to forbid using a variable defined on the same line for a CHECK-NOT directive? In all other cases we'll get a false negative (directive not satisfied) instead of a false positive (directive satisfied).

[jhenderson]

In D60392#2033995, @thopre wrote:

Perhaps the solution is to forbid using a variable defined on the same line for a CHECK-NOT directive? In all other cases we'll get a false negative (directive not satisfied) instead of a false positive (directive satisfied).

Hmm, you make a fair argument, and I think this suggestion might work. I'll need to give it more thought though. Perhaps it needs a mailing list thread to get more eyes on this?

[thopre]

In D60392#2035853, @jhenderson wrote:

In D60392#2033995, @thopre wrote:

Perhaps the solution is to forbid using a variable defined on the same line for a CHECK-NOT directive? In all other cases we'll get a false negative (directive not satisfied) instead of a false positive (directive satisfied).

Hmm, you make a fair argument, and I think this suggestion might work. I'll need to give it more thought though. Perhaps it needs a mailing list thread to get more eyes on this?

That was my intention. I'm focusing on D60390 for now, then I'll start a thread.

[thopre]

Status update: I've started manually rebasing this patch on top of latest FileCheck. Hopefully I'll update this diff in about 2 weeks. Sorry for the delay.

[thopre]

In D60392#2682536, @thopre wrote:

Status update: I've started manually rebasing this patch on top of latest FileCheck. Hopefully I'll update this diff in about 2 weeks. Sorry for the delay.

Sorry, I dropped the ball on that one but recently picked it up again and have a patch ready. It needs clean up but it's feature complete and all tests are already written. Please hang in there.

[thopre]

Rebase and fix issues with variables defined from an expression and with
verification failure failing to restore initial values of variables.

[jhenderson]

Sorry for ignoring this for a couple of weeks - it fell off my radar a bit, when I was swamped with deadline-realted tasks.

I recommend doing the llvm/test/FileCheck/numeric-expressions/numeric-expressions.txt renaming in a separate patch, as it's not really related to this patch.

I've not looked at the testing in depth yet, and I'm a little concerned that I'm very cold on the surrounding code. I think it would be useful if someone else could take a look too (@jdenny maybe?).

llvm/docs/CommandGuide/FileCheck.rst

661–671

Where this refers to "line" should it be "CHECK" directive?

Also, I think it would be valuable to give an example.

665–666

llvm/lib/FileCheck/FileCheck.cpp

382 ↗

(On Diff #381083)

My general rule is that consumeError should always have a comment explaining why you can just "ignore" the error and continue. In this case, I guess you're not ignoring the error per se, but you are throwing away the context, so a comment would still be useful.

411–415 ↗

(On Diff #381083)

Do you need this change? Can't you just return the result of EvalBinop directly?

449–450 ↗

(On Diff #381083)

1184 ↗

(On Diff #381083)

TODO that needs action still?

1307 ↗

(On Diff #381083)

(or possibly "using variables with empty expressions")

1311–1312 ↗

(On Diff #381083)

(or "using variables")

Should this definitely be "line"?

Perhaps change "Known issue" to FIXME and file a bug (bug filing to happen once this patch has landed, or shortly beforehand, with the FIXME to contain a link to that bug).

1322 ↗

(On Diff #381083)

1323–1324 ↗

(On Diff #381083)

1385–1386 ↗

(On Diff #381083)

1403 ↗

(On Diff #381083)

1424 ↗

(On Diff #381083)

1425–1426 ↗

(On Diff #381083)

1431 ↗

(On Diff #381083)

1433 ↗

(On Diff #381083)

No need for braces for single line if.

1438 ↗

(On Diff #381083)

Any reason we can't just use a for loop here?

1487 ↗

(On Diff #381083)

llvm/lib/FileCheck/FileCheckImpl.h

269 ↗

(On Diff #381083)

292–294 ↗

(On Diff #381083)

Seems like TentativeValue and StrTentativeValue are unnecessary? Could you not just set Value and StrValue in this context? If you need to distinguish a tentative versus concrete definition, add a boolean member or similar that states isTentative.

520 ↗

(On Diff #381083)

578 ↗

(On Diff #381083)

Nit: add new line after this method declaration.

[jdenny]

In D60392#3137020, @jhenderson wrote:

I've not looked at the testing in depth yet, and I'm a little concerned that I'm very cold on the surrounding code. I think it would be useful if someone else could take a look too (@jdenny maybe?).

Sorry for the delay, but I need a bit more time. I'll plan to take a look next week. Thanks for working on this.

[jdenny]

I understand the desire to be able to define and reference a variable within the same directive. However, as I understand it, this patch adds that ability in a way that introduces a new subtlety to FileCheck: the semantics of numeric variable references are then not always consistent. If you assume the wrong semantics, you can end up with a false pass of a test. I understand you've disabled the case of CHECK-NOT, but it does not appear to be the only case that can lead to false passes.

Example

Let's say we want to check that the very last value of y on an input line is equal to the first value of y plus 3. Thus, the following input line is then erroneous:

x=20, y=30, z=40, x=21, y=31, z=41, x=22, y=33, z=42, x=23, y=32, z=43,

Check version 1

CHECK: y=[[#VAR:]],{{.*}}y=[[#VAR+3]],

The behavior is:

• y=[[#VAR:]], matches y=30,.
• {{.*}} is maximal munch.
• So y=[[#VAR+3]], matches the last y= expression, which is y=32,.

The directive then fails because 32 is not 30+3. OK, that's the exact behavior I was going for. And now I think I understand how numeric variable references work.

(If the above is not the behavior you expect after applying this patch, let me know. Every example I tried after applying this patch produced seg faults or assertion failures, so I couldn't verify.)

Check version 2

Let's say I later realize I'd rather use VAR as defined to 30 by some earlier directive, so I rewrite check version 1 as follows:

CHECK: y=[[#VAR]],{{.*}}y=[[#VAR+3]],

The behavior is:

• y=[[#VAR]], is visibly differently than y=[[#VAR:]],, and its semantics are different too: it now matches a previously defined value instead of capturing a new value. Nevertheless, it still matches y=30,. So far, so good.
• {{.*}} is still maximal munch.
• Given that previous parts of the pattern match as before and VAR ends up with the same value as before, shouldn't the semantics of y=[[#VAR+3]], be the same as before? No. It can only match y=33, now, and so that's what it does.

The directive then accepts the erroneous input. That's pretty subtle.

Check version 3

Or, let's say I decide that splitting check version 1 into two directives would look better:

     CHECK: y=[[#VAR:]],
CHECK-SAME: {{.*}}y=[[#VAR+3]],

That's equivalent to version 1, right? Normally, I think it would be. Not in this case. Again, y=[[#VAR+3]], now has a different behavior: it only matches y=33,, and the erroneous input is now accepted.

Conclusion

I'm leery of letting the semantics of numeric variable references vary depending on where the variable happens to have been defined. It's too subtle.

I think it would be better to introduce the desired functionality with a different syntax to reflect the semantic shift. For example:

CHECK: y=[[#VAR1:]],{{.*}}y=[[#VAR2:]],[[!#VAR2 == VAR1 + 3]]

That introduces the concept of a post-match variable assertion. It would always appear at the end of a pattern to indicate it's evaluated last. It would always be delimited by [[! and ]], which hopefully doesn't collide with any existing syntax.

There's probably a better syntax I haven't thought of. My main point is to demonstrate how a different syntax could eliminate the inconsistent behavior I am concerned about.

[jdenny]

Or maybe a CHECK-ASSERT would be clearer and more generally useful:

CHECK: y=[[#VAR1:]],{{.*}}y=[[#VAR2:]]
CHECK-ASSERT: #VAR2 == VAR1 + 3

[thopre]

Hi Joel,

Thanks for your feedback, good job catching that!

In D60392#3181763, @jdenny wrote:

I understand the desire to be able to define and reference a variable within the same directive. However, as I understand it, this patch adds that ability in a way that introduces a new subtlety to FileCheck: the semantics of numeric variable references are then not always consistent. If you assume the wrong semantics, you can end up with a false pass of a test. I understand you've disabled the case of CHECK-NOT, but it does not appear to be the only case that can lead to false passes.

Example

Let's say we want to check that the very last value of y on an input line is equal to the first value of y plus 3. Thus, the following input line is then erroneous:

x=20, y=30, z=40, x=21, y=31, z=41, x=22, y=33, z=42, x=23, y=32, z=43,

Check version 1

CHECK: y=[[#VAR:]],{{.*}}y=[[#VAR+3]],

The behavior is:

• y=[[#VAR:]], matches y=30,.
• {{.*}} is maximal munch.
• So y=[[#VAR+3]], matches the last y= expression, which is y=32,.

The directive then fails because 32 is not 30+3. OK, that's the exact behavior I was going for. And now I think I understand how numeric variable references work.

(If the above is not the behavior you expect after applying this patch, let me know. Every example I tried after applying this patch produced seg faults or assertion failures, so I couldn't verify.)

I think the real issue here is using a wildcard pattern. I feel most often that not the user actually intended that. I remember someone asking for being able to use variables defined on the same line because using CHECK-SAME was different as pointed in your example below. See I think this is the behaviour people usually expect, judging from https://bugs.llvm.org/show_bug.cgi?id=30198 also about wildcard being a risky business.

Check version 2

Let's say I later realize I'd rather use VAR as defined to 30 by some earlier directive, so I rewrite check version 1 as follows:

CHECK: y=[[#VAR]],{{.*}}y=[[#VAR+3]],

The behavior is:

• y=[[#VAR]], is visibly differently than y=[[#VAR:]],, and its semantics are different too: it now matches a previously defined value instead of capturing a new value. Nevertheless, it still matches y=30,. So far, so good.
• {{.*}} is still maximal munch.
• Given that previous parts of the pattern match as before and VAR ends up with the same value as before, shouldn't the semantics of y=[[#VAR+3]], be the same as before? No. It can only match y=33, now, and so that's what it does.

The directive then accepts the erroneous input. That's pretty subtle.

I think the main issue is the regex was badly written in the first place but I agree this patch is making things worse by having inconsistent behaviour. Someone ought to have written either of those two regex instead depending on intended pattern to be matched:

CHECK: y=[[#VAR:]],{{.*}}y=[[#VAR+3]],{{[^y]*$}}

or

CHECK: y=[[#VAR:]],{{[^y]*}}y=[[#VAR+3]],

I like the suggestion of CHECK-ASSERT or a new syntax. I'll think about it and in the meantime anyone with an opinion on this feel free to chime in.

[jdenny]

In D60392#3182279, @thopre wrote:

Hi Joel,

Hi Thomas.

I like the suggestion of CHECK-ASSERT or a new syntax. I'll think about it

Thanks for considering that.

and in the meantime anyone with an opinion on this feel free to chime in.

Yes, it will be good to get other opinions. @jhenderson?

[jhenderson]

In D60392#3184072, @jdenny wrote:

In D60392#3182279, @thopre wrote:

Hi Joel,

Hi Thomas.

I like the suggestion of CHECK-ASSERT or a new syntax. I'll think about it

Thanks for considering that.

and in the meantime anyone with an opinion on this feel free to chime in.

Yes, it will be good to get other opinions. @jhenderson?

I think @jdenny has hit the nail on the head with regards to a fairly major problem with this patch. I don't think we want to introduce subtleties in behaviour like has been highlighted. Sorry for not considering this before.

Regarding the CHECK-ASSERT suggestion, do we need it? I was under the impression one of the earlier patches added a syntax like [[#VAR2, VAR1 + 3:]] or something to that effect, which achieves the same thing, right? Maybe I'm missing something though.

[thopre]

In D60392#3184688, @jhenderson wrote:

Regarding the CHECK-ASSERT suggestion, do we need it? I was under the impression one of the earlier patches added a syntax like [[#VAR2, VAR1 + 3:]] or something to that effect, which achieves the same thing, right? Maybe I'm missing something though.

You mean [[#VAR2:VAR1+3]]? Why does that prevent the need for something like CHECK-ASSERT? You still can only refer to a variable defined in an earlier line.

[jhenderson]

In D60392#3184854, @thopre wrote:

In D60392#3184688, @jhenderson wrote:

Regarding the CHECK-ASSERT suggestion, do we need it? I was under the impression one of the earlier patches added a syntax like [[#VAR2, VAR1 + 3:]] or something to that effect, which achieves the same thing, right? Maybe I'm missing something though.

You mean [[#VAR2:VAR1+3]]? Why does that prevent the need for something like CHECK-ASSERT? You still can only refer to a variable defined in an earlier line.

Looking back at the example, I guess it doesn't, although you could use CHECK-SAME to achieve the same effect, I think. More generally, I'm not sure how much I like the idea of checks that don't actually match anything, but I could be persuaded, especially if we can leverage the same functionality to support the CHECK-DEFINE concept I proposed previously (but never got as far as finishing off).

[jdenny]

In D60392#3188307, @jhenderson wrote:

In D60392#3184854, @thopre wrote:

In D60392#3184688, @jhenderson wrote:

Regarding the CHECK-ASSERT suggestion, do we need it? I was under the impression one of the earlier patches added a syntax like [[#VAR2, VAR1 + 3:]] or something to that effect, which achieves the same thing, right? Maybe I'm missing something though.

You mean [[#VAR2:VAR1+3]]? Why does that prevent the need for something like CHECK-ASSERT? You still can only refer to a variable defined in an earlier line.

Looking back at the example, I guess it doesn't, although you could use CHECK-SAME to achieve the same effect, I think. More generally, I'm not sure how much I like the idea of checks that don't actually match anything, but I could be persuaded, especially if we can leverage the same functionality to support the CHECK-DEFINE concept I proposed previously (but never got as far as finishing off).

I too would like a CHECK-DEFINE. Conceptually at least, I imagine CHECK-DEFINE or CHECK-ASSERT would match an empty string at the start of its search range (that is, end of the previous match or beginning of input if none). Giving it a match location in this way should help with diagnostics, such as -dump-input.

llvm/docs/CommandGuide/FileCheck.rst

 in which case the numeric expression constraint is checked and if verified the
 variable is assigned to the value. The unified syntax for both defining numeric
 variables and checking a numeric expression is thus
 ``[[#%<fmtspec>,<NUMVAR>: <constraint> <expr>]]`` with each element as
 described previously.
 
 The ``--enable-var-scope`` option has the same effect on numeric variables as
 on string variables.
 
-Important note: In its current implementation, a numeric expression cannot use
-a numeric variable with a non-empty numeric expression constraint defined on
-the same line.
+Important note: In its current implementation, a numeric expression using
+numeric variable defined on the same line can fail to match even though an
+input line satisfies it.  This happens when there is more than one possible
+match in the line that should match when ignoring the constraint and
+considering only the matching format (ie. matching an unsigned number with
+[[:digit:]]+ and the first match does not satisfy the constraint.  This is
+due to those numeric expressions constraints not being able to be expressed at
+the regular expression level and the regular expression engine not offering the
+possibility of getting all possible matches.
 
 FileCheck Pseudo Numeric Variables
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes there's a need to verify output that contains line numbers of the
 match file, e.g. when testing compiler diagnostics.  This introduces a certain
 fragility of the match file structure, as "``CHECK:``" lines contain absolute
 line numbers in the same file, which have to be updated whenever line numbers
 change due to text addition or deletion.

llvm/include/llvm/Support/FileCheck.h

union {
     int64_t SignedValue;
     uint64_t UnsignedValue;
   };
 
   /// Whether value is signed (and thus is stored in SignedValue) or not (in
   /// which case it is stored in UnsignedValue).
   bool Signed;
 
-  /// Whether this holds an actual value. Examples of where this would be
-  /// false:
-  /// - underflow or overflow of one of the binary operation in the expression;
-  /// - unset variable's value.
+  /// Whether this holds an actual value. Examples of where this would be false
+  /// include:
+  /// - underflow or overflow of one of the binary operation in the expression
+  /// - value of undefined variable
   bool Valid;
 
+  /// Whether this is a tentative value, ie other expressions on the same CHECK
+  /// line that might be using that value or other values that this value was
+  /// computed from have not yet been verified to match. True if any of the
+  /// operand involved in its computation had a tentative value.
+  bool Tentative;
+
 public:
   /// Constructor for an invalid value.
   FileCheckNumExprVal() : Valid(false) {}
 
   /// Constructor for a signed value.
   explicit FileCheckNumExprVal(int64_t Val)
-      : SignedValue(Val), Signed(true), Valid(true) {}
+      : SignedValue(Val), Signed(true), Valid(true), Tentative(false) {}
 
   /// Constructor for an unsigned value.
   explicit FileCheckNumExprVal(uint64_t Val)
-      : UnsignedValue(Val), Signed(false), Valid(true) {}
+      : UnsignedValue(Val), Signed(false), Valid(true), Tentative(false) {}
 
   /// Define equality to be true only if both values are valid and they have
   /// the same signedness and corresponding value.  Tentative bit is ignored to
   /// allow the evaluation of a numeric expression using variables with
   /// tentative value to compare equal to a matched value.
   bool operator==(const FileCheckNumExprVal &other);
   bool operator!=(const FileCheckNumExprVal &other) {
     return !(*this == other);
   }
 
   bool isSigned() const { return Signed; }
 
   bool isValid() const { return Valid; }
 
+  /// Whether this is a tentative value, ie other expressions on the same CHECK
+  /// line that might be using that value or other values that this value was
+  /// computed from have not yet been verified to match. True if any of the
+  /// operand involved in its computation had a tentative value.
+  bool isTentative() const { return Tentative; }
+
   /// Return the signed value. Must only be called if value is signed in the
   /// first place.
   int64_t getSignedValue() const {
     assert(Signed);
     return SignedValue;
   }
 
   /// Return the unsigned value. Must only be called if value is unsigned in
   /// the first place.
   uint64_t getUnsignedValue() const {
     assert(!Signed);
     return UnsignedValue;
   }
 
+  /// Change whether this value is tentative or not.
+  bool toggleTentative() { return Tentative = !Tentative; }
+
   /// Convert value to a signed value, or mark value invalid if not possible
   /// (original value was not within range for a signed integer).
   void convertSigned();
 
   /// Convert value to an unsigned value, or mark value invalid if not possible
   /// (original value was not within range for an unsigned integer).
   void convertUnsigned();
 

 private:
   /// Pointer to AST of the numeric expression.
   std::shared_ptr<FileCheckNumExprAST> AST;
 
   /// Matching format, i.e format (e.g. hex upper case letters) to use for the
   /// value when matching it.
   FileCheckNumExprFmt Fmt;
 
+  /// Whether the value of this numeric expression can be known at match time
+  /// (ie. when substituting it in the regular expression). If False, the
+  /// value is known after matching has happened.
+  bool MatchTimeKnown;
+
 public:
   /// Generic constructor for a numeric expression whose equality constraint is
-  /// represented by \p AST and matching format is \p Fmt. If matching format
-  /// is unset (ie. no explicit or implicit matching format), set it to default
-  /// one (unsigned decimal integer).
+  /// represented by \p AST, whose matching format is \p Fmt and which is known
+  /// at match time if \p MatchTimeKnown is true. If matching format is unset
+  /// (i.e. no explicit or implicit matching format), set it to default one
+  /// (unsigned decimal integer).
   FileCheckNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
-                   FileCheckNumExprFmt Fmt);
+                   FileCheckNumExprFmt Fmt, bool MatchTimeKnown);
 
   /// \returns pointer to AST of the numeric expression. Pointer is guaranteed
   /// to be valid as long as this object is.
   FileCheckNumExprAST *getAST() const { return AST.get(); }
 
+  /// \returns whether the value of this numeric expression can be known at
+  /// match time (ie. when substituting it in the regular expression). If
+  /// False, the value is known after matching has happened.
+  bool isMatchTimeKnown() const { return MatchTimeKnown; }
+
   /// \returns effective format of this numeric expression, ie (i) its explicit
   /// format if any, (ii) its implicit format if any or (iii) the default
   /// format.
   FileCheckNumExprFmt getEffectiveFmt() { return Fmt; }
+
+  /// Verify that the matched value in \p MatchedValue satisfies the constraint
+  /// expressed by this expression. Return true if constraint is not satisfied.
+  bool verifyConstraint(FileCheckNumExprVal MatchedValue) const;
 };
 
 /// Class representing a numeric variable with a given value in the AST of a
 /// numeric expression. Each definition of a variable gets its own instance of
 /// this class. Variable uses share the same instance as the respective
 /// definition.
 class FileCheckNumericVariable : public FileCheckNumExprAST {
 private:
public:
   bool isMatchTimeKnown() const;
 
   /// \returns implicit format of this numeric variable.
   FileCheckNumExprFmt getImplicitFmt() const;
 
   /// Appends numeric variable's name to UndefVarNames if undefined.
   void appendUndefVarNames(std::vector<StringRef> &UndefVarNames) const;
 
-  /// Sets value of this numeric variable if not defined. \returns whether the
-  /// variable was already defined.
-  bool setValue(FileCheckNumExprVal Value);
+  /// Sets value of this numeric variable if not defined. Value is set as a
+  /// tentative value if \p Tentative is true. \returns whether variable was
+  /// already definitely (i.e. not tentatively) defined.
+  bool setValue(FileCheckNumExprVal Value, bool Tentative);
+
+  /// Marks value as definitive (i.e. not tentative). Return whether value was
+  /// already definitive.
+  bool commitValue();
 
   /// Clears value of this numeric variable. \returns whether the variable was
   /// already undefined.
   bool clearValue();
 
   /// \returns the line number where this variable is defined.
   unsigned getDefLineNumber() { return DefLineNumber; }
 };
public:
 
   // \returns a description of \p Prefix.
   std::string getDescription(StringRef Prefix) const;
 };
 } // namespace Check
 
 struct FileCheckDiag;
 
-/// Structure representing the definition of a numeric variable in a pattern.
-/// It holds the pointer to the class instance holding the value and matching
-/// format of the numeric variable being defined and the parenthesized
-/// subexpression number in RegExStr to capture that value.
+/// Structure representing a numeric expression whose constraint needs to be
+/// verified after matching (either a variable definition with empty numeric
+/// expression or a numeric expression using directly or indirectly such a
+/// variable. It holds a pointer to the class instance holding the numeric
+/// expression to verify, a pointer to the class instance holding the value
+/// and matching format for the numeric variable being defined if any, and
+/// the parenthesized subexpression number in RegExStr to capture that value
+/// that needs verifying.
 struct FileCheckNumExprMatch {
+  /// Pointer to class instance representing the numeric expression to verify.
+  FileCheckNumExpr *NumExpr;
+
   /// Pointer to class instance holding the value and matching format of the
   /// numeric variable being defined.
   FileCheckNumericVariable *DefinedNumericVariable;
 
   /// Parenthesized subexpression number in RegExStr that captures the value
-  /// of this numeric variable definition.
+  /// that needs verifying.
   unsigned CaptureParen;
 };
 
 /// Class holding the FileCheckPattern global state, shared by all patterns:
 /// tables holding values of variables and whether they are defined or not at
 /// any given time in the matching process.
 class FileCheckPatternContext {
   friend class FileCheckPattern;
public:
   /// GlobalNumericVariableTable and for numeric variable clear their value as
   /// well.
   void clearLocalVars();
 
 private:
   /// Makes a new numeric expression instance and registers it for destruction
   /// when the context is destroyed.
   FileCheckNumExpr *makeNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
-                                FileCheckNumExprFmt Fmt);
+                                FileCheckNumExprFmt Fmt, bool MatchTimeKnown);
 
   /// Makes a new string substitution and registers it for destruction when the
   /// context is destroyed.
   FileCheckSubstitution *makeStringSubstitution(StringRef VarName,
                                                 size_t InsertIdx);
 
   /// Makes a new numeric substitution and registers it for destruction when
   /// the context is destroyed.
class FileCheckPattern {
   /// E.g. for the pattern "foo[[bar:.*]]baz[[bar]]quux[[bar:.*]]",
   /// VariableDefs will map "bar" to 2 corresponding to the second definition
   /// of "bar".
   ///
   /// Note: uses std::map rather than StringMap to be able to get the key when
   /// iterating over values.
   std::map<StringRef, unsigned> VariableDefs;
 
-  /// Holds the capture parentheses number and pointer to the corresponding
-  /// FileCheckNumericVariable class instance of all numeric variable
-  /// definitions. Used to set the matched value of all those variables.
-  StringMap<FileCheckNumExprMatch> NumericVariableDefs;
+  /// Holds the capture parentheses number, pointer to corresponding
+  /// FileCheckNumExpr class instance of the numeric expression to verify and
+  /// pointer to the FileCheckNumericVariable class instance of the variable
+  /// defined from this numeric expression, if any, for all numeric expressions
+  /// that needs to be verified after matching.
+  std::vector<FileCheckNumExprMatch> CapturedNumericExpressions;
 
   /// Pointer to a class instance holding the global state shared by all
   /// patterns:
   /// - separate tables with the values of live string and numeric variables
   ///   respectively at the start of any given CHECK line;
   /// - table holding whether a string variable has been defined at any given
   ///   point during the parsing phase.
   FileCheckPatternContext *Context;
public:
   /// The GlobalVariableTable StringMap in the FileCheckPatternContext class
   /// instance provides the current values of FileCheck string variables and
   /// is updated if this match defines new values. Likewise, the
   /// GlobalNumericVariableTable StringMap in the same class provides the
   /// current values of FileCheck numeric variables and is updated if this
   /// match defines new numeric values.
   size_t match(StringRef Buffer, size_t &MatchLen, const SourceMgr &SM) const;
   /// Prints the value of successful substitutions or the name of the undefined
-  /// pattern or numeric variable preventing such a successful substitution.
+  /// pattern or numeric variables preventing such a successful substitution.
+  ///
+  /// Note: Numeric expression using numeric variables defined on the same line
+  /// are not shown because we only have substitutions for numeric expressions
+  /// whose value can be known at match time.
   void printSubstitutions(const SourceMgr &SM, StringRef Buffer,
                           SMRange MatchRange = None) const;
   void printFuzzyMatch(const SourceMgr &SM, StringRef Buffer,
                        std::vector<FileCheckDiag> *Diags) const;
 
   bool hasVariable() const {
     return !(Substitutions.empty() && VariableDefs.empty());
   }
private:
   /// parsing the variable name. Otherwise, sets \p Name to the name of the
   /// parsed numeric variable.
   bool parseNumericVariable(StringRef &Expr, StringRef &Name, bool IsDefinition,
                             bool AcceptFail, const SourceMgr &SM) const;
   /// Parses \p Expr for use of a numeric operand. Accepts both literal values
   /// and numeric variables, depending on the value of \p AllowedOperandFlag.
   /// \returns the class representing that operand in the AST of the numeric
   /// expression or nullptr if parsing fails in which case errors are reported
-  /// on \p SM.
+  /// on \p SM. Also return in MatchTimeKnown if the operand's value is known
+  /// at match time.
   std::shared_ptr<FileCheckNumExprAST>
   parseNumericOperand(StringRef &Expr, enum AllowedOperand AO,
-                      const SourceMgr &SM) const;
-  /// Parses \p Expr for a binary operation. The left operand of this binary
-  /// operation is given in \p LeftOp and \p Legacy indicates whether we are
-  /// parsing a legacy numeric expression. \returns the class representing the
-  /// binary operation in the AST of the numeric expression, or nullptr if
-  /// parsing fails, in which case errors are reported on \p SM.
+                      bool &MatchTimeKnown, const SourceMgr &SM) const;
+  /// Parses \p Expr for a binary operation, as a legacy numeric expression if
+  /// \p Legacy is true. The left operand of this binary operation is given in
+  /// \p LeftOp with \p MatchTimeKnown holding whether the left operand's value
+  /// is known at match time. \returns the class representing the binary
+  /// operation in the AST of the numeric expression, or nullptr if parsing
+  /// fails, in which case errors are reported on \p SM. Also sets \p to
+  /// whether the binary operation can be evaluated at match time.
   std::shared_ptr<FileCheckNumExprAST>
-  parseBinop(StringRef &Expr, std::shared_ptr<FileCheckNumExprAST> LeftOp,
-             bool Legacy, const SourceMgr &SM) const;
+  parseBinop(StringRef &Expr, std::shared_ptr<FileCheckNumExprAST> Opl,
+             bool Legacy, bool &MatchTimeKnown, const SourceMgr &SM) const;
 };
 
 //===----------------------------------------------------------------------===//
 /// Summary of a FileCheck diagnostic.
 //===----------------------------------------------------------------------===//
 
 struct FileCheckDiag {
   /// What is the FileCheck directive for this diagnostic?

llvm/lib/Support/FileCheck.cpp

if (Op1.Signed) {
     // Op1 < Op2.
     if (Val1 < Val2)
       return FileCheckNumExprVal();
     return FileCheckNumExprVal(Val1 - Val2);
   }
 }
 
 FileCheckNumExpr::FileCheckNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
-                                   FileCheckNumExprFmt Fmt)
-    : AST(AST) {
+                                   FileCheckNumExprFmt Fmt, bool MatchTimeKnown)
+    : AST(AST), MatchTimeKnown(MatchTimeKnown) {
   if (Fmt.Valid)
     this->Fmt = Fmt;
   else
     this->Fmt = FmtUnsigned;
 }
 
 FileCheckNumExprVal FileCheckNumericVariable::eval() const {
   if (Value.isValid())
     return Value;
 
   if (NumExpr == nullptr || NumExpr->getAST() == nullptr)
     return FileCheckNumExprVal();
 
   return NumExpr->getAST()->eval();
 }
 
 bool FileCheckNumericVariable::isMatchTimeKnown() const {
   if (Value.isValid())
     return true;
 
-  return NumExpr != nullptr && NumExpr->getAST() != nullptr;
+  return NumExpr != nullptr && NumExpr->isMatchTimeKnown();
 }
 
 FileCheckNumExprFmt FileCheckNumericVariable::getImplicitFmt() const {
   if (NumExpr == nullptr)
     return FmtNone;
 
   return NumExpr->getEffectiveFmt();
 }
 
 void FileCheckNumericVariable::appendUndefVarNames(
     std::vector<StringRef> &UndefVarNames) const {
   if (!Value.isValid())
     UndefVarNames.emplace_back(Name);
 }
 
-bool FileCheckNumericVariable::setValue(FileCheckNumExprVal NewValue) {
-  if (Value.isValid())
+bool FileCheckNumericVariable::setValue(FileCheckNumExprVal NewValue,
+                                        bool Tentative) {
+  if (Value.isValid() && !Value.isTentative())
     return true;
+
+  assert(NumExpr != nullptr && "Setting value of parse time variable");
+  assert(!NewValue.isTentative() && "Setting value from tentative value");
   assert(NewValue.isValid() && "Setting invalid value");
+  if (Tentative)
+    NewValue.toggleTentative();
   Value = NewValue;
   return false;
 }
 
+bool FileCheckNumericVariable::commitValue() {
+  // Value is not a valid tentative value.
+  if (!Value.isValid() || !Value.isTentative())
+    return true;
+
+  Value.toggleTentative();
+  return false;
+}
+
 bool FileCheckNumericVariable::clearValue() {
   if (!Value.isValid())
     return true;
   Value = FileCheckNumExprVal();
   NumExpr = nullptr;
   return false;
 }
 
 FileCheckNumExprVal FileCheckASTBinop::eval() const {
   FileCheckNumExprVal LeftValue = LeftOp->eval();
   FileCheckNumExprVal RightValue = RightOp->eval();
 
   // Integer promotion.
   if (!LeftValue.isSigned() || !RightValue.isSigned()) {
     LeftValue.convertUnsigned();
     RightValue.convertUnsigned();
   }
 
   if (!LeftValue.isValid() || !RightValue.isValid())
     return FileCheckNumExprVal();
 
-  return EvalBinop(LeftValue, RightValue);
+  FileCheckNumExprVal ResultValue = EvalBinop(LeftValue, RightValue);
+
+  // Mark result as tentative if any of the operand was tentative. Assert
+  // result was not tentative beforehand to have that check only here.
+  assert((!ResultValue.isValid() || !ResultValue.isTentative()) &&
+         "Binary operation returned tentative value");
+  if (ResultValue.isValid() &&
+      (LeftValue.isTentative() || RightValue.isTentative()))
+    ResultValue.toggleTentative();
+  return ResultValue;
+}
+
+/// Verify that the matched value in \p MatchedValue satisfies the constraint
+/// expressed by this expression. Return true if constraint is not satisfied.
+bool FileCheckNumExpr::verifyConstraint(
+    FileCheckNumExprVal MatchedValue) const {
+  // Nothing to verify: numeric variable definition with empty numeric
+  // expression.
+  if (AST == nullptr)
+    return false;
+
+  // Will fail if evaluated value or MatchedValue are invalid.
+  return AST->eval() != MatchedValue;
 }
 
 FileCheckNumExprFmt FileCheckASTBinop::getImplicitFmt() const {
   FileCheckNumExprFmt LeftFmt = LeftOp->getImplicitFmt();
   FileCheckNumExprFmt RightFmt = RightOp->getImplicitFmt();
 
   FileCheckNumExprFmt Fmt = (LeftFmt.Valid) ? LeftFmt : RightFmt;
   if (LeftFmt.Valid && RightFmt.Valid && LeftFmt != RightFmt)
bool FileCheckPattern::parseNumericVariable(StringRef &Expr, StringRef &Name,
   // use.
   auto VarTableIter = Context->GlobalNumericVariableTable.find(Name);
   if (VarTableIter == Context->GlobalNumericVariableTable.end()) {
     SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
                     "using undefined numeric variable '" + Name + "'");
     return true;
   }
 
-  FileCheckNumericVariable *NumericVariable = VarTableIter->second.get();
-  if (!IsPseudo && NumericVariable->getDefLineNumber() == LineNumber &&
-      !NumericVariable->isMatchTimeKnown()) {
-    SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
-                    "numeric variable '" + NumericVariable->getName() +
-                        "' defined on the same line");
-    return true;
-  }
-
   return false;
 }
 
 std::shared_ptr<FileCheckNumExprAST>
 FileCheckPattern::parseNumericOperand(StringRef &Expr, enum AllowedOperand AO,
+                                      bool &MatchTimeKnown,
                                       const SourceMgr &SM) const {
 
   // Try to parse as a numeric variable use.
   if (AO == LegacyVar || AO == Any) {
     StringRef Name;
     // Error reporting done in ParseNumericVariable.
     if (!parseNumericVariable(Expr, Name, false /*IsDefinition*/,
                               true /*AcceptFail*/, SM)) {
       std::shared_ptr<FileCheckNumericVariable> NumericVariable =
           Context->GlobalNumericVariableTable.find(Name)->second;
+      MatchTimeKnown = NumericVariable->getDefLineNumber() < LineNumber ||
+                       NumericVariable->isMatchTimeKnown();
       return NumericVariable;
     }
   }
 
   // Otherwise, parse it as a literal.
   if (AO != LegacyLiteral && AO != Any)
     return nullptr;
   unsigned Radix = (AO == LegacyLiteral) ? 10 : 0;
   int64_t SignedLiteralValue;
   uint64_t UnsignedLiteralValue;
   StringRef SaveExpr = Expr;
   // Accept both signed and unsigned literal.
   if (AO != LegacyLiteral && !Expr.consumeInteger(Radix, SignedLiteralValue)) {
+    MatchTimeKnown = true;
     return std::make_shared<FileCheckNumExprLiteral>(SignedLiteralValue);
   } else {
     Expr = SaveExpr;
     if (!Expr.consumeInteger(Radix, UnsignedLiteralValue)) {
+      MatchTimeKnown = true;
       return std::make_shared<FileCheckNumExprLiteral>(UnsignedLiteralValue);
     } else
       return nullptr;
   }
 }
 
-std::shared_ptr<FileCheckNumExprAST>
-FileCheckPattern::parseBinop(StringRef &Expr,
-                             std::shared_ptr<FileCheckNumExprAST> LeftOp,
-                             bool Legacy, const SourceMgr &SM) const {
+std::shared_ptr<FileCheckNumExprAST> FileCheckPattern::parseBinop(
+    StringRef &Expr, std::shared_ptr<FileCheckNumExprAST> LeftOp, bool Legacy,
+    bool &MatchTimeKnown, const SourceMgr &SM) const {
   Expr = Expr.ltrim(SpaceChars);
   if (Expr.empty())
     return LeftOp;
 
   // Check if this is a supported operation and select a function to perform
   // it.
   SMLoc OpLoc = SMLoc::getFromPointer(Expr.data());
   char Operator = popFront(Expr);
std::shared_ptr<FileCheckNumExprAST> FileCheckPattern::parseBinop(
   Expr = Expr.ltrim(SpaceChars);
   if (Expr.empty()) {
     SM.PrintMessage(SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
                     "missing operand in numeric expression");
     return nullptr;
   }
   // Second operand in legacy numeric expression is a literal.
   enum AllowedOperand AO = Legacy ? LegacyLiteral : Any;
+  bool OprMatchTimeKnown;
   std::shared_ptr<FileCheckNumExprAST> RightOp =
-      parseNumericOperand(Expr, AO, SM);
+      parseNumericOperand(Expr, AO, OprMatchTimeKnown, SM);
   if (RightOp == nullptr) {
     SM.PrintMessage(SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
                     "invalid operand format '" + Expr + "'");
     return nullptr;
   }
   Expr = Expr.ltrim(SpaceChars);
+  MatchTimeKnown = MatchTimeKnown && OprMatchTimeKnown;
   return std::make_shared<FileCheckASTBinop>(EvalBinop, LeftOp, RightOp);
 }
 
 FileCheckNumExpr *FileCheckPattern::parseNumericSubstitution(
     StringRef Expr, FileCheckNumericVariable *&DefinedNumericVariable,
     bool Legacy, const SourceMgr &SM) const {
   std::shared_ptr<FileCheckNumExprAST> NumExprAST;
   StringRef DefExpr = StringRef();
FileCheckNumExpr *FileCheckPattern::parseNumericSubstitution(
   Expr = Expr.ltrim(SpaceChars);
   StringRef ConstraintExpr = Expr;
   if (stripFront(Expr, "==", true /*Optional*/))
     return nullptr;
 
   // Parse numeric expression itself.
   Expr = Expr.ltrim(SpaceChars);
   StringRef UseExpr = Expr;
+  bool MatchTimeKnown = false;
   if (Expr.empty()) {
     if (Expr.size() != ConstraintExpr.size()) {
       SM.PrintMessage(
           SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
           "Empty numeric expression should not have a matching constraint");
       return nullptr;
     }
   } else {
     // First operand in legacy numeric expression is the @LINE pseudo variable.
     enum AllowedOperand AO = Legacy ? LegacyVar : Any;
-    NumExprAST = parseNumericOperand(Expr, AO, SM);
+    NumExprAST = parseNumericOperand(Expr, AO, MatchTimeKnown, SM);
     while (NumExprAST != nullptr && !Expr.empty()) {
-      NumExprAST = parseBinop(Expr, NumExprAST, Legacy, SM);
+      NumExprAST = parseBinop(Expr, NumExprAST, Legacy, MatchTimeKnown, SM);
       // Legacy numeric expressions only allow 2 operands.
       if (Legacy && !Expr.empty()) {
         SM.PrintMessage(SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
                         "unexpected characters at end of numeric expression '" +
                             Expr + "'");
         return nullptr;
       }
     }
FileCheckNumExpr *FileCheckPattern::parseNumericSubstitution(
   else if (ImplicitFmt.Conflict) {
     SM.PrintMessage(
         SMLoc::getFromPointer(UseExpr.data()), SourceMgr::DK_Error,
         "variables with conflicting format specifier: need an explicit one");
     return nullptr;
   } else
     Fmt = ImplicitFmt;
 
-  FileCheckNumExpr *NumExpr = Context->makeNumExpr(NumExprAST, Fmt);
+  FileCheckNumExpr *NumExpr =
+      Context->makeNumExpr(NumExprAST, Fmt, MatchTimeKnown);
 
   // Parse numeric variable definition.
   if (!DefExpr.empty()) {
     DefExpr = DefExpr.ltrim(SpaceChars);
     StringRef Name;
     if (parseNumericVariable(DefExpr, Name, true /*IsDefinition*/,
                              false /*AcceptFail*/, SM)) {
       // Invalid variable definition. Error reporting done in parsing function.
if (PatternStr.startswith("[[")) {
       }
       // Strip the subtitution block we are parsing. End points to the start of
       // the "]]" closing the expression so account for it in computing the
       // index of the first unparsed character.
       PatternStr = UnparsedPatternStr.substr(End + 2);
 
       bool IsVarDef = false;
       bool SubstNeeded = false;
+      bool CaptureNeeded = false;
       bool Legacy = false;
       StringRef DefName;
       StringRef SubstStr;
       StringRef MatchRegexp = StringRef();
       size_t SubstInsertIdx = RegExStr.size();
 
       // Parse string variable or legacy numeric expression.
       if (!IsNumBlock) {
if (PatternStr.startswith("[[")) {
         if (parseVariable(MatchStr, IsPseudo, TrailIdx)) {
           SM.PrintMessage(SMLoc::getFromPointer(MatchStr.data()),
                           SourceMgr::DK_Error, "invalid variable name");
           return true;
         }
 
         StringRef Name = MatchStr.substr(0, TrailIdx);
         StringRef Trailer = MatchStr.substr(TrailIdx);
-        IsVarDef = (VarEndIdx != StringRef::npos);
+        CaptureNeeded = IsVarDef = (VarEndIdx != StringRef::npos);
         Legacy = IsNumBlock = IsPseudo;
         SubstNeeded = !IsVarDef;
 
         if (IsVarDef) {
           if ((IsPseudo || !Trailer.consume_front(":"))) {
             SM.PrintMessage(SMLoc::getFromPointer(MatchStr.data()),
                             SourceMgr::DK_Error,
                             "invalid name in string variable definition");
if (PatternStr.startswith("[[")) {
       FileCheckNumExpr *NumExpr;
       FileCheckNumericVariable *DefinedNumericVariable;
       if (IsNumBlock) {
         NumExpr = parseNumericSubstitution(MatchStr, DefinedNumericVariable,
                                            Legacy, SM);
         if (NumExpr == nullptr)
           return true;
         IsVarDef = DefinedNumericVariable != nullptr;
-        SubstNeeded = NumExpr->getAST() != nullptr;
         if (IsVarDef)
           DefName = DefinedNumericVariable->getName();
+        SubstNeeded = NumExpr->isMatchTimeKnown();
+        CaptureNeeded = IsVarDef || !NumExpr->isMatchTimeKnown();
         if (SubstNeeded)
           SubstStr = MatchStr;
         else {
           FileCheckNumExprFmt Fmt = NumExpr->getEffectiveFmt();
           MatchRegexp = Fmt.getWildcardRegex();
         }
       }
 
-      // Handle variable definition: [[<def>:(...)]] and [[#(...)<def>:(...)]].
-      if (IsVarDef) {
+      // Handle variable definition: [[<def>:(...)]] and [[#(...)<def>:(...)]]
+      // and numeric expression needing verifying.
+      if (CaptureNeeded) {
         RegExStr += '(';
         ++SubstInsertIdx;
 
         if (IsNumBlock) {
-          struct FileCheckNumExprMatch NumExprDef = {DefinedNumericVariable,
-                                                     CurParen};
-          NumericVariableDefs[DefName] = NumExprDef;
+          struct FileCheckNumExprMatch NumExprDef = {
+              NumExpr, DefinedNumericVariable, CurParen};
+          CapturedNumericExpressions.emplace_back(NumExprDef);
           // This store is done here rather than in match() to allow
           // ParseNumericVariable() to get the pointer to the class instance
           // of the right variable definition corresponding to a given numeric
           // variable use.
           Context->GlobalNumericVariableTable[DefName] =
               std::shared_ptr<FileCheckNumericVariable>(DefinedNumericVariable);
         } else {
           VariableDefs[DefName] = CurParen;
           // Mark string variable as defined to detect collisions between
           // string and numeric variable in parseNumericVariable and
           // DefineCmdlineVariables when the latter is created later than the
           // former. We cannot reuse GlobalVariableTable for that by populating
           // it with an empty string since we would then loose the ability to
           // detect the use of an undefined variable in match().
           Context->DefinedVariableTable[DefName] = true;
         }
 
         ++CurParen;
       }
 
       if (!MatchRegexp.empty() && AddRegExToRegEx(MatchRegexp, CurParen, SM))
         return true;
 
-      if (IsVarDef)
+      if (CaptureNeeded)
         RegExStr += ')';
 
       // Handle substitutions: [[foo]] and [[#<foo expr>]].
       if (SubstNeeded) {
         // Handle substitution of string variables that were defined earlier on
         // the same line by emitting a backreference. Numeric expressions do
         // not support substituting a numeric variable defined on the same
         // line.
size_t FileCheckPattern::match(StringRef Buffer, size_t &MatchLen,
   }
 
   // Regex match.
 
   // If there are substitutions, we need to create a temporary string with the
   // actual value.
   StringRef RegExToMatch = RegExStr;
   std::string TmpStr;
+
   if (!Substitutions.empty()) {
     TmpStr = RegExStr;
 
     size_t InsertOffset = 0;
     // Substitute all string variables and numeric expressions whose value is
     // only known now. Use of string variables defined on the same line are
-    // handled by back-references.
+    // handled by back-references. For numeric expressions using variable
+    // with empty numeric expression defined on the same line, RegExStr is
+    // filled with a suitable wildcard pattern and the constraint is then
+    // verified against the matched value.
+    //
+    // Known issue: Due to the constraints for numeric expressions using
+    // variable defined in earlier line not being expressed at the regular
+    // expression level, the wrong values might be matched, leading to a
+    // constraint check failure. An example of this is what happens when
+    // checking the line "1 3 4" against the directive
+    // "CHECK: [[#N:]] [[#N+1]]" which gets substituted by
+    // "[[:digit:]]+ [[:digit:]]+". This would match 1 3 which then fails the
+    // constraint checks.
+    //
+    // One possible solution would be to augment the regular expression engine
+    // to be able to return all matches for a given pattern. FileCheck could
+    // then test all possibility and fail the check only if none of them
+    // satisfy all numeric expressions. This would work both for numeric
+    // expression using a numeric variable defined on the same line as well as
+    // numeric expressions with a comparison rather than equality constraint.
     for (const auto &Substitution : Substitutions) {
       // Substitute and check for failure (e.g. use of undefined variable).
       llvm::Optional<std::string> Value = Substitution->getResult();
       if (!Value) {
         SM.PrintMessage(
             SMLoc::getFromPointer(Substitution->getFromString().data()),
             SourceMgr::DK_Error,
             "Unable to substitute variable or numeric expression");
         return StringRef::npos;
       }
 
       // Plop it into the regex at the adjusted offset.
       TmpStr.insert(TmpStr.begin() + Substitution->getIndex() + InsertOffset,
                     Value->begin(), Value->end());
       InsertOffset += Value->size();
     }
 
     // Match the newly constructed regex.
     RegExToMatch = TmpStr;
   }
 
+  bool AllNumExprSatisfied;
   SmallVector<StringRef, 4> MatchInfo;
+  do {
+    AllNumExprSatisfied = true;
     if (!Regex(RegExToMatch, Regex::Newline).match(Buffer, &MatchInfo))
       return StringRef::npos;
 
     // Successful regex match.
     assert(!MatchInfo.empty() && "Didn't get any match");
-  StringRef FullMatch = MatchInfo[0];
 
     // If this defines any string variables, remember their values.
     for (const auto &VariableDef : VariableDefs) {
       assert(VariableDef.second < MatchInfo.size() && "Internal paren error");
       Context->GlobalVariableTable[VariableDef.first] =
           MatchInfo[VariableDef.second];
     }
 
-  // If this defines any numeric variables, remember their values.
-  for (const auto &NumericVariableDef : NumericVariableDefs) {
-    assert(NumericVariableDef.getValue().CaptureParen < MatchInfo.size() &&
+    // Check the matched value satisfies the numeric expression constraint and
+    // store it in the numeric variable being defined if any.
+    for (const auto &CapturedNumericExpression : CapturedNumericExpressions) {
+      assert(CapturedNumericExpression.CaptureParen < MatchInfo.size() &&
              "Internal paren error");
-    unsigned CaptureParen = NumericVariableDef.getValue().CaptureParen;
+      unsigned CaptureParen = CapturedNumericExpression.CaptureParen;
+      StringRef MatchedStr = MatchInfo[CaptureParen];
+      FileCheckNumExpr *NumExpr = CapturedNumericExpression.NumExpr;
       FileCheckNumericVariable *DefinedNumericVariable =
-        NumericVariableDef.getValue().DefinedNumericVariable;
+          CapturedNumericExpression.DefinedNumericVariable;
 
       StringRef MatchedValue = MatchInfo[CaptureParen];
-    assert(DefinedNumericVariable->getNumExpr() != nullptr);
-    FileCheckNumExprFmt Fmt =
-        DefinedNumericVariable->getNumExpr()->getEffectiveFmt();
+      FileCheckNumExprFmt Fmt = NumExpr->getEffectiveFmt();
       FileCheckNumExprVal Value = Fmt.valueFromStringRepr(MatchedValue);
+
+      // Get numeric value from the matched string according to matching format
+      // of the numeric expression. Error out if this fails.
       if (!Value.isValid()) {
-      SM.PrintMessage(SMLoc::getFromPointer(MatchedValue.data()),
-                      SourceMgr::DK_Error, "Unable to represent numeric value");
+        SM.PrintMessage(SMLoc::getFromPointer(MatchedStr.data()),
+                        SourceMgr::DK_Error,
+                        "Unable to represent numeric value " + MatchedStr);
+        return StringRef::npos;
       }
-    if (DefinedNumericVariable->setValue(Value))
-      assert(false && "Numeric variable redefined");
+
+      // Now verify the constraint of the numeric expression.
+      if (NumExpr->verifyConstraint(Value)) {
+        AllNumExprSatisfied = false;
+        // Try matching again from the line following the one matched. There
+        // might be other possible matches in the current line but we have no
+        // way to ask for them to the regular expression engine. However this
+        // hazard can be alleviated if the user write a CHECK pattern that uses
+        // numeric expression for all numeric values the line to match contains.
+        Buffer = Buffer.substr(Buffer.find_first_of('\n')).substr(1);
+        break;
+      }
+
+      // Store verified matched value. This will set variable defined by the
+      // expression if any.
+      if (DefinedNumericVariable != nullptr &&
+          DefinedNumericVariable->setValue(Value, true /*Tentative*/))
+        assert(false && "Value of numeric variable already set");
+    }
+  } while (!AllNumExprSatisfied && !Buffer.empty());
+
+  // All constraints satisfied, make variable definitions definitive.
+  if (AllNumExprSatisfied) {
+    for (const auto &CapturedNumericExpression : CapturedNumericExpressions) {
+      FileCheckNumericVariable *DefinedNumericVariable =
+          CapturedNumericExpression.DefinedNumericVariable;
+      if (DefinedNumericVariable != nullptr)
+        DefinedNumericVariable->commitValue();
     }
+    // No line with all constraints satisfied found, return failure to match.
+  } else
+    return StringRef::npos;
 
   // Like CHECK-NEXT, CHECK-EMPTY's match range is considered to start after
   // the required preceding newline, which is consumed by the pattern in the
   // case of CHECK-EMPTY but not CHECK-NEXT.
   size_t MatchStartSkip = CheckTy == Check::CheckEmpty;
+  StringRef FullMatch = MatchInfo[0];
   MatchLen = FullMatch.size() - MatchStartSkip;
   return FullMatch.data() - Buffer.data() + MatchStartSkip;
 }
 
 unsigned FileCheckPattern::computeMatchDistance(StringRef Buffer) const {
   // Just compute the number of matching characters. For regular expressions, we
   // just compare against the regex itself and hope for the best.
   //
   // FIXME: One easy improvement here is have the regex lib generate a single
   // example regular expression which matches, and use that as the example
   // string.
   StringRef ExampleString(FixedStr);
   if (ExampleString.empty())
     ExampleString = RegExStr;
 
   // Only compare up to the first line in the buffer, or the string size.
   StringRef BufferPrefix = Buffer.substr(0, ExampleString.size());
   BufferPrefix = BufferPrefix.split('\n').first;
   return BufferPrefix.edit_distance(ExampleString);
 }
 
 void FileCheckPattern::printSubstitutions(const SourceMgr &SM, StringRef Buffer,
                                           SMRange MatchRange) const {
-  // Print what we know about substitutions.
+  // Print what we know about substitutions. This covers both string and
+  // numeric substitutions as long as they only use variables defined in
+  // earlier lines.
   if (!Substitutions.empty()) {
     for (const auto &Substitution : Substitutions) {
       SmallString<256> Msg;
       raw_svector_ostream OS(Msg);
       llvm::Optional<std::string> MatchedValue = Substitution->getResult();
 
       // Substitution failed or is not known at match time, print the undefined
       // variables it uses.
       if (!MatchedValue) {
         std::vector<StringRef> UndefVarNames;
         Substitution->getUndefVarNames(UndefVarNames);
         if (UndefVarNames.empty())
           continue;
         for (auto UndefVarName : UndefVarNames) {
           OS << "uses undefined variable \"";
           OS.write_escaped(UndefVarName) << "\"";
         }
       } else {
-        // Substitution succeeded. Print substituted value.
+        // Substitution succeeded and numeric expression satisfied its
+        // constraint. Print substituted value.
         OS << "with \"";
         OS.write_escaped(Substitution->getFromString()) << "\" equal to \"";
         OS.write_escaped(*MatchedValue) << "\"";
       }
 
       if (MatchRange.isValid())
         SM.PrintMessage(MatchRange.Start, SourceMgr::DK_Note, OS.str(),
                         {MatchRange});
FileCheckPatternContext::getPatternVarValue(StringRef VarName) {
   if (VarIter == GlobalVariableTable.end())
     return llvm::None;
 
   return VarIter->second;
 }
 
 FileCheckNumExpr *
 FileCheckPatternContext::makeNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
-                                     FileCheckNumExprFmt Fmt) {
-  NumExprs.push_back(llvm::make_unique<FileCheckNumExpr>(AST, Fmt));
+                                     FileCheckNumExprFmt Fmt,
+                                     bool MatchTimeKnown) {
+  NumExprs.push_back(
+      llvm::make_unique<FileCheckNumExpr>(AST, Fmt, MatchTimeKnown));
   return NumExprs.back().get();
 }
 
 FileCheckSubstitution *
 FileCheckPatternContext::makeStringSubstitution(StringRef VarName,
                                                 size_t InsertIdx) {
   Substitutions.push_back(
       llvm::make_unique<FileCheckStringSubstitution>(this, VarName, InsertIdx));
if (CmdlineDef[0] == '#') {
       FileCheckNumExprVal Value = NumExpr->getAST()->eval();
       if (!Value.isValid()) {
         SM.PrintMessage(SMLoc::getFromPointer(CmdlineDefExpr.data()),
                         SourceMgr::DK_Error,
                         "unable to represent numeric value");
         ErrorFound = true;
         continue;
       }
-      DefinedNumericVariable->setValue(Value);
+      DefinedNumericVariable->setValue(Value, false /*Tentative*/);
 
       // Record this variable definition.
       GlobalNumericVariableTable[DefinedNumericVariable->getName()] =
           std::shared_ptr<FileCheckNumericVariable>(DefinedNumericVariable);
     } else {
       // String variable definition.
       std::pair<StringRef, StringRef> CmdlineNameVal = CmdlineDef.split('=');
       StringRef Name = CmdlineNameVal.first;

llvm/test/FileCheck/numeric-expression.txt

 CHECK-LABEL: USE CONV FMT IMPL MATCH
 CHECK-NEXT: [[# %x, VAR1]]
 CHECK-NEXT: [[# %X, VAR1]]
 CHECK-NEXT: [[# %u, VAR3]]
 CHECK-NEXT: [[# %d, VAR3]]
 CHECK-NEXT: [[# %u, VAR4]]
 CHECK-NEXT: [[# %d, VAR4]]
 
+; Numeric expression with explicit and default matching format and implicit
+; matching rule using variables defined on same line
+USE SAME EXPL FMT IMPL MATCH
+11 10
+-11 C
+b c
+b 12 c
+C B
+B 10
+; CHECK-LABEL: USE SAME EXPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR5:]] [[#%u,VAR5-1]]
+; CHECK-NEXT: [[#%d,VAR6:]] [[#%X,VAR6+23]]
+; CHECK-NEXT: [[#%x,VAR7:]] [[#%x,VAR7+1]]
+; CHECK-NEXT: [[#%x,VAR8:]] [[#%u,VAR8+1]] [[#%x,VAR8+1]]
+; CHECK-NEXT: [[#%X,VAR9:]] [[#%X,VAR9-1]]
+; CHECK-NEXT: [[#%X,VAR10:]] [[#%d,VAR10-1]]
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables defined on same line
+USE SAME IMPL FMT IMPL MATCH
+11 10
+-11 -4
+b c
+b 12 c
+C B
+; CHECK-LABEL: USE SAME IMPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR5:]] [[#VAR5-1]]
+; CHECK-NEXT: [[#%d,VAR6:]] [[#VAR6+7]]
+; CHECK-NEXT: [[#%x,VAR7:]] [[#VAR7+1]]
+; CHECK-NEXT: [[#%x,VAR8:]] [[#%u,VAR8+1]] [[#VAR8+1]]
+; CHECK-NEXT: [[#%X,VAR9:]] [[#VAR9-1]]
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables defined on same line satisfying constraint is not the first
+; line encountered.
+REDEF SAME IMPL FMT IMPL MATCH NOT FIRST
+SAME LINE USE IMPL FMT IMPL MATCH NOT FIRST
+1 3
+1 2
+; CHECK-LABEL: SAME LINE USE IMPL FMT IMPL MATCH NOT FIRST
+; CHECK: [[#N:]] [[#N+1]]
+
+; Numeric expression with explicit matching format and implicit matching rule
+; using variables redefined on same line
+REDEF EXPL FMT IMPL MATCH
+21
+22 -31 -32
+AC/DC dc db
+; CHECK-LABEL: REDEF EXPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR11:]]
+; CHECK-NEXT: [[#%u,VAR11+1]] [[#%d,VAR11:]] [[#%d,VAR11-1]]
+; CHECK-NEXT: [[#%X,VAR12:]]/[[#%X,VAR12+48]] [[#%x,VAR12:]] [[#%x,VAR12-1]]
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables redefined on same line
+REDEF IMPL FMT IMPL MATCH
+21
+22 -31 -32
+AC/DC dc db
+; CHECK-LABEL: REDEF IMPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR11:]]
+; CHECK-NEXT: [[#VAR11+1]] [[#%d,VAR11:]] [[#VAR11-1]]
+; CHECK-NEXT: [[#%X,VAR12:]]/[[#VAR12+0x30]] [[#%x,VAR12:]] [[#VAR12-1]]
+
 ; Numeric variable definition with unsupported matching format
 RUN: not FileCheck -check-prefixes ERR,INVALID-FMT-SPEC1 -input-file %s %s 2>&1 \
 RUN:   | FileCheck -check-prefix INVALID-FMT-SPEC-MSG1 %s
 RUN: not FileCheck -check-prefixes ERR,INVALID-FMT-SPEC2 -input-file %s %s 2>&1 \
 RUN:   | FileCheck -check-prefix INVALID-FMT-SPEC-MSG2 %s
 
 DEF INVALID FMT
 INVVAR1=a

 
 EMPTY NUMEXPR DEF WITH CONSTRAINT
 18
 EMPTY-NUMDEF-CONSTRAINT-LABEL: EMPTY NUMEXPR CONSTRAINT
 EMPTY-NUMDEF-CONSTRAINT-NEXT: [[#VARDEF: ==]]
 EMPTY-NUMDEF-CONSTRAINT-MSG: numeric-expression.txt:[[#@LINE-1]]:44: error: Empty numeric expression should not have a matching constraint
 EMPTY-NUMDEF-CONSTRAINT-MSG-NEXT: {{E}}MPTY-NUMDEF-CONSTRAINT-NEXT: {{\[\[#VARDEF: ==\]\]}}
 EMPTY-NUMDEF-CONSTRAINT-MSG-NEXT: {{^                                           \^$}}
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables defined on same line satisfying constraint is not the first
+; line encountered.
+RUN: not FileCheck -check-prefix SAME-LINE-USE-FAIL -input-file %s %s
+
+SAME LINE USE IMPL FMT IMPL MATCH FAIL CONSTRAINT
+1 3
+1 4
+SAME-LINE-USE-FAIL-LABEL: SAME LINE USE IMPL FMT IMPL MATCH FAIL CONSTRAINT
+SAME-LINE-USE-FAIL-NEXT: [[#N:]] [[#N+1]]

llvm/test/FileCheck/verbose.txt

 VV-NEXT: verbose.txt:[[@LINE-17]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT: {{C}}HECK-NOT: raboof{{$}}
 VV-NEXT:  {{^           \^$}}
 VV-NEXT: verbose.txt:[[@LINE-22]]:1: note: scanning from here
 VV-NEXT: {{^}}bar{{$}}
 VV-NEXT: {{^}}^{{$}}
 
 NUMVAR=42
-NUMVAR - 1:41
+NUMVAR:42 NUMVAR=12 NUMVAR:12
 CHECK: NUMVAR=[[#NUMVAR:]]
 CHECK-NOT: [[#NUMVAR + 1]]
-CHECK-NEXT: NUMVAR - 1:[[#NUMVAR - 1]]
+CHECK-NEXT: NUMVAR:[[#NUMVAR]] NUMVAR=[[#NUMVAR:]] NUMVAR:[[#NUMVAR]]
 
 V:      verbose.txt:[[#@LINE-4]]:8: remark: {{C}}HECK: expected string found in input
 V-NEXT: {{C}}HECK: {{NUMVAR=[[][[]#NUMVAR:[]][]]$}}
 V-NEXT: {{^       \^$}}
 V-NEXT: verbose.txt:[[#@LINE-9]]:1: note: found here
 V-NEXT: {{^}}NUMVAR=42{{$}}
 V-NEXT: {{^}}^~~~~~~~~{{$}}
 
 V-NEXT: verbose.txt:[[#@LINE-9]]:13: remark: {{C}}HECK-NEXT: expected string found in input
-V-NEXT: {{C}}HECK-NEXT: {{NUMVAR - 1:[[][[]#NUMVAR - 1[]][]]$}}
-V-NEXT: {{^            \^$}}
-V-NEXT: verbose.txt:[[#@LINE-15]]:1: note: found here
-V-NEXT: {{^}}NUMVAR - 1:41{{$}}
-V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
-V-NEXT: verbose.txt:[[#@LINE-18]]:1: note: with "NUMVAR - 1" equal to "41"
-V-NEXT: {{^}}NUMVAR - 1:41{{$}}
-V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
+V-NEXT: {{C}}HECK-NEXT: {{NUMVAR:[[][[]#NUMVAR[]][]] NUMVAR=[[][[]#NUMVAR:[]][]] NUMVAR:[[][[]#NUMVAR[]][]]$}}
+V-NEXT:  {{^            \^$}}
+V-NEXT: verbose.txt:[[#VARLIN1:@LINE-15]]:1: note: found here
+V-NEXT: {{^}}NUMVAR:42 NUMVAR=12 NUMVAR:12{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~~~~~~~~~~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#VARLIN1]]:1: note: with "NUMVAR" equal to "42"
+V-NEXT: {{^}}NUMVAR:42 NUMVAR=12 NUMVAR:12{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~~~~~~~~~~~~~~~~~{{$}}
 
 VV-NEXT: verbose.txt:[[#@LINE-20]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT: {{C}}HECK-NOT: {{[[][[]#NUMVAR [+] 1[]][]]$}}
 VV-NEXT:  {{^           \^$}}
 VV-NEXT: verbose.txt:[[#@LINE-25]]:1: note: scanning from here
-VV-NEXT: {{^}}NUMVAR - 1:41{{$}}
+VV-NEXT: {{^}}NUMVAR:42 NUMVAR=12 NUMVAR:12{{$}}
 VV-NEXT: {{^}}^{{$}}
 
 before empty
 
 after empty
 CHECK: before empty
 CHECK-EMPTY:
 CHECK-NEXT: after empty

llvm/unittests/Support/FileCheckTest.cpp

TEST_F(FileCheckTest, Literal) {
   EXPECT_TRUE(UndefVarNames.empty());
   Ten.appendUndefVarNames(UndefVarNames);
   EXPECT_TRUE(UndefVarNames.empty());
 }
 
 TEST_F(FileCheckTest, NumericVariable) {
   // Undefined variable: eval and clearValue fails, appendUndefVarNames returns
   // the variable and setValue works.
-  auto NumVarExpr = FileCheckNumExpr(nullptr, FmtUnsigned);
+  auto NumVarExpr = FileCheckNumExpr(nullptr, FmtUnsigned, false);
   FileCheckNumericVariable FooVar =
       FileCheckNumericVariable(1, "FOO", &NumVarExpr);
   EXPECT_EQ("FOO", FooVar.getName());
   FileCheckNumExprVal Value = FooVar.eval();
   EXPECT_FALSE(Value.isValid());
   EXPECT_TRUE(FooVar.clearValue());
   std::vector<StringRef> UndefVarNames;
   EXPECT_TRUE(UndefVarNames.empty());
   FooVar.appendUndefVarNames(UndefVarNames);
   EXPECT_EQ(1U, UndefVarNames.size());
   EXPECT_EQ("FOO", UndefVarNames[0]);
   FileCheckNumExprVal LifeAnswer = FileCheckNumExprVal((uint64_t)42);
-  EXPECT_FALSE(FooVar.setValue(LifeAnswer));
+  EXPECT_FALSE(FooVar.setValue(LifeAnswer, false));
 
   // Defined variable: eval returns value set, setValue fails and
   // appendUndefVarNames is a nop.
   Value = FooVar.eval();
   EXPECT_TRUE(Value.isValid());
   EXPECT_EQ(42U, Value.getUnsignedValue());
   FileCheckNumExprVal FourThree = FileCheckNumExprVal((uint64_t)43);
-  EXPECT_TRUE(FooVar.setValue(FourThree));
+  EXPECT_TRUE(FooVar.setValue(FourThree, false));
   Value = FooVar.eval();
   EXPECT_TRUE(Value.isValid());
   EXPECT_EQ(42U, Value.getUnsignedValue());
   UndefVarNames.clear();
   EXPECT_TRUE(UndefVarNames.empty());
   FooVar.appendUndefVarNames(UndefVarNames);
   EXPECT_TRUE(UndefVarNames.empty());
 
if (Op1.isSigned()) {
     uint64_t Val1 = Op1.getUnsignedValue();
     uint64_t Val2 = Op2.getUnsignedValue();
 
     return FileCheckNumExprVal(Val1 + Val2);
   }
 }
 
 TEST_F(FileCheckTest, Binop) {
-  auto DefNumExpr = FileCheckNumExpr(nullptr, FmtUnsigned);
+  auto DefNumExpr = FileCheckNumExpr(nullptr, FmtUnsigned, false);
   auto FooVar =
       std::make_shared<FileCheckNumericVariable>(1, "FOO", &DefNumExpr);
   FileCheckNumExprVal LifeAnswer = FileCheckNumExprVal((uint64_t)42);
-  FooVar->setValue(LifeAnswer);
+  FooVar->setValue(LifeAnswer, false);
   auto BarVar =
       std::make_shared<FileCheckNumericVariable>(2, "BAR", &DefNumExpr);
   FileCheckNumExprVal OneEight = FileCheckNumExprVal((uint64_t)18);
-  BarVar->setValue(OneEight);
+  BarVar->setValue(OneEight, false);
   FileCheckASTBinop Binop = FileCheckASTBinop(doAdd, FooVar, BarVar);
 
   // Defined variable: eval returns right value, no undefined variable
   // returned.
   FileCheckNumExprVal Value = Binop.eval();
   EXPECT_TRUE(Value.isValid());
   EXPECT_EQ(60U, Value.getUnsignedValue());
   std::vector<StringRef> UndefVarNames;
TEST_F(FileCheckTest, Substitution) {
   std::vector<StringRef> UndefVarNames;
   EXPECT_TRUE(UndefVarNames.empty());
   StringSubstitution.getUndefVarNames(UndefVarNames);
   EXPECT_EQ(1U, UndefVarNames.size());
   EXPECT_EQ("VAR404", UndefVarNames[0]);
 
   // Substitution of defined pseudo and non-pseudo numeric variable returns the
   // right value, getUndefVarNames does not return any variable.
-  auto DefNumExpr = FileCheckNumExpr(nullptr, FmtUnsigned);
+  auto DefNumExpr = FileCheckNumExpr(nullptr, FmtUnsigned, false);
   auto LineVar =
       std::make_shared<FileCheckNumericVariable>(1, "@LINE", &DefNumExpr);
   auto NVar = std::make_shared<FileCheckNumericVariable>(1, "N", &DefNumExpr);
   FileCheckNumExprVal LifeAnswer = FileCheckNumExprVal((uint64_t)42);
-  LineVar->setValue(LifeAnswer);
+  LineVar->setValue(LifeAnswer, false);
   FileCheckNumExprVal Ten = FileCheckNumExprVal((uint64_t)10);
-  NVar->setValue(Ten);
-  FileCheckNumExpr NumExprLine = FileCheckNumExpr(LineVar, FmtUnsigned);
-  FileCheckNumExpr NumExprN = FileCheckNumExpr(NVar, FmtUnsigned);
+  NVar->setValue(Ten, false);
+  FileCheckNumExpr NumExprLine = FileCheckNumExpr(LineVar, FmtUnsigned, true);
+  FileCheckNumExpr NumExprN = FileCheckNumExpr(NVar, FmtUnsigned, true);
   FileCheckNumericSubstitution SubstitutionLine =
       FileCheckNumericSubstitution(&Context, "@LINE", &NumExprLine, 12);
   FileCheckNumericSubstitution SubstitutionN =
       FileCheckNumericSubstitution(&Context, "N", &NumExprN, 30);
   llvm::Optional<std::string> Value = SubstitutionLine.getResult();
   EXPECT_TRUE(Value);
   EXPECT_EQ("42", *Value);
   EXPECT_FALSE(UndefVarNames.empty());
TEST_F(FileCheckTest, UndefVars) {
 
   // getUndefVarName() on a numeric substitution with a defined variable
   // returns an empty string.
   FileCheckNumExprVal LifeAnswer = FileCheckNumExprVal((uint64_t)42);
   auto LineVar =
       std::make_shared<FileCheckNumericVariable>("@LINE", LifeAnswer);
   auto Zero = std::make_shared<FileCheckNumExprLiteral>((uint64_t)0);
   auto Binop = std::make_shared<FileCheckASTBinop>(doAdd, LineVar, Zero);
-  FileCheckNumExpr NumExpr = FileCheckNumExpr(Binop, FmtUnsigned);
+  FileCheckNumExpr NumExpr = FileCheckNumExpr(Binop, FmtUnsigned, true);
   FileCheckNumericSubstitution NumericSubstitution =
       FileCheckNumericSubstitution(&Context, "@LINE", &NumExpr, 12);
   UndefVarNames.clear();
   NumericSubstitution.getUndefVarNames(UndefVarNames);
   EXPECT_TRUE(UndefVarNames.empty());
 
   // getUndefVarName() on a numeric substitution with an undefined variable
   // returns that variable.