[FileCheck] Report captured variables

[jdenny]

Summary

Report captured variables in input dumps and traces. For example:

$ cat check
CHECK: hello [[WHAT:[a-z]+]]
CHECK: goodbye [[WHAT]]

$ FileCheck -dump-input=always -vv check < input |& tail -8
<<<<<<
           1: hello world
check:1'0     ^~~~~~~~~~~
check:1'1           ^~~~~ captured var "WHAT"
           2: goodbye world
check:2'0     ^~~~~~~~~~~~~
check:2'1                   with "WHAT" equal to "world"
>>>>>>

$ FileCheck -dump-input=never -vv check < input
check2:1:8: remark: CHECK: expected string found in input
CHECK: hello [[WHAT:[a-z]+]]
       ^
<stdin>:1:1: note: found here
hello world
^~~~~~~~~~~
<stdin>:1:7: note: captured var "WHAT"
hello world
      ^~~~~
check2:2:8: remark: CHECK: expected string found in input
CHECK: goodbye [[WHAT]]
       ^
<stdin>:2:1: note: found here
goodbye world
^~~~~~~~~~~~~
<stdin>:2:1: note: with "WHAT" equal to "world"
goodbye world
^

[thopre]

llvm/lib/Support/FileCheckImpl.h

117

I think I'd prefer this as a new field in NumericVariable. ExpressionValue is really an abstraction for signed/unsigned variable. It's used for all intermediate result of evaluating the AST of an expression and for all of those the StrValue means nothing and is of no interest. I realize the size penalty is small, it bothers me more on a conceptual level.

Does that make sense?

llvm/test/FileCheck/dump-input-annotations.txt

683

Although it will follow the same code path, I think I'd prefer a binary operation here rather than just a literal.

[jdenny]

Applied reviewer suggestions. Tidied up a test some.

[jdenny]

llvm/lib/Support/FileCheckImpl.h

117

That does make sense. Applied. Thanks!

[thopre]

Could you add an example where getStringValue() returns None?

llvm/lib/Support/FileCheckImpl.h

297

from it was parsed -> from which it was parsed?

[jdenny]

Address reviewer comments.

[jdenny]

In D83651#2149514, @thopre wrote:

Could you add an example where getStringValue() returns None?

It's set to None for @LINE and numeric variables defined on the command line. I've extended the comments to mention these cases.

Both of these cases are tested elsewhere in the test suite. getStringValue is not called for them, so there's nothing new to test, as far as I can tell.

Does that make sense?

[thopre]

In D83651#2149514, @thopre wrote:

Could you add an example where getStringValue() returns None?

Sorry, I meant in the testcases. Maybe something with @LINE since you mention it.

[jdenny]

In D83651#2150334, @thopre wrote:

In D83651#2149514, @thopre wrote:

Could you add an example where getStringValue() returns None?

Sorry, I meant in the testcases. Maybe something with @LINE since you mention it.

Ah, you mean the unittests, right? I keep forgetting about those for FileCheck.

[jdenny]

Extend unit tests for changes to NumericVariable.

@thopre Is that what you meant? I didn't see a way to make this specific to @LINE.

[thopre]

In D83651#2150335, @jdenny wrote:

In D83651#2150334, @thopre wrote:

In D83651#2149514, @thopre wrote:

Could you add an example where getStringValue() returns None?

Sorry, I meant in the testcases. Maybe something with @LINE since you mention it.

Ah, you mean the unittests, right? I keep forgetting about those for FileCheck.

Actually no, I was thinking of a case where printVariableDefs is called with a variable with no string value but I can't think of a case where this would happen. Having unittest for printVariableDefs is a good idea though.

[jdenny]

llvm/unittests/Support/FileCheckTest.cpp

686–687

Forgot to check getStringValue here. Will do before landing.

[thopre]

In D83651#2150410, @jdenny wrote:

Extend unit tests for changes to NumericVariable.

@thopre Is that what you meant? I didn't see a way to make this specific to @LINE.

Nvm, I didn't think enough about what I said. Do you think you could also unit test printVariableDefs?

[jdenny]

In D83651#2150412, @thopre wrote:

In D83651#2150335, @jdenny wrote:

In D83651#2150334, @thopre wrote:

In D83651#2149514, @thopre wrote:

Could you add an example where getStringValue() returns None?

Sorry, I meant in the testcases. Maybe something with @LINE since you mention it.

Ah, you mean the unittests, right? I keep forgetting about those for FileCheck.

Actually no, I was thinking of a case where printVariableDefs is called with a variable with no string value but I can't think of a case where this would happen.

I haven't thought of a case either. @LINE and command-line-defined numeric variables are never defined/captured by patterns, as I understand it.

[jdenny]

Removed an unused parameter from printVariableDefs.

Made unit testing of getStringValue stricter.

Added unit test for printVariableDefs. This is just a basic API test and doesn't check every aspect, such as sorting. More thorough testing appears in the FileCheck utility tests. Should we replicate all that here too? What's the general rule?

[thopre]

In D83651#2150645, @jdenny wrote:

Removed an unused parameter from printVariableDefs.

Made unit testing of getStringValue stricter.

Added unit test for printVariableDefs. This is just a basic API test and doesn't check every aspect, such as sorting. More thorough testing appears in the FileCheck utility tests. Should we replicate all that here too? What's the general rule?

Unfortunately there's no rule right now and there's already overlap. My personal feeling is:

• Every new code should have unit test with a focus on code coverage
• User visible behavior of a code should be tested in llvm/test (it does not need to aim at full coverage since there is unittest)
• Try to avoid duplication as much as possible
• Aim for readability

In this specific example the test in llvm/test shows the definition diagnostic alongside other diagnostic. The unit test would cover all possible case (e.g. loops taken/not taken). Unfortunately this already leads to some duplication so perhaps keep the sorting for the llvm/test which is much easier to read in llvm/test since output with several variable definition spans several lines.

I think the general rule should be discussed separately and I'm happy to approve the patch as it is.

[jdenny]

In D83651#2151432, @thopre wrote:

I think the general rule should be discussed separately

Agreed. Thanks for presenting your view.

and I'm happy to approve the patch as it is.

Thanks for the review. This cannot land without D83650, so we have some time still to extend the testing if needed.

llvm/lib/Support/FileCheck.cpp

NumericVariable *DefinedNumericVariable =
         NumericVariableMatch.DefinedNumericVariable;
 
     StringRef MatchedValue = MatchInfo[CaptureParenGroup];
     ExpressionFormat Format = DefinedNumericVariable->getImplicitFormat();
     Expected<ExpressionValue> Value =
         Format.valueFromStringRepr(MatchedValue, SM);
     if (!Value)
       return Value.takeError();
-    DefinedNumericVariable->setValue(*Value);
+    DefinedNumericVariable->setValue(*Value, MatchedValue);
   }
 
   // Like CHECK-NEXT, CHECK-EMPTY's match range is considered to start after
   // the required preceding newline, which is consumed by the pattern in the
   // case of CHECK-EMPTY but not CHECK-NEXT.
   size_t MatchStartSkip = CheckTy == Check::CheckEmpty;
   MatchLen = FullMatch.size() - MatchStartSkip;
   return FullMatch.data() - Buffer.data() + MatchStartSkip;
for (const auto &Substitution : Substitutions) {
         Diags->emplace_back(SM, CheckTy, getLoc(), MatchTy,
                             SMRange(Range.Start, Range.Start), OS.str());
       else
         SM.PrintMessage(Range.Start, SourceMgr::DK_Note, OS.str());
     }
   }
 }
 
+void Pattern::printVariableDefs(const SourceMgr &SM,
+                                FileCheckDiag::MatchType MatchTy,
+                                std::vector<FileCheckDiag> *Diags) const {
+  if (VariableDefs.empty() && NumericVariableDefs.empty())
+    return;
+  // Build list of variable captures.
+  struct VarCapture {
+    StringRef Name;
+    SMRange Range;
+  };
+  SmallVector<VarCapture, 2> VarCaptures;
+  for (const auto &VariableDef : VariableDefs) {
+    VarCapture VC;
+    VC.Name = VariableDef.first;
+    StringRef Value = Context->GlobalVariableTable[VC.Name];
+    SMLoc Start = SMLoc::getFromPointer(Value.data());
+    SMLoc End = SMLoc::getFromPointer(Value.data() + Value.size());
+    VC.Range = SMRange(Start, End);
+    VarCaptures.push_back(VC);
+  }
+  for (const auto &VariableDef : NumericVariableDefs) {
+    VarCapture VC;
+    VC.Name = VariableDef.getKey();
+    StringRef StrValue = VariableDef.getValue()
+                             .DefinedNumericVariable->getStringValue()
+                             .getValue();
+    SMLoc Start = SMLoc::getFromPointer(StrValue.data());
+    SMLoc End = SMLoc::getFromPointer(StrValue.data() + StrValue.size());
+    VC.Range = SMRange(Start, End);
+    VarCaptures.push_back(VC);
+  }
+  // Sort variable captures by the order in which they matched the input.
+  // Ranges shouldn't be overlapping, so we can just compare the start.
+  std::sort(VarCaptures.begin(), VarCaptures.end(),
+            [](const VarCapture &A, const VarCapture &B) {
+              assert(A.Range.Start != B.Range.Start &&
+                     "unexpected overlapping variable captures");
+              return A.Range.Start.getPointer() < B.Range.Start.getPointer();
+            });
+  // Create notes for the sorted captures.
+  for (const VarCapture &VC : VarCaptures) {
+    SmallString<256> Msg;
+    raw_svector_ostream OS(Msg);
+    OS << "captured var \"" << VC.Name << "\"";
+    if (Diags)
+      Diags->emplace_back(SM, CheckTy, getLoc(), MatchTy, VC.Range, OS.str());
+    else
+      SM.PrintMessage(VC.Range.Start, SourceMgr::DK_Note, OS.str(), VC.Range);
+  }
+}
+
 static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
                                   const SourceMgr &SM, SMLoc Loc,
                                   Check::FileCheckType CheckTy,
                                   StringRef Buffer, size_t Pos, size_t Len,
                                   std::vector<FileCheckDiag> *Diags,
                                   bool AdjustPrevDiags = false) {
   SMLoc Start = SMLoc::getFromPointer(Buffer.data() + Pos);
   SMLoc End = SMLoc::getFromPointer(Buffer.data() + Pos + Len);
if (ExpectedMatch) {
     // diagnostics.
     PrintDiag = !Diags;
   }
   FileCheckDiag::MatchType MatchTy = ExpectedMatch
                                          ? FileCheckDiag::MatchFoundAndExpected
                                          : FileCheckDiag::MatchFoundButExcluded;
   SMRange MatchRange = ProcessMatchResult(MatchTy, SM, Loc, Pat.getCheckTy(),
                                           Buffer, MatchPos, MatchLen, Diags);
-  if (Diags)
+  if (Diags) {
     Pat.printSubstitutions(SM, Buffer, MatchRange, MatchTy, Diags);
+    Pat.printVariableDefs(SM, MatchTy, Diags);
+  }
   if (!PrintDiag)
     return;
 
   std::string Message = formatv("{0}: {1} string found in input",
                                 Pat.getCheckTy().getDescription(Prefix),
                                 (ExpectedMatch ? "expected" : "excluded"))
                             .str();
   if (Pat.getCount() > 1)
     Message += formatv(" ({0} out of {1})", MatchedCount, Pat.getCount()).str();
 
   SM.PrintMessage(
       Loc, ExpectedMatch ? SourceMgr::DK_Remark : SourceMgr::DK_Error, Message);
   SM.PrintMessage(MatchRange.Start, SourceMgr::DK_Note, "found here",
                   {MatchRange});
   Pat.printSubstitutions(SM, Buffer, MatchRange, MatchTy, nullptr);
+  Pat.printVariableDefs(SM, MatchTy, nullptr);
 }
 
 static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
                        const FileCheckString &CheckStr, int MatchedCount,
                        StringRef Buffer, size_t MatchPos, size_t MatchLen,
                        FileCheckRequest &Req,
                        std::vector<FileCheckDiag> *Diags) {
   PrintMatch(ExpectedMatch, SM, CheckStr.Prefix, CheckStr.Loc, CheckStr.Pat,

llvm/lib/Support/FileCheckImpl.h

public:
   void log(raw_ostream &OS) const override { OS << "overflow error"; }
 };
 
 /// Class representing a numeric value.
 class ExpressionValue {
 private:
   uint64_t Value;
   bool Negative;
 
 public:
   template <class T>
   explicit ExpressionValue(T Val) : Value(Val), Negative(Val < 0) {}
 
   bool operator==(const ExpressionValue &Other) const {
     return Value == Other.Value && isNegative() == Other.isNegative();
   }
 
private:
 
   /// Format to use for expressions using this variable without an explicit
   /// format.
   ExpressionFormat ImplicitFormat;
 
   /// Value of numeric variable, if defined, or None otherwise.
   Optional<ExpressionValue> Value;
 
+  /// The input buffer's string from which Value was parsed, or None.  See
+  /// comments on getStringValue for a discussion of the None case.
+  Optional<StringRef> StrValue;
+
   /// Line number where this variable is defined, or None if defined before
   /// input is parsed. Used to determine whether a variable is defined on the
   /// same line as a given use.
   Optional<size_t> DefLineNumber;
 
 public:
   /// Constructor for a variable \p Name with implicit format \p ImplicitFormat
   /// defined at line \p DefLineNumber or defined before input is parsed if
   /// \p DefLineNumber is None.
   explicit NumericVariable(StringRef Name, ExpressionFormat ImplicitFormat,
                            Optional<size_t> DefLineNumber = None)
       : Name(Name), ImplicitFormat(ImplicitFormat),
         DefLineNumber(DefLineNumber) {}
 
   /// \returns name of this numeric variable.
   StringRef getName() const { return Name; }
 
   /// \returns implicit format of this numeric variable.
   ExpressionFormat getImplicitFormat() const { return ImplicitFormat; }
 
   /// \returns this variable's value.
   Optional<ExpressionValue> getValue() const { return Value; }
 
-  /// Sets value of this numeric variable to \p NewValue.
-  void setValue(ExpressionValue NewValue) { Value = NewValue; }
+  /// \returns the input buffer's string from which this variable's value was
+  /// parsed, or None if the value is not yet defined or was not parsed from the
+  /// input buffer.  For example, the value of @LINE is not parsed from the
+  /// input buffer, and some numeric variables are parsed from the command
+  /// line instead.
+  Optional<StringRef> getStringValue() const { return StrValue; }
+
+  /// Sets value of this numeric variable to \p NewValue, and sets the input
+  /// buffer string from which it was parsed to \p NewStrValue.  See comments on
+  /// getStringValue for a discussion of when the latter can be None.
+  void setValue(ExpressionValue NewValue,
+                Optional<StringRef> NewStrValue = None) {
+    Value = NewValue;
+    StrValue = NewStrValue;
+  }
 
   /// Clears value of this numeric variable, regardless of whether it is
   /// currently defined or not.
-  void clearValue() { Value = None; }
+  void clearValue() {
+    Value = None;
+    StrValue = None;
+  }
 
   /// \returns the line number where this variable is defined, if any, or None
   /// if defined before input is parsed.
   Optional<size_t> getDefLineNumber() const { return DefLineNumber; }
 };
 
 /// Class representing the use of a numeric variable in the AST of an
 /// expression.
void printSubstitutions(const SourceMgr &SM, StringRef Buffer,
                           SMRange MatchRange, FileCheckDiag::MatchType MatchTy,
                           std::vector<FileCheckDiag> *Diags) const;
   void printFuzzyMatch(const SourceMgr &SM, StringRef Buffer,
                        std::vector<FileCheckDiag> *Diags) const;
 
   bool hasVariable() const {
     return !(Substitutions.empty() && VariableDefs.empty());
   }
+  void printVariableDefs(const SourceMgr &SM, FileCheckDiag::MatchType MatchTy,
+                         std::vector<FileCheckDiag> *Diags) const;
 
   Check::FileCheckType getCheckTy() const { return CheckTy; }
 
   int getCount() const { return CheckTy.getCount(); }
 
 private:
   bool AddRegExToRegEx(StringRef RS, unsigned &CurParen, SourceMgr &SM);
   void AddBackrefToRegEx(unsigned BackrefNum);

llvm/test/FileCheck/dump-input-annotations.txt

 ; SUBST-NEG-NEXT:not:2'0     !~~~~~~~~~~~~~~~~~~~~ error: no match expected
 ; SUBST-NEG-NEXT:not:2'1                           with "DEF_MATCH1" equal to "def-match1"
 ; SUBST-NEG-NEXT:not:2'2                           with "DEF_MATCH2" equal to "def-match2"
 ; SUBST-NEG-NEXT:         3: END
 ; SUBST-NEG-NEXT:check:3     ^~~
 ; SUBST-NEG-NEXT:>>>>>>
 
 ;--------------------------------------------------
-; Substitutions: CHECK-NEXT, CHECK-SAME, CHECK-DAG fixups.
+; Captured variables
+;--------------------------------------------------
+
+; RUN: echo 'strvar: foo'           >  %t.in
+; RUN: echo 'numvar no expr: 51'    >> %t.in
+; RUN: echo 'numvar expr: -49'      >> %t.in
+; RUN: echo 'many: foo 100 8 bar'   >> %t.in
+; RUN: echo 'var in neg match: foo' >> %t.in
+; RUN: echo 'END'                   >> %t.in
+
+; RUN: echo 'CHECK: strvar: [[STRVAR:[a-z]+]]'                         >  %t.chk
+; RUN: echo 'CHECK: numvar no expr: [[#NUMVAR_NO_EXPR:]]'              >> %t.chk
+; RUN: echo 'CHECK: numvar expr: [[#%d,NUMVAR_EXPR:2-NUMVAR_NO_EXPR]]' >> %t.chk
+
+; Capture many variables of different kinds in a different order than their
+; names sort alphabetically to ensure they're sorted in capture order.
+; RUN: echo 'CHECK: many: [[VAR1:foo]] [[#%d,VAR3:]] [[#VAR2:]] [[VAR4:bar]]' \
+; RUN:      >> %t.chk
+
+; RUN: echo 'CHECK-NOT: var in neg match: [[VAR:foo]]' >> %t.chk
+; RUN: echo 'CHECK: END'                               >> %t.chk
+
+; RUN: %ProtectFileCheckOutput \
+; RUN: not FileCheck -dump-input=always -vv -input-file=%t.in %t.chk 2>&1 \
+; RUN: | FileCheck -strict-whitespace -match-full-lines %s -check-prefix=CAPTURE-NEG
+
+;      CAPTURE-NEG:<<<<<<
+; CAPTURE-NEG-NEXT:           1: strvar: foo
+; CAPTURE-NEG-NEXT:check:1'0     ^~~~~~~~~~~
+; CAPTURE-NEG-NEXT:check:1'1             ^~~ captured var "STRVAR"
+; CAPTURE-NEG-NEXT:           2: numvar no expr: 51
+; CAPTURE-NEG-NEXT:check:2'0     ^~~~~~~~~~~~~~~~~~
+; CAPTURE-NEG-NEXT:check:2'1                     ^~ captured var "NUMVAR_NO_EXPR"
+; CAPTURE-NEG-NEXT:           3: numvar expr: -49
+; CAPTURE-NEG-NEXT:check:3'0     ^~~~~~~~~~~~~~~~
+; CAPTURE-NEG-NEXT:check:3'1                      with "%d,NUMVAR_EXPR:2-NUMVAR_NO_EXPR" equal to "-49"
+; CAPTURE-NEG-NEXT:check:3'2                  ^~~ captured var "NUMVAR_EXPR"
+; CAPTURE-NEG-NEXT:           4: many: foo 100 8 bar
+; CAPTURE-NEG-NEXT:check:4'0     ^~~~~~~~~~~~~~~~~~~
+; CAPTURE-NEG-NEXT:check:4'1           ^~~           captured var "VAR1"
+; CAPTURE-NEG-NEXT:check:4'2               ^~~       captured var "VAR3"
+; CAPTURE-NEG-NEXT:check:4'3                   ^     captured var "VAR2"
+; CAPTURE-NEG-NEXT:check:4'4                     ^~~ captured var "VAR4"
+; CAPTURE-NEG-NEXT:           5: var in neg match: foo
+; CAPTURE-NEG-NEXT:not:5'0       !~~~~~~~~~~~~~~~~~~~~ error: no match expected
+; CAPTURE-NEG-NEXT:not:5'1                         !~~ captured var "VAR"
+; CAPTURE-NEG-NEXT:           6: END
+; CAPTURE-NEG-NEXT:check:6       ^~~
+; CAPTURE-NEG-NEXT:>>>>>>
+
+;--------------------------------------------------
+; CHECK-NEXT, CHECK-SAME, CHECK-DAG note fixups.
 ;
 ; When CHECK-NEXT or CHECK-SAME fails for the wrong line, or when a CHECK-DAG
 ; match is discarded, the associated diagnostic type must be converted from
-; successful to failed or discarded.  However, any substitution diagnostics must
-; be traversed to find that diagnostic.
+; successful to failed or discarded.  However, any note annotation must be
+; traversed to find that diagnostic.  We check this behavior here only for
+; substitutions, but it's the same mechanism for all note annotations.
 ;--------------------------------------------------
 
 ;- - - - - - - - - - - - - - - - - - - - - - - - -
 ; CHECK-NEXT.
 ;- - - - - - - - - - - - - - - - - - - - - - - - -
 
 ; RUN: echo 'pre var' > %t.in
 

llvm/test/FileCheck/verbose.txt

 
 VV-NEXT: verbose.txt:[[@LINE-17]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT:  {{C}}HECK-NOT: raboof{{$}}
 VV-NEXT: {{^}}           ^{{$}}
 VV-NEXT: verbose.txt:[[@LINE-22]]:1: note: scanning from here
 VV-NEXT: {{^}}bar{{$}}
 VV-NEXT: {{^}}^{{$}}
 
+STRVAR=foobar
+STRVAR:foobar
+CHECK: STRVAR=[[STRVAR:[a-z]+]]
+CHECK-NEXT: STRVAR:[[STRVAR]]
+
+     V: verbose.txt:[[#@LINE-3]]:8: remark: {{C}}HECK: expected string found in input
+V-NEXT: {{C}}HECK: {{STRVAR=\[\[STRVAR:\[a-z\]\+\]\]}}
+V-NEXT:  {{^}}       ^{{$}}
+V-NEXT: verbose.txt:[[#@LINE-8]]:1: note: found here
+V-NEXT: {{^}}STRVAR=foobar{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#@LINE-11]]:8: note: captured var "STRVAR"
+V-NEXT: {{^}}STRVAR=foobar{{$}}
+V-NEXT: {{^}}       ^~~~~~{{$}}
+
+V-NEXT: verbose.txt:[[#@LINE-12]]:13: remark: {{C}}HECK-NEXT: expected string found in input
+V-NEXT: {{C}}HECK-NEXT: {{STRVAR:\[\[STRVAR\]\]}}
+V-NEXT:  {{^}}            ^{{$}}
+V-NEXT: verbose.txt:[[#@LINE-17]]:1: note: found here
+V-NEXT: {{^}}STRVAR:foobar{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#@LINE-20]]:1: note: with "STRVAR" equal to "foobar"
+V-NEXT: {{^}}STRVAR:foobar{{$}}
+V-NEXT: {{^}}^{{$}}
+
 NUMVAR=42
 NUMVAR - 1:41
 CHECK: NUMVAR=[[#NUMVAR:]]
 CHECK-NOT: [[#NUMVAR + 1]]
 CHECK-NEXT: NUMVAR - 1:[[#NUMVAR - 1]]
 
 V:      verbose.txt:[[#@LINE-4]]:8: remark: {{C}}HECK: expected string found in input
 V-NEXT: {{C}}HECK: {{NUMVAR=[[][[]#NUMVAR:[]][]]$}}
 V-NEXT:  {{^}}       ^{{$}}
 V-NEXT: verbose.txt:[[#@LINE-9]]:1: note: found here
 V-NEXT: {{^}}NUMVAR=42{{$}}
 V-NEXT: {{^}}^~~~~~~~~{{$}}
+V-NEXT: verbose.txt:[[#@LINE-12]]:8: note: captured var "NUMVAR"
+V-NEXT: NUMVAR=42
+V-NEXT:        ^~
 
-V-NEXT: verbose.txt:[[#@LINE-9]]:13: remark: {{C}}HECK-NEXT: expected string found in input
+V-NEXT: verbose.txt:[[#@LINE-12]]:13: remark: {{C}}HECK-NEXT: expected string found in input
 V-NEXT: {{C}}HECK-NEXT: {{NUMVAR - 1:[[][[]#NUMVAR - 1[]][]]$}}
 V-NEXT:  {{^}}            ^{{$}}
-V-NEXT: verbose.txt:[[#@LINE-15]]:1: note: found here
+V-NEXT: verbose.txt:[[#@LINE-18]]:1: note: found here
 V-NEXT: {{^}}NUMVAR - 1:41{{$}}
 V-NEXT: {{^}}^~~~~~~~~~~~~{{$}}
-V-NEXT: verbose.txt:[[#@LINE-18]]:1: note: with "NUMVAR - 1" equal to "41"
+V-NEXT: verbose.txt:[[#@LINE-21]]:1: note: with "NUMVAR - 1" equal to "41"
 V-NEXT: {{^}}NUMVAR - 1:41{{$}}
 V-NEXT: {{^}}^{{$}}
 
-VV-NEXT: verbose.txt:[[#@LINE-20]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
+VV-NEXT: verbose.txt:[[#@LINE-23]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
 VV-NEXT: {{C}}HECK-NOT: {{[[][[]#NUMVAR [+] 1[]][]]$}}
 VV-NEXT:   {{^}}           ^{{$}}
-VV-NEXT: verbose.txt:[[#@LINE-25]]:1: note: scanning from here
+VV-NEXT: verbose.txt:[[#@LINE-28]]:1: note: scanning from here
 VV-NEXT: {{^}}NUMVAR - 1:41{{$}}
 VV-NEXT: {{^}}^{{$}}
 
 before empty
 
 after empty
 CHECK: before empty
 CHECK-EMPTY:

llvm/unittests/Support/FileCheckTest.cpp

TEST_F(FileCheckTest, NumericVariable) {
   NumericVariableUse FooVarUse("FOO", &FooVar);
   Expected<ExpressionFormat> ImplicitFormat = FooVarUse.getImplicitFormat(SM);
   ASSERT_THAT_EXPECTED(ImplicitFormat, Succeeded());
   EXPECT_EQ(*ImplicitFormat, ExpressionFormat::Kind::Unsigned);
   EXPECT_FALSE(FooVar.getValue());
   Expected<ExpressionValue> EvalResult = FooVarUse.eval();
   expectUndefErrors({"FOO"}, EvalResult.takeError());
 
+  // Defined variable without string: only getValue and eval return value set.
   FooVar.setValue(ExpressionValue(42u));
-
-  // Defined variable: getValue and eval return value set.
   Optional<ExpressionValue> Value = FooVar.getValue();
   ASSERT_TRUE(Value);
   EXPECT_EQ(42, cantFail(Value->getSignedValue()));
+  EXPECT_FALSE(FooVar.getStringValue());
   EvalResult = FooVarUse.eval();
   ASSERT_THAT_EXPECTED(EvalResult, Succeeded());
   EXPECT_EQ(42, cantFail(EvalResult->getSignedValue()));
 
+  // Defined variable with string: getValue, eval, and getStringValue return
+  // value set.
+  StringRef StringValue = "925";
+  FooVar.setValue(ExpressionValue(925u), StringValue);
+  Value = FooVar.getValue();
+  ASSERT_TRUE(Value);
+  EXPECT_EQ(925, cantFail(Value->getSignedValue()));
+  // getStringValue should return the same memory not just the same characters.
+  EXPECT_EQ(StringValue.begin(), FooVar.getStringValue().getValue().begin());
+  EXPECT_EQ(StringValue.end(), FooVar.getStringValue().getValue().end());
+  EvalResult = FooVarUse.eval();
+  ASSERT_THAT_EXPECTED(EvalResult, Succeeded());
+  EXPECT_EQ(925, cantFail(EvalResult->getSignedValue()));
+  EXPECT_EQ(925, cantFail(EvalResult->getSignedValue()));
+
   // Clearing variable: getValue and eval fail. Error returned by eval holds
   // the name of the cleared variable.
   FooVar.clearValue();
   EXPECT_FALSE(FooVar.getValue());
+  EXPECT_FALSE(FooVar.getStringValue());
   EvalResult = FooVarUse.eval();
   expectUndefErrors({"FOO"}, EvalResult.takeError());
 }
 
 TEST_F(FileCheckTest, Binop) {
   SourceMgr SM;
 
   StringRef ExprStr = bufferize(SM, "FOO+BAR");
bool parsePattern(StringRef Pattern) {
     return P.parsePattern(PatBufferRef, "CHECK", SM, Req);
   }
 
   Expected<size_t> match(StringRef Buffer) {
     StringRef BufferRef = bufferize(SM, Buffer);
     size_t MatchLen;
     return P.match(BufferRef, MatchLen, SM);
   }
+
+  void printVariableDefs(FileCheckDiag::MatchType MatchTy,
+                         std::vector<FileCheckDiag> &Diags) {
+    P.printVariableDefs(SM, MatchTy, &Diags);
+  }
 };
 
 TEST_F(FileCheckTest, ParseNumericSubstitutionBlock) {
   PatternTester Tester;
 
   // Variable definition.
 
   expectDiagnosticError("invalid variable name",
#endif
   ExpressionPointer = P.parseNumericSubstitutionBlock(
       GlobalNumVarRef, DefinedNumericVariable, /*IsLegacyLineExpr=*/false,
       LineNumber, &Cxt, SM);
   ASSERT_THAT_EXPECTED(ExpressionPointer, Succeeded());
   ExpressionVal = (*ExpressionPointer)->getAST()->eval();
   ASSERT_THAT_EXPECTED(ExpressionVal, Succeeded());
   EXPECT_EQ(cantFail(ExpressionVal->getSignedValue()), 36);
 }
+
+TEST_F(FileCheckTest, CapturedVarDiags) {
+  PatternTester Tester;
+  ASSERT_FALSE(Tester.parsePattern("[[STRVAR:[a-z]+]] [[#NUMVAR:@LINE]]"));
+  EXPECT_THAT_EXPECTED(Tester.match("foobar 2"), Succeeded());
+  std::vector<FileCheckDiag> Diags;
+  Tester.printVariableDefs(FileCheckDiag::MatchFoundAndExpected, Diags);
+  EXPECT_EQ(Diags.size(), 2ul);
+  for (FileCheckDiag Diag : Diags) {
+    EXPECT_EQ(Diag.CheckTy, Check::CheckPlain);
+    EXPECT_EQ(Diag.MatchTy, FileCheckDiag::MatchFoundAndExpected);
+    EXPECT_EQ(Diag.InputStartLine, 1u);
+    EXPECT_EQ(Diag.InputEndLine, 1u);
+  }
+  EXPECT_EQ(Diags[0].InputStartCol, 1u);
+  EXPECT_EQ(Diags[0].InputEndCol, 7u);
+  EXPECT_EQ(Diags[1].InputStartCol, 8u);
+  EXPECT_EQ(Diags[1].InputEndCol, 9u);
+  EXPECT_EQ(Diags[0].Note, "captured var \"STRVAR\"");
+  EXPECT_EQ(Diags[1].Note, "captured var \"NUMVAR\"");
+}
 } // namespace