[clang] Show verify prefix in error messages

[aaron.ballman]

I'm not opposed but I am wondering what the need is for this? (It'd be good to capture that information in the patch summary as well.) Some examples showing where this helps to correct the issue would be useful.

[tbaeder]

When passing a different prefix via -verify=foo, the error messages now say "error: 'foo-error' diagnostics seen but not expected", etc.

I'm often working in test files where two different prefixes are used and I'm always confused about which one of the two the error messages are talking about.

[aaron.ballman]

In D154688#4497967, @tbaeder wrote:

When passing a different prefix via -verify=foo, the error messages now say "error: 'foo-error' diagnostics seen but not expected", etc.

I'm often working in test files where two different prefixes are used and I'm always confused about which one of the two the error messages are talking about.

What I'm confused by is that we list the line numbers of the failures, so the prefix only seems like it's helpful in a case where two prefixes use the same message and the same severity on the same line. e.g., // foo-error {{whatever}} bar-error {{whatever}}. In the other cases, either the line number is different, or the severity is different, or the message is different which I thought was giving sufficient context.

[tbaeder]

In D154688#4498398, @aaron.ballman wrote:

In D154688#4497967, @tbaeder wrote:

When passing a different prefix via -verify=foo, the error messages now say "error: 'foo-error' diagnostics seen but not expected", etc.

I'm often working in test files where two different prefixes are used and I'm always confused about which one of the two the error messages are talking about.

What I'm confused by is that we list the line numbers of the failures, so the prefix only seems like it's helpful in a case where two prefixes use the same message and the same severity on the same line. e.g., // foo-error {{whatever}} bar-error {{whatever}}. In the other cases, either the line number is different, or the severity is different, or the message is different which I thought was giving sufficient context.

This is also reported as being on line 4:

// RUN: %clang_cc1 -fsyntax-only -fdiagnostics-print-source-range-info -verify=bar %s 2>&1


static_assert(true); // foo \
                     // bar-error {{failed}}

which is also the case if line 4 contained another foo-error {{failed}} which didn't trigger, leaving developers wondering which one it is.

[tbaeder]

Ping

[aaron.ballman]

In D154688#4499797, @tbaeder wrote:

In D154688#4498398, @aaron.ballman wrote:

In D154688#4497967, @tbaeder wrote:

When passing a different prefix via -verify=foo, the error messages now say "error: 'foo-error' diagnostics seen but not expected", etc.

I'm often working in test files where two different prefixes are used and I'm always confused about which one of the two the error messages are talking about.

What I'm confused by is that we list the line numbers of the failures, so the prefix only seems like it's helpful in a case where two prefixes use the same message and the same severity on the same line. e.g., // foo-error {{whatever}} bar-error {{whatever}}. In the other cases, either the line number is different, or the severity is different, or the message is different which I thought was giving sufficient context.

This is also reported as being on line 4:

// RUN: %clang_cc1 -fsyntax-only -fdiagnostics-print-source-range-info -verify=bar %s 2>&1


static_assert(true); // foo \
                     // bar-error {{failed}}

which is also the case if line 4 contained another foo-error {{failed}} which didn't trigger, leaving developers wondering which one it is.

Okay, we do that often enough in our tests (along with two diagnostics on the same line directly) that this seems pretty reasonable to me.

Can you add some test coverage where the prefix is not the default expected?

[MaskRay]

The change looks great: (from error: 'error' diagnostics seen but not expected: to error: 'expected-error' diagnostics seen but not expected).
When I read Clang tests, I was confused by what error meant. I eventually figured out -verify is special. The new diagnostic is clearer and more helpful.

[aaron.ballman]

Precommit CI found relevant failures that should be addressed.

[MaskRay]

clang/lib/Frontend/VerifyDiagnosticConsumer.cpp

881

The variables are immediately used. I think in this case our convention is omit the blank line (and in general I think we omit blank lines for many more cases, compared to some other projects (GNU, Linux kernel, etc)).

clang/test/Frontend/verify.c

159

This may need a comment explaining that this is not recognized as a directive.

[tbaeder]

clang/test/Frontend/verify.c

159

Not sure I follow, what isn't being recognized as a directive?

[tbaeder]

Ping

[tbaeder]

Ping

[aaron.ballman]

LGTM!

clang/lib/Frontend/VerifyDiagnosticConsumer.cpp

else {
       if (const FileEntry *File = SourceMgr->getFileEntryForID(
                                                 SourceMgr->getFileID(I->first)))
         OS << " File " << File->getName();
       OS << " Line " << SourceMgr->getPresumedLineNumber(I->first);
     }
     OS << ": " << I->second;
   }
 
+  std::string Prefix = *Diags.getDiagnosticOptions().VerifyPrefixes.begin();
+  std::string KindStr = Prefix + "-" + Kind;
   Diags.Report(diag::err_verify_inconsistent_diags).setForceEmit()
-    << Kind << /*Unexpected=*/true << OS.str();
+      << KindStr << /*Unexpected=*/true << OS.str();
   return std::distance(diag_begin, diag_end);
 }
 
 /// Takes a list of diagnostics that were expected to have been generated
 /// but were not and produces a diagnostic to the user from this.
 static unsigned PrintExpected(DiagnosticsEngine &Diags,
                               SourceManager &SourceMgr,
                               std::vector<Directive *> &DL, const char *Kind) {
else
       OS << " Line " << SourceMgr.getPresumedLineNumber(D->DiagnosticLoc);
     if (D->DirectiveLoc != D->DiagnosticLoc)
       OS << " (directive at "
          << SourceMgr.getFilename(D->DirectiveLoc) << ':'
          << SourceMgr.getPresumedLineNumber(D->DirectiveLoc) << ')';
     OS << ": " << D->Text;
   }
 
+  std::string Prefix = *Diags.getDiagnosticOptions().VerifyPrefixes.begin();
+  std::string KindStr = Prefix + "-" + Kind;
   Diags.Report(diag::err_verify_inconsistent_diags).setForceEmit()
-    << Kind << /*Unexpected=*/false << OS.str();
+      << KindStr << /*Unexpected=*/false << OS.str();
   return DL.size();
 }
 
 /// Determine whether two source locations come from the same file.
 static bool IsFromSameFile(SourceManager &SM, SourceLocation DirectiveLoc,
                            SourceLocation DiagnosticLoc) {
   while (DiagnosticLoc.isMacroID())
     DiagnosticLoc = SM.getImmediateMacroCallerLoc(DiagnosticLoc);

clang/test/ARCMT/verify.m

 // RUN: %clang_cc1 -arcmt-action=check -verify %s
 // RUN: not %clang_cc1 -arcmt-action=check -verify %t.invalid 2>&1 | FileCheck %s
 
 #if 0
 // expected-error {{should be ignored}}
 #endif
 
 #error should not be ignored
 // expected-error@-1 {{should not be ignored}}
 
 #error
 // expected-error@-1 {{}}
 
 //      CHECK: error: no expected directives found: consider use of 'expected-no-diagnostics'
-// CHECK-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK-NEXT:   (frontend): error reading '{{.*}}verify.m.tmp.invalid'
 // CHECK-NEXT: 2 errors generated.

clang/test/Frontend/verify-any-file.c

 // RUN: not %clang_cc1 -verify %s 2>&1 | FileCheck %s
 
 #include "verify-any-file.h"
 // expected-error@*:* {{unknown type name 'unexpected'}}
 
 // expected-error@*:* {{missing error}}
 
 // expected-error@*:123 {{invalid line : "*" required}}
 //
-//      CHECK: error: 'error' diagnostics expected but not seen:
+//      CHECK: error: 'expected-error' diagnostics expected but not seen:
 // CHECK-NEXT:   File * Line * (directive at {{.*}}verify-any-file.c:6): missing error
-// CHECK-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK-NEXT:   File {{.*}}verify-any-file.c Line 8: missing or invalid line number following '@' in expected '*'
 // CHECK-NEXT: 2 errors generated.

clang/test/Frontend/verify-fatal.c

 // RUN: not %clang_cc1 -Wfatal-errors -verify %s 2>&1 | FileCheck %s
 
 #error first fatal
 // expected-error@-1 {{first fatal}}
 
 #error second fatal
 // expected-error@-1 {{second fatal}}
 
 
-//      CHECK: error: 'error' diagnostics expected but not seen:
+//      CHECK: error: 'expected-error' diagnostics expected but not seen:
 // CHECK-NEXT:   Line 6 (directive at {{.*}}verify-fatal.c:7): second fatal
 // CHECK-NEXT: 1 error generated.

clang/test/Frontend/verify-ignore-unexpected.c

 // RUN: not %clang_cc1 -DTEST_SWITCH -verify-ignore-unexpected=remark,aoeu,note -verify %s 2>&1 \
 // RUN:     | FileCheck -check-prefix=CHECK-BAD-SWITCH %s
 #ifdef TEST_SWITCH
 // expected-no-diagnostics
 #endif
-// CHECK-BAD-SWITCH: error: 'error' diagnostics seen but not expected:
+// CHECK-BAD-SWITCH: error: 'expected-error' diagnostics seen but not expected:
 // CHECK-BAD-SWITCH-NEXT: (frontend): invalid value 'aoeu' in '-verify-ignore-unexpected='
 
 // RUN: %clang_cc1 -DTEST1 -verify %s
 // RUN: %clang_cc1 -DTEST1 -verify -verify-ignore-unexpected %s
 #ifdef TEST1
 #warning MyWarning1
     // expected-warning@-1 {{MyWarning1}}
 int x; // expected-note {{previous definition is here}}

 // RUN: not %clang_cc1 -DTEST2 -verify -verify-ignore-unexpected=error %s 2>&1 \
 // RUN:     | FileCheck -check-prefix=CHECK-ERR %s
 #ifdef TEST2
 #warning MyWarning2
 int x;
 float x;
 #endif
 // CHECK-UNEXP: no expected directives found
-// CHECK-UNEXP-NEXT: 'error' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: 'expected-error' diagnostics seen but not expected
 // CHECK-UNEXP-NEXT: Line {{[0-9]+}}: redefinition of 'x'
-// CHECK-UNEXP-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: 'expected-warning' diagnostics seen but not expected
 // CHECK-UNEXP-NEXT: Line {{[0-9]+}}: MyWarning2
-// CHECK-UNEXP-NEXT: 'note' diagnostics seen but not expected
+// CHECK-UNEXP-NEXT: 'expected-note' diagnostics seen but not expected
 // CHECK-UNEXP-NEXT: Line {{[0-9]+}}: previous definition is here
 // CHECK-UNEXP-NEXT: 4 errors generated.
 
 // CHECK-NOTE: no expected directives found
-// CHECK-NOTE-NEXT: 'error' diagnostics seen but not expected
+// CHECK-NOTE-NEXT: 'expected-error' diagnostics seen but not expected
 // CHECK-NOTE-NEXT: Line {{[0-9]+}}: redefinition of 'x'
-// CHECK-NOTE-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-NOTE-NEXT: 'expected-warning' diagnostics seen but not expected
 // CHECK-NOTE-NEXT: Line {{[0-9]+}}: MyWarning2
 // CHECK-NOTE-NEXT: 3 errors generated.
 
 // CHECK-WARN: no expected directives found
-// CHECK-WARN-NEXT: 'error' diagnostics seen but not expected
+// CHECK-WARN-NEXT: 'expected-error' diagnostics seen but not expected
 // CHECK-WARN-NEXT: Line {{[0-9]+}}: redefinition of 'x'
-// CHECK-WARN-NEXT: 'note' diagnostics seen but not expected
+// CHECK-WARN-NEXT: 'expected-note' diagnostics seen but not expected
 // CHECK-WARN-NEXT: Line {{[0-9]+}}: previous definition is here
 // CHECK-WARN-NEXT: 3 errors generated.
 
 // CHECK-ERR: no expected directives found
-// CHECK-ERR-NEXT: 'warning' diagnostics seen but not expected
+// CHECK-ERR-NEXT: 'expected-warning' diagnostics seen but not expected
 // CHECK-ERR-NEXT: Line {{[0-9]+}}: MyWarning2
-// CHECK-ERR-NEXT: 'note' diagnostics seen but not expected
+// CHECK-ERR-NEXT: 'expected-note' diagnostics seen but not expected
 // CHECK-ERR-NEXT: Line {{[0-9]+}}: previous definition is here
 // CHECK-ERR-NEXT: 3 errors generated.
 
 // RUN: not %clang_cc1 -DTEST3 -verify -verify-ignore-unexpected %s 2>&1 \
 // RUN:     | FileCheck -check-prefix=CHECK-EXP %s
 #ifdef TEST3
 // expected-error {{test3}}
 #endif
-// CHECK-EXP: 'error' diagnostics expected but not seen
+// CHECK-EXP: 'expected-error' diagnostics expected but not seen
 // CHECK-EXP-NEXT: Line {{[0-9]+}}: test3
 
 // RUN: not %clang_cc1 -DTEST4 -verify -verify-ignore-unexpected %s 2>&1 \
 // RUN:     | FileCheck -check-prefix=CHECK-NOEXP %s
 // RUN: not %clang_cc1 -DTEST4 -verify -verify-ignore-unexpected=warning,error,note %s 2>&1 \
 // RUN:     | FileCheck -check-prefix=CHECK-NOEXP %s
 #ifdef TEST4
 #warning MyWarning4
 int x;
 float x;
 #endif
 // CHECK-NOEXP: error: no expected directives found
 // CHECK-NOEXP-NEXT: 1 error generated

clang/test/Frontend/verify-unknown-arg.c

 // RUN: not %clang_cc1 -asdf -verify %s 2>&1 | FileCheck %s
 
 // expected-no-diagnostics
 
-//      CHECK: error: 'error' diagnostics seen but not expected:
+//      CHECK: error: 'expected-error' diagnostics seen but not expected:
 // CHECK-NEXT: (frontend): unknown argument: '-asdf'

clang/test/Frontend/verify.c

 #line -2                 // expected-error {{line_error}}
 #error AAA               // expected-error {{BBB}} <- this shall be part of diagnostic
 #warning CCC             // expected-warning {{DDD}} <- this shall be part of diagnostic
 
 #if 0
 // This is encapsulated in "#if 0" so that the expected-* checks below
 // are not inadvertently included in the diagnostic checking!
 
-//      CHECK2: error: 'error' diagnostics expected but not seen:
+//      CHECK2: error: 'expected-error' diagnostics expected but not seen:
 // CHECK2-NEXT:   Line 41: define_error
 // CHECK2-NEXT:   Line 43: line_error
-// CHECK2-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK2-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK2-NEXT:   Line 43: #line directive requires a positive integer argument
 // CHECK2-NEXT:   Line 44: AAA // expected-error {{[{][{]BBB[}][}]}} <- this shall be part of diagnostic
-// CHECK2-NEXT: error: 'warning' diagnostics expected but not seen:
+// CHECK2-NEXT: error: 'expected-warning' diagnostics expected but not seen:
 // CHECK2-NEXT:   Line 42: undef_error
-// CHECK2-NEXT: error: 'warning' diagnostics seen but not expected:
+// CHECK2-NEXT: error: 'expected-warning' diagnostics seen but not expected:
 // CHECK2-NEXT:   Line 42: extra tokens at end of #undef directive
 // CHECK2-NEXT:   Line 45: CCC // expected-warning {{[{][{]DDD[}][}]}} <- this shall be part of diagnostic
 // CHECK2-NEXT: 7 errors generated.
 #endif
 #endif
 
 #ifdef TEST3
 #ifndef TEST3         // expected-note {{line_67}}
                       // expected-note {{line_68_ignored}}
 # ifdef UNDEFINED     // expected-note {{line_69_ignored}}
 # endif               // expected-note {{line_70_ignored}}
 #elif defined(TEST3)  // expected-note {{line_71}}
 # if 1                // expected-note {{line_72}}
                       // expected-note {{line_73}}
 # else                // expected-note {{line_74}}
                       // expected-note {{line_75_ignored}}
 #  ifndef TEST3       // expected-note {{line_76_ignored}}
 #  endif              // expected-note {{line_77_ignored}}
 # endif               // expected-note {{line_78}}
 #endif
 
-//      CHECK3: error: 'note' diagnostics expected but not seen:
+//      CHECK3: error: 'expected-note' diagnostics expected but not seen:
 // CHECK3-NEXT:   Line 67: line_67
 // CHECK3-NEXT:   Line 71: line_71
 // CHECK3-NEXT:   Line 72: line_72
 // CHECK3-NEXT:   Line 73: line_73
 // CHECK3-NEXT:   Line 74: line_74
 // CHECK3-NEXT:   Line 78: line_78
 // CHECK3-NEXT: 6 errors generated.
 #endif
 
 #ifdef TEST4
 #include "missing_header_file.include" // expected-error {{include_error}}
 
-//      CHECK4: error: 'error' diagnostics expected but not seen:
+//      CHECK4: error: 'expected-error' diagnostics expected but not seen:
 // CHECK4-NEXT:   Line 92: include_error
-// CHECK4-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK4-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK4-NEXT:   Line 92: 'missing_header_file.include' file not found
 // CHECK4-NEXT: 2 errors generated.
 #endif
 
 #ifdef TEST5
 #include "verify-directive.h"
 // expected-error@50 {{source file test}}
 
-//      CHECK5: error: 'error' diagnostics expected but not seen:
+//      CHECK5: error: 'expected-error' diagnostics expected but not seen:
 // CHECK5-NEXT:   Line 1 (directive at {{.*}}verify-directive.h:2): include file test
 // CHECK5-NEXT:   Line 50 (directive at {{.*}}verify.c:103): source file test
 // CHECK5-NEXT: 2 errors generated.
 #endif
 
 #if 0
 // RUN: not %clang_cc1 -verify %t.invalid 2>&1 | FileCheck -check-prefix=CHECK6 %s
 
 //      CHECK6: error: no expected directives found: consider use of 'expected-no-diagnostics'
-// CHECK6-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK6-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK6-NEXT:   (frontend): error reading '{{.*}}verify.c.tmp.invalid'
 // CHECK6-NEXT: 2 errors generated.
 
 // RUN: printf '//expected-error@2{{1}}\n#error 2\n' | not %clang_cc1 -verify 2>&1 | FileCheck -check-prefix=CHECK7 %s
 
-//      CHECK7: error: 'error' diagnostics expected but not seen:
+//      CHECK7: error: 'expected-error' diagnostics expected but not seen:
 // CHECK7-NEXT:   Line 2 (directive at <stdin>:1): 1
-// CHECK7-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK7-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK7-NEXT:   Line 2: 2
 // CHECK7-NEXT: 2 errors generated.
 #endif
 
 #ifdef TEST8
 // RUN: not %clang_cc1 -DTEST8 -verify %s 2>&1 | FileCheck -check-prefix=CHECK8 %s
 
 // expected-warning@nonexistent-file:1 {{ }}
 // expected-error@-1 {{file 'nonexistent-file' could not be located}}
 
 // expected-warning@verify-directive.h: {{ }}
 // expected-error@-1 {{missing or invalid line number}}
 
 // expected-warning@verify-directive.h:0 {{ }}
 // expected-error@-1 {{missing or invalid line number}}
 
 // expected-warning@verify-directive.h:0*{{ }}
 // expected-error@-1 {{missing or invalid line number}}
 
 // expected-warning@verify-directive.h:*0{{ }}
 // syntactically ok -- means match in any line for 0 occurrences.
 
 // expected-warning@verify-directive.h:1 {{diagnostic}}
 
-//      CHECK8: error: 'warning' diagnostics expected but not seen:
+//      CHECK8: error: 'expected-warning' diagnostics expected but not seen:
 // CHECK8-NEXT:   File {{.*}}verify-directive.h Line 1 (directive at {{.*}}verify.c:146): diagnostic
 // CHECK8-NEXT: 1 error generated.
 #endif
+
+
+#ifdef TEST9
+// RUN: not %clang_cc1 -DTEST9 -verify=what %s 2>&1 | FileCheck -check-prefix=CHECK9 %s
+
+// what-error {{huh?}}
+// CHECK9: error: 'what-error' diagnostics expected but not seen:
+#endif

clang/test/Frontend/verify2.c

 #if 0
 // RUN: not %clang_cc1 -verify %s 2>&1 | FileCheck %s
 
 // Please note that all comments are inside "#if 0" blocks so that
 // VerifyDiagnosticConsumer sees no comments while processing this
 // test-case (and hence no expected-* directives).
 #endif
 
 #include "verify2.h"
 #error source
 
 #if 0
 // expected-error {{should be ignored}}
 
 //      CHECK: error: no expected directives found: consider use of 'expected-no-diagnostics'
-// CHECK-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK-NEXT:   Line 5: header
 // CHECK-NEXT:   Line 10: source
 // CHECK-NEXT: 3 errors generated.
 #endif
 
 #ifdef CHECK2
 // RUN: not %clang_cc1 -DCHECK2 -verify %s 2>&1 | FileCheck -check-prefix=CHECK2 %s
 
 // The following checks that -verify can match "any line" in an included file.
 // The location of the diagnostic need therefore only match in the file, not to
 // a specific line number.  This is useful where -verify is used as a testing
 // tool for 3rd-party libraries where headers may change and the specific line
 // number of a diagnostic in a header is not important.
 
 // expected-error@verify2.h:* {{header}}
 // expected-error@verify2.h:* {{unknown}}
 
-//      CHECK2: error: 'error' diagnostics expected but not seen:
+//      CHECK2: error: 'expected-error' diagnostics expected but not seen:
 // CHECK2-NEXT:   File {{.*}}verify2.h Line * (directive at {{.*}}verify2.c:32): unknown
-// CHECK2-NEXT: error: 'error' diagnostics seen but not expected:
+// CHECK2-NEXT: error: 'expected-error' diagnostics seen but not expected:
 // CHECK2-NEXT:   File {{.*}}verify2.c Line 10: source
 // CHECK2-NEXT: 2 errors generated.
 #endif

clang/test/Frontend/verify3.c

 // This test-case runs several sub-tests on -verify to ensure that correct
 // diagnostics are generated in relation to the mis-use and non-use of the
 // 'expected-no-diagnostics' directive.
 
 // RUN: not %clang_cc1 -DTEST1 -verify %s 2>&1 | FileCheck -check-prefix=CHECK1 %s
 #ifdef TEST1
 // expected-no-diagnostics
 // expected-note {{}}
 
-//      CHECK1: error: 'error' diagnostics seen but not expected:
+//      CHECK1: error: 'expected-error' diagnostics seen but not expected:
 // CHECK1-NEXT:   Line 8: expected directive cannot follow 'expected-no-diagnostics' directive
 // CHECK1-NEXT: 1 error generated.
 #endif
 
 // RUN: not %clang_cc1 -DTEST2 -verify %s 2>&1 | FileCheck -check-prefix=CHECK2 %s
 #ifdef TEST2
 #warning X
 // expected-warning@-1 {{X}}
 // expected-no-diagnostics
 
-//      CHECK2: error: 'error' diagnostics seen but not expected:
+//      CHECK2: error: 'expected-error' diagnostics seen but not expected:
 // CHECK2-NEXT:   Line 19: 'expected-no-diagnostics' directive cannot follow other expected directives
 // CHECK2-NEXT: 1 error generated.
 #endif
 
 // RUN: not %clang_cc1 -DTEST3 -verify %s 2>&1 | FileCheck -check-prefix=CHECK3 %s
 // RUN: not %clang_cc1 -verify %s 2>&1 | FileCheck -check-prefix=CHECK3 %s
 #ifdef TEST3
 // no directives

clang/test/Misc/diag-verify.cpp

void test() {
   x = y; // expected-error{{ use of undeclared identifier 'y' }}
   x = y; // expected-error{{use of undeclared identifier 'y' is disallowed}}
   x = y; // expected-error{{please don't use of undeclared identifier 'y'}}
   x = y; // expected-error{{use of undeclared identifier 'y'; please declare y before use}}
   x = y; // expected-error{{use of use of undeclared identifier 'y'}}
   x = y; // expected-error{{use of undeclared identifier 'y' identifier 'y'}}
 }
 
-//CHECK: error: 'error' diagnostics expected but not seen: 
+//CHECK: error: 'expected-error' diagnostics expected but not seen: 
 //CHECK:   Line 17: use of undeclared identifier 'y' is fine
 //CHECK:   Line 18: abuse of undeclared identifier 'y'
 //CHECK:   Line 19: good use of undeclared identifier 'y' in code
 //CHECK:   Line 20:  use of undeclared identifier 'y' 
 //CHECK:   Line 21: use of undeclared identifier 'y' is disallowed
 //CHECK:   Line 22: please don't use of undeclared identifier 'y'
 //CHECK:   Line 23: use of undeclared identifier 'y'; please declare y before use
 //CHECK:   Line 24: use of use of undeclared identifier 'y'
 //CHECK:   Line 25: use of undeclared identifier 'y' identifier 'y'
-//CHECK: error: 'error' diagnostics seen but not expected: 
+//CHECK: error: 'expected-error' diagnostics seen but not expected: 
 //CHECK:   Line 17: use of undeclared identifier 'y'
 //CHECK:   Line 18: use of undeclared identifier 'y'
 //CHECK:   Line 19: use of undeclared identifier 'y'
 //CHECK:   Line 20: use of undeclared identifier 'y'
 //CHECK:   Line 21: use of undeclared identifier 'y'
 //CHECK:   Line 22: use of undeclared identifier 'y'
 //CHECK:   Line 23: use of undeclared identifier 'y'
 //CHECK:   Line 24: use of undeclared identifier 'y'
 //CHECK:   Line 25: use of undeclared identifier 'y'
 //CHECK: 18 errors generated.