[clang] Fix a bug that allowed some overflowing octal escape sequences

[shafik]

clang/lib/Lex/LiteralSupport.cpp

266–267

Maybe the comment should be more specific such as "one of the top three bits are set" or something similar.

[barannikov88]

Update the comment

[cor3ntin]

I wonder if a better fix would be change ResultChar to be a 64 bits integer, although I think it might be over killed (ie, i'm not aware of a good use case for wanting to store values that large in a 32 bits wide char, so this looks fine.
Thanks for the fix :)

clang/lib/Lex/LiteralSupport.cpp

while (ThisTokBuf != ThisTokEnd) {
         HadError = true;
         if (Diags)
           Diag(Diags, Features, Loc, ThisTokBegin, EscapeBegin, ThisTokBuf,
                diag::err_delimited_escape_invalid)
               << StringRef(ThisTokBuf, 1);
         ThisTokBuf++;
         continue;
       }
-      if (ResultChar & 0x020000000)
+      // Check if one of the top three bits is set before shifting them out.
+      if (ResultChar & 0xE0000000)
         Overflow = true;
 
       ResultChar <<= 3;
       ResultChar |= *ThisTokBuf++ - '0';
     }
     // Check for overflow.  Reject '\777', but not L'\777'.
     if (!HadError &&
         (Overflow || (CharWidth != 32 && (ResultChar >> CharWidth) != 0))) {

clang/test/Lexer/char-escapes-delimited.c

 
 void octal(void) {
   char a = '\o{1}';              // ext-warning {{extension}} cxx2b-warning {{C++2b}}
   char b = '\o{12345678881238}'; // expected-error 4{{invalid digit '8' in escape sequence}}
   char c = '\o{777}';            // //expected-error {{octal escape sequence out of range}}
 #if __WCHAR_MAX__ > 0xFFFF
   unsigned d = L'\o{37777777777}'; // ext-warning {{extension}} cxx2b-warning {{C++2b}}
   unsigned e = L'\o{40000000000}'; // expected-error {{octal escape sequence out of range}}
+  unsigned f = L'\o{100000000000}'; // expected-error {{octal escape sequence out of range}}
+  unsigned g = L'\o{200000000000}'; // expected-error {{octal escape sequence out of range}}
 #else
   unsigned d = L'\o{177777}'; // ext-warning {{extension}} cxx2b-warning {{C++2b}}
   unsigned e = L'\o{200000}'; // expected-error {{octal escape sequence out of range}}
 #endif
 }
 
 void concat(void) {
   (void)"\x{" "12}"; // expected-error {{expected '}'}}