[clang][lex] Speculative fix for buffer overrun on raw string parse

[jansvoboda11]

Summary

This attempts to fix a (non-deterministic) buffer overrun when parsing raw string literals during modular build.

Similar fix to 4e5b5c36f47c9a406ea7f6b4f89fae477693973a.

[beccadax]

Looks good. Thanks for implementing this!

clang/lib/Lex/LiteralSupport.cpp

1651–1652

In your shoes, I would also have promoted this to a real check, but if you think that's overkill I'm fine with leaving it as it is.

[beccadax]

clang/lib/Lex/LiteralSupport.cpp

1643

Nit: "16" is a magic number; it might be better to use a constant or comment to document its significance (raw strings can only have 16-character delimiters).

(I believe this is C++11 [lex.string]p2, but I've never written one of the citation comments you see in clang, so I'm not sure if you should cite a later standard.)

[jansvoboda11]

Implement suggested improvements

clang/lib/Lex/LiteralSupport.cpp

1643

Good point on the magic number, I'm going to extract that.

I'll check how the citations are written and add one here.

1651–1652

I think that's sensible, thanks!

clang/lib/Lex/LiteralSupport.cpp

if (ThisTokBuf[0] == 'L' || ThisTokBuf[0] == 'u' || ThisTokBuf[0] == 'U') {
       ++ThisTokBuf;
       // Skip 8 of u8 marker for utf8 strings.
       if (ThisTokBuf[0] == '8')
         ++ThisTokBuf;
     }
 
     // Check for raw string
     if (ThisTokBuf[0] == 'R') {
+      if (ThisTokBuf[1] != '"') {
+        // The file may have come from PCH and then changed after loading the
+        // PCH; Fail gracefully.
+        return DiagnoseLexingError(StringToks[i].getLocation());
+      }
       ThisTokBuf += 2; // skip R"
 
+      // C++11 [lex.string]p2: A `d-char-sequence` shall consist of at most 16
+      // characters.
+      constexpr unsigned RawStrDelimiterLen = 16;
+
       const char *Prefix = ThisTokBuf;
-      while (ThisTokBuf[0] != '(')
+      while (ThisTokBuf - Prefix < RawStrDelimiterLen && ThisTokBuf[0] != '(')
         ++ThisTokBuf;
+      if (ThisTokBuf[0] != '(')
+        return DiagnoseLexingError(StringToks[i].getLocation());
       ++ThisTokBuf; // skip '('
 
       // Remove same number of characters from the end
       ThisTokEnd -= ThisTokBuf - Prefix;
-      assert(ThisTokEnd >= ThisTokBuf && "malformed raw string literal");
+      if (ThisTokEnd < ThisTokBuf)
+        return DiagnoseLexingError(StringToks[i].getLocation());
 
       // C++14 [lex.string]p4: A source-file new-line in a raw string literal
       // results in a new-line in the resulting execution string-literal.
       StringRef RemainingTokenSpan(ThisTokBuf, ThisTokEnd - ThisTokBuf);
       while (!RemainingTokenSpan.empty()) {
         // Split the string literal on \r\n boundaries.
         size_t CRLFPos = RemainingTokenSpan.find("\r\n");
         StringRef BeforeCRLF = RemainingTokenSpan.substr(0, CRLFPos);