[Lex/DependencyDirectivesScanner] Handle the case where the source line starts with a `tok::hashhash`

[jansvoboda11]

Could you explain why this is necessary and even correct? I'd expect Clang to give an error when seeing ## in this position, and I'd expect the scanner to do the same.

[akyrtzi]

In D133674#3784602, @jansvoboda11 wrote:

Could you explain why this is necessary and even correct? I'd expect Clang to give an error when seeing ## in this position, and I'd expect the scanner to do the same.

## is lexed as tok::hashhash; it is ignored by the preprocessor (it is not treated as the start of a preprocessor directive) and passed on to the parser to interpret, like any other token.

[akyrtzi]

Add comment to clarify why we skip if tok::hashhash is encountered.

[akyrtzi]

In D133674#3784710, @akyrtzi wrote:

In D133674#3784602, @jansvoboda11 wrote:

Could you explain why this is necessary and even correct? I'd expect Clang to give an error when seeing ## in this position, and I'd expect the scanner to do the same.

## is lexed as tok::hashhash; it is ignored by the preprocessor (it is not treated as the start of a preprocessor directive) and passed on to the parser to interpret, like any other token.

@jansvoboda11 I've added a comment in code to make it clear why it skips.

[jansvoboda11]

Thank you! LGTM.

clang/lib/Lex/DependencyDirectivesScanner.cpp

bool Scanner::lexPPLine(const char *&First, const char *const End) {
   // Handle preprocessing directives.
 
   TheLexer.setParsingPreprocessorDirective(true);
   auto ScEx2 = make_scope_exit(
       [&]() { TheLexer.setParsingPreprocessorDirective(false); });
 
   // Lex '#'.
   const dependency_directives_scan::Token &HashTok = lexToken(First, End);
+  if (HashTok.is(tok::hashhash)) {
+    // A \p tok::hashhash at this location is passed by the preprocessor to the
+    // parser to interpret, like any other token. So for dependency scanning
+    // skip it like a normal token not affecting the preprocessor.
+    skipLine(First, End);
+    assert(First <= End);
+    return false;
+  }
   assert(HashTok.is(tok::hash));
   (void)HashTok;
 
   Optional<StringRef> FoundId = tryLexIdentifierOrSkipLine(First, End);
   if (!FoundId)
     return false;
 
   StringRef Id = *FoundId;

clang/unittests/Lex/DependencyDirectivesScannerTest.cpp

 TEST(MinimizeSourceToDependencyDirectivesTest, EmptyHash) {
   SmallVector<char, 128> Out;
 
   ASSERT_FALSE(
       minimizeSourceToDependencyDirectives("#\n#define MACRO a\n", Out));
   EXPECT_STREQ("#define MACRO a\n", Out.data());
 }
 
+TEST(MinimizeSourceToDependencyDirectivesTest, HashHash) {
+  SmallVector<char, 128> Out;
+
+  StringRef Source = R"(
+    #define S
+    #if 0
+      ##pragma cool
+      ##include "t.h"
+    #endif
+    #define E
+    )";
+  ASSERT_FALSE(minimizeSourceToDependencyDirectives(Source, Out));
+  EXPECT_STREQ("#define S\n#if 0\n#endif\n#define E\n", Out.data());
+}
+
 TEST(MinimizeSourceToDependencyDirectivesTest, Define) {
   SmallVector<char, 128> Out;
   SmallVector<dependency_directives_scan::Token, 4> Tokens;
   SmallVector<Directive, 4> Directives;
 
   ASSERT_FALSE(minimizeSourceToDependencyDirectives("#define MACRO", Out,
                                                     Tokens, Directives));
   EXPECT_STREQ("#define MACRO\n", Out.data());