Skip Unicode character expansion in assembly files

[salari01]

Summary

When using the C preprocessor with assembly files, either with a capital S file extension, or with -xassembler-with-cpp, the Unicode escape sequence \u is ignored. The \u pattern can be used for expanding a macro argument that starts with u.

[olista01]

The change itself looks fine, but I have a few comments on the test.

Tests of clang shouldn't actually run the assembler, as I think this one will fail if clang is built without the ARM backend. A better way to write the test would be to just run the preprocessor, and check that the occurrences of \u make it through unmodified.

You should also be able to use clang's -verify option to check the diagnostics, so that you don't need a dummy warning to avoid empty FileCheck input. Grep for expected-no-diagnostics in the existing tests for examples of this.

[salari01]

Updated test to check preprocessed output instead of the assembled file. Cannot use -verify with the driver, but with -E and -o -, there is no longer a need to have the dummy warning to avoid the FileCheck error.

[olista01]

LGTM

lib/Lex/Lexer.cpp

case '@':
     if (CurPtr[-1] == '@' && LangOpts.ObjC1)
       Kind = tok::at;
     else
       Kind = tok::unknown;
     break;
 
   // UCNs (C99 6.4.3, C++11 [lex.charset]p2)
   case '\\':
+    if (!LangOpts.AsmPreprocessor) {
       if (uint32_t CodePoint = tryReadUCN(CurPtr, BufferPtr, &Result)) {
         if (CheckUnicodeWhitespace(Result, CodePoint, CurPtr)) {
           if (SkipWhitespace(Result, CurPtr, TokAtPhysicalStartOfLine))
             return true; // KeepWhitespaceMode
 
           // We only saw whitespace, so just try again with this lexer.
           // (We manually eliminate the tail call to avoid recursion.)
           goto LexNextToken;
         }
 
         return LexUnicode(Result, CodePoint, CurPtr);
       }
+    }
 
     Kind = tok::unknown;
     break;
 
   default: {
     if (isASCII(Char)) {
       Kind = tok::unknown;
       break;

test/Lexer/asm-preproc-no-unicode.s

+// RUN: %clang -E -xassembler-with-cpp %s -o - 2>&1 | FileCheck %s
+
+// CHECK-NOT: warning: \u used with no following hex digits
+// CHECK: .word \u
+
+    .macro foo, u
+        .word \u
+    .endm