[MC] Fix regression tests on Windows when git “core.autocrlf” is set to true

[caoz]

Summary

Fix regression tests on Windows when git “core.autocrlf” is set to true.

See: https://bugs.llvm.org/show_bug.cgi?id=35095

[rnk]

Please add a regression test that tests your code change when the repo is checked out with normal line feeds. That way we can catch regressions for this on Linux.

lib/MC/MCParser/AsmLexer.cpp

213

This assumes the next character is '\n', which was not the case on Mac OS 9. Please check this assumption.

624

Indentation should use two spaces, not four, here and above.

[caoz]

Thanks Reid. The fix doesn't have regressions on Linux, as it supports both styles of line ending ("\r\n" and "\n").

[rnk]

In D39737#918425, @caoz wrote:

Thanks Reid. The fix doesn't have regressions on Linux, as it supports both styles of line ending ("\r\n" and "\n").

I'm sure it's possible to add regression tests that run on Linux with .gitattributes.

[caoz]

Thank the reviewers for your comments. I added a regression test for Linux. This solves the line ending issue for git. I would appreciate it If anyone could let me know a solution for svn.

[efriedma]

With svn, this is controlled by svn:eol-style and svn:mime-type.

That said, we don't have to worry about any of this if you just configure your git or svn client not to mess with line endings.

[caoz]

Thank Eli and Zachary for your help on source control system settings.

[zturner]

Ahh sorry, I may have misunderstood your previous question. I thought we were talking about setting core.autocrlf=false at the global LLVM .gitattributes file, and then not having anything special for this one particular test.

[caoz]

In D39737#921245, @zturner wrote:

Ahh sorry, I may have misunderstood your previous question. I thought we were talking about setting core.autocrlf=false at the global LLVM .gitattributes file, and then not having anything special for this one particular test.

Do you think we should add the .gitattributes file? Thanks.

[rnk]

lgtm

.gitattributes

+# binary files
+test/Object/Inputs/*.a-* binary
+test/tools/dsymutil/Inputs/* binary
+test/tools/llvm-ar/Inputs/*.lib binary
+test/tools/llvm-objdump/Inputs/*.a binary
+test/tools/llvm-rc/Inputs/* binary
+test/tools/llvm-strings/Inputs/numbers binary
+test/MC/AsmParser/incbin_abcd binary
+test/YAMLParser/spec-09-02.test binary
+
+# Windows line ending test
+test/MC/AsmParser/preserve-comments-crlf.s text eol=crlf

docs/GettingStartedVS.rst

* With the distributed files:
 
       1. ``cd <where-you-want-llvm-to-live>``
       2. ``gunzip --stdout llvm-VERSION.tar.gz | tar -xvf -``
          (*or use WinZip*)
       3. ``cd llvm``
 
    * With anonymous Subversion access:
 
+     *Note:* some regression tests require Linux style line ending (``\n``). To
+     pass all regression tests, please add two lines *enable-auto-props = yes*
+     and *\* = svn:mime-type=application/octet-stream* to 
+     ``C:\Users\<username>\AppData\Roaming\Subversion\config``.
+
       1. ``cd <where-you-want-llvm-to-live>``
       2. ``svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm``
       3. ``cd llvm``
 
 5. Use `CMake <http://www.cmake.org/>`_ to generate up-to-date project files:
 
    * Once CMake is installed then the simplest way is to just start the
      CMake GUI, select the directory where you have LLVM extracted to, and

lib/MC/MCParser/AsmLexer.cpp

AsmToken AsmLexer::LexLineComment() {
   // Mark This as an end of statement with a body of the
   // comment. While it would be nicer to leave this two tokens,
   // backwards compatability with TargetParsers makes keeping this in this form
   // better.
   const char *CommentTextStart = CurPtr;
   int CurChar = getNextChar();
   while (CurChar != '\n' && CurChar != '\r' && CurChar != EOF)
     CurChar = getNextChar();
+  if (CurChar == '\r' && CurPtr != CurBuf.end() && *CurPtr == '\n')
+    ++CurPtr;
 
   // If we have a CommentConsumer, notify it about the comment.
   if (CommentConsumer) {
     CommentConsumer->HandleComment(
         SMLoc::getFromPointer(CommentTextStart),
         StringRef(CommentTextStart, CurPtr - 1 - CommentTextStart));
   }
 
if (CurPtr != CurBuf.end() && *CurPtr == '\n')
       ++CurPtr;
     return AsmToken(AsmToken::EndOfStatement,
                     StringRef(TokStart, CurPtr - TokStart));
   }
   case '\n':
     IsAtStartOfLine = true;
     IsAtStartOfStatement = true;
     return AsmToken(AsmToken::EndOfStatement, StringRef(TokStart, 1));
   case ':': return AsmToken(AsmToken::Colon, StringRef(TokStart, 1));
   case '+': return AsmToken(AsmToken::Plus, StringRef(TokStart, 1));
   case '-': return AsmToken(AsmToken::Minus, StringRef(TokStart, 1));
   case '~': return AsmToken(AsmToken::Tilde, StringRef(TokStart, 1));
   case '(': return AsmToken(AsmToken::LParen, StringRef(TokStart, 1));
   case ')': return AsmToken(AsmToken::RParen, StringRef(TokStart, 1));
   case '[': return AsmToken(AsmToken::LBrac, StringRef(TokStart, 1));
   case ']': return AsmToken(AsmToken::RBrac, StringRef(TokStart, 1));
   case '{': return AsmToken(AsmToken::LCurly, StringRef(TokStart, 1));

test/MC/AsmParser/preserve-comments-crlf.s

+  #RUN: llvm-mc -preserve-comments -n -triple i386-linux-gnu < %s > %t
+  #RUN: diff %s %t
+  .text
+
+foo:  #Comment here
+  #comment here
+  nop
+  #if DIRECTIVE COMMENT
+  ## WHOLE LINE COMMENT
+  cmpl  $196, %eax  ## EOL COMMENT
+  #endif
+  .ident  "clang version 3.9.0"
+  .section  ".note.GNU-stack","",@progbits