[flang] Fix handling of files without terminating newlines.

[sscalpone]

flang/lib/Parser/source.cpp

129–130

Could you merge these two sections since they both deal with files that don't end with a newline character?

[DavidTruby]

Added test and unified handling of empty files with non-newline terminated files.

[DavidTruby]

@sscalpone I've mostly added this for discussion; I don't think there's another way of doing this with the way we currently handle newlines but I also don't think doing this is the best way forward. We might want to merge it to fix the bug in the short term while we look at newline handling though.

[sscalpone]

Please add a test case to cover "if we have spare memory"

[DavidTruby]

Well, currently this will crash if there isn't enough free memory with an error in the MemoryBuffer code. What other sensible behaviour is there? We can't continue if there's no memory as we'll have an infinite loop in the parser so crashing out seems the only option?

[sscalpone]

I'm asking about the case where "(content().size() >= buf_->getBufferSize())" is false.

[DavidTruby]

Oh, right, I see. I'll add that to the test case!

[DavidTruby]

Added test case including carriage returns; this tests the case where we already
have enough memory to add a newline at the end since we've moved the carriage
returns there.

[sscalpone]

Thanks David!

flang/lib/Parser/source.cpp

bool SourceFile::ReadStandardInput(llvm::raw_ostream &error) {
   auto inbuf = std::move(buf_or.get());
   buf_ =
       llvm::WritableMemoryBuffer::getNewUninitMemBuffer(inbuf->getBufferSize());
   llvm::copy(inbuf->getBuffer(), buf_->getBufferStart());
   ReadFile();
   return true;
 }
 
 void SourceFile::ReadFile() {
-  if (buf_->getBuffer().size() == 0) {
+  buf_end_ = RemoveCarriageReturns(buf_->getBuffer());
+  if (content().size() == 0 || content().back() != '\n') {
+    // Don't bother to copy if we have spare memory
+    if (content().size() >= buf_->getBufferSize()) {
+      auto tmp_buf{llvm::WritableMemoryBuffer::getNewUninitMemBuffer(
+          content().size() + 1)};
+      llvm::copy(content(), tmp_buf->getBufferStart());
       Close();
-    buf_ = llvm::WritableMemoryBuffer::getNewUninitMemBuffer(1);
-    buf_->getBuffer()[0] = '\n';
+      buf_ = std::move(tmp_buf);
+    }
+    buf_end_++;
+    buf_->getBuffer()[buf_end_ - 1] = '\n';
   }
-  buf_end_ = RemoveCarriageReturns(buf_->getBuffer());
   IdentifyPayload();
   RecordLineStarts();
 }
 
 void SourceFile::Close() {
   path_.clear();
   buf_.reset();
 }

flang/test/Semantics/missing_newline.f90

+! RUN: echo -n "end program" > %t.f90
+! RUN: %f18 -fparse-only %t.f90
+! RUN: echo -ne "\rend program" > %t.f90
+! RUN: %f18 -fparse-only %t.f90