[libc++] Set __file_ to 0 in basic_filebuf::close() even if fclose fails

[phosek]

Summary

This issue was detected by ASan in one of our tests. This test manually
invokes basic_filebuf::cloe(). fclose(h.release() returned a non-zero
exit status, so file_ wasn't set to 0. Later when basic_filebuf
destructor ran, we would enter the if (__file_) block again leading to
heap-use-after-free error.

The POSIX specification for fclose says that independently of the return
value, fclose closes the underlying file descriptor and any further
access (including another call to fclose()) to the stream results in
undefined behavior. This is exactly what happened in our test case.

To avoid this issue, we have to always set __file_ to 0 independently of
the fclose return value.

[ldionne]

Is it possible to add a test case?

[phosek]

In D64979#1593299, @ldionne wrote:

Is it possible to add a test case?

Yes, but I need to figure out a way to make fclose fail reliably inside the test.

[phosek]

Test added, would it be possible to take a look?

[ldionne]

LGTM if you leave a comment explaining the test.

libcxx/test/std/input.output/file.streams/fstreams/filebuf.members/close.pass.cpp

11 ↗

(On Diff #210927)

typo: close();

libcxx/trunk/include/fstream

 {
     basic_filebuf<_CharT, _Traits>* __rt = 0;
     if (__file_)
     {
         __rt = this;
         unique_ptr<FILE, int(*)(FILE*)> __h(__file_, fclose);
         if (sync())
             __rt = 0;
-        if (fclose(__h.release()) == 0)
-            __file_ = 0;
-        else
+        if (fclose(__h.release()))
             __rt = 0;
+        __file_ = 0;
         setbuf(0, 0);
     }
     return __rt;
 }
 
 template <class _CharT, class _Traits>
 typename basic_filebuf<_CharT, _Traits>::int_type
 basic_filebuf<_CharT, _Traits>::underflow()

libcxx/trunk/test/std/input.output/file.streams/fstreams/filebuf.members/close.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+// <fstream>
+
+// basic_filebuf<charT,traits>* close();
+
+#include <fstream>
+#include <cassert>
+#if defined(__unix__)
+#include <fcntl.h>
+#include <unistd.h>
+#endif
+#include "test_macros.h"
+#include "platform_support.h"
+
+int main(int, char**)
+{
+    std::string temp = get_temp_file_name();
+    {
+        std::filebuf f;
+        assert(!f.is_open());
+        assert(f.open(temp.c_str(), std::ios_base::out) != 0);
+        assert(f.is_open());
+        assert(f.close() != nullptr);
+        assert(!f.is_open());
+        assert(f.close() == nullptr);
+        assert(!f.is_open());
+    }
+#if defined(__unix__)
+    {
+        std::filebuf f;
+        assert(!f.is_open());
+        // Use open directly to get the file descriptor.
+        int fd = open(temp.c_str(), O_RDWR);
+        assert(fd >= 0);
+        // Use the internal method to create filebuf from the file descriptor.
+        assert(f.__open(fd, std::ios_base::out) != 0);
+        assert(f.is_open());
+        // Close the file descriptor directly to force filebuf::close to fail.
+        assert(close(fd) == 0);
+        // Ensure that filebuf::close handles the failure.
+        assert(f.close() == nullptr);
+        assert(!f.is_open());
+        assert(f.close() == nullptr);
+    }
+#endif
+    std::remove(temp.c_str());
+
+    return 0;
+}