[libc++][static linking] std streams are not initialized prior to their use in static object constructors

[ldionne]

Summary

When an executable (e.g. for bare metal environment) is statically linked with libc++ the following code might not work as expected:

#include <iostream>
 
class Foo {
public:
  Foo(int n) {
    std::cout << "Hello World - " << n << std::endl;
  }
};
 
Foo f(5);
 
int main() {
  return 0;
}

The program hangs or crashes because 'std::cout' is not initialized before 'Foo f(5)'.

The standard says:

27.3 Standard iostream objects
Header <iostream> synopsis
(2) ... The objects are constructed, and the associations are established at some time prior to or during first
time an object of class basic_ios<charT,traits>::Init is constructed, and in any case before the body of main
begins execution [264]. The objects are not destroyed during program execution [265].

And footnote 265 says:

Constructors and destructors for static objects can access these objects to read input from stdin or write output
to stdout or stderr.

The similar issue was raised more than year ago. A patch was proposed but not committed. See discussion of it here:

initial patch
review, suggestion for #ifdef

The proposed patch caused initialization/deinitialization of the std streams as many times as the static Init objects were created.
The current patch fixes this issue. A number of uses is counted by means of __shared_count. It is used instead of a 'int' variable because it is thread-safe. The standard allows multi-threaded initialization of static objects from different compilation modules.

[rnk]

I think a better approach would be to use __attribute__((init_priority(101))) on Linux and #pragma init_seg(lib) on Windows to ensure that libc++'s iostream initializer runs earlier.

[eastig]

In D12689#243295, @rnk wrote:

I think a better approach would be to use __attribute__((init_priority(101))) on Linux and #pragma init_seg(lib) on Windows to ensure that libc++'s iostream initializer runs earlier.

Thank you for advice.

I found this discussion of init_priority: Clarification of attribute init_priority
https://gcc.gnu.org/ml/gcc-help/2011-05/msg00220.html

It looks like it is not reliable method (https://gcc.gnu.org/ml/gcc-help/2011-05/msg00221.html):

Note that although the documentation doesn't seem to mention it, the
init_priority attribute only works correctly when using the GNU linker
or gold.

Gcc libc++ does not use it.

[rsmith]

Can we instead fix this in Clang by ensuring that libc++ is put at the right position in the static link order so that its initializers run first? libc++'s avoidance of running iostreams init code from every translation unit is a significant startup performance feature, and we shouldn't abandon it unless we really have to.

[jroelofs]

A testcase would be good, regardless of which of the proposed fixes ends up being chosen.

[eastig]

In D12689#243592, @rsmith wrote:

Can we instead fix this in Clang by ensuring that libc++ is put at the right position in the static link order so that its initializers run first? libc++'s avoidance of running iostreams init code from every translation unit is a significant startup performance feature, and we shouldn't abandon it unless we really have to.

Let me give more details about the problem we have.

Clang generates a __cxx_global_var_init<X> function for each global variable that needs to be set-up (initialized) before firing the main routine of the user program. Those initializer functions are then grouped under a parent initializer function named after the corresponding translation unit: _GLOBAL__sub_I_<translation_unit>. Those _GLOBAL__sub_I_<translation_unit> entries then get stuffed into the .init_array tables of individual object files.

In ARM compiler toolchains armlink is used for linking, not GNU ld. libc++ is a part of the toolchain static system libraries. So armlink logic should be updated to change the order in which the .init_array entries from system libraries are added to the final image's .init_array table. This can be a problem because of an assumption that there are no dependencies among initializers from different translation units.

IMHO if the programming language has a means of resolving such problems it's better to use it instead of hacking a linker.

About impact on startup performance, I don't see why it will be significant. Initialization is done once. Other times it is simply a call to increase a counter. To be significant there should be millions of calls. Why does gnu libc++ use a similar way if it hurts performance?

In the patch the __APPLE__ macro is used to have the old behaviour. Maybe instead of it another macro, e.g. __STATIC_BUILD__, can be use when we want to build a static library.

[eastig]

In D12689#243674, @jroelofs wrote:

A testcase would be good, regardless of which of the proposed fixes ends up being chosen.

I will write it.

[eastig]

Added tests

[eastig]

Ping.

[rnk]

I think you need to address the feedback about avoiding dynamic
initialization on stock non-Mac systems.

Sent from phone

[eastig]

In D12689#249899, @rnk wrote:

I think you need to address the feedback about avoiding dynamic
initialization on stock non-Mac systems.

Sent from phone

In my comment above I proposed a special macro to distinguish between dynamic and static linking:

In the patch the __APPLE__ macro is used to have the old behaviour. Maybe instead of it another macro, e.g. __STATIC_BUILD__, can be use when we want to build a static library.

Is it good to have such a macro?

[eastig]

The issue is reported as:
https://llvm.org/bugs/show_bug.cgi?id=28954

[hfinkel]

In D12689#515120, @eastig wrote:

The issue is reported as:
https://llvm.org/bugs/show_bug.cgi?id=28954

We do need to fix this bug; I posted to the PR so we can discuss approach there.

[ldionne]

This is superseded by https://reviews.llvm.org/D31413. Closing to clear up the review queue.

[eastig]

This comment was removed by eastig.

[ldionne]

In D12689#2277225, @eastig wrote:

Abandoned in favour of https://reviews.llvm.org/D31413

Sorry, we had a race condition :-)

include/ios

     virtual ~failure() throw();
 };
 
+class __shared_count;
+
 class _LIBCPP_TYPE_VIS ios_base::Init
 {
+private:
+    static __shared_count *UseCount;
 public:
     Init();
     ~Init();
 };
 
+#ifndef __APPLE__
+// Apple linker has a guarantee for an initialization order:
+// if A links against B, B's initializer will be run before A's.
+// I.e. if you link to libc++.dylib, then cout et al. are guaranteed
+// to be constructed before your initializers run. This way, definition
+// of __start_std_streams might remain in src/iostream.cpp, and linker arranges
+// its initialization before any stream usage.
+// Without Apple linker, that definition must be here,
+// so its initialization will be explicitly executed before any stream usage.
+// For construction of cout, cin, cerr etc.
+static ios_base::Init __start_std_streams;
+#endif
+
 // fmtflags
 
 inline _LIBCPP_INLINE_VISIBILITY

src/iostream.cpp

 #include "__std_stream"
 #include "string"
 #include "new"
+#include "cassert"
+#include "memory"
 
 _LIBCPP_BEGIN_NAMESPACE_STD
 

 _ALIGNAS_TYPE (ostream)  _LIBCPP_FUNC_VIS char clog[sizeof(ostream)];
 _ALIGNAS_TYPE (wostream) _LIBCPP_FUNC_VIS char wclog[sizeof(wostream)];
 
-ios_base::Init __start_std_streams;
-
-ios_base::Init::Init()
+static void init_std_streams()
 {
 #ifndef _LIBCPP_HAS_NO_STDIN
     istream* cin_ptr  = ::new(cin)  istream(::new(__cin)  __stdinbuf <char>(stdin, &mb_cin));

 #endif
 }
 
-ios_base::Init::~Init()
+static void fini_std_streams()
 {
 #ifndef _LIBCPP_HAS_NO_STDOUT
     ostream* cout_ptr = reinterpret_cast<ostream*>(cout);

     wclog_ptr->flush();
 }
 
+/// __Init is a utility class that counts the number of ios_base::Init
+/// objects. It automatically deinitializes the std streams when the
+/// last ios_base::Init object is destroyed.
+class __Init: public __shared_count {
+private:
+  void __on_zero_shared() _NOEXCEPT {
+    fini_std_streams();
+  }
+public:
+  __Init(long count = 0): __shared_count(count) {
+    init_std_streams();
+  }
+};
+
+_ALIGNAS_TYPE (__Init) _LIBCPP_FUNC_VIS static char __init_storage[sizeof(__Init)];
+
+__shared_count *ios_base::Init::UseCount = nullptr;
+
+ios_base::Init::Init()
+{
+  //local static object is initialized only once
+  struct __S {
+    __S() {
+      Init::UseCount = ::new(__init_storage) __Init(-1);
+    }
+  };
+  static __S __s;
+  UseCount->__add_shared();
+}
+
+ios_base::Init::~Init()
+{
+  assert(UseCount);
+  UseCount->__release_shared();
+}
+
+#ifdef __APPLE__
+// Apple linker has a guarantee for an initialization order:
+// if A links against B, B's initializer will be run before A's.
+// I.e. if you link to libc++.dylib, then cout et al. are guaranteed
+// to be constructed before your initializers run. This way, initialization
+// of __start_std_streams might remain here in module initializer
+// Without Apple linker, the following definition must be in header <ios>,
+// so its initialization will be explicitly executed before any stream usage.
+ios_base::Init __start_std_streams;
+#endif
+
 _LIBCPP_END_NAMESPACE_STD

test/std/input.output/iostream.objects/narrow.stream.objects/cerr_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::cerr' is the same in a static object constructor
+// and in the main function.
+// It dumps std::cerr memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::cerr it must be the same.
+
+struct A {
+  char *cerr_mem_dump;
+
+  A(): cerr_mem_dump(new char[sizeof(std::cerr)]) {
+    std::memcpy(cerr_mem_dump, (char *)&std::cerr, sizeof(std::cerr));
+  }
+  ~A() {
+    delete [] cerr_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.cerr_mem_dump, (const char *)&std::cerr, sizeof(std::cerr)) == 0);
+}
+

test/std/input.output/iostream.objects/narrow.stream.objects/cin_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::cin' is the same in a static object constructor
+// and in the main function.
+// It dumps std::cin memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::cin it must be the same.
+
+struct A {
+  char *cin_mem_dump;
+
+  A(): cin_mem_dump(new char[sizeof(std::cin)]) {
+    std::memcpy(cin_mem_dump, (char *)&std::cin, sizeof(std::cin));
+  }
+  ~A() {
+    delete [] cin_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.cin_mem_dump, (char *)&std::cin, sizeof(std::cin)) == 0);
+}
+

test/std/input.output/iostream.objects/narrow.stream.objects/clog_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::clog' is the same in a static object constructor
+// and in the main function.
+// It dumps std::clog memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::clog it must be the same.
+
+struct A {
+  char *clog_mem_dump;
+
+  A(): clog_mem_dump(new char[sizeof(std::clog)]) {
+    std::memcpy(clog_mem_dump, (char *)&std::clog, sizeof(std::clog));
+  }
+  ~A() {
+    delete [] clog_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.clog_mem_dump, (char *)&std::clog, sizeof(std::clog)) == 0);
+}
+

test/std/input.output/iostream.objects/narrow.stream.objects/cout_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::cout' is the same in a static object constructor
+// and in the main function.
+// It dumps std::cout memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::cout it must be the same.
+
+struct A {
+  char *cout_mem_dump;
+
+  A(): cout_mem_dump(new char[sizeof(std::cout)]) {
+    std::memcpy(cout_mem_dump, (char *)&std::cout, sizeof(std::cout));
+  }
+  ~A() {
+    delete [] cout_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.cout_mem_dump, (char *)&std::cout, sizeof(std::cout)) == 0);
+}
+

test/std/input.output/iostream.objects/wide.stream.objects/wcerr_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::wcerr' is the same in a static object constructor
+// and in the main function.
+// It dumps std::wcerr memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::wcerr it must be the same.
+
+struct A {
+  char *wcerr_mem_dump;
+
+  A(): wcerr_mem_dump(new char[sizeof(std::wcerr)]) {
+    std::memcpy(wcerr_mem_dump, (char *)&std::wcerr, sizeof(std::wcerr));
+  }
+  ~A() {
+    delete [] wcerr_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.wcerr_mem_dump, (const char *)&std::wcerr, sizeof(std::wcerr)) == 0);
+}
+

test/std/input.output/iostream.objects/wide.stream.objects/wcin_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::wcin' is the same in a static object constructor
+// and in the main function.
+// It dumps std::wcin memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::wcin it must be the same.
+
+struct A {
+  char *wcin_mem_dump;
+
+  A(): wcin_mem_dump(new char[sizeof(std::wcin)]) {
+    std::memcpy(wcin_mem_dump, (char *)&std::wcin, sizeof(std::wcin));
+  }
+  ~A() {
+    delete [] wcin_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.wcin_mem_dump, (char *)&std::wcin, sizeof(std::wcin)) == 0);
+}
+

test/std/input.output/iostream.objects/wide.stream.objects/wclog_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::wclog' is the same in a static object constructor
+// and in the main function.
+// It dumps std::wclog memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::wclog it must be the same.
+
+struct A {
+  char *wclog_mem_dump;
+
+  A(): wclog_mem_dump(new char[sizeof(std::wclog)]) {
+    std::memcpy(wclog_mem_dump, (char *)&std::wclog, sizeof(std::wclog));
+  }
+  ~A() {
+    delete [] wclog_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.wclog_mem_dump, (char *)&std::wclog, sizeof(std::wclog)) == 0);
+}
+

test/std/input.output/iostream.objects/wide.stream.objects/wcout_init.pass.cpp

+//===----------------------------------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <iostream>
+#include <cassert>
+#include <cstring>
+
+// The test checks that 'std::wcout' is the same in a static object constructor
+// and in the main function.
+// It dumps std::wcout memory in the static object constructor and compares it
+// with the memory in the main function.
+// Assumption: if there are no uses of std::wcout it must be the same.
+
+struct A {
+  char *wcout_mem_dump;
+
+  A(): wcout_mem_dump(new char[sizeof(std::wcout)]) {
+    std::memcpy(wcout_mem_dump, (char *)&std::wcout, sizeof(std::wcout));
+  }
+  ~A() {
+    delete [] wcout_mem_dump;
+  }
+};
+
+static A a;
+
+int main()
+{
+  assert(memcmp(a.wcout_mem_dump, (char *)&std::wcout, sizeof(std::wcout)) == 0);
+}
+