[libcxx] Create internal namespace that hides all symbols.

[EricWF]

Summary

Currently functions and types that are pure implementation must be individually annotated with a DSO visibility attribute. This requires tedious repetition and is prone to errors.

As it stands there are a number of symbols within libc++ that do not have a visibility attribute. These symbols may or may be compiled into the DSO depending on the compilation flags. This leads to the DSO providing an unstable and unmanageable set of symbols.

This patch creates a nested namespace __libcpp_internal that sets the visibility to "hidden" for all symbols within it. Classes and functions that are pure implementation details should live within this namespace. This way the symbols don't have to be individually annotated.

Slowly but surely symbols should be moved to this namespace. Migrating code into this namespace will be tricky. We will have to ensure that no platform already ships a DSO that contains the candidate symbols.

[mclow.lists]

This would be more compelling if you had examples of such symbols, and moved a few of them into this namespace.

[EricWF]

Address @mclow.lists comments. Use the three symbols from D8651 as examples.

Also add tests using sym_check that ensure no symbols within the __libcpp_internal namespace are visible in the resulting dylib.

[EricWF]

GCC 4.0 seems to have attributes on namespaces and we should require GCC >= 4.7.
I can't seem to find any documentation about when Clang supports visibility attributes on namespaces but it I think it works for all clang versions.

[EricWF]

Confirmed working on GCC 4.6 and Clang 3.2.

[danalbert]

test/libcxx/symbols/inline_namespace.sh.cpp

2

within

test/libcxx/test/config.py

640

Why can't we check the static lib? The symbols will still be hidden either way.

643

This is always true.

670

Why sys.path (PYTHONPATH)?

[EricWF]

Address @danalbert's comments.

test/libcxx/test/config.py

640

We can check a static library. This branch selects libc++.a for checking if it exists. Otherwise we try libc++.dylib or libc++.so.

643

Good catch. That should be an 'else'.

670

Probably a mistake. I thought sys.path and os.environ['PATH'] were essentially the same. I'll remove it.

[EricWF]

Make changes requested by @danalbert.

[EricWF]

There hasn't been much interest in this approach and it won't work with the new linkage attributes.

include/__config

 
 #endif // __clang__ || __GNUC__ || _MSC_VER || __IBMCPP__
 
+#define _LIBCPP_BEGIN_NAMESPACE_INTERNAL namespace __libcpp_internal _LIBCPP_HIDDEN {
+#define _LIBCPP_END_NAMESPACE_INTERNAL }
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+  _LIBCPP_BEGIN_NAMESPACE_INTERNAL
+  _LIBCPP_END_NAMESPACE_INTERNAL
+  using namespace __libcpp_internal;
+_LIBCPP_END_NAMESPACE_STD
+
 #ifdef _LIBCPP_HAS_NO_UNICODE_CHARS
 typedef unsigned short char16_t;
 typedef unsigned int   char32_t;

include/locale

 #endif
 }
 
+_LIBCPP_BEGIN_NAMESPACE_INTERNAL
+
 inline
 int __snprintf_l(char *__s, size_t __n, locale_t __l, const char *__format, ...) {
   va_list __va;

   return __res;
 }
 
+_LIBCPP_END_NAMESPACE_INTERNAL
+
 #endif  // __linux__
 
 // __scan_keyword

test/libcxx/symbols/inline_namespace.sh.cpp

+// Test that symbols withing the __libcpp_internal namespace are given hidden
+// visibility and do not appear in the dylib.
+
+// Check that these example symbols do not appear.
+// RUN: %env not sym_match.py %libcxx \
+// RUN:  [_]+ZNSt3110sscanf_lEPKcP15__locale_structS1_z \
+// RUN:  [_]+ZNSt3__117__libcpp_internal10__sscanf_lEPKcP15__locale_structS2_z \
+// RUN:  [_]+ZNSt3112asprintf_lEPPcP15__locale_structPKcz \
+// RUN:  [_]+ZNSt3__117__libcpp_internal12__asprintf_lEPPcP15__locale_structPKcz \
+// RUN:  [_]+ZNSt3112snprintf_lEPcmP15__locale_structPKcz \
+// RUN:  [_]+ZNSt3__117__libcpp_internal12__snprintf_lEPcmP15__locale_structPKcz
+
+// Check that no symbols in the __libcpp_internal namespace have been emitted
+// into the dylib.
+// RUN: %env not sym_match.py %libcxx .*__libcpp_internal
+

test/libcxx/test/config.py

         sub = self.config.substitutions
         # Configure compiler substitions
         sub.append(('%cxx', self.cxx.path))
+        # Add %libcxx as a substition for the full path to the created libc++
+        # library.
+        libcxx_root = self.cxx_library_root
+        assert libcxx_root
+        libcxx_lib = os.path.join(libcxx_root, 'libc++.a')
+        if os.path.isfile(libcxx_lib):
+            pass # Use libc++.a as libcxx_lib.
+        elif self.target_info.platform() == 'darwin':
+            libcxx_lib = os.path.join(libcxx_root, 'libc++.dylib')
+        else:
+            libcxx_lib = os.path.join(libcxx_root, 'libc++.so')
+        if not os.path.isfile(libcxx_lib):
+            self.lit_config.error("Failed to find libc++ library in %s" %
+                                  libcxx_root)
+        sub.append(('%libcxx', str(libcxx_lib)))
         # Configure flags substitutions
         flags_str = ' '.join(self.cxx.flags)
         compile_flags_str = ' '.join(self.cxx.compile_flags)

         sub.append(('%compile', compile_str))
         sub.append(('%link', link_str))
         sub.append(('%build', build_str))
+        # Build the PATH variable from sys.path and self.lit_config.path
+        # and self.env['PATH'].  Also add libcxx/utils/sym_check to the path.
+        path_list = []
+        if 'PATH' in os.environ:
+            path_list += os.environ['PATH'].split(':')
+        path_list += self.lit_config.path
+        if 'PATH' in self.env:
+            path_list += self.env['PATH'].split(':')
+        path_list += [os.path.join(self.libcxx_src_root, 'utils', 'sym_check')]
+        env_cpy = dict(self.env)
+        env_cpy['PATH'] = ':'.join(path_list)
         # Configure exec prefix substitutions.
-        exec_env_str = 'env ' if len(self.env) != 0 else ''
-        for k, v in self.env.items():
+        exec_env_str = 'env ' if len(env_cpy) != 0 else ''
+        for k, v in env_cpy.items():
             exec_env_str += ' %s=%s' % (k, v)
+        sub.append(('%env', exec_env_str))
         # Configure run env substitution.
         exec_str = ''
         if self.lit_config.useValgrind:

utils/sym_check/sym_match.py

     sys.stdout.write(report)
     if matching_count != 0:
         print('%d matching symbols found...' % matching_count)
+        sys.exit(0)
+    else:
+        sys.exit(1)
 
 
 if __name__ == '__main__':