Visibility attributes on windows (dllexport, dllimport) don't only have an effect when compiling the definitions of symbols, but also affect how undefined references to symbols behave (expecting them to live in the same DLL or not).
If we are building libcxxabi statically, with the intent to bundle it into the built libcxx (which is the only form that really works on windows, due to their interdependencies), we should have it flag its symbols as if it was being built in the same way as libcxx.
I.e., if libcxx is being built shared and libcxxabi is built static (to be linked into libcxx), libcxxabi should include dllexport on its symbols and don't import symbols from libcxx with dllimport as they are going to reside in the same DLL. And if building a static libcxx and going to bundle libcxxabi into it, it needs to access cxxabi symbols without dllimport attributes. (libcxxabi doesn't add #define _LIBCXXABI_DISABLE_VISIBILITY_ANNOTATIONS to any header like libcxx does, if configured for a static-only build.)
Why does _LIBCXXABI_BUILDING_LIBRARY have an impact when building libc++?