[libunwind][AIX] Save/restore errno before/after system calls dlopen/dlsym/dlclose

[xingxue]

Summary

libunwind on AIX calls dlopen()/dlsym()/dlclose() to dynamically load libc++abi and get the personality for state table EH when it is running against the legacy xlcang++ compiler genereated applications. dlopen() sets errno to 0 when it is successful, which clobbers the value in errno from the user code. This seems to be an AIX bug that it should not set errno to 0 according to POSIX. We will open a bug report to AIX but in the mean time there won't be time line when AIX will have a fix and even AIX does fix it, it won't help earlier AIX releases in the field. This patch saves and restores errno before and after these calls so that user code can work as expected.

[compnerd]

I'm not opposed to the change, but I wonder if we should consider placing this in an #if defined(_AIX) guard. This would avoid touching errno on other platforms, which would likely be a TLS access. It would also make it more obvious that this is a workaround for AIX. At the very least, I think that we should add some comments to explain the issue so that it is obvious at the site rather than having to go through the change log.

[hubert.reinterpretcast]

In D131292#3706835, @compnerd wrote:

I'm not opposed to the change, but I wonder if we should consider placing this in an #if defined(_AIX) guard. This would avoid touching errno on other platforms, which would likely be a TLS access. It would also make it more obvious that this is a workaround for AIX. At the very least, I think that we should add some comments to explain the issue so that it is obvious at the site rather than having to go through the change log.

errno for other platforms should be untouched anyway... _LIBUNWIND_SUPPORT_TBTAB_UNWIND is only true (AFAIK) for AIX.

[xingxue]

I'm not opposed to the change, but I wonder if we should consider placing this in an #if defined(_AIX) guard. This would avoid touching errno on other platforms, which would likely > be a TLS access. It would also make it more obvious that this is a workaround for AIX. At the very least, I think that we should add some comments to explain the issue so that it is > obvious at the site rather than having to go through the change log.

Thanks for your comments, Saleem! I've added comments to describe the workaround. As to guarding the workaround with #if defined(_AIX), I thought the code block containing the workaround is guarded by #if defined(_LIBUNWIND_SUPPORT_TBTAB_UNWIND), which is only enabled on AIX so it won't affect other platforms. If you think it is better to use #if defined(_AIX) to highlight it is for AIX only, I can do that.

[compnerd]

Thanks for adding the comment! I think that being part of the traceback table support should be sufficiently isolated and that the additional conditional would be superfluous.

libunwind/src/UnwindCursor.hpp

if (xlcPersonalityV0 == NULL) {
         // If libc++abi is statically linked in, symbol __xlcxx_personality_v0
         // has been resolved at the link time.
         xlcPersonalityV0 = &__xlcxx_personality_v0;
         if (xlcPersonalityV0 == NULL) {
           // libc++abi is dynamically linked. Resolve __xlcxx_personality_v0
           // using dlopen().
           const char libcxxabi[] = "libc++abi.a(libc++abi.so.1)";
           void *libHandle;
+          // The AIX dlopen() sets errno to 0 when it is successful, which
+          // clobbers the value of errno from the user code. This is an AIX
+          // bug because according to POSIX it should not set errno to 0. To
+          // workaround before AIX fixes the bug, errno is saved and restored.
+          int saveErrno = errno;
           libHandle = dlopen(libcxxabi, RTLD_MEMBER | RTLD_NOW);
           if (libHandle == NULL) {
             _LIBUNWIND_TRACE_UNWINDING("dlopen() failed with errno=%d\n",
                                        errno);
             assert(0 && "dlopen() failed");
           }
           xlcPersonalityV0 = reinterpret_cast<__xlcxx_personality_v0_t *>(
               dlsym(libHandle, "__xlcxx_personality_v0"));
           if (xlcPersonalityV0 == NULL) {
             _LIBUNWIND_TRACE_UNWINDING("dlsym() failed with errno=%d\n", errno);
             assert(0 && "dlsym() failed");
           }
           dlclose(libHandle);
+          errno = saveErrno;
         }
       }
       xlcPersonalityV0InitLock.unlock();
     }
     handler = reinterpret_cast<unw_word_t>(xlcPersonalityV0);
     _LIBUNWIND_TRACE_UNWINDING("State table: LSDA=%p, Personality=%p\n",
                                reinterpret_cast<void *>(lsda),
                                reinterpret_cast<void *>(handler));