Coming from the unixy side (musl), there is no reasonable way short of self-exec with LD_PRELOAD (which has reasonableness problems itself since it'd be inherited by child processes) to make a -fintegrated-crt-alloc where replacement of malloc is runtime-conditional. And the only way to (non-statically) initialize a replacement is init-on-first-use (call_once style; this can be done without barriers if needed using a not-widely-known algorithm you can dig up). The better way is just making the statically initialized state a valid initial state.

Forgive my jumping into this without knowing all the details, but is there a public intended-for-upstreamability hexagon arch definition for musl? If not, are there assumptions about musl and the libc-defined part of the ABI in this changeset that might depend on or conflict with what's eventually added upstream?

I'm pretty sure copy relocations would also interact really badly with STV_PROTECTED TLS objects - the shared library would be accessing its own copy via local-dynamic model while the main program would be accessing the copy it made through the copy relocation. While STV_PROTECTED has all sorts of problems, prior to the introduction of TLS copy relocations, it *did* work as intended for TLS, even if not for global data and functions.

Copy relocations are extremely bad, in all places. They waste memory, possibly massive amounts (e.g. const char libfoo_table[1<<24] = { ... }; in a library transforms from 16M of shared text to 16M of per-instance data), and they make the size of structures and arrays exported from shared libraries part of the ABI (precluding extending them at the end in future library versions). They should not be used anywhere, and the only historical reason they exist at all is that, prior to the existence of shared libraries, the existing ABIs on platforms at the time allowed absolute references to data objects embedded in program text. There is no good reason to allow this on new architectures not encumbered by such legacy ABIs, and certainly no reason to introduce it for TLS, where the extreme memory waste is not per-process but per-thread.

That's a separate issue of clang having a slight types-ABI mismatch with some 32-bit archs whose original ABIs used long instead of int for wchar_t. The wrong header order makes it quickly apparent, but these are independent; wrong header order is still wrong and will break (other) things even without this type mismatch. Backport of the fix would be much appreciated.

It's not just infeasible; it's impossible by design. musl's headers do not support any contract with compiler-provided versions of the std headers, and won't. Attempting to use them in place of the libc ones is unsupported usage and will periodically break.

Just chiming in to confirm that the requested change is correct for musl. musl does not support use/mixing of compiler-provided std headers with its headers, and intentionally has no mechanism for communicating with such headers as to which types have already been defined or still need to be defined. If the current include order, with clang's headers before the libc ones, works in some situations, it's only by accident.

Have you checked what gcc does? It should be safe to do the same, no? Or is the issue that gcc leaves it to gas, and it's reasonably assumed that gas and ld.bfd versions will match, whereas clang uses an internal assembler that might not match?

I second the question, especially since I'm pretty sure that everything the existing asan code here is doing that makes it "work" on musl is poking at implementation internals that are not public interfaces and have no reason to continue "working" however they are now. A long time ago the question was raised of whether/how libcs could provide mechanisms for sanitizers to work, but the ideas I saw at the time were based on hooks, which we didn't consider reasonable from the musl side - as soon as hooks are offered as stable API, they WILL be used not just by sanitizers but by applications.

I don't immediately see any problem with adding a dummy .sdata, though I think the linker should probably also be able to do this. defining a dummy __global_pointer with SHN_ABS makes no sense. The bfd linker doing this looks like a bug to me, and the bfd linker failing to produce an error when linking it looks like another bug.

I'm not sure how LLVM internals work, but if it's anything like GCC's, I think you just need to make it take a register operand for the GOT address as an input/operand. Then the higher-level code should ensure it's available, and then you just use it in the code emitted. Touching higher-level code shouldn't be needed.

Hardcoding ebx defeats most of the purpose of no-plt, which was motivated by: https://ewontfix.com/18/

I'm not sure if/where it's documented, but the congruence requirement is the only reasonable interpretation of p_align. We're fixing it in musl according to this interpretation.

The waste is not just 48 bytes in the ELF image on disk, but up to 63 bytes in each thread. Depending on how that falls, the waste is likely actually either 0 bytes or 4096 bytes per thread, since the memory is allocated with page granularity.

I do not see how .tdata happening to be the first thing in the LOAD segment is relevant or helpful at all. Any "accidental alignment" as a result of that is not part of the alignment that will affect the runtime addresses of TLS objects.

Also, I think the description "make glibc happy" is possibly wrong. It looks like, while musl and some other implementations work with the current hack in LLD, it's only due to a bug that they work. The real problem is that the hack is emitting bad ELF.

I'm reasonably happy as long as this is fixed, but I would really rather see the logic reversed: rather than special casing each !glibc, all of these nonstandard functions should be enabled only if(T.isGlibc) to begin with.

This commit in musl has details on why calling memset, etc. doesn't work and violates the ABI:

Looking further at this, it looks like your current implementations of the __aeabi_* functions are simply wrong - they clobber registers that the ABI does not allow them to clobber.

In musl we do not define the __aeabi_* functions for the sake of having optimized implementations but simply for ABI conformance. They are specifically and intentionally NOT optimized since their ABIs (special rules about which registers they don't clobber) preclude writing them in C, and also make it hard to do the potentially-more-worthwhile optimizations like vectorization (since you don't have vector scratch registers). With this in mind, I would strongly recommend against ever generating calls to them. Our view on the musl side is that they exist just for minimal (i.e. non-performance) ABI compat with code that might have been compiled with a legacy compiler making use of them.

Are you sure you'd have to wrap all of them? Most of the "xlocale" functions are just standard functions that are in locale.h since POSIX 2008. Only a few are nonstandard. Of course from a practical standpoint it should work to start out just testing the ones you've observed missing on some systems, then add more later as needed.

I realize your build system might not be setup to deal with this situation any better, so if so, I apologize in advance for the noise. Much of this patch, however, is exactly the reason why musl does not provide a __MUSL__ macro (see the musl FAQ): in short, these hard-coded assumptions about what functions musl does not have will break the build if/when musl adds them. The right way, from our perspective, to do this would be with build-time detection for nonstandard functions like strtoll_l. By unconditionally defining them as static-inline when targeting musl, the build will break if musl's locale.h ever adds a declaration for an extern version.