Can you make sure that we handle the older ARM versions correctly as well, i.e. v4, v5 and v6? I take it we still have test cases for the arm <-> thumb transition? That's the one part of the triple logic that is really non-trivial.

The "struct object" is an implementation detail of the unwind implementation. You are guaranteed historically to get at least 8 longs / 8 pointers for internal use statically allocated in each object. What is stored inside is up to the unwind implementation.

GCC supports -mbss-plt to get the legacy behavior. Not sure if anyone actually uses it though.

Given that some people like to post-process assembler files, using the section symbol directly is a bad idea. Adding the local symbols is fine.

Oh, we certainly should never be hitting an assertion on front-end flags. As such, there is a problem to fix here. I still maintain that the combination of flags is non-sense, so the question is:

It should be kept in mind that secure PLT is desirable for certain cases with non-position independent code as well. Even in static binaries it can be desirable... But that is for a follow-up patch.

IMO we should explicitly error out. That combination is nonsense to me. Creating useless JSON database fragments is not an improvement.

The difference is that modsi3 etc are all paired instructions. A backend should not be lowering to one of them if a real division instruction exists and it should be consistent in the lowering.

ARM and x86 implement different chars, don't they?

We tried to keep the condition simple. I.e. does the compiler on any of those platforms ever use the libcall? If not, it is IMO not worth the complexity.

Please stop adding complexity to doctor around the symptoms. There are two real fixes here and this change doesn't help with either:
(1) Emit cross-section pointers as indirect. This increases the binary size, but otherwise ensures that any linker can create read-only .eh_frame on MIPS.
(2) Teach lld on MIPS to properly reassemble the DWARF instructions, similar to what GNU ld can do. The latter is a bit stupid and needs a good kick to work properly, but this is the correct approach forward.

This is not acceptable. If anything, the encoding should be switched to indirect, but that should already be the case.

I really don't like ignoring options that are supposed to provide actual functionality. Most of the other options are for pointless fine tuning and workarounds for broken gcc behavior in ancient versions.

I really, really dislike this patch. It is using very blunt force to workaround a GCC bug. The comment is too verbose as well. Please try the following change from NetBSD instead:

Good enough for me.

Do you see the comment just following the code? The patch completely violates that basic design principle. It would be perfectly sensible to hard-code a list of dumb terminals and explicitly default to no colors for them. The reverse (hard-coding a list and assuming it is fine for everything else) is not.

That's no excuse for making the situation even worse.

I completely disagree with this approach. A lot of GNU tools (including GCC) are completely broken. We shouldn't follow them. There are a lot more terminals around than just "dumb", "xterm" and "linux". It is completely non-acceptable to just assume ANSI escape sequences work. If Android doesn't ship a usable terminfo implementation, I consider that an Android bug. Wouldn't be the first portability nightmare with Android.

This works in 32bit mode as well? I'm suprised.

I'm not really a fan of linking libutil into all binaries. Why is this code using forkpty in first place and not posix_openpt/grantpt?

Instead of computing and storing the modulus directly, it is likely better to precompute the inverse and use that to improve the performance of the operation in first place. Consider using fast_remainder32 and associated functions.

So the next steps if you have the time would IMO be:

Split into verbose conditional register names into a separate function. We likely want to remove them going forward as they are a specific feature of the Darwin assembler and not wildly supported.

The public interface for obtaining the TLS storage is the combination of reading the DTV vector of a thread in combination with dl_iterate_phdr to find the size of the TLS block of a specific module. That gives you all that you need to know. It is important to keep in mind that the vector can be initialized lazily, so __tls_get_addr and friends will have to be intercepted to update the global view.

I really dislike this direction. fallocate can double the amount of disk IO and increase cache trashing, especially when linking large programs with debug information. Keeping more things in memory doesn't sound like an actual improvement either. If the goal is really only to improve the diagnostics in tools, I think a better idea would be to figure out a good way to handle this from a SIGBUS handler based on the passed in siginfo_t.

No need for a custom container, just allocate the vector dynamically and free it when it becomes empty.

No, __cxa_atexit will always reference the DSO handle. That exists even in the main executable.

Is there any reason why keeping at_exit and __cxa_atexit handling merged? They are pretty much disjunct code paths, especially since the at_exit stack means that the real at_exit can be used.

In D39079#905468, @rnk wrote:

In D39079#905454, @joerg wrote:

It also increases the pressure on the branch predictor, so it is not really black and white.

I don't understand this objection. I'm assuming that the PLT stub is an indirect jump through the PLTGOT,
not a hotpatched stub that jumps directly to the definition chosen by the loader. This is the ELF model
that I'm familiar with, especially since calls to code more than 2GB away generally need to be indirect anyway.

In D39079#905396, @rnk wrote:

In D39079#905372, @joerg wrote:

Why again is this a good idea?

It saves the direct jump to the PLT, reducing icache pressure, which is a major cost in some workloads.

Let me phrase it differently. What is this patch (and the matching backend PR) supposed to achieve? There are effectively two ways to get rid of PLT entries:
(1) Bind references locally. This is effectively what -Bsymbolic does and what is breaking the ELF interposition rules.
(2) Do an indirect call via the GOT. Requires knowing what an external symbol is, making it non-attractive for anything but LTO, since it will create performance issues for all non-local accesses (i.e. anything private).

Why again is this a good idea? This is an even worse hack than -Bsymbolic, the latter at least is visible in ELF header without code inspection. This is breaking core premises of ELF.

Almost.

In D39016#903937, @alexandreyy wrote:

Because it is not possible to change the cl option by LLDB.

This is even worse. You can't new[] and then free(). Please follow the suggestion on just embedding the prefix directly, if desirable.

Even a full static binary will have a PLT when IFUNC is used. As such, a linker has to deal with conversion between direct and PLT branches anyway.

A PLT is used not only by PIC code. It is required for all dynamic entry points and that's not limited to PIC. It's not even limited to dynamically linked binaries. There is no support for the embedded ABIs as I said before. I'm going to stop responding since it is rather pointless now. My objection stands.

A working linker is certainly expected to use the most efficient binding. The assembler (and by extension the compiler) can't tell. This applies in both directions -- in your example, the linker might need to translate the direct call into a PLT call, depending on the target. It all goes back to "use a working linker". So yes, at this point in time the existance of two different relocations for call instructions can be clearly seen as a historic artifact. This doesn't change anything of what I have been saying since the beginning. The PowerPC ABIs are heavily biases towards position independent code and this is just adding complications for no good reason.

I've looked at this in some detail now. I'm not exactly sure yet why it is broken. The patch seems quite wrong to me. DW_CFA_GNU_args_size should be applied only when unwinding a call instruction and that regard, the commit message of the original change is quite correct. What I am still trying to understand is how the precise unwind frame disagrees with the unwinder.

I think we should special case Darwin and Windows and fall-back to LD_LIBRARY_PATH for the rest. Can't remember if there is a UNIX-like platform left where it doesn't work.

There is no layer of indirection here. The call gets resolved to the local symbol by a working linker.

On the contrary, I find little reason for adding the complexity here. That's even ignoring the question of whether the suggested check even covers the relocation models correctly.

The existing code works fine. What you have demonstrated so far is more an issue with lld and not a problem in the code. GNU ld is perfectly fine with translating the PLT references to direct calls. As such, I see no need for changes here.

The patch is not acceptable in the current form. This includes fixing the memory leaks.

This is not about any operating system, but basic consistent behavior. either do the canonicalisation or not. Doing it sometimes is just bogus. You've effectively implemented -fcanonical-system-headers=sometimes.

The behavior of sometimes transforming the path and sometimes not is IMO completely unacceptable. I don't care if GCC created that bug first.

In that case: the short version is that there is no EABI support at the moment. That patch changes the behavior for SYSV ABI, it is not acceptable as such. The test case doesn't reflect the ABI variance either.

Let's start with the obvious: I have no idea why you need this code at all. NetBSD builds a mix of static, PIC and PIE code using GNU ld without hitting any unsupported relocations. As I said before, the normal ABI on PowerPC is mostly PIC by default. I wonder if the root of your problem is that you want EABI and not the SYSV ABI.

This doesn't look right to me at all. The normal SYSV ABI on PowerPC is effectively PIC, with a few edge cases. This really looks like a step backwards.

Well, the background for the use of the option in NetBSD is related to inducted differences in reproducable builds. From that perspective, it is even worth to sometimes shorten the dependency and sometimes not.

ninja is not the only consumer of this. For human inspection, it can be quite surprising to see symlinks resolved, i.e. /usr/include/machine on NetBSD. Build systems have different ideas on whether they want absolute resolved paths or not, so it seems like ninja should be able to cope with either format. This is a lossy transformation, so I'm somewhat reluctant to agree with it.

The comments at the very least are misleading. Resolving the path doesn't necessary shorten the string at all, in many cases it will be longer. I'm really not sure if clang should resolve symlinks like this as it can be quite surprising.

So what about targets that don't support subnormals? I'm moderately sure ARM falls into this category given the right phase of the moon.

I can't really comment on the Linux interface, but we generally ignore the system call error in NetBSD where one is necessary. While aborting might be legal, it seems to be the worst possible way of dealing with questionable arguments.

This version is fine with me. The only contentious part is whether it should be opt-in or opt-out for platforms, so getting this version in and revisiting the issue again later is OK from my perspective.

Create a function that does check the guard variable and make sure that it has the correct visibility declaration and/or access.

Well, the libexecinfo one exists as fallback because gcc doesn't provide one.

Kamil, which unwind.h are you using? The outdated copy in libexecinfo.h or the modern one used by libunwind? I see little reason to cater to the bugs in the former...

BTW, I recently spend some time slapping GNU ld in NetBSD into shape so that it can properly support read-only .eh_frame even on MIPS. You might want to look at adopting similar changes.

divtc3 and friends.

Because PPC uses the TC variant.

Just assume that the full 32bit address space is available.

The primary reason for using mmap is not so much performance, but reduced memory foot print.

@spatel: I don't see a reason why we can't (or shouldn't) try to do common-prefix elimination for the memcmp intrinsic. It certainly seems to be better to me to preserve the intrinsics in your case as they should be easier to reason about. That's kind of my question for here too -- why does the expansion allow better code?

What is the advantage of expanding the memcpy intrinsic in InstCombine vs doing it later in the target-specific code?

In D34158#837444, @jyknight wrote:

Now, the "/tmp/foo-XXX.sh" also has a line labeled "Driver args: " with the original command-line on it. If I understand correctly, you then like to take this
simpler Driver command-line, and edit it manually: add -save-temps, and change the input filename to the "/tmp/foo-XXX.c" file, and run that, instead
of actually invoking the reproducer foo-XXX.sh.

In D34158#837281, @hfinkel wrote:

In D34158#837130, @joerg wrote:

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

Don't you use preprocessed source files from a crash?

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

I don't see any reason why zero-initialised constants should be emitted in BSS. I know that GCC does that and I just fixed bugs in that because created wrong section flags for it. So yes, I'd prefer to revert this and fix the real problem.

I'm not sure I want anything like this unconditionally. It is going to waste quite a bit of space, i.e. 2KB per executable and shared library sums up.

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

Merging would be reasonable, yes.

Warning for .ctor/.dtor use would IMO be completely bogus. They can easily be translated and they are kind of the LCD for "portable" assembler, i.e. much less problematic to deal with than plain .init/.fini segments.

I don't really like this. The reason why -lm is added explicitly on many targets is because the C++ STL typically depends on it and that means for static linking and broken ELF linkers, it will be necessary to link against it explicitly.
There is also the question on whether any platform we have currently uses separate STL and ABI libraries and it is not clear whether the flag should handle both.

While more localized, this seems to be an even greater hack than adopting the JIT users to use AllocateRWXMemory.

In D35558#816891, @lhames wrote:

They are not allocated RWX, they are allocated RW with the option for later X. I.e. the kernel enforces W^X, but you can request additional protections for later use.

When you say "additional protections" do you mean you can add the 'X' permission later (to get RWX), or just that you can later toggle from RW- to R-X?

Keep in mind that the SELinux case in libffi is not fork-safe. One important part LLVM needs to consider is
whether it wants to enshrine the performance penalty of mprotect-after-commit in its APIs or not. The second part
is whether platforms should aim to support hot-patchable JIT for multi-threaded environments or not. If the latter is
not considered relevant, the API only needs to provide a function to allocate JIT-safe memory and a function to make
it executable. If the latter is relevant, the current AllocateRWX is the interface you will end up with, one way or the other.

In D35558#815442, @lhames wrote:

Right, the pages are allocated RWX.

It seems odd to prevent the flipping of the W/X bits while allowing RWX pages. I assume there are additional privileges
required (granted only to JITs and debuggers?) to get RWX pages? Is there a good source for me to read up about the PaX MPROTECT design?