Well, the libexecinfo one exists as fallback because gcc doesn't provide one.

Kamil, which unwind.h are you using? The outdated copy in libexecinfo.h or the modern one used by libunwind? I see little reason to cater to the bugs in the former...

BTW, I recently spend some time slapping GNU ld in NetBSD into shape so that it can properly support read-only .eh_frame even on MIPS. You might want to look at adopting similar changes.

divtc3 and friends.

Because PPC uses the TC variant.

Just assume that the full 32bit address space is available.

The primary reason for using mmap is not so much performance, but reduced memory foot print.

@spatel: I don't see a reason why we can't (or shouldn't) try to do common-prefix elimination for the memcmp intrinsic. It certainly seems to be better to me to preserve the intrinsics in your case as they should be easier to reason about. That's kind of my question for here too -- why does the expansion allow better code?

What is the advantage of expanding the memcpy intrinsic in InstCombine vs doing it later in the target-specific code?

In D34158#837444, @jyknight wrote:

Now, the "/tmp/foo-XXX.sh" also has a line labeled "Driver args: " with the original command-line on it. If I understand correctly, you then like to take this
simpler Driver command-line, and edit it manually: add -save-temps, and change the input filename to the "/tmp/foo-XXX.c" file, and run that, instead
of actually invoking the reproducer foo-XXX.sh.

In D34158#837281, @hfinkel wrote:

In D34158#837130, @joerg wrote:

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

Don't you use preprocessed source files from a crash?

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

I don't see any reason why zero-initialised constants should be emitted in BSS. I know that GCC does that and I just fixed bugs in that because created wrong section flags for it. So yes, I'd prefer to revert this and fix the real problem.

I'm not sure I want anything like this unconditionally. It is going to waste quite a bit of space, i.e. 2KB per executable and shared library sums up.

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

Merging would be reasonable, yes.

Warning for .ctor/.dtor use would IMO be completely bogus. They can easily be translated and they are kind of the LCD for "portable" assembler, i.e. much less problematic to deal with than plain .init/.fini segments.

I don't really like this. The reason why -lm is added explicitly on many targets is because the C++ STL typically depends on it and that means for static linking and broken ELF linkers, it will be necessary to link against it explicitly.
There is also the question on whether any platform we have currently uses separate STL and ABI libraries and it is not clear whether the flag should handle both.

While more localized, this seems to be an even greater hack than adopting the JIT users to use AllocateRWXMemory.

In D35558#816891, @lhames wrote:

They are not allocated RWX, they are allocated RW with the option for later X. I.e. the kernel enforces W^X, but you can request additional protections for later use.

When you say "additional protections" do you mean you can add the 'X' permission later (to get RWX), or just that you can later toggle from RW- to R-X?

Keep in mind that the SELinux case in libffi is not fork-safe. One important part LLVM needs to consider is
whether it wants to enshrine the performance penalty of mprotect-after-commit in its APIs or not. The second part
is whether platforms should aim to support hot-patchable JIT for multi-threaded environments or not. If the latter is
not considered relevant, the API only needs to provide a function to allocate JIT-safe memory and a function to make
it executable. If the latter is relevant, the current AllocateRWX is the interface you will end up with, one way or the other.

In D35558#815442, @lhames wrote:

Right, the pages are allocated RWX.

It seems odd to prevent the flipping of the W/X bits while allowing RWX pages. I assume there are additional privileges
required (granted only to JITs and debuggers?) to get RWX pages? Is there a good source for me to read up about the PaX MPROTECT design?

Test cases?

i386: code requires three push instructions + call + potential stack cleanup.
x86_64: code requires three register loads + call

This doesn't make sense to me, both the existing and new code. off_t is a signed type.

Please mark it as alias for -nopie, otherwise fine.

What if the constexpr function is called with in a non-constexpr context, e.g. with a random global variable as argument?

Strictly speaking, I think this comes from csh, but LGTM.

A better construct would be to compute the full length and change both it and Str in the loop. No point in the complex arithmetic. I generally do prefer this as it improves interaction with pipes etc.

If you want to go this way, rename them consistently and use a different prefix (e.g. AUXV_*) please.

LGTM, as Kamil said, we have the same change in pkgsrc for Illumos.

The bigger question for me is: what depends on the RTLD_GLOBAL behavior? I generally consider using it a design bug. I.e. what breaks if we just change to RTLD_LOCAL?

Lock-free atomic operations are also signal safe. Your code is not. While I don't know whether all this functions are not required to be signal safe, the general assertion is certainly questionable.

LGTM, but please cut down the test cases by hand. Most of the attributes and module flags should be unnecessary.

As I said in the discussion about similar flags for GVN, I'm generally fine with. We should have a big fat warning in the Release Notes that all -fexperimental-* flags are exactly that -- temporary and not intended for long term consumption. I.e. "we can and will remove them whenever it creates the most havoc".

Do you need someone to commit this?

I don't think it is a good idea to make this function non-transitive.

It would be nice to make the stackprobesize a proper TLI attribute, so that OSes can decide how much guarding they are willing to guarantee. I think I would also like to have an option to inline the probing without the complexity of the function call. But both can be done as a separate step.

In D34409#786482, @yuyichao wrote:

If the jump table is writable

FWIW, that also sounds like a hack and a (minor?) security issue.

If the jump table is writable, you can just use the absolute pointers, PIC or no PIC. That would be the "block address" encoding.

In D34397#786261, @mkazantsev wrote:

In D34397#785264, @joerg wrote:

What about the old TODO item of extending powi to integer types and folding them together in the middle end?

Joerg, what TODO item do you mean? Where can I read about it?
Anyways, even if we have a concept of powi in IR, it does not automatically mean that we have it in SCEV. It would be extremely good to have it, though, but it looks like a massive task that takes time.

If you want to create a stop-gap solution that bad, I would make it emit the jump table as writable for non-PIC && largeish code model. That's a much more contained change. It would be really better to work on the real fix and not add more hacks...

For some reason we are using large code model with PIC in the JIT and this is the only issue I see on x86-64 so far.

Go ahead.

The x86_64 code is mixed: it is correct for non-PIC but large, it doesn't work correctly for PIC. That covers the cases most people have been interested in, especially since large model and PIC runs into various other issues too from what I remember.

In D34409#785891, @t.p.northover wrote:

The entirely sensible assumption of the PPC backend is that a single function is no longer than 2GB/4GB.

What's being produced is an offset from the basic blocks (.text) to the Jump table (.rodata). That's not necessarily 32-bits, and I think there are entirely legitimate reasons for putting jump tables in .rodata.

At least the PPC change is definitely wrong. AArch64 should be wrong as well from what I discussed with Tim.

What about the old TODO item of extending powi to integer types and folding them together in the middle end?

Not parsers, encoders. Note that the escaping is not correct for control characters other than \n.

I don't disagree with you, but please see the referenced review for further details.

My suggestion would be to just use the YAML writer for now and leave a comment to make it clear that this is really JSON only.

Please see the discussion in D31992. This patch seems to go in the wrong direction.

As I said, I don't see the point in pretending we support float128 when the runtime doesn't contain the necessary pieces.

This fixes the problem in mbedtls, thanks.

Soft-float on the runtime environment part. I.e. non-trivial operations are lowered to library calls.

At the very least, missing test case.

In D33726#774105, @ruiu wrote:

I'm totally against adding per-OS path knowledge to our linker. Compilers already know include paths and I don't want to maintain another list of paths in the linker. Also this can be more confusing than useful when you are doing cross-linking.

Actually, for NetBSD we want to use -fomit-frame-pointer by default whenever the implicit -funwind-tables is also present. In general, that should be the justification for the default behavior: "Can I reliably unwind a binary with the available information?" Performance of stack unwinding is IMO secondary and users of sanitizers can disable it where necessary.

In D33874#774445, @krytarowski wrote:

In D33874#774433, @joerg wrote:

Given that W^X is becoming a lot more popular across systems including SELinux and other variants, I think it would be better to extend MemoryBlock to store separate pointers for W and X mappings. That avoids the complexity of storing the pointer directly in the allocation.

I was thinking about this and I was evaluating this option. My concern whether it is appropriate to include additional internal pointer for all targets unconditionally or just one extra for NetBSD (and other who can catch up).

I will add this pointer without ifdefs.

Given that W^X is becoming a lot more popular across systems including SELinux and other variants, I think it would be better to extend MemoryBlock to store separate pointers for W and X mappings. That avoids the complexity of storing the pointer directly in the allocation.

A small subset can be found by searching for LINKER_RPATH_FLAG in pkgsrc. A classic offender is Emacs. For more, I would have to systematically search.

sysroot is already handled in NetBSD.cpp line 118 or so.

Such knowledge is necessary anyway. There is enough software that wants to use the linker directly.

I'm against it. I consider this a strong bug on the LLD side and the behavior of clang correct.

Strictly speaking, the glibc attribute violates the specification of const. I.e. we could turn a call for pthread_self() into a once() initialised static variable and that transform would be valid under the const rules. That clearly doesn't reflect the intention...

In D33372#760060, @pcc wrote:

Non-LTO pipeline allows the user to put PIC code in a non-PIE executable, or non-PIC code in a shared library if they wish so. Maybe we should completely ignore the output type here and strictly go by module flags, to be consistent.

Disabling PIC can sometimes act as an optimization.

I find the use of "must" at the very least inappropriate. If there was no use case for not including it, it wouldn't be an option. There is also nothing really Android-specific here beside maybe the open64 mess.

Can this use ErrorOr?

Lock-free operations provide two major advantages. You don't need to worry about signal safety and they can be mixed with certain non-atomic operations without creating havoc. All I said is that the presence of a 32bit CAS is enough to ensure lock-free atomic operations for (appropriately aligned) 8bit and 16bit values can be implemented. That will not be worse than any locked atomic operation on any non-brain-dead platform. E.g. it might not be true on SPARC v7 and v8 or the VAX, but then they don't have a 32bit CAS. It doesn't mean that you have to implement 16bit atomics using a 32bit CAS. I would hope the webassembly backend exposes atomics as precise operations, but if it only provides a 32bit CAS, that's still good enough for the needs of any frontend language.

While I fully agree that the current fallback-to-GCC behavior is far from helpful, I'm not sure how much sense providing linker support for bare-metal makes. I would expect this to changes wildly depending on the specific environment.

I don't get this. If you have a lock-free 32bit CAS, you can implement all the 16bit operations on top of that and they are still lock-free.

In D32605#755237, @evstupac wrote:

In D32605#755223, @joerg wrote:

Can you check the countable part in one of the cases? Otherwise it looks good.

If I get yuo right, "ctlz_and_other" checks conversion to countable loop and that CPUs that not support ctlz will not insert it in this case.

Can you check the countable part in one of the cases? Otherwise it looks good.

At least NetBSD doesn't support/use DF_TEXTREL and only DT_TEXTREL.

It's actually more a diagnostic hint. Due to things like Nvidia's binary-only libGL, this is actually supported by most TLS implementations as long as no initializers are needed.

Just as a side note: this is not only ARMv4. Older SPARC, M68K, SH, IA64, Alpha all seem to lack a FFS / CLZ instruction from cursory check.