Actually, since it used to be part of XSI, I would include it unconditionally and only conditionalize it if a system starts to actually drop it.

The description is certainly not right. IE can be used from dlopen'd objects without problems as long as there is still reserved space around and the data is not initialized non-trivially or the dynamic linker is willing to fix up all threads. The point of the flag is to allow the dynamic linker to make a quick decision without having to scan the relocation table first. A classic user of this was libGL. It also applies to many platforms, not just i386. As such, please update the description before any commit.

Neither GNU ld nor gold implement really useful behavior here. Ideally, every library would be linked with -z defs, but that normally doesn't work because on many systems certain symbols are imported from the main program by default. A good example are the entry points of the dynamic linker. A bad example historically speaking is libwrap. From a user perspective it would be desirable to have a way to say "This symbol will be supplied by someone else" when linking a shared library or potentially even the main binary. That would allow whitelisting intentionally undefined symbols and allow avoiding any symbol resolution dance for shared libraries. A single pass over the symbol tables is still necessary for the sake of knowing the defined symbols and for potential copy relocations, but no further work should be needed.

This seems to imply hard-float support though? Accessing floating point registers when using soft-float is not going to get very far.

I'd really prefer to keep the argv[0] code as is. I'm not sure what that test case is supposed to do, but it seems quite questionable as "check" is not a valid language frontend nor a version suffix. It should not work.

For the clang side, I don't understand why Driver::GetFilePath is not good enough. This shouldn't need all toolchain changes at all.

As discussed with dankm on IRC, I still would like to see the correct behavior going into 8.0, i.e. not change it later. Since this also matters for potential faster implementations later, it seems like a good idea to do it now. The changes are well-localized.

As first step, it goes into the right direction. I would explicitly set --as-needed for all those indirectly loaded objects. If people want to retain the questionable behavior of newer GNU tools, it could be a separate flag so that a final round can warn if an indirectly pulled library is necessary, but that behavior doesn't IMO make much sense. Full version has to look at DT_RUNPATH/DT_RPATH and also --rpath-link.

Right, I'm not aware of anyone but FreeBSD really using the OSABI field. For FreeBSD it was a long standing hack around limitations in the ELF kernel loader. I'm not even sure if FreeBSD use(d) to set the OSABI field for the intermediate object files.

That's the other reason why I find the GCC specification as string prefix confusing. I still say we should just go with mapping of path names and then the order question mostly goes away.

@ruiu: No, it is exactly what you want, since it allows you to point lld into the normal sysroot. Cross-compiling is the default case for the NetBSD toolchain.

Thanks, this looks like a good starting point.

It can certainly make it worse, e.g. if you have one input section with half page size and page alignment followed by another input section with half page size. Depending on the precise size and order, it will fit into one page or two. It's a bit constructured, but gives the general idea.

I think this should factor in any larger-than-normal alignment, otherwise it could easily result in a significant increase in size?

Talking from the perspective of having had to deal with thousands of packages in pkgsrc over the years: it is naive to believe that there isn't a lot of software that calls the linker directly for various reasons. As such, it is very important to have a useful configuration in the linker by default. Such a configuration is by its very nature target specific. This doesn't mean it can't be done in a cross-compiler friendly way, on the contrary. Over the years NetBSD has been pushing its toolchain to be as similar for the native build and a cross-target as reasonable possible. Modulo the build time choices in the config.h sense, the only difference between the native and cross tools is the built-in default of the former.

In D56215#1344279, @krytarowski wrote:

In D56215#1344183, @joerg wrote:

This doesn't seem a reasonable approach at all:

(1) It breaks cross-linking.
(2) It is not correct for any target architecture, e.g. /usr/local/lib certainly doesn't belong on this list and /lib doesn't either.
(3) The correct list depends not only on the target architecture, but also the active emulation.

Is it acceptable to pass all the paths through configure/build phase of lld? It's done this way in GNU ld in the NetBSD distribution. If we need or want to hardcode all the specific paths it will be harder to maintain the proper list and behavior inside lld.

This doesn't seem a reasonable approach at all:

I foundamentally don't understand what you are trying to do. You can either look at the executable from a segment perspective or from a section perspective. But trying to mix the views is bound to give bogus results.

In D55998#1340448, @jankratochvil wrote:

it is in theory possible to create an elf file where only a part of a section would belong to some segment (and another part to a different one).

ELF standard: "An object file segment contains one or more sections.", "An object file segment comprises one or more sections"

This header is used on systems without glibc. So please don't argue about behavior based only on that. Granted, most other libc implementation are less annoying when it comes to free and malloc, but still.

In D55167#1326120, @vitalybuka wrote:

In D55167#1324855, @joerg wrote:

I don't see why this interceptor needs to know about the internals of struct cdbr or struct cdbw at all. They are fully opaque. All memory accessed by users is explicitly sized as argument to the functions or returned with the size.

Then we give up on TSAN check of fieds, or msan check if caller will try to read particular fields

I don't see why this interceptor needs to know about the internals of struct cdbr or struct cdbw at all. They are fully opaque. All memory accessed by users is explicitly sized as argument to the functions or returned with the size.

I'd recomment another pass to reduce the now fully redundant () pairs around % string substitutions, but in principle LGTM.

LGTM

LGTM

LGTM

LGTM

Can you split off the pure modernisation changes like new exception or print ? Those are completely non-contentious changes after all. I generally do not like the range and list related changes as many instances are clear regressions for the 2.x case. filter to list comprehension should IMO be a separate change as well, but those are much less problematic and often an improvement in terms of both performance and readability.

I'm sorry, but it still sounds to me like you want to address badly written build rules by making the driver more complicated. I don't see that is a reasonable goal forward.

I don't understand the point here. Why would you want to include pre-processing-only commands in the compilation database?

Well, I do like the general idea. I would go with a sorted static array and a binary search though. This is not really time critical, but the array version would be smaller in terms of code without involving run-time allocations.

I would actually rule out subshells as well. I see no reason why a test case should have to use them, i.e. compared to just change the working directory back afterwards. As I said, I want to be able to repeat a problematic command manually and interactively as easy as possible. Using non-portable shell features is a problem for that. Depending on state between commands is also annoying. Let's really try to keep it minimal and where necessary, adjust test case guidelines and test cases to simplify things.

As I said: because it makes it harder to rerun parts of the test interactively. Which is a good enough reason to forbid cd - as well as any directory stack operations, IMO.

Personally, I would strongly prefer if the test cases are as explicit as possible. That makes debugging based on the llvm-lit output a lot easier. Ignoring anything else, this feature would make it a lot more tricky to figure out what exactly is going on.

I'm quite against this. "cd -" is a non-portable extension to shell and should never be used in first place.

I think I would enumerate the BSDs here (and Apple) explicitly and not depend on BSD.

Nothing changed. I don't see how catering to the broken libstdc++ self-configuration helps. So no, I still object to this change.

__m68k__ and __ia64__ too, just for completeness.

To summarize the discussion on IRC, from my perspective this change is wrong. PPC as architecture is strongly geared towards PIC code. We used to create quite bad position dependent code and I generally don't see the advantage. I can understand wanting to default to PIE (i.e. position independent code with main binary assumption as far as interception is considered). In other words, the middle end generally tended to create better results for position independent codegen. The default behavior of GCC is ephemeral here, IMO.

Excuse me for bring this up so late, but why do we want to make any such promises? As in: fundamentally, LLVM IR doesn't have any order property on the module level. I have yet so seen reasonable code where the order of functions matters for anything but performance. I've seen a few things that required -funit-at-a-time, most noticable GCC's CRT implementation. But those are all major hacks. So under what premise is it useful to have to even pretend to honor source code order?

Yeah, I would restrict it to just mention that it depends on the target, link time editor and runtime linker. Even the concrete feature set on Linux changes with glibc versions.

I think this is still too optimistic. Full support for ifunc seems to be generally limited to x86. Most other architectures lack even definitions for anonymous ifunc relocations or support proper relaxation only in limited forms. That's especially annoying when looking at static linking.

I find this warning confusing. I find a4 to be perfectly expected. IMO this warning should be applied only, if the effective value of the expression is not the same as in the modulo-n arithmetic. This means that if (-x) is explicitly or implicitly cast to a less wide unsigned type, it should not warn. It would consider a warning for the case of using (-x) if integer promotion rules makes it negative though. The question is, how to best patch around the warning though. What options does MSVC have for that? I.e. what equivalent expressions do not trigger this warning?

Correct. The protected name is double underscore as both suffix and prefix.

Please check the history of the file for some of the problems with the redefinition. I'm quite against this change.

Every system call has a public and internal variant. The former might be replaced by libpthread etc for thread cancellation support, but that's a different topic.

Yes, it is optional, but on most architectures, the builtin variant is much cheaper. That said, I'm not sure what the situation is on SPARC with the necessary register window flush.

I don't understand why most of this symbols don't reference the plain system call directly, i.e. _sys_read etc.

There is one user of builtin_setjmp/builtin_longjmp that should be kept in mind: Ruby.

Is there a reason for defining them? As in: does anything outside libunwind use them? I haven't seen such software yet.

Why do we need to allocate memory in this case at all? I.e. why can't this just be:

if (S.empty())
  return StringRef("", 0);
...

If a build against compiler-rt works, that's ok. It wasn't clear from the diff.

I don't understand the desire for this logic. Why can't wasm override the rest of the types if it wants to have something special?

This hard-coding seems to be counter-productive, it could be compiler-rt just as well.

There are two different considerations here:
(1) Create less target code
(2) Create less IR

Depends a bit on the platform, __cxa_atexit on most modern ELF systems, fallback to atexit. If the global dtor is run too late, it smells like a missing library dependency. They are executed in topological order after all.

Can this ever end up in a shared library? If yes, please use the normal logic for creating a global destructor. atexit is not very friendly to dlopen...

Both needs a test case :)

It seems to miss most of the interesting checks, i.e. crt files. Compare with any of the entries on netbsd.c for example.

This is absolutely not how the clang driver is supposed to work. No conditional compilation.

Looking further, at least on NetBSD libgcc seems to always include both the "normal" and the .rem/.urem routines. While we currently don't replace __umodsi3 and friends, that looks more like an oversight on our part.
I asked the Sparc folks and they can't remember any special reason for why .urem should be used. I.e. it follows the normal Sparc ABI.
As such, I'm mostly ambivalent on this change and the rest of the block.

Not what I mean. Certain platforms like Sparc and SH link a copy of certain routines into every DSO. This is the so-called milli code. They sometimes use special calling conventions as well. That's different from the "normal" helper routines in libgcc, which are shared by all libraries.

That would be the milli code version, wouldn't it?

Please update the summary / title

After a careful review of newer GCC / libgcc and the assembler annotations from LLVM, I have come to the following conclusions:

This still needs a test case?

This looks sensible, but I don't know what PoisonShadow will do for the rest of the memory block.

"-z textrel" can also be used in build instructions if some platforms will need it, even if not all of them do.

Yes, if that option is enabled explicitly or implicitly in the frontend, the expectation is that all compiler-generated functions have uwtable as well.

Given that .eh_frame sections can be used to create backtraces i.e. from signal handlers, this seems to be undesirable in the generality. Shouldn't this attribute be conditional on whether functions are normally supposed to have unwind data?

I'm back to the point where I can't reproduce the problem :( Can we start providing an actual failing test case? It's annoying to debug a problem when you can't reproduce it.

Things are different for a libgcc-based toolchain and a compiler-rt based toolchain.