<strings.h> should be included unconditionally.

Please enable the tests for NetBSD as well.

bcmp is BSD and it first appeard in 4.2BSD... it's not specific to GNU source but compat layer for BSD software.

@mgorny let's go through tech-kern@ and later checking FreeBSD/Darwin/OpenBSD. I think it's worth to clarify this in the documentation.

In D58230#1398529, @mgorny wrote:

In D58230#1398020, @krytarowski wrote:

For EINTR we shall use llvm::sys::RetryAfterSignal

kevent() man page indicates:

All changes contained in the changelist are applied before any pending events are read from the queue.

Also:

[EINTR] A signal was delivered before the timeout expired and before any events were placed on the kqueue for return.

So while it's not stated explicitly, I think in_events is always consumed, even if EINTR is returned. In which case, llvm::sys::RetryAfterSignal would be wrong whenever in_events is not empty.

It looks good to me.

'attaching a debugger produces an observable side-effect (EINTR) in the debugged process is considered a bug by the linux kernel folks'

For EINTR we shall use llvm::sys::RetryAfterSignal

Short term this looks fine.

In D57959#1390992, @mgorny wrote:

But to what purpose? I think it's better to use consistent macros to refer to the same scenario.

In D57959#1390929, @mgorny wrote:

Ok, I see that _WIN32 actually redefines sigset_t, so I've added a separate branch for it.

@labath our short-term goal is to enable execution of LLDB tests on the NetBSD buildbot. We are stuck temporarily with an older release of NetBSD on the machine for some time (1-2 months) so we need to live with it for now. No need to make it better than sufficient as of now.

I think that sigemptyset(2) is unsupported on Windows.

The NetBSD buildbot breaks in these tests now:

The NetBSD buildbot is affected and red for some time now.

chandlerc added a comment.

There was a long discussion between two NetBSD maintainers about this (both already in the reviewers list of this patch). I'm not sure if there is an existing thread that would be better to follow up on as opposed to starting a fresh thread.

In D56650#1377546, @ruiu wrote:

If you still need to patch GNU ld, it doesn't seems like this patch makes things easier for you. (But even if this would make it easier for you, this patch's approach is not okay by design though.)

If we pass flags from clang, we sacrifice:

@rui we need some resolution here. We have stronger feelings from the community to customize the linker directly based on detected triple.

In D57412#1376056, @cryptoad wrote:

In D57412#1376046, @krytarowski wrote:

I have got a scudo support patch for NetBSD locally.. but the original tests are havily prepared against GNU malloc. This makes me uncertain whether scudo really works. Are there plans to make the tests more generic?

That should be the case for the unit-tests. The new way things are done will clearly separate the C/C++ wrappers.
I can add you on the standalone check-ins for you to chime in on the direction we are headed to.
All the development efforts so far has targeted Linux/Android/Fuchsia, and I unfortunately have no BSD knowledge, but we are definitely open to contributions.
Initial check-ins will be Linux specific though since it's my workstation and that's what I am testing things against.

I wouldn't pursue work on your side on the current non-standalone Scudo, as the standalone version will be better (more performant, smaller memory footprint, etc) on all fronts.

I have got a scudo support patch for NetBSD locally.. but the original tests are havily prepared against GNU malloc. This makes me uncertain whether scudo really works. Are there plans to make the tests more generic?

In D57303#1373077, @phosek wrote:

In D57303#1373061, @krytarowski wrote:

How do you resolve paths? Linker cache with registry of libraries?

DT_NEEDED aren't treated as paths, they are used as object names (keys); dynamic linker passes those to the loader service which is responsible for resolving them and returning back the corresponding memory objects.

In D57303#1373057, @phosek wrote:

In D57303#1373035, @krytarowski wrote:

+ vitalybuka

Can we fix it for everybody? It's certainly not restricted to NetBSD.

As a point of reference, we use shared runtimes on Fuchsia but we don't use rpath so this is not something we want for every system.

Can we fix it for everybody? It's certainly not restricted to NetBSD.

NetBSD is affected in the same way.

In D56975#1364959, @joerg wrote:

I'd really prefer to keep the argv[0] code as is. I'm not sure what that test case is supposed to do, but it seems quite questionable as "check" is not a valid language frontend nor a version suffix. It should not work.

In D56554#1354885, @ruiu wrote:

In D56554#1353572, @krytarowski wrote:

What do you think about this new logic:

1. specified -z execstack -> do not emit GNU STACK segment
2. specified -z noexecstack -> emit GNU STACK segment
3. no option specified -> detect findSection(".note.GNU-stack") (I might miss offhand some details here to be sure if it is reliable) 3.1. detected -> emit GNU STACK segment 3.2. not detected -> do not emit GNU STACK segment
   
   Additionally we will specify explicitly in the clang driver for Linux and FreeBSD -z noexecstack.

I think I don't like the very idea of using "marker sections" in input object files to change the linker behavior. That's too subtle and seems error-prone to me.

After thinking for a while, I started thinking that the first version of this patch is probably exactly what we want. I had been thinking that -z {no,}execstack and -z {no,}gnustack are four different options, but what we actually want to get is tri-state:

• emit PT_GNU_STACK with RWX permission
• emit PT_GNU_STACK with RW permission
• do not emit PT_GNU_STACK
  
  So we could map them to -z {execstack,noexecstack,nognustack}, respectively, with default set to -z noexecstack. You guys can pass -z nognustack to the linker to tell the linker to not emit it at all. That's exactly this patch does.

In D56215#1350179, @ruiu wrote:

To be honest, I don't think I would lgtm a patch that changes lld's default behavior depending on host/target only for NetBSD, and it seems like a NetBSD's issue rather than an lld's issue. As I said, could you please make an effort to pass the flags you need from the compiler driver to the linker just like other systems do? That is easy to do, doesn't hurt anyone, probably a good thing to do anyway as it makes options explicit rather than implicit, and solves at least most of the problems you have.

Actually probably Linux MIPS used it and some proprietary OSes.. but we skip them for now. Only FreeBSD sets OSABI in lld and uses an emulation name detection hack. Once we will get merged TripleTarget detection we can switch it to use proper Triple check.

In D56215#1354603, @krytarowski wrote:

@mgorny could you check if we can get crossbuilding functional for:

• to !NetBSD from NetBSD
• from !NetBSD to NetBSD.
• from NetBSD/amd64 to NetBSD/aarch64
  
  I wonder whether it can work if we will keep using 'ld' file name for a linker.

@mgorny could you check if we can get crossbuilding functional for:

@ruiu actually if we can get D56215 merged we will be able to tune it specifically for NetBSD (with if (Config->TargetTriple.isOSNetBSD()) {) and retain intact the current Linux-biased logic for everybody who deserves to use it.

Please check these options in regression test-suite. There are also some missing entries and please add them too.

What do you think about this new logic:

I understand the concerns, but lld is biased with being a Linux linker. We need to customize it (restore defaults?) for NetBSD.

Shouldn't the logic of emitting/not-emitting be based on findSection(".note.GNU-stack")? @mgorny can you please investigate it?

Actually looking at GNU ld(1) source code, the logic is a little bit more complex and it depends on more aspects like relocations.

In D56554#1353513, @ruiu wrote:

In D56554#1353487, @krytarowski wrote:

In D56554#1353368, @ruiu wrote:

The absence of PT_GNU_STACK segment makes stack area executable on systems that recognizes PT_GNU_STACK segment. So, I think if -z execstack is specified, we should omit PT_GNU_STACK segment rather than adding it, which I think you guys want. If we do that, it seems -z nognustack is a redundant option. That option name is unfortunate (you don't really mean you want an executable stack area), but that's I think still better than adding an option that is very similar to an existing feature.

If we are going to change the meaning of -z execstack, can we rename the option in lld? Probably to -z gnustack vs -z nognustack, probably there is no other use-case than RWX->RW protection change.

Both -z execstack and -z noexecstack are supported by GNU linkers, so we can't simply rename them. We might be able to define aliases to the options. But I'm not sure if we want it if the only reason to do so is aesthetic reason.

In D56554#1353368, @ruiu wrote:

The absence of PT_GNU_STACK segment makes stack area executable on systems that recognizes PT_GNU_STACK segment. So, I think if -z execstack is specified, we should omit PT_GNU_STACK segment rather than adding it, which I think you guys want. If we do that, it seems -z nognustack is a redundant option. That option name is unfortunate (you don't really mean you want an executable stack area), but that's I think still better than adding an option that is very similar to an existing feature.

Looks good, we don't GNU STACK on NetBSD.

@joerg if you think that this patch is OK, please click accept so we can be aware that this is what you want.

In D56215#1350134, @joerg wrote:

Thanks, this looks like a good starting point.

In D56141#1348702, @eugenis wrote:

In D56141#1342105, @krytarowski wrote:

Can we handle with this change a similar scenario that proc map is too large to be retrieved? If so can we generalize no-/proc to no-memorymap?

Does this look like an error return from internal_sysctl in ReadProcMaps? Sure, we can mock that condition with the same flag.

On NetBSD the 'new' semantics was already implemented with 'old' RPATH. I don't know the timing whether there already existed RUNPATH, but the result is that we never implemented it and never needed it. The core of NetBSD was convinced to add an alias as it was very difficult in some examples (probably due to Rust) to rollback to 'old' RPATH.

In D56288#1348148, @mgorny wrote:

Ok, maybe I'm being silly but since clang driver has to pass --enable-new-dtags for GNU ld compatibility anyway, wouldn't it make sense to keep the default as disabled in order to match GNU ld behavior?

In D56064#1347535, @EricWF wrote:

In D56064#1347410, @krytarowski wrote:

I'm going to revert it soon and see what happens. Please keep an eye on http://lab.llvm.org:8011/builders/lldb-amd64-ninja-netbsd8

I was receiving complains from other developers about flakiness.

Were the complaints about the lldb-amd64-ninja-netbsd8 bot specifically? Because if the tests are flaky on all NetBSD system, regardless of hardware and load, then we should find out why.
If it's just the one bot under load, we should target our fix to that.

I've grepped FreeBSD, OpenBSD and pkgsrc.

Reverted in r350477.

I'm going to revert it soon and see what happens. Please keep an eye on http://lab.llvm.org:8011/builders/lldb-amd64-ninja-netbsd8

In D56064#1347394, @EricWF wrote:

I would rather revert this, and fix the flaky tests. And if there are flaky tests that aren't marked flaky then we need to do that.

In D56215#1346342, @arichardson wrote:

In D56215#1346061, @mgorny wrote:

For the record, another option is to actually fix other software not to call LD directly.

Or if you really need to call the linker directly without specifying search paths you could also install a shell script as /usr/bin/ld that passes the appropriate flags to /usr/bin/ld.lld?

I don't think ifdefs in the source code are a good idea. Cross linking should just work as expected. But you could look at the OS field in the first input ELF file to choose default config options/a different emulation for NetBSD.

The approach we are using in CheriBSD to differentiate between MIPS and CHERI pure-capability to either pass an explicitly -m elf_btsmip_cheri_fbsd/ -m elf_btsmip_fbsd or infer whether it is CHERI or MIPS based on the e_flags field of the first input file.

In D56215#1345845, @ruiu wrote:

Not sure what I understand the point, but as to make lld work on/for NetBSD, I wonder if you can just add -L<path> to the command line in the compiler driver. Isn't a NetBSD triple passed to the compiler driver? If so, I wonder if you can add a few extra options when invoking the linker.

Actually I think that we can handle two cases witch a combination of proposals:

• native mode
• cross mode

In D56215#1345695, @ruiu wrote:

In D56215#1345417, @joerg wrote:

Talking from the perspective of having had to deal with thousands of packages in pkgsrc over the years: it is naive to believe that there isn't a lot of software that calls the linker directly for various reasons. As such, it is very important to have a useful configuration in the linker by default. Such a configuration is by its very nature target specific. This doesn't mean it can't be done in a cross-compiler friendly way, on the contrary. Over the years NetBSD has been pushing its toolchain to be as similar for the native build and a cross-target as reasonable possible. Modulo the build time choices in the config.h sense, the only difference between the native and cross tools is the built-in default of the former.

It might be naive but I don't think it's too naive. Most programs use ld via cc, and I don't think it is too unreasonable to not implement a host-specific logic for a very small percentage of programs that directly use ld. If we do, that logic would be the same or very similar to the one that we already have in cc. I think we should avoid that duplication of the host-specific config in the toolchain.

I see there are pros and cons to not have host-specific config in ld. That said, if other operating systems can live without host-specific config in lld, why can't NetBSD do? I really don't like to be in a situation where only one operating system have a host-specific config while others don't.

On Thu, Jan 03, 2019 at 08:31:53PM +0000, Kamil Rytarowski via Phabricator via cfe-commits wrote:

krytarowski added a comment.

On 03.01.2019 21:19, Joerg Sonnenberger wrote:

On Thu, Jan 03, 2019 at 06:34:22PM +0000, Kamil Rytarowski via Phabricator via cfe-commits wrote:

krytarowski added a comment.

Actually I find it frustrating and unfair as GNU ld doesn't have builtin

knowledge either.. it's passed with gross 'super hack' comments from build scripts... but we are forced to push it to lld in order to move on.

I'm puzzled. Seriously, when was the last time you actually checked how

much customization contains on a per-OS base in GNU ld? Yes, I'm
including the various build scripts because GNU ld is generally build by
a mix of hard-coded logic in the tool itself and various adjustments in
the linker scripts it is shipped with. But they are all a builtin part
of GNU ld as far as the end user is concerned. It is pretty much
irrelevant from a point of functionality where that logic is, but
skipping is a serious usability issue.

Joerg

I'm referring to code '/usr/src/external/gpl3/binutils/usr.bin/ld/Makefile':

I think that ifdefining the linker options with __NetBSD__ is no-go.

We want to keep it disabled by default on NetBSD.. but it would be to keep dynamic detection of target NetBSD rather than hardcoded ifdef.

On 03.01.2019 21:19, Joerg Sonnenberger wrote:

On Thu, Jan 03, 2019 at 06:34:22PM +0000, Kamil Rytarowski via Phabricator via cfe-commits wrote:

krytarowski added a comment.

Actually I find it frustrating and unfair as GNU ld doesn't have builtin
knowledge either.. it's passed with gross 'super hack' comments from build scripts... but we are forced to push it to lld in order to move on.

I'm puzzled. Seriously, when was the last time you actually checked how
much customization contains on a per-OS base in GNU ld? Yes, I'm
including the various build scripts because GNU ld is generally build by
a mix of hard-coded logic in the tool itself and various adjustments in
the linker scripts it is shipped with. But they are all a builtin part
of GNU ld as far as the end user is concerned. It is pretty much
irrelevant from a point of functionality where that logic is, but
skipping is a serious usability issue.

Joerg

I'm referring to code '/usr/src/external/gpl3/binutils/usr.bin/ld/Makefile':