LGTM, Nice work John.

LGTM but waiting on Akira would be nice imho.

@ahatanak how does this diff look to you?

In D86049#3821183, @plotfi wrote:

In D86049#3819109, @mwyman wrote:

In D86049#3818981, @plotfi wrote:

@ahatanak I can revive some of what I was working on from https://reviews.llvm.org/D86049?id=285923 if we think we need a thunk for the checks as @rjmccall mentioned.

I believe the generated direct methods already handle the null checks and class init in CGObjCCommonMac::GenerateDirectMethodPrologue, meaning the thunks aren't strictly necessary for the callee to handle them.

Could the thunks instead allow us to have publicly-visible mangled names (something akin to the new selector stubs _objc_msgSend$selectorName but for _objc_direct$Class_selectorName) while leaving the actual impl name alone, letting the stack traces see normal ObjC symbol names?

I think the square brackets are still problematic for linking, so is LLVM's handling of \01 (I believe).

In D86049#3819109, @mwyman wrote:

In D86049#3818981, @plotfi wrote:

@ahatanak I can revive some of what I was working on from https://reviews.llvm.org/D86049?id=285923 if we think we need a thunk for the checks as @rjmccall mentioned.

I believe the generated direct methods already handle the null checks and class init in CGObjCCommonMac::GenerateDirectMethodPrologue, meaning the thunks aren't strictly necessary for the callee to handle them.

Could the thunks instead allow us to have publicly-visible mangled names (something akin to the new selector stubs _objc_msgSend$selectorName but for _objc_direct$Class_selectorName) while leaving the actual impl name alone, letting the stack traces see normal ObjC symbol names?

@ahatanak I can revive some of what I was working on from https://reviews.llvm.org/D86049?id=285923 if we think we need a thunk for the checks as @rjmccall mentioned.

In D86049#3818923, @ahatanak wrote:

In D86049#3818696, @plotfi wrote:

In D86049#3818435, @mwyman wrote:

In D86049#3816006, @plotfi wrote:

@dmaclach @ahatanak @mwyman How do things look from here? Do you want something for properties as well or would it be ok if we did this in a later commit?

Huh, for some reason I thought when I'd last poked at using the visibility attribute it wasn't allowed on ObjC methods, which is why I'd suggested adding the enum on objc_direct, but as that no longer appears to be the case yes I like this approach better.

@dmaclach @mwyman I am also very happy with the fact that we can just reuse the regular visibility attribute. In the future we can decide on revised behavior for hasMethodVisibilityDefault.

@ahatanak Do you have feedback on this?

The visibility attribute changes look good to me.

Do we have the answers to the last two questions John raised in https://reviews.llvm.org/D86049#2255738? I think we should get it right the first time since, once we make the direct methods visible, it'd be hard to change where the null check or class initialization is done since that would be an ABI change. Should we run experiments to measure the impact on code size?

In D86049#3818435, @mwyman wrote:

In D86049#3816006, @plotfi wrote:

@dmaclach @ahatanak @mwyman How do things look from here? Do you want something for properties as well or would it be ok if we did this in a later commit?

Huh, for some reason I thought when I'd last poked at using the visibility attribute it wasn't allowed on ObjC methods, which is why I'd suggested adding the enum on objc_direct, but as that no longer appears to be the case yes I like this approach better.

@dmaclach @ahatanak @mwyman How do things look from here? Do you want something for properties as well or would it be ok if we did this in a later commit?

Updated based on @ahatanak's feedback.

In D86049#3814111, @ahatanak wrote:

In D86049#3812412, @plotfi wrote:

In D86049#3812068, @ahatanak wrote:

In D86049#3806898, @plotfi wrote:

1. Do we change the existing visibility behavior of objc methods? Yes / No

I don't think we want to change the existing visibility behavior of non-direct objc methods. Is there a use reason or use case for making them visible outside the linkage unit?

2. If we leave hidden as the default do we change the behavior for objc_direct? Yes / No

I think direct methods shouldn't be hidden by default (i.e., they should get the default visibility). But it's not clear to me whether we should make that change right away as I've heard concerns from people internally. I think I need more time to understand what exactly their concerns are.

3. If we leave objc_direct as hidden by default do we expand the existing objc_direct attr to have the enum as you said so __attribute__((objc_direct("visible"))) or do we add a new attr as I have done so far?

I wasn't sure why it wasn't possible to use the existing __attribute__((visibility("default"))) attribute. Is it not possible to make only the direct methods get the default visibility?

This is not possible because default visibility is implicit (as far as I understand). It can not be checked if __attribute__((visibility("default"))) is set because it is always set unless -fvisibility=hidden is passed on the command line. So either we treat direct methods like everything else, and hide them when __attribute__((visibility("hidden"))) or the command line to hide everything by default is used, or we need a new attr or a new enum on the existing objc_direct attr.

Does this make sense or am I missing some details?

But there are ways to check whether the user explicitly added a visibility attribute (e.g., __attribute__((visibility("default")))), right? For example, NamedDecl::getExplicitVisibility.

I'm just not sure whether we want to add support for a new attribute like __attribute__((objc_direct("default"))) since it seems equivalent to __attribute__((objc_direct, visibility("default"))).

In D86049#3812068, @ahatanak wrote:

In D86049#3806898, @plotfi wrote:

1. Do we change the existing visibility behavior of objc methods? Yes / No

I don't think we want to change the existing visibility behavior of non-direct objc methods. Is there a use reason or use case for making them visible outside the linkage unit?

2. If we leave hidden as the default do we change the behavior for objc_direct? Yes / No

I think direct methods shouldn't be hidden by default (i.e., they should get the default visibility). But it's not clear to me whether we should make that change right away as I've heard concerns from people internally. I think I need more time to understand what exactly their concerns are.

3. If we leave objc_direct as hidden by default do we expand the existing objc_direct attr to have the enum as you said so __attribute__((objc_direct("visible"))) or do we add a new attr as I have done so far?

I wasn't sure why it wasn't possible to use the existing __attribute__((visibility("default"))) attribute. Is it not possible to make only the direct methods get the default visibility?

In D86049#3812068, @ahatanak wrote:

In D86049#3806898, @plotfi wrote:

1. Do we change the existing visibility behavior of objc methods? Yes / No

I don't think we want to change the existing visibility behavior of non-direct objc methods. Is there a use reason or use case for making them visible outside the linkage unit?

2. If we leave hidden as the default do we change the behavior for objc_direct? Yes / No

I think direct methods shouldn't be hidden by default (i.e., they should get the default visibility). But it's not clear to me whether we should make that change right away as I've heard concerns from people internally. I think I need more time to understand what exactly their concerns are.

3. If we leave objc_direct as hidden by default do we expand the existing objc_direct attr to have the enum as you said so __attribute__((objc_direct("visible"))) or do we add a new attr as I have done so far?

I wasn't sure why it wasn't possible to use the existing __attribute__((visibility("default"))) attribute. Is it not possible to make only the direct methods get the default visibility?

I think the decision tree goes

@ahatanak @mwyman @rjmccall @dmaclach I am going to work on a version of this patch where the visibility can be encoded into the objc_direct attribute. Does that seem reasonable to do from here?

In D131424#3803190, @mwyman wrote:

Hi Akira,

I'd reached out to John offline and he'd mentioned you might be able to help on some of these objc_direct reviews; if so, that would be wonderful!

-Michael

Adding a test case to cover @protocol methods not being allowed to contain direct

Update test.

Updated to use mangleObjCMethodName in clang/lib/AST/Mangle.cpp to set the _+<a: > direct method name.

Updated with some of @mwyman's feedback.

Updated D86049.

Updating implementation to use an objc_direct_visible attr to explicitly mark when we want objc_direct to be exposed outside of the link unit.

I tried running the following on some example code and got a stacktrace:

In D86049#3663726, @mwyman wrote:

Hi, I work with @dmaclach. I know this has been sitting around without much activity for two years, but we believe there is a solid use-case for this on our end and I'd like to help get the ABI nailed down and land this change (or one accomplishing the same goal). Do you still have interest in this?

In D128688#3619734, @smeenai wrote:

In D128688#3619638, @plotfi wrote:

In D128688#3619619, @MaskRay wrote:

@plotfi Do you have zlib-gnu usage?

Hi all thanks for reaching out. I will confirm with some folks who I wrote this zlib-gnu support for to confirm it is not needed.

We don't need zlib-gnu anymore.

In D128688#3619619, @MaskRay wrote:

@plotfi Do you have zlib-gnu usage?

Thanks for the feedback on corner cases @aaron.ballman, this will give me more concrete things to think about here.

This is pretty incomplete. Please ignore for now.

@rjmccall would you be able to review this patch? What do you think of this approach? This change is to support C++-Interop on the Swift side.

In D86049#2844766, @dmaclach wrote:

Has anything happened with this at all or did it get dropped?

LGTM once you've addressed @phosek's nits.

I will take another look over this commit in the morning. Overall looks good.

@jkorous thoughts?

This makes sense to me. I approve. Can we move the 3/60 seconds number to a const int value set somewhere higher up in the file as a global with a comment explaining this as well?

If @compnerd has no gripes go ahead and land. LGTM.
Before landing, please make sure to run the check-clang tests as well because the interface stubs generation on the clang driver side invokes llvm-ifs.

Ping? Haven't seen any chatter in a while. Anything new here? I don't want to hold you folks up.

@wolfgangp I get this warning on macOS. I noticed it wasn't fixed in https://github.com/llvm/llvm-project/commit/c6e8f81410a2942b5abd112aa6e468268e01d946#diff-67f7be4aff998787c9c4efcb95e5d0dc225e1dcd37b9b005a3db5493f02efdc3 so I posted this small diff.

Is this still open? Seems good to go.

In D94461#2494710, @compnerd wrote:

You mean use armv7---elf as a triple? While that is possible, keep in mind that it might change the ABI of the generated interfaces. While it is unlikely to matter as the ifso cannot be used at runtime, if the linker attempts to do something like symbolic resolution of ABI dependent specialization (e.g. strcmp.eabi vs strcmp where the latter is generic), that could matter. However, adding support to explicitly override the encoded triple based on the user's input is reasonable (though might deserve a warning that the triple is being overridden).

In D94461#2494667, @compnerd wrote:

The full triple is needed in some cases: armv7-unknown-linux-gnueabi vs armv7-unknown-linux-gnueabihf vs armv7-unknown-linux-eabihf vs armv7-unknown-linux-eabi are all different. Additionally, there is nothing that prevents you from doing something like: armv7-unknown-linux-gnueabihf-coff or armv7-unknown-linux-gnueabihf-macho to generate COFF or MachO object files that are reading in additional bits of ABI information and the libc from the target triple. I really think that preserving the target triple is best.

In D94461#2492369, @phosek wrote:

In D94461#2491947, @plotfi wrote:

In D94461#2491946, @plotfi wrote:

1. I think for unification, the first thing would be unifying the text abi file format. Currently, the YAML format used by ifs is very similar to the one used in InterfaceStub(elfabi), however there are not compatible. One thing is that ifs embeds llvm triples in the text ifs file while elfabi does not do that. Elfabi get platform and endianness information from command line instead of getting them from the text stub file. We did that because there are use cases that we generate ELF stubs for multiple platforms using a single set of text abi files. Is there a specific reason why ifs text files require llvm triples? Is it feasible to make llvm-ifs to get the platform information from the command line options?

The triple-ness was added because originally IFS was intended to be a tool that the clang driver invokes. The idea was that you build some project with IFS enabled and you automatically get the stubs for shipping or for CI purposes to track what new exposure was done in recent builds of said project. I think it would be ok to add a flag to override the triple if that isn't there already. @compnerd
thoughts?

I meant to add, as part of llvm-ifs being invoked by clang the idea was that clang would just forward its triple and llvm-ifs would decide what to do with it because we wanted the ifs format to be kinda platform agnostic but we wanted llvm-ifs to potentially be able to generate stubs for multiple platforms in the future (like MS .libs and tbds etc too).

Would you need a full triple in that case? Other binary tools like linkers doesn't use a full triple either, they use just e_machine and e_ident. Would that be sufficient?

In D94461#2491946, @plotfi wrote:

1. I think for unification, the first thing would be unifying the text abi file format. Currently, the YAML format used by ifs is very similar to the one used in InterfaceStub(elfabi), however there are not compatible. One thing is that ifs embeds llvm triples in the text ifs file while elfabi does not do that. Elfabi get platform and endianness information from command line instead of getting them from the text stub file. We did that because there are use cases that we generate ELF stubs for multiple platforms using a single set of text abi files. Is there a specific reason why ifs text files require llvm triples? Is it feasible to make llvm-ifs to get the platform information from the command line options?

The triple-ness was added because originally IFS was intended to be a tool that the clang driver invokes. The idea was that you build some project with IFS enabled and you automatically get the stubs for shipping or for CI purposes to track what new exposure was done in recent builds of said project. I think it would be ok to add a flag to override the triple if that isn't there already. @compnerd
thoughts?

1. I think for unification, the first thing would be unifying the text abi file format. Currently, the YAML format used by ifs is very similar to the one used in InterfaceStub(elfabi), however there are not compatible. One thing is that ifs embeds llvm triples in the text ifs file while elfabi does not do that. Elfabi get platform and endianness information from command line instead of getting them from the text stub file. We did that because there are use cases that we generate ELF stubs for multiple platforms using a single set of text abi files. Is there a specific reason why ifs text files require llvm triples? Is it feasible to make llvm-ifs to get the platform information from the command line options?

Updated based on @paquette's feedback. This only includes the StableHashTree data structure and a unit test.

Nice. Will you folks be updating llvm-ifs to use this support instead of yaml2obj soon? If so add me to the review, I will be happy to take a look.

Thanks for fixing this @cishida!

I think I am overall ok with this for now until the TBE folks post some more patches to integrate ifs and tbe.

@vsk any update on this? Is there anything we can do to help in landing this patch in llvm-project/main?

Overall looks good.

comments added

clang-tidy

spelling and grammar

Cleaning up patch to be easier to understand.

@sdesmalen @efriedma @dmgreen Hi Sander, Eli, Dave. Would any of you have some time to help review this Prolog Epilog Size optimization patch? Much appreciated if you do. Me and @kyulee would be available to chat on IRC or discord about it for more info.

@thegameg: @paquette tells be you might have some ideas on a better format than this json business going on here (based on work you've done on remarks). What do you think?

@mpaszkowski ping

In D86049#2253705, @plotfi wrote:

I've updated the diff to reflect the alternate non-wrapper exposure. This way requires the sanitizing of the exported objc_direct method name.

In D86952#2254880, @paquette wrote:

Oh wait the tests already test what I was concerned about. LGTM with minor nits.

(although decoupling the tests from mir-canon would be nice, since this is supposed to be standalone)

In D86952#2254859, @paquette wrote:

Is there any way to test the hash itself on certain instructions?

e.g. verify that two instructions with different address spaces get a different hash?