This is an archive of the discontinued LLVM Phabricator instance.

[clangd] JSON <-> XPC conversions
AbandonedPublic

Authored by jkorous on Jun 25 2018, 1:05 PM.

Details

Summary

This is a self-contained pair of utility functions for the XPC transport layer.

It's not dependent on but following the refactoring patch:
https://reviews.llvm.org/D48559

Diff Detail

Event Timeline

jkorous created this revision.Jun 25 2018, 1:05 PM
sammccall accepted this revision.Jul 16 2018, 2:36 AM
sammccall added a subscriber: sammccall.

Sorry it took so long for these to get attention.
Starting here because it's simple and helps me understand the bigger patches....
This one looks good, really just nits.

clangd/xpc/CMakeLists.txt
1

I think it'd be helpful for some of us confused linux people to have a brief README in this directory giving some context about XPC in clangd.

Anything you want to write about the architecture is great but the essentials would be:

  • alternative transport layer to JSON-RPC
  • semantically equivalent, uses same LSP request/response message types (?)
  • MacOS only. Feature is guarded by CLANGD_BUILD_XPC_SUPPORT, including whole xpc/ dir.
15

this is done :-)

xpc/XPCJSONConversions.cpp
22

The underlying JSON library actually has both int64_t and double representations, and I see XPC has both as well.
If double works for all the data sent over this channel, then this approach is simplest. But for completeness, this is also possible:

if (auto I = json.asInteger())
  return xpc_int64_create(*I);
return xpc_double_create(*json.asNumber());

(I don't know of any reason clangd would use large integers today unless they arrived as incoming JSON-RPC request IDs or similar)

40

(you could also pre-size the vector, or use a deque)

62

hmm, I think this shouldn't compile anymore, rather require you to explicitly do the narrowing conversion to int64 or double.

69

this looks like objC syntax, I'm not familiar with it, but should this file be .mm?

Alternatively, it seems like you could write a C++ loop with xpc_array_get_value (though I don't know if there are performance concerns).

(and dictionary)

87

there are other data types (error, fd, ...) and this data presumably comes over a socket or something, so you may not actually want UB here.
Consider an assert and returning json::Expr(nullptr)

xpc/XPCJSONConversions.h
1

nit: header names wrong file.

1

consider just calling this Conversion.h or similar - it's already in the xpc/ dir and, it seems like XPC conversions that weren't JSON-related would be likely to go here anyhow?

19

you could consider calling these json::Expr toJSON(const xpc_object_t&) and bool fromJSON(const json::Expr&, xpc_object_t&) to fit in with the json lib's convention - not sure if it actually buys you anything though.

(If so they should be in the global namespace so ADL works)

This revision is now accepted and ready to land.Jul 16 2018, 2:36 AM
jkorous marked an inline comment as done.Jul 16 2018, 1:47 PM

Hi Sam, no worries! Thanks for making time to take a look! I am going to rebase all my patches on top of JSON library moved to llvm/Support and process your comments.

clangd/xpc/CMakeLists.txt
1

That's a good idea, thanks.

15

Yes, I know - thanks for the good work with lifting the JSON library! I am going to rebase my patches.

xpc/XPCJSONConversions.cpp
22

I was initially thinking along the same line but got stuck on the fact that JSON standard (modelled by llvm::json::Value) doesn't distinguish numeric types (it only has Number).

Relevant part of llvm::json::Value interface is:

llvm::Optional<double> getAsNumber() const {
    if (LLVM_LIKELY(Type == T_Double))
      return as<double>();
    if (LLVM_LIKELY(Type == T_Integer))
      return as<int64_t>();
    return llvm::None;
  }
  // Succeeds if the Value is a Number, and exactly representable as int64_t.
  llvm::Optional<int64_t> getAsInteger() const {
    if (LLVM_LIKELY(Type == T_Integer))
      return as<int64_t>();
    if (LLVM_LIKELY(Type == T_Double)) {
      double D = as<double>();
      if (LLVM_LIKELY(std::modf(D, &D) == 0.0 &&
                      D >= double(std::numeric_limits<int64_t>::min()) &&
                      D <= double(std::numeric_limits<int64_t>::max())))
        return D;
    }
    return llvm::None;
  }

while both

enum ValueType

and

template<typename T> T &as() const

are private which makes sense since those are implementation details.

Basically it seems to me there's no reliable way how to tell if Value is double or not (my example is 2.0) which means we couldn't preserve the "static" numeric type (it would be value dependent). I don't think changing the JSON library because of this would be a good idea. It's not a big deal for us and it's possible we'll skip JSON completely in XPC transport layer in the future.

40

Ok, I'll preallocate necessary space. I remember Bjarne Stroustrup saying he doesn't bother with reserve() anymore but it (most probably) won't do any harm here.

http://www.stroustrup.com/bs_faq2.html#slow-containers
"People sometimes worry about the cost of std::vector growing incrementally. I used to worry about that and used reserve() to optimize the growth. After measuring my code and repeatedly having trouble finding the performance benefits of reserve() in real programs, I stopped using it except where it is needed to avoid iterator invalidation (a rare case in my code). Again: measure before you optimize."

69

Oh, sorry. These are actually C blocks - a nonstandard C extension.
https://en.wikipedia.org/wiki/Blocks_(C_language_extension)
https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Blocks/Articles/00_Introduction.html

There was no performance concern on my side - just liked this approach better. Didn't realize how confusing it might be, will rewrite this.

87

That's a good point. Thanks!

xpc/XPCJSONConversions.h
1

Good catch, thanks!
All right, I don't have any strong opinion about the filename so might as well rename it.

19

I don't have any strong opinion about this - just that I would slightly prefer consistency. I will process other comments first and decide in the meantime.

jkorous marked 12 inline comments as done.Jul 19 2018, 7:50 AM
jkorous added inline comments.
xpc/XPCJSONConversions.cpp
62

Explicitly narroving now. Thanks.

69

Allright, by trying to get rid of C block in dictionary conversion I remembered that there's unfortunately no sane reason how to iterate XPC dictionary without using xpc_dictionary_apply() which uses C block for functor parameter.

https://developer.apple.com/documentation/xpc/1505404-xpc_dictionary_apply?language=objc
https://developer.apple.com/documentation/xpc/xpc_dictionary_applier_t?language=objc

Anyway, even had we succeeded in removing C blocks from conversion they still would be necessary for dispatch as xpc_connection_set_event_handler() is another part of XPC API that uses it.

I guess that there's no point in removing the xpc_array_apply() then. Is that ok with you?

jkorous marked 7 inline comments as done.Jul 19 2018, 8:19 AM

BTW Just for the record - I put the rebased & updated patch here: [clangd] XPC WIP https://reviews.llvm.org/D49548

jkorous planned changes to this revision.Jul 25 2018, 6:41 AM
jkorous abandoned this revision.Apr 23 2019, 5:29 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 23 2019, 5:29 PM
Herald added a subscriber: kadircet. · View Herald Transcript