Page MenuHomePhabricator

[WIP] Add concurrent packets support to gdb-remote client
AbandonedPublic

Authored by labath on Jul 28 2016, 5:39 AM.

Details

Reviewers
clayborg
Summary

This change adds the ability to the client to send multiple packets without waiting for the
response to the first one. The individual senders use a queue to register themselves when they
have sent a packet. When they reach the front of the queue it's their turn to read a packet.

This does not change the protocol in any way -- a well-behaved stub which just sits in a
ReadPacketAndSendResponse() loop will not notice any difference. It does however make a huge
difference on connections with a non-negligible latency (anything over 10ms certainly fits the
bill), because we will have multiple packets in flight at the same time. The queueing system
avoids the need for a separate thread doing the reading/writing and should add no overhead to the
common case of sequential packet streams.

No code except the unit test actually uses the new functionality yet, as the default is for all
senders to acquire an exclusive connection lock. Any packets using this will need to be enabled
on a one-off basis, making sure that there can be no bad interactions between other packets which
can get sent concurrently. Some obvious cases where this can *not* be enabled are:

  • AckMode: when using unreliable connections, we can never be sure that the other side has recieved our packet and we may need to do retransmissions.
  • packets which need to be sent as an atomic sequence (e.g. if we don't have thread suffix enabled, then we need to send Hg plus another packet together).

The first packet which I would try to enable is qModuleInfo, which is a read-only packet with no
side-effects, and we need to send a lot of them (over 150 when attaching to a typical Android
application), so it can give a big speed boost there. However, I can see this being useful in
other places as well (parallel thread backtrace computation?).

To make the code thread-sanitizer clean, I've needed to add a lock to the packet history class,
as now we can be sending and receiving a packet simultaneously.

  • WIP ----------

This code does not actually work yet (apart from the unittests), because the read/write lock does
not like to be locked recursively. I'll need to weed those out first (which should be pretty
orthogonal to the changes here).

  • WIP ----------

Diff Detail

Event Timeline

labath updated this revision to Diff 65917.Jul 28 2016, 5:39 AM
labath retitled this revision from to [WIP] Add concurrent packets support to gdb-remote client.
labath updated this object.
labath added a reviewer: clayborg.
labath added a subscriber: lldb-commits.
clayborg edited edge metadata.Jul 28 2016, 10:49 AM

A few things that worry me:

  • There are no sequence IDs on any packets so when receiving responses, I am not sure how you are going to match up packets with their responses especially when you are sending from multiple threads. I didn't look through your implementation, but I am guessing you didn't add that?
  • What not just make a qModulesInfo that gets all module info from a process and avoid the many callbacks? When we were reducing packets for Apple Watch we did a lot of consolidation packets where we get all of the data we needed in fewer packets (we only get 22-50 packet per second on the watch!).

So I am kind of against this patch because there is no way to take advantage of it from a API standpoint. Take qModuleInfo for example. ProcessGDBRemote::GetModuleSpec() and PlatformRemoteGDBServer::GetModuleSpec() are the only two callers of this. How would you ever take advantage of sending multiple packets at the same time? Make the access multi-threaded? I was sooner see a new functions that get more than one module spec at a time:

ProcessGDBRemote::GetModuleSpecs(...)
PlatformRemoteGDBServer::GetModuleSpecs(...)

Then this could call the qModulesInfo (note the extra "s") with all of the module infos encoded as JSON, and the reply can contain all module infos needed.

As soon as we start adding sequence numbers and extra stuff, this stops becoming GDB remote protocol and becomes our own. We might as well switch over to a new JSON based protocol that fixes all of the issues with GDB remote...

I am going to put my responses inilne:

A few things that worry me:

  • There are no sequence IDs on any packets so when receiving responses, I am not sure how you are going to match up packets with their responses especially when you are sending from multiple threads. I didn't look through your implementation, but I am guessing you didn't add that?

I did not add packet sequence number, but I don't think they are actually necessary. Matching requests with responses is easy. The responses have to come in the order in which the requests were sent. So the thread which first sent the request will be the first one to read the response. After it is done, the next one gets a chance to read it's response. The whole implementation is very straight forward (see SendPacketAndWaitForResponseNoLock) and completely transparent to the server.

The interactions between this and the packet verifier and echo synchronization features you have added (we don't actually rely on those) are a bit more complicated and we can discuss how exactly they should interact, but I am confident it can be done in a way they do not interfere.

  • What not just make a qModulesInfo that gets all module info from a process and avoid the many callbacks? When we were reducing packets for Apple Watch we did a lot of consolidation packets where we get all of the data we needed in fewer packets (we only get 22-50 packet per second on the watch!).

    So I am kind of against this patch because there is no way to take advantage of it from a API standpoint. Take qModuleInfo for example. ProcessGDBRemote::GetModuleSpec() and PlatformRemoteGDBServer::GetModuleSpec() are the only two callers of this. How would you ever take advantage of sending multiple packets at the same time? Make the access multi-threaded? I was sooner see a new functions that get more than one module spec at a time:

    ProcessGDBRemote::GetModuleSpecs(...) PlatformRemoteGDBServer::GetModuleSpecs(...)

    Then this could call the qModulesInfo (note the extra "s") with all of the module infos encoded as JSON, and the reply can contain all module infos needed.

I had considered this approach, and actually started implementing it, but then I sort of became less convinced that it is a good idea. The thing is that the place where I get the list of modules I want to query is buried inside DynamicLoaderPOSIXDYLD::LoadAllCurrentModules, which then calls DynamicLoader::LoadModuleAtAddress, which goes into Target::GetSharedModule, then Platform::GetSharedModule (which is a virtual function implemented in about 10 classes), and eventually ending in PlatformRemoteGDBServer::GetModuleSpecs. To pipe that information through I'd need all of these functions to process modules in bulk, instead of one-by-one. This is certainly possible, but it won't make the inferfaces very nice.

The other option I see would be to make some kind of a shortcut and have the DynamicLoader issue some kind of a "prefetch" call to the target/platform/whatever, which would trigger the qModulesInfo packet, and then we would go about as before, processing them individually. This also does not seem like a good interface, as it complicates the already quite complicated interactions between these classes. I guess these don't sound like very convincing arguments against these approaches, but I have several interesting reasons to try to go the other way.

Yes, my idea was to have the Dynamic Loader spin up some threads which will do the processing of individual modules in parallel. AFAICS, there is nothing about getting a module that would make it inherently serial, so it should be possible to achieve that. Granted, this will need to be done with a very steady hand, reviewing the assumptions the functions I mentioned are making, but I think that's doable, and I'm in no rush to get that done quickly.

What I like about his idea, is that it's not just limited to qModuleInfo packets. To find out which modules are loaded we traverse a linked list in the inferior memory. This is an inherently serial process, but if we have this, we could interleave traversing the list with processing the elements of the list that we have already found. Next, if you remember the parallel module _download_ discussion from the beginning of this year, I was able to obtain about 5x speed improvement by using this approach, so this would be a step towards that. The possibilities are very widely open. (Computing summaries of complicated frame variables in parallel, why not?) All without changing the protocol or even server used.

Additionally, although I have no data to back that up, I believe the pipelining approach will bring superior performance compared to the bulk mode. This is because the bulk mode creates one huge packet, which takes a lot of time to process (disk access), during which nothing is happening. Whereas here, you are overlapping work on the server, with network communication, with client workload.

As soon as we start adding sequence numbers and extra stuff, this stops becoming GDB remote protocol and becomes our own. We might as well switch over to a new JSON based protocol that fixes all of the issues with GDB remote...

There are things which I don't like about the gdb-remote protocol, but the (lack of) sequence numbers is not one of them. I saw that you are having a lot of problems with the packet sequences getting out of sync, which quite surprises me, as we haven't seen such issues with lldb-server. So actually, moving towards a different protocol is a non-goal for us (although we would be open to discussion if someone wanted to push that).

Let me know what you think.
PS: I will be away next week, so I may not respond until I am back.

labath added a comment.Aug 8 2016, 9:22 AM

Any thoughts on this, Greg?

I was out on vacation, sorry for the delay. I will get back to you later today.

One thing that should be tested before we go farther, can you check the packet speed test with and without this patch? We really need to make sure this doesn't regress. Adding thread synchronization above and beyond taking a mutex could really impact performance.

Just run to a breakpoint in a release build and type:

(lldb) process plugin packet speed-test

labath added a comment.Aug 9 2016, 9:54 AM

These are the results before and after applying this change (on linux):
Before:

(lldb) process plugin packet speed-test
Testing sending 1000 packets of various sizes:
qSpeedTest(send=0      , recv=0      ) in 0.057456000 sec for  17404.62 packets/sec (  0.057456 ms per packet) with standard deviation of   0.012368 ms
qSpeedTest(send=0      , recv=4      ) in 0.048495000 sec for  20620.68 packets/sec (  0.048495 ms per packet) with standard deviation of   0.002189 ms
qSpeedTest(send=0      , recv=8      ) in 0.048270000 sec for  20716.80 packets/sec (  0.048270 ms per packet) with standard deviation of   0.001979 ms
qSpeedTest(send=0      , recv=16     ) in 0.048686000 sec for  20539.79 packets/sec (  0.048686 ms per packet) with standard deviation of   0.001992 ms
qSpeedTest(send=0      , recv=32     ) in 0.050556000 sec for  19780.04 packets/sec (  0.050556 ms per packet) with standard deviation of   0.005152 ms
qSpeedTest(send=0      , recv=64     ) in 0.051095000 sec for  19571.39 packets/sec (  0.051095 ms per packet) with standard deviation of   0.004969 ms
qSpeedTest(send=0      , recv=128    ) in 0.056365000 sec for  17741.51 packets/sec (  0.056365 ms per packet) with standard deviation of   0.007733 ms
qSpeedTest(send=0      , recv=256    ) in 0.061426000 sec for  16279.75 packets/sec (  0.061426 ms per packet) with standard deviation of   0.003777 ms
qSpeedTest(send=0      , recv=512    ) in 0.065135000 sec for  15352.73 packets/sec (  0.065135 ms per packet) with standard deviation of   0.003205 ms
qSpeedTest(send=0      , recv=1024   ) in 0.070767000 sec for  14130.88 packets/sec (  0.070767 ms per packet) with standard deviation of   0.005090 ms
qSpeedTest(send=4      , recv=0      ) in 0.053847000 sec for  18571.14 packets/sec (  0.053847 ms per packet) with standard deviation of   0.003236 ms
qSpeedTest(send=4      , recv=4      ) in 0.054817000 sec for  18242.52 packets/sec (  0.054817 ms per packet) with standard deviation of   0.002795 ms
qSpeedTest(send=4      , recv=8      ) in 0.054378000 sec for  18389.79 packets/sec (  0.054378 ms per packet) with standard deviation of   0.002432 ms
qSpeedTest(send=4      , recv=16     ) in 0.054126000 sec for  18475.41 packets/sec (  0.054126 ms per packet) with standard deviation of   0.005307 ms
qSpeedTest(send=4      , recv=32     ) in 0.047904000 sec for  20875.08 packets/sec (  0.047904 ms per packet) with standard deviation of   0.003132 ms
qSpeedTest(send=4      , recv=64     ) in 0.051456000 sec for  19434.08 packets/sec (  0.051456 ms per packet) with standard deviation of   0.004501 ms
qSpeedTest(send=4      , recv=128    ) in 0.054608000 sec for  18312.34 packets/sec (  0.054608 ms per packet) with standard deviation of   0.003507 ms
qSpeedTest(send=4      , recv=256    ) in 0.054924000 sec for  18206.98 packets/sec (  0.054924 ms per packet) with standard deviation of   0.005625 ms
qSpeedTest(send=4      , recv=512    ) in 0.064180000 sec for  15581.18 packets/sec (  0.064180 ms per packet) with standard deviation of   0.005875 ms
qSpeedTest(send=4      , recv=1024   ) in 0.068276000 sec for  14646.44 packets/sec (  0.068276 ms per packet) with standard deviation of   0.003039 ms
qSpeedTest(send=8      , recv=0      ) in 0.053046000 sec for  18851.56 packets/sec (  0.053046 ms per packet) with standard deviation of   0.003500 ms
qSpeedTest(send=8      , recv=4      ) in 0.054089000 sec for  18488.05 packets/sec (  0.054089 ms per packet) with standard deviation of   0.006284 ms
qSpeedTest(send=8      , recv=8      ) in 0.049932000 sec for  20027.24 packets/sec (  0.049932 ms per packet) with standard deviation of   0.004150 ms
qSpeedTest(send=8      , recv=16     ) in 0.058777000 sec for  17013.46 packets/sec (  0.058777 ms per packet) with standard deviation of   0.008652 ms
qSpeedTest(send=8      , recv=32     ) in 0.051253000 sec for  19511.05 packets/sec (  0.051253 ms per packet) with standard deviation of   0.008256 ms
qSpeedTest(send=8      , recv=64     ) in 0.052551000 sec for  19029.13 packets/sec (  0.052551 ms per packet) with standard deviation of   0.006581 ms
qSpeedTest(send=8      , recv=128    ) in 0.049986000 sec for  20005.60 packets/sec (  0.049986 ms per packet) with standard deviation of   0.005448 ms
qSpeedTest(send=8      , recv=256    ) in 0.054557000 sec for  18329.46 packets/sec (  0.054557 ms per packet) with standard deviation of   0.006423 ms
qSpeedTest(send=8      , recv=512    ) in 0.052093000 sec for  19196.44 packets/sec (  0.052093 ms per packet) with standard deviation of   0.002142 ms
qSpeedTest(send=8      , recv=1024   ) in 0.057268000 sec for  17461.76 packets/sec (  0.057268 ms per packet) with standard deviation of   0.001820 ms
qSpeedTest(send=16     , recv=0      ) in 0.044685000 sec for  22378.87 packets/sec (  0.044685 ms per packet) with standard deviation of   0.001565 ms
qSpeedTest(send=16     , recv=4      ) in 0.045271000 sec for  22089.20 packets/sec (  0.045271 ms per packet) with standard deviation of   0.001261 ms
qSpeedTest(send=16     , recv=8      ) in 0.050808000 sec for  19681.94 packets/sec (  0.050808 ms per packet) with standard deviation of   0.007900 ms
qSpeedTest(send=16     , recv=16     ) in 0.051924000 sec for  19258.92 packets/sec (  0.051924 ms per packet) with standard deviation of   0.005277 ms
qSpeedTest(send=16     , recv=32     ) in 0.049635000 sec for  20147.07 packets/sec (  0.049635 ms per packet) with standard deviation of   0.001714 ms
qSpeedTest(send=16     , recv=64     ) in 0.049719000 sec for  20113.04 packets/sec (  0.049719 ms per packet) with standard deviation of   0.001488 ms
qSpeedTest(send=16     , recv=128    ) in 0.051774000 sec for  19314.71 packets/sec (  0.051774 ms per packet) with standard deviation of   0.002255 ms
qSpeedTest(send=16     , recv=256    ) in 0.053197000 sec for  18798.05 packets/sec (  0.053197 ms per packet) with standard deviation of   0.001695 ms
qSpeedTest(send=16     , recv=512    ) in 0.058951000 sec for  16963.24 packets/sec (  0.058951 ms per packet) with standard deviation of   0.005022 ms
qSpeedTest(send=16     , recv=1024   ) in 0.061830000 sec for  16173.38 packets/sec (  0.061830 ms per packet) with standard deviation of   0.002527 ms
qSpeedTest(send=32     , recv=0      ) in 0.047818000 sec for  20912.63 packets/sec (  0.047818 ms per packet) with standard deviation of   0.002138 ms
qSpeedTest(send=32     , recv=4      ) in 0.048937000 sec for  20434.44 packets/sec (  0.048937 ms per packet) with standard deviation of   0.002160 ms
qSpeedTest(send=32     , recv=8      ) in 0.049310000 sec for  20279.86 packets/sec (  0.049310 ms per packet) with standard deviation of   0.001967 ms
qSpeedTest(send=32     , recv=16     ) in 0.049303000 sec for  20282.74 packets/sec (  0.049303 ms per packet) with standard deviation of   0.001901 ms
qSpeedTest(send=32     , recv=32     ) in 0.048884000 sec for  20456.59 packets/sec (  0.048884 ms per packet) with standard deviation of   0.002039 ms
qSpeedTest(send=32     , recv=64     ) in 0.049960000 sec for  20016.01 packets/sec (  0.049960 ms per packet) with standard deviation of   0.002190 ms
qSpeedTest(send=32     , recv=128    ) in 0.051732000 sec for  19330.39 packets/sec (  0.051732 ms per packet) with standard deviation of   0.002192 ms
qSpeedTest(send=32     , recv=256    ) in 0.053717000 sec for  18616.08 packets/sec (  0.053717 ms per packet) with standard deviation of   0.002260 ms
qSpeedTest(send=32     , recv=512    ) in 0.056674000 sec for  17644.78 packets/sec (  0.056674 ms per packet) with standard deviation of   0.002280 ms
qSpeedTest(send=32     , recv=1024   ) in 0.061923000 sec for  16149.09 packets/sec (  0.061923 ms per packet) with standard deviation of   0.002552 ms
qSpeedTest(send=64     , recv=0      ) in 0.048675000 sec for  20544.43 packets/sec (  0.048675 ms per packet) with standard deviation of   0.002477 ms
qSpeedTest(send=64     , recv=4      ) in 0.050288000 sec for  19885.46 packets/sec (  0.050288 ms per packet) with standard deviation of   0.004986 ms
qSpeedTest(send=64     , recv=8      ) in 0.049818000 sec for  20073.07 packets/sec (  0.049818 ms per packet) with standard deviation of   0.001913 ms
qSpeedTest(send=64     , recv=16     ) in 0.049816000 sec for  20073.87 packets/sec (  0.049816 ms per packet) with standard deviation of   0.001994 ms
qSpeedTest(send=64     , recv=32     ) in 0.049917000 sec for  20033.26 packets/sec (  0.049917 ms per packet) with standard deviation of   0.002194 ms
qSpeedTest(send=64     , recv=64     ) in 0.050215000 sec for  19914.37 packets/sec (  0.050215 ms per packet) with standard deviation of   0.002086 ms
qSpeedTest(send=64     , recv=128    ) in 0.051921000 sec for  19260.03 packets/sec (  0.051921 ms per packet) with standard deviation of   0.002006 ms
qSpeedTest(send=64     , recv=256    ) in 0.053772000 sec for  18597.04 packets/sec (  0.053772 ms per packet) with standard deviation of   0.002288 ms
qSpeedTest(send=64     , recv=512    ) in 0.056476000 sec for  17706.64 packets/sec (  0.056476 ms per packet) with standard deviation of   0.002386 ms
qSpeedTest(send=64     , recv=1024   ) in 0.063685000 sec for  15702.28 packets/sec (  0.063685 ms per packet) with standard deviation of   0.005601 ms
qSpeedTest(send=128    , recv=0      ) in 0.050048000 sec for  19980.82 packets/sec (  0.050048 ms per packet) with standard deviation of   0.003461 ms
qSpeedTest(send=128    , recv=4      ) in 0.050134000 sec for  19946.54 packets/sec (  0.050134 ms per packet) with standard deviation of   0.002229 ms
qSpeedTest(send=128    , recv=8      ) in 0.056629000 sec for  17658.80 packets/sec (  0.056629 ms per packet) with standard deviation of   0.008926 ms
qSpeedTest(send=128    , recv=16     ) in 0.053294000 sec for  18763.84 packets/sec (  0.053294 ms per packet) with standard deviation of   0.007024 ms
qSpeedTest(send=128    , recv=32     ) in 0.053769000 sec for  18598.08 packets/sec (  0.053769 ms per packet) with standard deviation of   0.003904 ms
qSpeedTest(send=128    , recv=64     ) in 0.053158000 sec for  18811.84 packets/sec (  0.053158 ms per packet) with standard deviation of   0.004076 ms
qSpeedTest(send=128    , recv=128    ) in 0.054059000 sec for  18498.31 packets/sec (  0.054059 ms per packet) with standard deviation of   0.004584 ms
qSpeedTest(send=128    , recv=256    ) in 0.055685000 sec for  17958.16 packets/sec (  0.055685 ms per packet) with standard deviation of   0.004079 ms
qSpeedTest(send=128    , recv=512    ) in 0.058741000 sec for  17023.88 packets/sec (  0.058741 ms per packet) with standard deviation of   0.004337 ms
qSpeedTest(send=128    , recv=1024   ) in 0.067375000 sec for  14842.30 packets/sec (  0.067375 ms per packet) with standard deviation of   0.003557 ms
qSpeedTest(send=256    , recv=0      ) in 0.050108000 sec for  19956.89 packets/sec (  0.050108 ms per packet) with standard deviation of   0.003281 ms
qSpeedTest(send=256    , recv=4      ) in 0.051799000 sec for  19305.39 packets/sec (  0.051799 ms per packet) with standard deviation of   0.003626 ms
qSpeedTest(send=256    , recv=8      ) in 0.052719000 sec for  18968.49 packets/sec (  0.052719 ms per packet) with standard deviation of   0.004331 ms
qSpeedTest(send=256    , recv=16     ) in 0.052387000 sec for  19088.71 packets/sec (  0.052387 ms per packet) with standard deviation of   0.004896 ms
qSpeedTest(send=256    , recv=32     ) in 0.052822000 sec for  18931.51 packets/sec (  0.052822 ms per packet) with standard deviation of   0.005342 ms
qSpeedTest(send=256    , recv=64     ) in 0.059212000 sec for  16888.47 packets/sec (  0.059212 ms per packet) with standard deviation of   0.007692 ms
qSpeedTest(send=256    , recv=128    ) in 0.055576000 sec for  17993.38 packets/sec (  0.055576 ms per packet) with standard deviation of   0.004692 ms
qSpeedTest(send=256    , recv=256    ) in 0.062278000 sec for  16057.03 packets/sec (  0.062278 ms per packet) with standard deviation of   0.006579 ms
qSpeedTest(send=256    , recv=512    ) in 0.065064000 sec for  15369.48 packets/sec (  0.065064 ms per packet) with standard deviation of   0.003246 ms
qSpeedTest(send=256    , recv=1024   ) in 0.070101000 sec for  14265.13 packets/sec (  0.070101 ms per packet) with standard deviation of   0.002870 ms
qSpeedTest(send=512    , recv=0      ) in 0.055987000 sec for  17861.29 packets/sec (  0.055987 ms per packet) with standard deviation of   0.007052 ms
qSpeedTest(send=512    , recv=4      ) in 0.050394000 sec for  19843.63 packets/sec (  0.050394 ms per packet) with standard deviation of   0.002432 ms
qSpeedTest(send=512    , recv=8      ) in 0.050362000 sec for  19856.24 packets/sec (  0.050362 ms per packet) with standard deviation of   0.001982 ms
qSpeedTest(send=512    , recv=16     ) in 0.050612000 sec for  19758.16 packets/sec (  0.050612 ms per packet) with standard deviation of   0.001977 ms
qSpeedTest(send=512    , recv=32     ) in 0.050446000 sec for  19823.18 packets/sec (  0.050446 ms per packet) with standard deviation of   0.001591 ms
qSpeedTest(send=512    , recv=64     ) in 0.050786000 sec for  19690.46 packets/sec (  0.050786 ms per packet) with standard deviation of   0.001496 ms
qSpeedTest(send=512    , recv=128    ) in 0.052420000 sec for  19076.69 packets/sec (  0.052420 ms per packet) with standard deviation of   0.001954 ms
qSpeedTest(send=512    , recv=256    ) in 0.053425000 sec for  18717.83 packets/sec (  0.053425 ms per packet) with standard deviation of   0.002012 ms
qSpeedTest(send=512    , recv=512    ) in 0.058670000 sec for  17044.49 packets/sec (  0.058670 ms per packet) with standard deviation of   0.004880 ms
qSpeedTest(send=512    , recv=1024   ) in 0.062009000 sec for  16126.69 packets/sec (  0.062009 ms per packet) with standard deviation of   0.001987 ms
qSpeedTest(send=1024   , recv=0      ) in 0.053092000 sec for  18835.23 packets/sec (  0.053092 ms per packet) with standard deviation of   0.003347 ms
qSpeedTest(send=1024   , recv=4      ) in 0.054213000 sec for  18445.76 packets/sec (  0.054213 ms per packet) with standard deviation of   0.001928 ms
qSpeedTest(send=1024   , recv=8      ) in 0.053599000 sec for  18657.06 packets/sec (  0.053599 ms per packet) with standard deviation of   0.002024 ms
qSpeedTest(send=1024   , recv=16     ) in 0.056705000 sec for  17635.13 packets/sec (  0.056705 ms per packet) with standard deviation of   0.005067 ms
qSpeedTest(send=1024   , recv=32     ) in 0.056331000 sec for  17752.21 packets/sec (  0.056331 ms per packet) with standard deviation of   0.005956 ms
qSpeedTest(send=1024   , recv=64     ) in 0.060983000 sec for  16398.01 packets/sec (  0.060983 ms per packet) with standard deviation of   0.009666 ms
qSpeedTest(send=1024   , recv=128    ) in 0.052064000 sec for  19207.13 packets/sec (  0.052064 ms per packet) with standard deviation of   0.002793 ms
qSpeedTest(send=1024   , recv=256    ) in 0.057106000 sec for  17511.29 packets/sec (  0.057106 ms per packet) with standard deviation of   0.002904 ms
qSpeedTest(send=1024   , recv=512    ) in 0.073955000 sec for  13521.74 packets/sec (  0.073955 ms per packet) with standard deviation of   0.013739 ms
qSpeedTest(send=1024   , recv=1024   ) in 0.074225000 sec for  13472.55 packets/sec (  0.074225 ms per packet) with standard deviation of   0.015451 ms
Testing receiving 4.0MB of data using varying receive packet sizes:
qSpeedTest(send=0      , recv=32     ) 131072 packets needed to receive 4.0MB in 6.696583000 sec for 0.597320 MB/sec for  19572.97 packets/sec (  0.051091 ms per packet)
qSpeedTest(send=0      , recv=64     )  65536 packets needed to receive 4.0MB in 3.230414000 sec for 1.238231 MB/sec for  20287.18 packets/sec (  0.049292 ms per packet)
qSpeedTest(send=0      , recv=128    )  32768 packets needed to receive 4.0MB in 1.686612000 sec for 2.371619 MB/sec for  19428.30 packets/sec (  0.051471 ms per packet)
qSpeedTest(send=0      , recv=256    )  16384 packets needed to receive 4.0MB in 0.845153000 sec for 4.732871 MB/sec for  19385.84 packets/sec (  0.051584 ms per packet)
qSpeedTest(send=0      , recv=512    )   8192 packets needed to receive 4.0MB in 0.421588000 sec for 9.487936 MB/sec for  19431.29 packets/sec (  0.051463 ms per packet)
qSpeedTest(send=0      , recv=1024   )   4096 packets needed to receive 4.0MB in 0.232212000 sec for 17.225637 MB/sec for  17639.05 packets/sec (  0.056692 ms per packet)

After:

(lldb) process plugin packet speed-test
Testing sending 1000 packets of various sizes:
qSpeedTest(send=0      , recv=0      ) in 0.047088000 sec for  21236.83 packets/sec (  0.047088 ms per packet) with standard deviation of   0.006629 ms
qSpeedTest(send=0      , recv=4      ) in 0.045426000 sec for  22013.82 packets/sec (  0.045426 ms per packet) with standard deviation of   0.001687 ms
qSpeedTest(send=0      , recv=8      ) in 0.045206000 sec for  22120.96 packets/sec (  0.045206 ms per packet) with standard deviation of   0.001241 ms
qSpeedTest(send=0      , recv=16     ) in 0.045476000 sec for  21989.62 packets/sec (  0.045476 ms per packet) with standard deviation of   0.001294 ms
qSpeedTest(send=0      , recv=32     ) in 0.045909000 sec for  21782.22 packets/sec (  0.045909 ms per packet) with standard deviation of   0.001929 ms
qSpeedTest(send=0      , recv=64     ) in 0.046741000 sec for  21394.49 packets/sec (  0.046741 ms per packet) with standard deviation of   0.001344 ms
qSpeedTest(send=0      , recv=128    ) in 0.048040000 sec for  20815.99 packets/sec (  0.048040 ms per packet) with standard deviation of   0.001184 ms
qSpeedTest(send=0      , recv=256    ) in 0.049685000 sec for  20126.80 packets/sec (  0.049685 ms per packet) with standard deviation of   0.001366 ms
qSpeedTest(send=0      , recv=512    ) in 0.052725000 sec for  18966.34 packets/sec (  0.052725 ms per packet) with standard deviation of   0.001989 ms
qSpeedTest(send=0      , recv=1024   ) in 0.059048000 sec for  16935.38 packets/sec (  0.059048 ms per packet) with standard deviation of   0.001811 ms
qSpeedTest(send=4      , recv=0      ) in 0.044740000 sec for  22351.36 packets/sec (  0.044740 ms per packet) with standard deviation of   0.001847 ms
qSpeedTest(send=4      , recv=4      ) in 0.045536000 sec for  21960.65 packets/sec (  0.045536 ms per packet) with standard deviation of   0.001471 ms
qSpeedTest(send=4      , recv=8      ) in 0.045569000 sec for  21944.74 packets/sec (  0.045569 ms per packet) with standard deviation of   0.001264 ms
qSpeedTest(send=4      , recv=16     ) in 0.045332000 sec for  22059.47 packets/sec (  0.045332 ms per packet) with standard deviation of   0.001097 ms
qSpeedTest(send=4      , recv=32     ) in 0.046639000 sec for  21441.28 packets/sec (  0.046639 ms per packet) with standard deviation of   0.004458 ms
qSpeedTest(send=4      , recv=64     ) in 0.050692000 sec for  19726.98 packets/sec (  0.050692 ms per packet) with standard deviation of   0.003364 ms
qSpeedTest(send=4      , recv=128    ) in 0.050537000 sec for  19787.48 packets/sec (  0.050537 ms per packet) with standard deviation of   0.001669 ms
qSpeedTest(send=4      , recv=256    ) in 0.052030000 sec for  19219.68 packets/sec (  0.052030 ms per packet) with standard deviation of   0.001357 ms
qSpeedTest(send=4      , recv=512    ) in 0.057476000 sec for  17398.57 packets/sec (  0.057476 ms per packet) with standard deviation of   0.005383 ms
qSpeedTest(send=4      , recv=1024   ) in 0.062974000 sec for  15879.57 packets/sec (  0.062974 ms per packet) with standard deviation of   0.006826 ms
qSpeedTest(send=8      , recv=0      ) in 0.048881000 sec for  20457.85 packets/sec (  0.048881 ms per packet) with standard deviation of   0.007901 ms
qSpeedTest(send=8      , recv=4      ) in 0.049233000 sec for  20311.58 packets/sec (  0.049233 ms per packet) with standard deviation of   0.006031 ms
qSpeedTest(send=8      , recv=8      ) in 0.049758000 sec for  20097.27 packets/sec (  0.049758 ms per packet) with standard deviation of   0.007929 ms
qSpeedTest(send=8      , recv=16     ) in 0.049089000 sec for  20371.16 packets/sec (  0.049089 ms per packet) with standard deviation of   0.006375 ms
qSpeedTest(send=8      , recv=32     ) in 0.050110000 sec for  19956.10 packets/sec (  0.050110 ms per packet) with standard deviation of   0.008473 ms
qSpeedTest(send=8      , recv=64     ) in 0.050779000 sec for  19693.18 packets/sec (  0.050779 ms per packet) with standard deviation of   0.005380 ms
qSpeedTest(send=8      , recv=128    ) in 0.053557000 sec for  18671.70 packets/sec (  0.053557 ms per packet) with standard deviation of   0.008242 ms
qSpeedTest(send=8      , recv=256    ) in 0.054547000 sec for  18332.81 packets/sec (  0.054547 ms per packet) with standard deviation of   0.007676 ms
qSpeedTest(send=8      , recv=512    ) in 0.058448000 sec for  17109.22 packets/sec (  0.058448 ms per packet) with standard deviation of   0.009949 ms
qSpeedTest(send=8      , recv=1024   ) in 0.064296000 sec for  15553.07 packets/sec (  0.064296 ms per packet) with standard deviation of   0.007377 ms
qSpeedTest(send=16     , recv=0      ) in 0.047807000 sec for  20917.44 packets/sec (  0.047807 ms per packet) with standard deviation of   0.008033 ms
qSpeedTest(send=16     , recv=4      ) in 0.047000000 sec for  21276.60 packets/sec (  0.047000 ms per packet) with standard deviation of   0.005585 ms
qSpeedTest(send=16     , recv=8      ) in 0.047780000 sec for  20929.26 packets/sec (  0.047780 ms per packet) with standard deviation of   0.006055 ms
qSpeedTest(send=16     , recv=16     ) in 0.048940000 sec for  20433.18 packets/sec (  0.048940 ms per packet) with standard deviation of   0.007877 ms
qSpeedTest(send=16     , recv=32     ) in 0.048452000 sec for  20638.98 packets/sec (  0.048452 ms per packet) with standard deviation of   0.006351 ms
qSpeedTest(send=16     , recv=64     ) in 0.053125000 sec for  18823.53 packets/sec (  0.053125 ms per packet) with standard deviation of   0.007059 ms
qSpeedTest(send=16     , recv=128    ) in 0.054247000 sec for  18434.20 packets/sec (  0.054247 ms per packet) with standard deviation of   0.006543 ms
qSpeedTest(send=16     , recv=256    ) in 0.050342000 sec for  19864.13 packets/sec (  0.050342 ms per packet) with standard deviation of   0.003944 ms
qSpeedTest(send=16     , recv=512    ) in 0.056098000 sec for  17825.95 packets/sec (  0.056098 ms per packet) with standard deviation of   0.004119 ms
qSpeedTest(send=16     , recv=1024   ) in 0.058348000 sec for  17138.55 packets/sec (  0.058348 ms per packet) with standard deviation of   0.002004 ms
qSpeedTest(send=32     , recv=0      ) in 0.044705000 sec for  22368.86 packets/sec (  0.044705 ms per packet) with standard deviation of   0.003299 ms
qSpeedTest(send=32     , recv=4      ) in 0.046046000 sec for  21717.41 packets/sec (  0.046046 ms per packet) with standard deviation of   0.001855 ms
qSpeedTest(send=32     , recv=8      ) in 0.045633000 sec for  21913.96 packets/sec (  0.045633 ms per packet) with standard deviation of   0.001534 ms
qSpeedTest(send=32     , recv=16     ) in 0.045522000 sec for  21967.40 packets/sec (  0.045522 ms per packet) with standard deviation of   0.001016 ms
qSpeedTest(send=32     , recv=32     ) in 0.046046000 sec for  21717.41 packets/sec (  0.046046 ms per packet) with standard deviation of   0.001172 ms
qSpeedTest(send=32     , recv=64     ) in 0.046843000 sec for  21347.91 packets/sec (  0.046843 ms per packet) with standard deviation of   0.001441 ms
qSpeedTest(send=32     , recv=128    ) in 0.047837000 sec for  20904.32 packets/sec (  0.047837 ms per packet) with standard deviation of   0.001571 ms
qSpeedTest(send=32     , recv=256    ) in 0.049504000 sec for  20200.39 packets/sec (  0.049504 ms per packet) with standard deviation of   0.001091 ms
qSpeedTest(send=32     , recv=512    ) in 0.052681000 sec for  18982.18 packets/sec (  0.052681 ms per packet) with standard deviation of   0.001701 ms
qSpeedTest(send=32     , recv=1024   ) in 0.058205000 sec for  17180.65 packets/sec (  0.058205 ms per packet) with standard deviation of   0.002028 ms
qSpeedTest(send=64     , recv=0      ) in 0.045595000 sec for  21932.23 packets/sec (  0.045595 ms per packet) with standard deviation of   0.001447 ms
qSpeedTest(send=64     , recv=4      ) in 0.048343000 sec for  20685.52 packets/sec (  0.048343 ms per packet) with standard deviation of   0.004192 ms
qSpeedTest(send=64     , recv=8      ) in 0.052826000 sec for  18930.07 packets/sec (  0.052826 ms per packet) with standard deviation of   0.006674 ms
qSpeedTest(send=64     , recv=16     ) in 0.053331000 sec for  18750.82 packets/sec (  0.053331 ms per packet) with standard deviation of   0.006791 ms
qSpeedTest(send=64     , recv=32     ) in 0.053790000 sec for  18590.82 packets/sec (  0.053790 ms per packet) with standard deviation of   0.005458 ms
qSpeedTest(send=64     , recv=64     ) in 0.049696000 sec for  20122.34 packets/sec (  0.049696 ms per packet) with standard deviation of   0.002810 ms
qSpeedTest(send=64     , recv=128    ) in 0.050803000 sec for  19683.88 packets/sec (  0.050803 ms per packet) with standard deviation of   0.001376 ms
qSpeedTest(send=64     , recv=256    ) in 0.052227000 sec for  19147.18 packets/sec (  0.052227 ms per packet) with standard deviation of   0.001451 ms
qSpeedTest(send=64     , recv=512    ) in 0.056122000 sec for  17818.32 packets/sec (  0.056122 ms per packet) with standard deviation of   0.002323 ms
qSpeedTest(send=64     , recv=1024   ) in 0.061925000 sec for  16148.57 packets/sec (  0.061925 ms per packet) with standard deviation of   0.002727 ms
qSpeedTest(send=128    , recv=0      ) in 0.048665000 sec for  20548.65 packets/sec (  0.048665 ms per packet) with standard deviation of   0.002070 ms
qSpeedTest(send=128    , recv=4      ) in 0.049676000 sec for  20130.45 packets/sec (  0.049676 ms per packet) with standard deviation of   0.001542 ms
qSpeedTest(send=128    , recv=8      ) in 0.049602000 sec for  20160.48 packets/sec (  0.049602 ms per packet) with standard deviation of   0.001353 ms
qSpeedTest(send=128    , recv=16     ) in 0.049804000 sec for  20078.71 packets/sec (  0.049804 ms per packet) with standard deviation of   0.001733 ms
qSpeedTest(send=128    , recv=32     ) in 0.050268000 sec for  19893.37 packets/sec (  0.050268 ms per packet) with standard deviation of   0.001648 ms
qSpeedTest(send=128    , recv=64     ) in 0.050944000 sec for  19629.40 packets/sec (  0.050944 ms per packet) with standard deviation of   0.001650 ms
qSpeedTest(send=128    , recv=128    ) in 0.052144000 sec for  19177.66 packets/sec (  0.052144 ms per packet) with standard deviation of   0.001757 ms
qSpeedTest(send=128    , recv=256    ) in 0.053151000 sec for  18814.32 packets/sec (  0.053151 ms per packet) with standard deviation of   0.008508 ms
qSpeedTest(send=128    , recv=512    ) in 0.056988000 sec for  17547.55 packets/sec (  0.056988 ms per packet) with standard deviation of   0.002518 ms
qSpeedTest(send=128    , recv=1024   ) in 0.062617000 sec for  15970.10 packets/sec (  0.062617 ms per packet) with standard deviation of   0.002400 ms
qSpeedTest(send=256    , recv=0      ) in 0.049461000 sec for  20217.95 packets/sec (  0.049461 ms per packet) with standard deviation of   0.001476 ms
qSpeedTest(send=256    , recv=4      ) in 0.050928000 sec for  19635.56 packets/sec (  0.050928 ms per packet) with standard deviation of   0.001747 ms
qSpeedTest(send=256    , recv=8      ) in 0.050779000 sec for  19693.18 packets/sec (  0.050779 ms per packet) with standard deviation of   0.001450 ms
qSpeedTest(send=256    , recv=16     ) in 0.050916000 sec for  19640.19 packets/sec (  0.050916 ms per packet) with standard deviation of   0.002230 ms
qSpeedTest(send=256    , recv=32     ) in 0.051062000 sec for  19584.04 packets/sec (  0.051062 ms per packet) with standard deviation of   0.001511 ms
qSpeedTest(send=256    , recv=64     ) in 0.052163000 sec for  19170.68 packets/sec (  0.052163 ms per packet) with standard deviation of   0.001650 ms
qSpeedTest(send=256    , recv=128    ) in 0.053406000 sec for  18724.49 packets/sec (  0.053406 ms per packet) with standard deviation of   0.001675 ms
qSpeedTest(send=256    , recv=256    ) in 0.055275000 sec for  18091.36 packets/sec (  0.055275 ms per packet) with standard deviation of   0.006017 ms
qSpeedTest(send=256    , recv=512    ) in 0.058381000 sec for  17128.86 packets/sec (  0.058381 ms per packet) with standard deviation of   0.006529 ms
qSpeedTest(send=256    , recv=1024   ) in 0.062474000 sec for  16006.66 packets/sec (  0.062474 ms per packet) with standard deviation of   0.002550 ms
qSpeedTest(send=512    , recv=0      ) in 0.050066000 sec for  19973.63 packets/sec (  0.050066 ms per packet) with standard deviation of   0.001578 ms
qSpeedTest(send=512    , recv=4      ) in 0.051145000 sec for  19552.25 packets/sec (  0.051145 ms per packet) with standard deviation of   0.002413 ms
qSpeedTest(send=512    , recv=8      ) in 0.051583000 sec for  19386.23 packets/sec (  0.051583 ms per packet) with standard deviation of   0.001808 ms
qSpeedTest(send=512    , recv=16     ) in 0.052346000 sec for  19103.66 packets/sec (  0.052346 ms per packet) with standard deviation of   0.005228 ms
qSpeedTest(send=512    , recv=32     ) in 0.054020000 sec for  18511.66 packets/sec (  0.054020 ms per packet) with standard deviation of   0.004604 ms
qSpeedTest(send=512    , recv=64     ) in 0.052584000 sec for  19017.19 packets/sec (  0.052584 ms per packet) with standard deviation of   0.001935 ms
qSpeedTest(send=512    , recv=128    ) in 0.054611000 sec for  18311.33 packets/sec (  0.054611 ms per packet) with standard deviation of   0.001507 ms
qSpeedTest(send=512    , recv=256    ) in 0.055602000 sec for  17984.96 packets/sec (  0.055602 ms per packet) with standard deviation of   0.004139 ms
qSpeedTest(send=512    , recv=512    ) in 0.060131000 sec for  16630.36 packets/sec (  0.060131 ms per packet) with standard deviation of   0.004732 ms
qSpeedTest(send=512    , recv=1024   ) in 0.063374000 sec for  15779.34 packets/sec (  0.063374 ms per packet) with standard deviation of   0.002498 ms
qSpeedTest(send=1024   , recv=0      ) in 0.058207000 sec for  17180.06 packets/sec (  0.058207 ms per packet) with standard deviation of   0.004969 ms
qSpeedTest(send=1024   , recv=4      ) in 0.057945000 sec for  17257.74 packets/sec (  0.057945 ms per packet) with standard deviation of   0.002644 ms
qSpeedTest(send=1024   , recv=8      ) in 0.056318000 sec for  17756.31 packets/sec (  0.056318 ms per packet) with standard deviation of   0.001681 ms
qSpeedTest(send=1024   , recv=16     ) in 0.057504000 sec for  17390.10 packets/sec (  0.057504 ms per packet) with standard deviation of   0.003238 ms
qSpeedTest(send=1024   , recv=32     ) in 0.058180000 sec for  17188.04 packets/sec (  0.058180 ms per packet) with standard deviation of   0.001859 ms
qSpeedTest(send=1024   , recv=64     ) in 0.057998000 sec for  17241.97 packets/sec (  0.057998 ms per packet) with standard deviation of   0.002039 ms
qSpeedTest(send=1024   , recv=128    ) in 0.058390000 sec for  17126.22 packets/sec (  0.058390 ms per packet) with standard deviation of   0.001929 ms
qSpeedTest(send=1024   , recv=256    ) in 0.061801000 sec for  16180.97 packets/sec (  0.061801 ms per packet) with standard deviation of   0.004318 ms
qSpeedTest(send=1024   , recv=512    ) in 0.063850000 sec for  15661.71 packets/sec (  0.063850 ms per packet) with standard deviation of   0.002689 ms
qSpeedTest(send=1024   , recv=1024   ) in 0.068770000 sec for  14541.22 packets/sec (  0.068770 ms per packet) with standard deviation of   0.002334 ms
Testing receiving 4.0MB of data using varying receive packet sizes:
qSpeedTest(send=0      , recv=32     ) 131072 packets needed to receive 4.0MB in 5.729221000 sec for 0.698175 MB/sec for  22877.80 packets/sec (  0.043710 ms per packet)
qSpeedTest(send=0      , recv=64     )  65536 packets needed to receive 4.0MB in 3.002543000 sec for 1.332204 MB/sec for  21826.83 packets/sec (  0.045815 ms per packet)
qSpeedTest(send=0      , recv=128    )  32768 packets needed to receive 4.0MB in 1.634748000 sec for 2.446860 MB/sec for  20044.68 packets/sec (  0.049889 ms per packet)
qSpeedTest(send=0      , recv=256    )  16384 packets needed to receive 4.0MB in 0.834877000 sec for 4.791125 MB/sec for  19624.45 packets/sec (  0.050957 ms per packet)
qSpeedTest(send=0      , recv=512    )   8192 packets needed to receive 4.0MB in 0.462364000 sec for 8.651193 MB/sec for  17717.64 packets/sec (  0.056441 ms per packet)
qSpeedTest(send=0      , recv=1024   )   4096 packets needed to receive 4.0MB in 0.243027000 sec for 16.459076 MB/sec for  16854.09 packets/sec (  0.059333 ms per packet)

The results seem to be within error margin or even slightly faster (although I can't explain why).

The mac build is still ongiong. I'll have the results tomorrow, but I wouldn't expect this to affect them to be any different. In the uncontended case, all this is doing is replacing a recursive mutex with a read/write lock + a normal mutex. All of these operations should be extremely fast compared to the rest of the things we are doing here(*). If we wanted to speed things up here, we should probably try to avoid all the string copying going on when building packets.

(*) There is also the push/pop of an item on the queue, but again that should be very fast in case of an empty queue. And we could even optimize that away in the case we have an exclusive lock, but it does not seem to be necessary.

Let me know how things look on Mac. Our lock primitives are not as fast as linux. If all looks good on Mac speed wise, we should be good to go.

Let me know how things look on Mac. Our lock primitives are not as fast as linux. If all looks good on Mac speed wise, we should be good to go.

These are the results on Mac. Before:

Testing sending 1000 packets of various sizes:
qSpeedTest(send=0      , recv=0      ) in 0.069825000 sec for  14321.52 packets/sec (  0.069825 ms per packet) with standard deviation of   0.072367 ms
qSpeedTest(send=0      , recv=4      ) in 0.059877000 sec for  16700.90 packets/sec (  0.059877 ms per packet) with standard deviation of   0.052829 ms
qSpeedTest(send=0      , recv=8      ) in 0.062646000 sec for  15962.71 packets/sec (  0.062646 ms per packet) with standard deviation of   0.053817 ms
qSpeedTest(send=0      , recv=16     ) in 0.057945000 sec for  17257.74 packets/sec (  0.057945 ms per packet) with standard deviation of   0.056490 ms
qSpeedTest(send=0      , recv=32     ) in 0.065491000 sec for  15269.27 packets/sec (  0.065491 ms per packet) with standard deviation of   0.052040 ms
qSpeedTest(send=0      , recv=64     ) in 0.069541000 sec for  14380.01 packets/sec (  0.069541 ms per packet) with standard deviation of   0.057329 ms
qSpeedTest(send=0      , recv=128    ) in 0.065865000 sec for  15182.57 packets/sec (  0.065865 ms per packet) with standard deviation of   0.052399 ms
qSpeedTest(send=0      , recv=256    ) in 0.086535000 sec for  11556.02 packets/sec (  0.086535 ms per packet) with standard deviation of   0.064901 ms
qSpeedTest(send=0      , recv=512    ) in 0.093280000 sec for  10720.41 packets/sec (  0.093280 ms per packet) with standard deviation of   0.062687 ms
qSpeedTest(send=0      , recv=1024   ) in 0.120688000 sec for   8285.83 packets/sec (  0.120688 ms per packet) with standard deviation of   0.077364 ms
qSpeedTest(send=4      , recv=0      ) in 0.071374000 sec for  14010.70 packets/sec (  0.071374 ms per packet) with standard deviation of   0.058664 ms
qSpeedTest(send=4      , recv=4      ) in 0.056071000 sec for  17834.53 packets/sec (  0.056071 ms per packet) with standard deviation of   0.049469 ms
qSpeedTest(send=4      , recv=8      ) in 0.063151000 sec for  15835.06 packets/sec (  0.063151 ms per packet) with standard deviation of   0.043858 ms
qSpeedTest(send=4      , recv=16     ) in 0.059600000 sec for  16778.52 packets/sec (  0.059600 ms per packet) with standard deviation of   0.041999 ms
qSpeedTest(send=4      , recv=32     ) in 0.067713000 sec for  14768.21 packets/sec (  0.067713 ms per packet) with standard deviation of   0.044975 ms
qSpeedTest(send=4      , recv=64     ) in 0.062116000 sec for  16098.91 packets/sec (  0.062116 ms per packet) with standard deviation of   0.051030 ms
qSpeedTest(send=4      , recv=128    ) in 0.076669000 sec for  13043.08 packets/sec (  0.076669 ms per packet) with standard deviation of   0.049543 ms
qSpeedTest(send=4      , recv=256    ) in 0.089484000 sec for  11175.18 packets/sec (  0.089484 ms per packet) with standard deviation of   0.060211 ms
qSpeedTest(send=4      , recv=512    ) in 0.092912000 sec for  10762.87 packets/sec (  0.092912 ms per packet) with standard deviation of   0.054948 ms
qSpeedTest(send=4      , recv=1024   ) in 0.127601000 sec for   7836.93 packets/sec (  0.127601 ms per packet) with standard deviation of   0.058131 ms
qSpeedTest(send=8      , recv=0      ) in 0.071355000 sec for  14014.43 packets/sec (  0.071355 ms per packet) with standard deviation of   0.057814 ms
qSpeedTest(send=8      , recv=4      ) in 0.060355000 sec for  16568.64 packets/sec (  0.060355 ms per packet) with standard deviation of   0.043343 ms
qSpeedTest(send=8      , recv=8      ) in 0.067887000 sec for  14730.36 packets/sec (  0.067887 ms per packet) with standard deviation of   0.054229 ms
qSpeedTest(send=8      , recv=16     ) in 0.068483000 sec for  14602.16 packets/sec (  0.068483 ms per packet) with standard deviation of   0.063420 ms
qSpeedTest(send=8      , recv=32     ) in 0.062375000 sec for  16032.06 packets/sec (  0.062375 ms per packet) with standard deviation of   0.062549 ms
qSpeedTest(send=8      , recv=64     ) in 0.063213000 sec for  15819.53 packets/sec (  0.063213 ms per packet) with standard deviation of   0.048372 ms
qSpeedTest(send=8      , recv=128    ) in 0.080731000 sec for  12386.82 packets/sec (  0.080731 ms per packet) with standard deviation of   0.074979 ms
qSpeedTest(send=8      , recv=256    ) in 0.089141000 sec for  11218.18 packets/sec (  0.089141 ms per packet) with standard deviation of   0.056186 ms
qSpeedTest(send=8      , recv=512    ) in 0.097054000 sec for  10303.54 packets/sec (  0.097054 ms per packet) with standard deviation of   0.080488 ms
qSpeedTest(send=8      , recv=1024   ) in 0.114285000 sec for   8750.05 packets/sec (  0.114285 ms per packet) with standard deviation of   0.062551 ms
qSpeedTest(send=16     , recv=0      ) in 0.062928000 sec for  15891.18 packets/sec (  0.062928 ms per packet) with standard deviation of   0.053606 ms
qSpeedTest(send=16     , recv=4      ) in 0.065235000 sec for  15329.20 packets/sec (  0.065235 ms per packet) with standard deviation of   0.058759 ms
qSpeedTest(send=16     , recv=8      ) in 0.058764000 sec for  17017.22 packets/sec (  0.058764 ms per packet) with standard deviation of   0.053362 ms
qSpeedTest(send=16     , recv=16     ) in 0.064264000 sec for  15560.81 packets/sec (  0.064264 ms per packet) with standard deviation of   0.055768 ms
qSpeedTest(send=16     , recv=32     ) in 0.060461000 sec for  16539.59 packets/sec (  0.060461 ms per packet) with standard deviation of   0.055025 ms
qSpeedTest(send=16     , recv=64     ) in 0.068768000 sec for  14541.65 packets/sec (  0.068768 ms per packet) with standard deviation of   0.044804 ms
qSpeedTest(send=16     , recv=128    ) in 0.071216000 sec for  14041.79 packets/sec (  0.071216 ms per packet) with standard deviation of   0.068766 ms
qSpeedTest(send=16     , recv=256    ) in 0.082106000 sec for  12179.38 packets/sec (  0.082106 ms per packet) with standard deviation of   0.048975 ms
qSpeedTest(send=16     , recv=512    ) in 0.087569000 sec for  11419.57 packets/sec (  0.087569 ms per packet) with standard deviation of   0.057329 ms
qSpeedTest(send=16     , recv=1024   ) in 0.130835000 sec for   7643.21 packets/sec (  0.130835 ms per packet) with standard deviation of   0.062011 ms
qSpeedTest(send=32     , recv=0      ) in 0.072672000 sec for  13760.46 packets/sec (  0.072672 ms per packet) with standard deviation of   0.047426 ms
qSpeedTest(send=32     , recv=4      ) in 0.072757000 sec for  13744.38 packets/sec (  0.072757 ms per packet) with standard deviation of   0.055688 ms
qSpeedTest(send=32     , recv=8      ) in 0.063909000 sec for  15647.25 packets/sec (  0.063909 ms per packet) with standard deviation of   0.052424 ms
qSpeedTest(send=32     , recv=16     ) in 0.069800000 sec for  14326.65 packets/sec (  0.069800 ms per packet) with standard deviation of   0.051824 ms
qSpeedTest(send=32     , recv=32     ) in 0.062857000 sec for  15909.13 packets/sec (  0.062857 ms per packet) with standard deviation of   0.068030 ms
qSpeedTest(send=32     , recv=64     ) in 0.071279000 sec for  14029.38 packets/sec (  0.071279 ms per packet) with standard deviation of   0.056358 ms
qSpeedTest(send=32     , recv=128    ) in 0.075480000 sec for  13248.54 packets/sec (  0.075480 ms per packet) with standard deviation of   0.062057 ms
qSpeedTest(send=32     , recv=256    ) in 0.081876000 sec for  12213.59 packets/sec (  0.081876 ms per packet) with standard deviation of   0.055678 ms
qSpeedTest(send=32     , recv=512    ) in 0.096283000 sec for  10386.05 packets/sec (  0.096283 ms per packet) with standard deviation of   0.070404 ms
qSpeedTest(send=32     , recv=1024   ) in 0.127398000 sec for   7849.42 packets/sec (  0.127398 ms per packet) with standard deviation of   0.063599 ms
qSpeedTest(send=64     , recv=0      ) in 0.077976000 sec for  12824.46 packets/sec (  0.077976 ms per packet) with standard deviation of   0.042989 ms
qSpeedTest(send=64     , recv=4      ) in 0.061040000 sec for  16382.70 packets/sec (  0.061040 ms per packet) with standard deviation of   0.038150 ms
qSpeedTest(send=64     , recv=8      ) in 0.064183000 sec for  15580.45 packets/sec (  0.064183 ms per packet) with standard deviation of   0.051014 ms
qSpeedTest(send=64     , recv=16     ) in 0.064531000 sec for  15496.43 packets/sec (  0.064531 ms per packet) with standard deviation of   0.050704 ms
qSpeedTest(send=64     , recv=32     ) in 0.067908000 sec for  14725.80 packets/sec (  0.067908 ms per packet) with standard deviation of   0.055231 ms
qSpeedTest(send=64     , recv=64     ) in 0.067957000 sec for  14715.19 packets/sec (  0.067957 ms per packet) with standard deviation of   0.051354 ms
qSpeedTest(send=64     , recv=128    ) in 0.079725000 sec for  12543.12 packets/sec (  0.079725 ms per packet) with standard deviation of   0.051747 ms
qSpeedTest(send=64     , recv=256    ) in 0.073793000 sec for  13551.42 packets/sec (  0.073793 ms per packet) with standard deviation of   0.054190 ms
qSpeedTest(send=64     , recv=512    ) in 0.088459000 sec for  11304.67 packets/sec (  0.088459 ms per packet) with standard deviation of   0.063903 ms
qSpeedTest(send=64     , recv=1024   ) in 0.127690000 sec for   7831.47 packets/sec (  0.127690 ms per packet) with standard deviation of   0.068364 ms
qSpeedTest(send=128    , recv=0      ) in 0.064236000 sec for  15567.60 packets/sec (  0.064236 ms per packet) with standard deviation of   0.056044 ms
qSpeedTest(send=128    , recv=4      ) in 0.062882000 sec for  15902.80 packets/sec (  0.062882 ms per packet) with standard deviation of   0.050858 ms
qSpeedTest(send=128    , recv=8      ) in 0.063884000 sec for  15653.37 packets/sec (  0.063884 ms per packet) with standard deviation of   0.054283 ms
qSpeedTest(send=128    , recv=16     ) in 0.066950000 sec for  14936.52 packets/sec (  0.066950 ms per packet) with standard deviation of   0.051539 ms
qSpeedTest(send=128    , recv=32     ) in 0.058177000 sec for  17188.92 packets/sec (  0.058177 ms per packet) with standard deviation of   0.049257 ms
qSpeedTest(send=128    , recv=64     ) in 0.068716000 sec for  14552.65 packets/sec (  0.068716 ms per packet) with standard deviation of   0.059560 ms
qSpeedTest(send=128    , recv=128    ) in 0.069624000 sec for  14362.86 packets/sec (  0.069624 ms per packet) with standard deviation of   0.072935 ms
qSpeedTest(send=128    , recv=256    ) in 0.083122000 sec for  12030.51 packets/sec (  0.083122 ms per packet) with standard deviation of   0.054313 ms
qSpeedTest(send=128    , recv=512    ) in 0.094324000 sec for  10601.76 packets/sec (  0.094324 ms per packet) with standard deviation of   0.070021 ms
qSpeedTest(send=128    , recv=1024   ) in 0.128338000 sec for   7791.92 packets/sec (  0.128338 ms per packet) with standard deviation of   0.067563 ms
qSpeedTest(send=256    , recv=0      ) in 0.066692000 sec for  14994.30 packets/sec (  0.066692 ms per packet) with standard deviation of   0.061524 ms
qSpeedTest(send=256    , recv=4      ) in 0.072561000 sec for  13781.51 packets/sec (  0.072561 ms per packet) with standard deviation of   0.051529 ms
qSpeedTest(send=256    , recv=8      ) in 0.062888000 sec for  15901.29 packets/sec (  0.062888 ms per packet) with standard deviation of   0.053264 ms
qSpeedTest(send=256    , recv=16     ) in 0.065313000 sec for  15310.89 packets/sec (  0.065313 ms per packet) with standard deviation of   0.049048 ms
qSpeedTest(send=256    , recv=32     ) in 0.059649000 sec for  16764.74 packets/sec (  0.059649 ms per packet) with standard deviation of   0.055966 ms
qSpeedTest(send=256    , recv=64     ) in 0.067059000 sec for  14912.24 packets/sec (  0.067059 ms per packet) with standard deviation of   0.049511 ms
qSpeedTest(send=256    , recv=128    ) in 0.074379000 sec for  13444.66 packets/sec (  0.074379 ms per packet) with standard deviation of   0.052978 ms
qSpeedTest(send=256    , recv=256    ) in 0.073528000 sec for  13600.26 packets/sec (  0.073528 ms per packet) with standard deviation of   0.075635 ms
qSpeedTest(send=256    , recv=512    ) in 0.086456000 sec for  11566.58 packets/sec (  0.086456 ms per packet) with standard deviation of   0.068442 ms
qSpeedTest(send=256    , recv=1024   ) in 0.116261000 sec for   8601.34 packets/sec (  0.116261 ms per packet) with standard deviation of   0.071226 ms
qSpeedTest(send=512    , recv=0      ) in 0.060334000 sec for  16574.40 packets/sec (  0.060334 ms per packet) with standard deviation of   0.049805 ms
qSpeedTest(send=512    , recv=4      ) in 0.064310000 sec for  15549.68 packets/sec (  0.064310 ms per packet) with standard deviation of   0.050966 ms
qSpeedTest(send=512    , recv=8      ) in 0.061415000 sec for  16282.67 packets/sec (  0.061415 ms per packet) with standard deviation of   0.070034 ms
qSpeedTest(send=512    , recv=16     ) in 0.065371000 sec for  15297.30 packets/sec (  0.065371 ms per packet) with standard deviation of   0.047046 ms
qSpeedTest(send=512    , recv=32     ) in 0.065901000 sec for  15174.28 packets/sec (  0.065901 ms per packet) with standard deviation of   0.055894 ms
qSpeedTest(send=512    , recv=64     ) in 0.064459000 sec for  15513.74 packets/sec (  0.064459 ms per packet) with standard deviation of   0.061275 ms
qSpeedTest(send=512    , recv=128    ) in 0.073455000 sec for  13613.78 packets/sec (  0.073455 ms per packet) with standard deviation of   0.047663 ms
qSpeedTest(send=512    , recv=256    ) in 0.071227000 sec for  14039.62 packets/sec (  0.071227 ms per packet) with standard deviation of   0.055198 ms
qSpeedTest(send=512    , recv=512    ) in 0.078342000 sec for  12764.54 packets/sec (  0.078342 ms per packet) with standard deviation of   0.050908 ms
qSpeedTest(send=512    , recv=1024   ) in 0.122088000 sec for   8190.81 packets/sec (  0.122088 ms per packet) with standard deviation of   0.064947 ms
qSpeedTest(send=1024   , recv=0      ) in 0.081843000 sec for  12218.52 packets/sec (  0.081843 ms per packet) with standard deviation of   0.061668 ms
qSpeedTest(send=1024   , recv=4      ) in 0.070711000 sec for  14142.07 packets/sec (  0.070711 ms per packet) with standard deviation of   0.055051 ms
qSpeedTest(send=1024   , recv=8      ) in 0.083947000 sec for  11912.28 packets/sec (  0.083947 ms per packet) with standard deviation of   0.072992 ms
qSpeedTest(send=1024   , recv=16     ) in 0.075772000 sec for  13197.49 packets/sec (  0.075772 ms per packet) with standard deviation of   0.076053 ms
qSpeedTest(send=1024   , recv=32     ) in 0.085752000 sec for  11661.54 packets/sec (  0.085752 ms per packet) with standard deviation of   0.054003 ms
qSpeedTest(send=1024   , recv=64     ) in 0.066813000 sec for  14967.15 packets/sec (  0.066813 ms per packet) with standard deviation of   0.062023 ms
qSpeedTest(send=1024   , recv=128    ) in 0.070897000 sec for  14104.97 packets/sec (  0.070897 ms per packet) with standard deviation of   0.046704 ms
qSpeedTest(send=1024   , recv=256    ) in 0.071602000 sec for  13966.09 packets/sec (  0.071602 ms per packet) with standard deviation of   0.060842 ms
qSpeedTest(send=1024   , recv=512    ) in 0.083723000 sec for  11944.15 packets/sec (  0.083723 ms per packet) with standard deviation of   0.072209 ms
qSpeedTest(send=1024   , recv=1024   ) in 0.111374000 sec for   8978.76 packets/sec (  0.111374 ms per packet) with standard deviation of   0.079672 ms
Testing receiving 4.0MB of data using varying receive packet sizes:
qSpeedTest(send=0      , recv=32     ) 131072 packets needed to receive 4.0MB in 7.028567000 sec for 0.569106 MB/sec for  18648.47 packets/sec (  0.053624 ms per packet)
qSpeedTest(send=0      , recv=64     )  65536 packets needed to receive 4.0MB in 4.003298000 sec for 0.999176 MB/sec for  16370.50 packets/sec (  0.061085 ms per packet)
qSpeedTest(send=0      , recv=128    )  32768 packets needed to receive 4.0MB in 2.007883000 sec for 1.992148 MB/sec for  16319.68 packets/sec (  0.061276 ms per packet)
qSpeedTest(send=0      , recv=256    )  16384 packets needed to receive 4.0MB in 1.046918000 sec for 3.820738 MB/sec for  15649.74 packets/sec (  0.063899 ms per packet)
qSpeedTest(send=0      , recv=512    )   8192 packets needed to receive 4.0MB in 0.676827000 sec for 5.909929 MB/sec for  12103.54 packets/sec (  0.082620 ms per packet)
qSpeedTest(send=0      , recv=1024   )   4096 packets needed to receive 4.0MB in 0.505348000 sec for 7.915338 MB/sec for   8105.31 packets/sec (  0.123376 ms per packet)

After:

Testing sending 1000 packets of various sizes:
qSpeedTest(send=0      , recv=0      ) in 0.073414000 sec for  13621.38 packets/sec (  0.073414 ms per packet) with standard deviation of   0.062441 ms
qSpeedTest(send=0      , recv=4      ) in 0.064519000 sec for  15499.31 packets/sec (  0.064519 ms per packet) with standard deviation of   0.059963 ms
qSpeedTest(send=0      , recv=8      ) in 0.059424000 sec for  16828.22 packets/sec (  0.059424 ms per packet) with standard deviation of   0.056796 ms
qSpeedTest(send=0      , recv=16     ) in 0.063597000 sec for  15724.01 packets/sec (  0.063597 ms per packet) with standard deviation of   0.059944 ms
qSpeedTest(send=0      , recv=32     ) in 0.066860000 sec for  14956.63 packets/sec (  0.066860 ms per packet) with standard deviation of   0.058246 ms
qSpeedTest(send=0      , recv=64     ) in 0.063729000 sec for  15691.44 packets/sec (  0.063729 ms per packet) with standard deviation of   0.057115 ms
qSpeedTest(send=0      , recv=128    ) in 0.070575000 sec for  14169.32 packets/sec (  0.070575 ms per packet) with standard deviation of   0.059277 ms
qSpeedTest(send=0      , recv=256    ) in 0.070976000 sec for  14089.27 packets/sec (  0.070976 ms per packet) with standard deviation of   0.061671 ms
qSpeedTest(send=0      , recv=512    ) in 0.082607000 sec for  12105.51 packets/sec (  0.082607 ms per packet) with standard deviation of   0.065792 ms
qSpeedTest(send=0      , recv=1024   ) in 0.099353000 sec for  10065.12 packets/sec (  0.099353 ms per packet) with standard deviation of   0.049501 ms
qSpeedTest(send=4      , recv=0      ) in 0.056516000 sec for  17694.11 packets/sec (  0.056516 ms per packet) with standard deviation of   0.054699 ms
qSpeedTest(send=4      , recv=4      ) in 0.065968000 sec for  15158.86 packets/sec (  0.065968 ms per packet) with standard deviation of   0.067132 ms
qSpeedTest(send=4      , recv=8      ) in 0.058242000 sec for  17169.74 packets/sec (  0.058242 ms per packet) with standard deviation of   0.063960 ms
qSpeedTest(send=4      , recv=16     ) in 0.067658000 sec for  14780.22 packets/sec (  0.067658 ms per packet) with standard deviation of   0.059159 ms
qSpeedTest(send=4      , recv=32     ) in 0.064881000 sec for  15412.83 packets/sec (  0.064881 ms per packet) with standard deviation of   0.055007 ms
qSpeedTest(send=4      , recv=64     ) in 0.062119000 sec for  16098.13 packets/sec (  0.062119 ms per packet) with standard deviation of   0.056333 ms
qSpeedTest(send=4      , recv=128    ) in 0.070882000 sec for  14107.95 packets/sec (  0.070882 ms per packet) with standard deviation of   0.062611 ms
qSpeedTest(send=4      , recv=256    ) in 0.066560000 sec for  15024.04 packets/sec (  0.066560 ms per packet) with standard deviation of   0.063069 ms
qSpeedTest(send=4      , recv=512    ) in 0.072380000 sec for  13815.97 packets/sec (  0.072380 ms per packet) with standard deviation of   0.056864 ms
qSpeedTest(send=4      , recv=1024   ) in 0.094272000 sec for  10607.60 packets/sec (  0.094272 ms per packet) with standard deviation of   0.066426 ms
qSpeedTest(send=8      , recv=0      ) in 0.059004000 sec for  16948.00 packets/sec (  0.059004 ms per packet) with standard deviation of   0.047414 ms
qSpeedTest(send=8      , recv=4      ) in 0.062650000 sec for  15961.69 packets/sec (  0.062650 ms per packet) with standard deviation of   0.056580 ms
qSpeedTest(send=8      , recv=8      ) in 0.062791000 sec for  15925.85 packets/sec (  0.062791 ms per packet) with standard deviation of   0.054558 ms
qSpeedTest(send=8      , recv=16     ) in 0.069848000 sec for  14316.80 packets/sec (  0.069848 ms per packet) with standard deviation of   0.060661 ms
qSpeedTest(send=8      , recv=32     ) in 0.060621000 sec for  16495.93 packets/sec (  0.060621 ms per packet) with standard deviation of   0.048762 ms
qSpeedTest(send=8      , recv=64     ) in 0.067326000 sec for  14853.10 packets/sec (  0.067326 ms per packet) with standard deviation of   0.051635 ms
qSpeedTest(send=8      , recv=128    ) in 0.065161000 sec for  15346.60 packets/sec (  0.065161 ms per packet) with standard deviation of   0.032026 ms
qSpeedTest(send=8      , recv=256    ) in 0.058371000 sec for  17131.79 packets/sec (  0.058371 ms per packet) with standard deviation of   0.012066 ms
qSpeedTest(send=8      , recv=512    ) in 0.066125000 sec for  15122.87 packets/sec (  0.066125 ms per packet) with standard deviation of   0.018305 ms
qSpeedTest(send=8      , recv=1024   ) in 0.077604000 sec for  12885.93 packets/sec (  0.077604 ms per packet) with standard deviation of   0.007076 ms
qSpeedTest(send=16     , recv=0      ) in 0.049753000 sec for  20099.29 packets/sec (  0.049753 ms per packet) with standard deviation of   0.008330 ms
qSpeedTest(send=16     , recv=4      ) in 0.049638000 sec for  20145.86 packets/sec (  0.049638 ms per packet) with standard deviation of   0.010060 ms
qSpeedTest(send=16     , recv=8      ) in 0.048579000 sec for  20585.03 packets/sec (  0.048579 ms per packet) with standard deviation of   0.005388 ms
qSpeedTest(send=16     , recv=16     ) in 0.049604000 sec for  20159.66 packets/sec (  0.049604 ms per packet) with standard deviation of   0.005625 ms
qSpeedTest(send=16     , recv=32     ) in 0.051731000 sec for  19330.77 packets/sec (  0.051731 ms per packet) with standard deviation of   0.010170 ms
qSpeedTest(send=16     , recv=64     ) in 0.054381000 sec for  18388.78 packets/sec (  0.054381 ms per packet) with standard deviation of   0.010240 ms
qSpeedTest(send=16     , recv=128    ) in 0.052270000 sec for  19131.43 packets/sec (  0.052270 ms per packet) with standard deviation of   0.005497 ms
qSpeedTest(send=16     , recv=256    ) in 0.062754000 sec for  15935.24 packets/sec (  0.062754 ms per packet) with standard deviation of   0.016370 ms
qSpeedTest(send=16     , recv=512    ) in 0.060756000 sec for  16459.28 packets/sec (  0.060756 ms per packet) with standard deviation of   0.009633 ms
qSpeedTest(send=16     , recv=1024   ) in 0.090837000 sec for  11008.73 packets/sec (  0.090837 ms per packet) with standard deviation of   0.025883 ms
qSpeedTest(send=32     , recv=0      ) in 0.049081000 sec for  20374.48 packets/sec (  0.049081 ms per packet) with standard deviation of   0.007256 ms
qSpeedTest(send=32     , recv=4      ) in 0.047704000 sec for  20962.60 packets/sec (  0.047704 ms per packet) with standard deviation of   0.004123 ms
qSpeedTest(send=32     , recv=8      ) in 0.050561000 sec for  19778.09 packets/sec (  0.050561 ms per packet) with standard deviation of   0.010482 ms
qSpeedTest(send=32     , recv=16     ) in 0.051938000 sec for  19253.73 packets/sec (  0.051938 ms per packet) with standard deviation of   0.008375 ms
qSpeedTest(send=32     , recv=32     ) in 0.051384000 sec for  19461.31 packets/sec (  0.051384 ms per packet) with standard deviation of   0.008739 ms
qSpeedTest(send=32     , recv=64     ) in 0.058200000 sec for  17182.13 packets/sec (  0.058200 ms per packet) with standard deviation of   0.013832 ms
qSpeedTest(send=32     , recv=128    ) in 0.052266000 sec for  19132.90 packets/sec (  0.052266 ms per packet) with standard deviation of   0.005973 ms
qSpeedTest(send=32     , recv=256    ) in 0.053797000 sec for  18588.40 packets/sec (  0.053797 ms per packet) with standard deviation of   0.005621 ms
qSpeedTest(send=32     , recv=512    ) in 0.071394000 sec for  14006.78 packets/sec (  0.071394 ms per packet) with standard deviation of   0.018682 ms
qSpeedTest(send=32     , recv=1024   ) in 0.080212000 sec for  12466.96 packets/sec (  0.080212 ms per packet) with standard deviation of   0.012564 ms
qSpeedTest(send=64     , recv=0      ) in 0.048958000 sec for  20425.67 packets/sec (  0.048958 ms per packet) with standard deviation of   0.007057 ms
qSpeedTest(send=64     , recv=4      ) in 0.048045000 sec for  20813.82 packets/sec (  0.048045 ms per packet) with standard deviation of   0.005918 ms
qSpeedTest(send=64     , recv=8      ) in 0.053285000 sec for  18767.01 packets/sec (  0.053285 ms per packet) with standard deviation of   0.011614 ms
qSpeedTest(send=64     , recv=16     ) in 0.052871000 sec for  18913.96 packets/sec (  0.052871 ms per packet) with standard deviation of   0.011088 ms
qSpeedTest(send=64     , recv=32     ) in 0.054907000 sec for  18212.61 packets/sec (  0.054907 ms per packet) with standard deviation of   0.011033 ms
qSpeedTest(send=64     , recv=64     ) in 0.050928000 sec for  19635.56 packets/sec (  0.050928 ms per packet) with standard deviation of   0.006639 ms
qSpeedTest(send=64     , recv=128    ) in 0.052239000 sec for  19142.79 packets/sec (  0.052239 ms per packet) with standard deviation of   0.005014 ms
qSpeedTest(send=64     , recv=256    ) in 0.055185000 sec for  18120.87 packets/sec (  0.055185 ms per packet) with standard deviation of   0.007720 ms
qSpeedTest(send=64     , recv=512    ) in 0.058989000 sec for  16952.31 packets/sec (  0.058989 ms per packet) with standard deviation of   0.007119 ms
qSpeedTest(send=64     , recv=1024   ) in 0.076331000 sec for  13100.84 packets/sec (  0.076331 ms per packet) with standard deviation of   0.005977 ms
qSpeedTest(send=128    , recv=0      ) in 0.051201000 sec for  19530.87 packets/sec (  0.051201 ms per packet) with standard deviation of   0.009745 ms
qSpeedTest(send=128    , recv=4      ) in 0.049078000 sec for  20375.73 packets/sec (  0.049078 ms per packet) with standard deviation of   0.007373 ms
qSpeedTest(send=128    , recv=8      ) in 0.048959000 sec for  20425.25 packets/sec (  0.048959 ms per packet) with standard deviation of   0.006551 ms
qSpeedTest(send=128    , recv=16     ) in 0.053476000 sec for  18699.98 packets/sec (  0.053476 ms per packet) with standard deviation of   0.010515 ms
qSpeedTest(send=128    , recv=32     ) in 0.050877000 sec for  19655.25 packets/sec (  0.050877 ms per packet) with standard deviation of   0.006253 ms
qSpeedTest(send=128    , recv=64     ) in 0.051106000 sec for  19567.17 packets/sec (  0.051106 ms per packet) with standard deviation of   0.006136 ms
qSpeedTest(send=128    , recv=128    ) in 0.053410000 sec for  18723.09 packets/sec (  0.053410 ms per packet) with standard deviation of   0.007398 ms
qSpeedTest(send=128    , recv=256    ) in 0.053649000 sec for  18639.68 packets/sec (  0.053649 ms per packet) with standard deviation of   0.005514 ms
qSpeedTest(send=128    , recv=512    ) in 0.061364000 sec for  16296.20 packets/sec (  0.061364 ms per packet) with standard deviation of   0.011483 ms
qSpeedTest(send=128    , recv=1024   ) in 0.076944000 sec for  12996.46 packets/sec (  0.076944 ms per packet) with standard deviation of   0.006585 ms
qSpeedTest(send=256    , recv=0      ) in 0.050720000 sec for  19716.09 packets/sec (  0.050720 ms per packet) with standard deviation of   0.009306 ms
qSpeedTest(send=256    , recv=4      ) in 0.051201000 sec for  19530.87 packets/sec (  0.051201 ms per packet) with standard deviation of   0.008261 ms
qSpeedTest(send=256    , recv=8      ) in 0.048852000 sec for  20469.99 packets/sec (  0.048852 ms per packet) with standard deviation of   0.005998 ms
qSpeedTest(send=256    , recv=16     ) in 0.049769000 sec for  20092.83 packets/sec (  0.049769 ms per packet) with standard deviation of   0.005678 ms
qSpeedTest(send=256    , recv=32     ) in 0.051966000 sec for  19243.35 packets/sec (  0.051966 ms per packet) with standard deviation of   0.006798 ms
qSpeedTest(send=256    , recv=64     ) in 0.051089000 sec for  19573.69 packets/sec (  0.051089 ms per packet) with standard deviation of   0.005669 ms
qSpeedTest(send=256    , recv=128    ) in 0.053214000 sec for  18792.05 packets/sec (  0.053214 ms per packet) with standard deviation of   0.006338 ms
qSpeedTest(send=256    , recv=256    ) in 0.054821000 sec for  18241.18 packets/sec (  0.054821 ms per packet) with standard deviation of   0.005743 ms
qSpeedTest(send=256    , recv=512    ) in 0.060782000 sec for  16452.24 packets/sec (  0.060782 ms per packet) with standard deviation of   0.008451 ms
qSpeedTest(send=256    , recv=1024   ) in 0.081474000 sec for  12273.85 packets/sec (  0.081474 ms per packet) with standard deviation of   0.013432 ms
qSpeedTest(send=512    , recv=0      ) in 0.049354000 sec for  20261.78 packets/sec (  0.049354 ms per packet) with standard deviation of   0.006584 ms
qSpeedTest(send=512    , recv=4      ) in 0.051040000 sec for  19592.48 packets/sec (  0.051040 ms per packet) with standard deviation of   0.009613 ms
qSpeedTest(send=512    , recv=8      ) in 0.050489000 sec for  19806.29 packets/sec (  0.050489 ms per packet) with standard deviation of   0.008578 ms
qSpeedTest(send=512    , recv=16     ) in 0.050745000 sec for  19706.38 packets/sec (  0.050745 ms per packet) with standard deviation of   0.008309 ms
qSpeedTest(send=512    , recv=32     ) in 0.051861000 sec for  19282.31 packets/sec (  0.051861 ms per packet) with standard deviation of   0.008396 ms
qSpeedTest(send=512    , recv=64     ) in 0.050947000 sec for  19628.24 packets/sec (  0.050947 ms per packet) with standard deviation of   0.005801 ms
qSpeedTest(send=512    , recv=128    ) in 0.076467000 sec for  13077.54 packets/sec (  0.076467 ms per packet) with standard deviation of   0.045961 ms
qSpeedTest(send=512    , recv=256    ) in 0.083959000 sec for  11910.58 packets/sec (  0.083959 ms per packet) with standard deviation of   0.048536 ms
qSpeedTest(send=512    , recv=512    ) in 0.086383000 sec for  11576.35 packets/sec (  0.086383 ms per packet) with standard deviation of   0.044516 ms
qSpeedTest(send=512    , recv=1024   ) in 0.105816000 sec for   9450.37 packets/sec (  0.105816 ms per packet) with standard deviation of   0.050868 ms
qSpeedTest(send=1024   , recv=0      ) in 0.054342000 sec for  18401.97 packets/sec (  0.054342 ms per packet) with standard deviation of   0.012813 ms
qSpeedTest(send=1024   , recv=4      ) in 0.052497000 sec for  19048.71 packets/sec (  0.052497 ms per packet) with standard deviation of   0.011391 ms
qSpeedTest(send=1024   , recv=8      ) in 0.051663000 sec for  19356.21 packets/sec (  0.051663 ms per packet) with standard deviation of   0.007735 ms
qSpeedTest(send=1024   , recv=16     ) in 0.057973000 sec for  17249.41 packets/sec (  0.057973 ms per packet) with standard deviation of   0.013560 ms
qSpeedTest(send=1024   , recv=32     ) in 0.053555000 sec for  18672.39 packets/sec (  0.053555 ms per packet) with standard deviation of   0.008089 ms
qSpeedTest(send=1024   , recv=64     ) in 0.054065000 sec for  18496.25 packets/sec (  0.054065 ms per packet) with standard deviation of   0.006756 ms
qSpeedTest(send=1024   , recv=128    ) in 0.059441000 sec for  16823.41 packets/sec (  0.059441 ms per packet) with standard deviation of   0.014128 ms
qSpeedTest(send=1024   , recv=256    ) in 0.057164000 sec for  17493.53 packets/sec (  0.057164 ms per packet) with standard deviation of   0.006987 ms
qSpeedTest(send=1024   , recv=512    ) in 0.061649000 sec for  16220.86 packets/sec (  0.061649 ms per packet) with standard deviation of   0.014232 ms
qSpeedTest(send=1024   , recv=1024   ) in 0.092846000 sec for  10770.52 packets/sec (  0.092846 ms per packet) with standard deviation of   0.028475 ms
Testing receiving 4.0MB of data using varying receive packet sizes:
qSpeedTest(send=0      , recv=32     ) 131072 packets needed to receive 4.0MB in 6.936484000 sec for 0.576661 MB/sec for  18896.03 packets/sec (  0.052921 ms per packet)
qSpeedTest(send=0      , recv=64     )  65536 packets needed to receive 4.0MB in 3.770819000 sec for 1.060777 MB/sec for  17379.78 packets/sec (  0.057538 ms per packet)
qSpeedTest(send=0      , recv=128    )  32768 packets needed to receive 4.0MB in 2.111190000 sec for 1.894666 MB/sec for  15521.10 packets/sec (  0.064428 ms per packet)
qSpeedTest(send=0      , recv=256    )  16384 packets needed to receive 4.0MB in 1.039231000 sec for 3.849000 MB/sec for  15765.50 packets/sec (  0.063430 ms per packet)
qSpeedTest(send=0      , recv=512    )   8192 packets needed to receive 4.0MB in 0.505272000 sec for 7.916528 MB/sec for  16213.05 packets/sec (  0.061679 ms per packet)
qSpeedTest(send=0      , recv=1024   )   4096 packets needed to receive 4.0MB in 0.334587000 sec for 11.955037 MB/sec for  12241.96 packets/sec (  0.081686 ms per packet)

I think they look pretty much within error margins.

labath updated this revision to Diff 67507.Aug 10 2016, 5:04 AM
labath edited edge metadata.

The patch I used to run the tests. Still very much WIP.

clayborg accepted this revision.Aug 10 2016, 9:18 AM
clayborg edited edge metadata.

OK. I can't complain if it doesn't affect performance and could allow multiple packets to be in flight with threading, though I think solving this issue with threading is a bit of a hack, but it should work. Thanks for running the perf tests.

This revision is now accepted and ready to land.Aug 10 2016, 9:18 AM

OK. I can't complain if it doesn't affect performance and could allow multiple packets to be in flight with threading, though I think solving this issue with threading is a bit of a hack, but it should work. Thanks for running the perf tests.

Thanks. I understand your reluctance. I am not completely convinced of the optimality of the solution, but to me it looks like the best way forward at the moment. I want to proceed slowly with this, to make it as little "hacky" and obtrusive as possible.

I can't commit this patch yet because it replaces the sequence mutex with a non-recursive rwlock. The biggest offender using recursive locks here is the gdb-remote register context, so I have started to do some cleanups there first, so I am able to get rid of the recursion. (The motivation for D23553).

labath abandoned this revision.Sep 5 2016, 8:16 AM

I am abandoning this for the time being in favour of D24236. The actual implementation of it turned out to be more hackish then I anticipated.