Comments for paper cruncher

Comment on The Hashing Trick by diet

diet — Thu, 16 May 2013 15:44:40 +0000

This can be the ideal blog for anybody who wants to keep in mind this kind of topic. Anyone recognize a substantial amount of the practically not easy to argue readily available (not far too I really will want…HaHa). You actually put a new spin spanning a topic that’s been said about for some time. Amazing stuff, merely fantastic!

Comment on Being truly asynchronous with Tornado by breaking news

breaking news — Thu, 07 Feb 2013 10:57:46 +0000

Attractive part of content. I simply stumbled upon your weblog and in accession capital to say that I acquire in fact loved account your blog posts. Anyway I will be subscribing to your augment and even I success you get admission to persistently quickly.

Comment on Being truly asynchronous with Tornado by papercruncher

papercruncher — Sat, 19 Jan 2013 02:56:51 +0000

Yield as a keyword is built in, but the whole yield Task() model is actually a really clever hack in Tornado that handles the callback soup for you.

Comment on Power of k random choices by papercruncher

papercruncher — Sat, 19 Jan 2013 02:55:26 +0000

Hi Mike,

The case where k=n can actually lead to herding when the server information is stale. The Mitzenmacher shows that and I’ve “proven” the same thing with simulations. I’m not a fan of the least connections algorithm especially for the case where not all calls carry the same weight. For example, if some requests take a long time to process and others don’t, the nginx algorithm fails spectacularly (unless of course you are over provisioned). The reason for that is because not every connection weighs the same so the information nginx is acting upon is not accurate.

In real life, this algorithm is shown to work best when k << n

Comment on Being truly asynchronous with Tornado by yavorgeorgiev

yavorgeorgiev — Thu, 17 Jan 2013 02:59:13 +0000

Is the yield/task model built into python or does Tornado compile that down into some callback soup?

Comment on Power of k random choices by Mike K Tung (@mikektung)

Mike K Tung (@mikektung) — Wed, 16 Jan 2013 01:44:32 +0000

Hey Marios, I didn’t examine the paper, but (4) can’t dominate (3) because they are equivalent in the case that k=n, right? For estimating which server is least loaded, nginx has a pretty good option in the latest build of the upstream module called ‘least_conn’ that works reasonably well.

Comment on Q-Digest by ekirpichov

ekirpichov — Wed, 27 Jun 2012 18:16:40 +0000

I’m not sure if that particular digest can indeed be the result of a sequence of inserts; this is a crafted example, simplified from a more complex example that actually resulted from a sequence of inserts (I found that in the debugger and then hand-simplified it while preserving the bug).

Comment on Q-Digest by papercruncher

papercruncher — Wed, 27 Jun 2012 08:23:56 +0000

I’ve been looking at the picture you posted on your G+ and I’m having a hard time figuring out how you would end up with that initial digest to begin with. While you are right that the specific snapshot does not violate any of the two properties, the authors only examine building the q-digest in the case where all the data is available. They handle the case of more incoming data by merging Q-Digest as they say in the first paragraph in section 3.3. There is no INSERT() operation. Even if that element in the first top diagram came via MERGE(), it wouldn’t be there as a COMPRESS() operation would have moved it to the root.

I apologize if you already covered this in your post, I failed at visualizing the other tree q-digest you described.

Comment on Q-Digest by Eugene Kirpichov

Eugene Kirpichov — Tue, 26 Jun 2012 21:44:46 +0000

They promised to, but eventually didn’t In the meantime, I had a few discussions about this with my friends and ended up with this implementation: https://github.com/clearspring/stream-lib/blob/master/src/main/java/com/clearspring/analytics/stream/quantile/QDigest.java

Comment on Q-Digest by papercruncher

papercruncher — Tue, 26 Jun 2012 20:29:19 +0000

Thanks Eugene, I’ll look into it. Did the authors get back to you?