The arrow of time

Ivan Voras' blog

Bullet Cache - RC1 and main features

I've just uploaded the first Release Candidate version of Bullet Cache! It is basically feature complete and done, and I'm happy to say that it looks like I have a small number of users and also some feedback on the project - so keep it up :) At this point I'd like to shortly talk about what made me write Bullet Cache - which also leads to why it was done the way it is and what are its main strengths. For the impatient, these are flexibility in cached data expiry and performance, but read on for the details...

Previous posts in the Bullet Cache series:

I have been using memcached almost as soon as it appeared, and I can testify that the improvements it brought to the game were impressive. But that early version of memcached was single-threaded, ran a slow ASCII-based network protocol and had very little options for advanced data managament. As I was slow in writing Bullet Cache (hey, life got in the way), memcached gradually fixed all but the last quibble: it is still a pure key-value data store.

One thing in particular always irked me: the inability to delete data from the cache (i.e. "expire" it) by some criteria external to the usual "list of keys" approach. What matters in real-life application is the ability to e.g. expire all cached records belonging to a certain web page, or to a certain user, or whatever other group, without actually maintaining a list of keys which contain such information (because then you need to store this list somewhere...). This was my primary motivation, feature-wise, and Bullet Cache now contains very flexible bulk data management functionalities.

The key to this ability is that Bullet Cache records have simple metadata "tags" associated with them, and these tags can be used to perform simple data queries. By using record tags, loosely defined groups of records can be expired (or indeed retrieved) in a single efficient operation. This gives the application developers freedom to implement functions like: "fetch all cached data relevant to a single Web page" and "expire all cached data relevant to a single user." Since records can have an arbitrary number of tags assigned to them, complex behaviour can be implemented, for example by "tainting" certain composite records with tags representing user data or page data, and then expiring (or retrieving) in bulk all records having specific tags. See the User Guide for some more examples!

And of course, Bullet Cache is very fast, of which I'm especially proud of. There are no single global locks for reading or writing, and concurrency is extreme.

Try it! If the SourceForge download servers are busy, try again :D

#1 Re: Bullet Cache - RC1 and main features

Added on 2011-11-10T11:38 by devilping

Nice. Thanks. I have carried out some tests and is happy in the speed. Whether there are chances to see it in FreeBSD ports collection ?

#2 Re: Bullet Cache - RC1 and main features

Added on 2011-11-10T12:03 by Ivan Voras

The chances are excellent - this is the first thing I will do as soon as the ports tree is unfrozen from the 9.0 release cycle (i.e. after FreeBSD 9.0 is released).

#3 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T00:52 by devilping

Some benchmarks of RC1 on 24 core system ( -t 24 for srv and clients):

1)

BULLET:

Generated 50000 records: 5589 KB (819 KB keys, 4770 KB data). Starting 24

threads.

Average data size: 97 bytes.

24 clients * 500000 requests = 12000000 requests: 0.00 seconds. That's inf

requests/s

50000 records from the record pool have been put.

32.704u 117.467s 0:42.63

12'000'000 requests by 97 bytes in 42 seconds (??!!)

MEMCACHED:

Generated 50000 records: 5589 KB (819 KB keys, 4770 KB data). Starting 24

threads.

Average data size: 97 bytes.

24 clients * 500000 requests = 12000000 requests: 0.00 seconds. That's inf

requests/s

50000 records from the record pool have been put.

60.183u 228.645s 4:00.66

12'000'000 requests by 97 bytes in 4 minutes 42 seconds

REDIS:

Generated 50000 records: 5589 KB (819 KB keys, 4770 KB data). Starting 24

threads.

Average data size: 97 bytes.

24 clients * 500000 requests = 12000000 requests: 0.00 seconds. That's inf

requests/s

50000 records from the record pool have been put.

44.840u 275.863s 4:43.10

12'000'000 requests by 97 bytes in 4 minutes 43 seconds.



And my personal benchmark via php module on one thread
(100 request for 100000 bytes data)

Bytes count: 10011400

PUTs count: 100

Memcache PUTs time: 5.60080099

Memcache GETs time: 0.57081795

Redis PUTs time: 0.20418906

Redis GETs time: 0.07123494

Bullet PUTs time: 0.11589503

Bullet GETs time: 0.12635112

BTW:

If i kill one copy of mdcached opened a unix socket, the next process it will be started but probably doesn't check that /tmp/mdcached.sock already exist and socket on this way doesn't open. Accordingly, clients can't connect before:
killall -9 mdcached ; rm -f /tmp/mdcached.sock; mdcached

#4 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T00:53 by jgh

any chance in providing a java api?

#5 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T02:31 by Ivan Voras

Thanks for posting this, these are some interesting results - and thanks for the unix socket suggestion. Can you contact me via email (ivoras@ gmail), I'm interested in some details.

#6 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T14:35 by Ivan Voras

@jgh: Java API would be possible but it would be much slower than any other API since the network protocol is binary; the Java library would have to do much more processing than the languages using native C code. But on the other hand, there could be a JNI library which would avoid all this processing. I'll consider writing a Java library if there are interested users.

#7 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T23:21 by jgh

Thanks for the update.

JNI looks like the way to go. I think you may attract more users if there was a a Java library

#8 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T23:34 by Ivan Voras

Certainly, Java is very wide-spread, but it's really inconvenient.For JNI to be wide-sread, I would have to offer dynamic libraries (DLLs and SOs) for many environments: Windows, Linux, MacOS X, FreeBSD...

And on the other hand native (interpreted) Java uses big endian for integer encoding, and every single number I get from the network I need to transcode from little endian into big endian, then have Java read it and convert it back into little endian so it can do something with it (assuming an x86 CPU) - it will be VERY slow. Even if the server is fast enough, people doing benchmarks with the slow client will think everything is slow.

Java, basically, sucks.

#9 Re: Bullet Cache - RC1 and main features

Added on 2011-11-11T23:49 by Ivan Voras

I've just looked at the state of native (interpreted) Java binary protocol encoding / decoding and it's very bad - I would have to either do everything manually or redeclare / reimplement every single used data structure with something like Preon.

JNI will probably be the only way to do it with reasonable effort (of course, sponsors are welcome :) ).

Post your comment here!

Your name:
Comment title:
Text:
Type "xxx" here:

Comments are subject to moderation and will be deleted if deemed inappropriate. All content is © Ivan Voras. Comments are owned by their authors... who agree to basically surrender all rights by publishing them here :)