Running a successful open-source project is very much like running a business: it takes skill, time and money (there may not be a direct monetary cost, but there’s an opportunity cost in everything). You need to market your product: why should people use your software rather than anyone else’s? In fact, with open-source software development, you need to explain to people why they should spend their own free time helping you make your product better.

You need to do what businesses have been doing for years: you need to engage with your customers.

You can barely walk down the street today without being told that your favourite local café now has an account on Twitter that you should follow. Thousands of articles keep popping up everywhere on how businesses can “make the most” of their presence on Twitter by monitoring what people are saying about them, and by engaging with their customers and clients. Twitter even have a microsite and a 20-page downloadable guide to their service for small businesses.

Developers are, more often than not, inherently early-adopters of any new technology, and use of Twitter is no exception¹. As such, it seems strange to think that businesses have caught a jump on open-source developers when it comes to using the real-time web to engage with the people that use their product. But from certain quarters, this does appear to be changing.

Redis: seriously proactive community engagement

Redis, in case you weren’t aware, is an in-memory key-value store. “What, like memcached?” Yes, but super-charged with data-structures. Your values aren’t simply limited to being a string: they can also be hashes, lists, sets and sorted sets.

Redis’ primary developer is Salvatore Sanfilippo (antirez on Twitter). Here is one man who, as far as I’ve seen, has gone further than any other open-source developer in terms of proactively engaging with the community that’s grown up around his project. And this is no small project: sites which use it for some serious crunching include heavyweights such as Instagram, Twitter, Craigslist and GitHub. Last week, we saw first-hand exactly how far Salvatore goes to keep the community happy.

We’ve recently been having somewhat of a spring clean of some of the parts of our White Label Dating platform, and last week it was the turn of the data we’re storing in Redis. We use Redis for a number of transient caching tasks, as well as for stats collection for our background-processing jobs (using the redistat gem). Over the last few months, our memory usage through Redis had reached a whopping 29Gb, so we felt it was definitely time to find out exactly what we were throwing in there.

After a while auditing what we were putting in to Redis, and with some help from the excellent redis-rdb-tools, we’d soon taken our Redis usage down from 29Gb to a much more managable 2.5Gb (as well as making some code changes to make better use of Redis’ key expiration so we wouldn’t end up in the same situation again). Of course, this kind of graph cliff is the sort of thing you instantly share with the world:

Today's graphing cliff: finding the 240 rogue keys (out of ~800k) that were taking up 24Gb (90%) of RAM in Redis yfrog.com/kkx3hcp

— Global Personals Dev (@globaldev) April 18, 2012

There were a couple of replies from people who’d been in a similar position, but other than giving ourselves a small pat on the back, we thought no more about it. Until this message popped up in our Twitter feed:

@Joolz @globaldev @boffbowsh what about if I add (today) a --find-big-keys to redis-cli? May be very useful in this cases.

— Salvatore Sanfilippo (@antirez) April 18, 2012

Well, what can you say to that? Two hours later:

redis-cli --bigkeys implementation pushed into unstable (2.6 / 2.4 soon), example: bit.ly/HTkpin /cc @Joolz @globaldev @boffbowsh

— Salvatore Sanfilippo (@antirez) April 18, 2012

Another hour later, and the change had been backported into the current maintenance (2.4) and upcoming (2.6) branches; the following day Redis 2.4.11 was released with the new --bigkeys command. From an off-the-cuff tweet about digging around for big keys, the maintainer of the project had seen the conversation, responded, suggested something he could do to help (immediately, not “at some point in the future”,) and committed the change in two hours.

Now that’s what you call customer engagement.