Memcached and You
Tweemotional.com was not doing so well. It had too much information in the database which was also missing an index on the date causing the SELECT to do a file sort on millions of rows.
So I took the opportunity to flush the database since raw tweet data that is more than 24 hours old is ignored anyway and added in the index.
However, I also plugged in Memcached support. With Memcached you can do the query on the database once, take the result and stuff it in memcached. The next time you need the data you simply query memcached which returns the result instantly.
So now when the backend scripts for Tweemotional.com are gather information from Twitter they also update the Memcached versions of what the site needs to display the data. So now the site itself runs very quickly since it’s not hitting the database nearly as much.
If you want to make use of Memcached for your site you’ll need pretty much any computer you can get. I have it running on a PIII 533Mhz system with 512MB of memory. Obviously if you have a site that could make use of more space to store cached objects then you’ll need a system with more RAM or diskspace. For best performance you want enough memory that memcached doesn’t have to pull information from the disk before sending it over.
There are Windows binaries available but they are for an older version. If you want to run the latest version you’ll probably want to just use some version of Linux. I prefer Ubuntu. The 8.04 version installed and ran just fine on my system. I turned off the graphical flair for the GUI and it runs rather smoothly.
There are existing libraries for Memcached for PHP and C# as well as other languages. Enyim.Caching is what I use with C#. I then wrote a simple static class wrapper around that so I can just use memcached.get, memcached.set and memcached.remove and not have to deal with any initialization.
Another reason for using a wrapper is that memcached doesn’t like keys with certain characters. To avoid this problem the wrapper automatically MD5’s the key I pass in before it passed it along to memcached. In my code I can then use whatever key I want.
The other thing you have to remember with C# is that you must explicity make your classes Serializable or they cannot be cached through memcached. I don’t believe PHP has this issue. I’m not sure if there’s anything in PHP that isn’t serializable by default.
If you’re running any sort of website that is doing some heavy lifting with queries it would be to your benefit to take a look at memcached to see what it can do for you.
