The site I work on fetches significant amounts of data on-demand from a remote data centre via a JSON feed. In order to provide resilience against the site being unavailable I implemented Redis on Windows as a read-through cache. (None Windows operating systems were not permitted at the time). I also implemented a refresh-ahead cache service which would ensure ‘hot’ content was always fresh.
Although the solution worked I was recently asked to look at replacements for several reasons
- Redis on Windows wasn’t a supported platform
- We couldn’t cluster it trivially as members within a distributed Redis cluster were not equal so some would need to be read slaves and some write masters. This would introduce more complexity to our code, deployment and could have introduced a single point of failure.
- When a Redis process reached it’s 32-bit Windows memory limit of 2GB it would crash. We had deployed 4 instances to each server and we distributed the read/writes across these instances.
I chose AppFabric Caching Services from Microsoft’s for a few reasons, and within an hour or so had replaced Redis with AppFabric.
- It supports equal cluster members, so all members are read/write
- Native 64-bit support meant no 2GB process limit
- Active support and community
- Like Redis, it’s free
Having deployed AppFabric in a production environment for several months now, I thought i’d write a few findings:
- Installation is not entirely scriptable. Despite efforts by Microsoft to make everything scriptable with PowerShell the very first installation and subsequent upgrades had to be done via the Wizard. After that, the PowerShell commandlets were available for use.
- There can be long waits (5 minutes or so) during startup when using a distributed cache. It makes sense that the various nodes require time to synchronise, but the error messages that are reported are cryptic.
- It’s rock solid once you get it running. We’ve used it now for about 3 months in a 3 server cluster storing items in the cache of upto 50MB and it’s fast.
- You can’t enumerate the cache keys. With Redis our refresh-ahead service would iterate over the keys and freshen content if necessary, we’ve had to drop this functionality and take a small performance hit with AppFabric.
- Upgrading isn’t entirely trivial. I upgraded our servers from v1.1 to v1.2 and although it’s designed to have heterogenous nodes in the cluster with differing versions, some of the security settings appeared to have changed between versions meaning that our website ‘client’ was not permitted access to the cluster. Luckily we still had Redis running in the background and a config switch directed the website at that temporarily.
- Changing cache settings requires you to delete the cache. Carefully consider the largest size object you want in the cache when you create the cache – if you want to change it in future, you’ll need to either start a new cache and update your config, or take the site off-line whilst you upgrade your cache.
- You need to specify the CacheItemVersion when deleting items. Use the following to remove from the cache. When I tried Remove(key) I always got a false returned and the item remained. This makes sense that in a distributed cache it needs to know explicitly which version to remove, but seemed to only occur when we had a cache cluster.
public bool Remove(string key)
var dataCacheItem = _cache.Get(key);
return _cache.Remove(key, dataCacheItem.Version);
Overall I’m pretty happy with AppFabric, but some of these gotchas, the general lack of enthusiasm on forums etc. concerns me slightly about the future. But hey, it didn’t take long to write the concrete implementation, tests and config switches needed to get it into the codebase, so it won’t take long to replace it if it does get dropped.