More Facebook data management adventures; check the link below for more details and a tech talk video
Social plugins have become an important and growing source of traffic for millions of websites over the past year. We released a new version of Insights for Websites last week to give site owners better analytics on how people interact with their content and to help them optimize their websites in real time.
To accomplish this, we had to engineer a system that could process over 20 billion events per day (200,000 events per second) with a lag of less than 30 seconds. This system had to be able to process many different types of events, and do so in a way that accounted for an uneven distribution of keys.