Saving Twitter in Three Easy Steps
I don’t think I’m alone in seeing Twitter’s service deteriorate to the point where euthanasia seems like the only humane option.
Without any obvious intimate knowledge of how the service is architected, these are my three easy steps to hunt and kill the fail whale.
1. Stop accepting new registrations
In the current situation I can’t for the life of me understand why they are still accepting new people in. It is obvious the system can not handle the strain on peak times (its bad enough throughout the rest of the day).
Also worth noting is that a significant portion of those new registrations are spambots, I’ve got at least 5 or 6 new ones of those following me each day.
2. No friends timeline on the home page
When you’re logged in, the first page you see is the “what are you doing” input box and your friends timeline, how many followers, direct messages you have etc.
Just think of the amount of excess queries that is for people that just want to use the web interface to leave a new message. Why put strain on the database for that? Keep it simple and use a Google like interface with just the input box and a submit button on the home page.
3. Enforce caching
Many people use Twitter clients like Twitterific, Twhirl etc. and get annoyed about the API limit that gets enforced. There doesn’t appear to be anything like that in place when using the Twitter web interface.
By enforcing caching you can make sure database queries only get run every x minutes, this can be throttled based on the current server load (much like there now is an API call to determine the request limit).
These three steps are nothing more than common sense really but should go a long way into making Twitter usable again. Moving forward though they should really start looking at data push strategies to optimize things further. I’ve had some interesting discussions about that possibility and hope to do a separate blog post about it.
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.0 Belgium License.
Twitter rocks, but yeah, it’s buckling under the weight of its own success. Action needs to be taken one way or another. These ideas would be a good start.
The whole thing is coded in Ruby on Rails with MySQL as the database server. The poor performance of RoR notably in database-related areas (http://www.infoq.com/articles/Rails-Performance) combined with MySQL not being a good high-performance database is killing them.
MySQL only has one-way replication. This means Twitter can’t build a proper round-robin system and all updates have to go through a Master database server. All reads, however, can go through any number of slaves. But it’s the updates that kill Twitter, I’m sure, not the reads. Even MySQL with their poor support for indexes should be able to handle that many reads with a database farm.
Twitter wrote a blog entry on their database system explaining what their problems were and how they were going to tackle those problems.
Sorry Peter–but that’s sort of silly. I mean, sure, I suppose if everyone toned down how they use it it might help–but these are their problems to solve–and they’re totally solve-able.
If it was simply a problem of how we use it then it wouldn’t be us.. it’d be people like scoble.
Here’s my vid that explains it so clearly: http://www.youtube.com/watch?v=93dGW_hDuQ0
I see your point Phillip but lets be a bit pragmatic about this — twitter needs to take some measures to keep the service up and running.
Obviously they ultimately need to be able to handle the load and as many Scoble’s as gets thrown at them but the number one goal should be to make sure they continue to have a service for them to fix.
The approach right now with randomly disabling features while keeping new registrations flooding in is just pissing off users.
Loved that vid by the way