Comments on: Scaling http://www.dronamraju.com/blog/2005/10/scaling.html Opinions on Business Technology & Life from Ravi Dronamraju Tue, 07 Oct 2008 13:41:38 +0000 http://wordpress.org/?v=2.6.2 By: Ravi Dronamraju http://www.dronamraju.com/blog/2005/10/scaling.html#comment-56 Ravi Dronamraju Sat, 29 Oct 2005 18:48:43 +0000 http://www.dronamraju.com/blog/?p=64#comment-56 Yes. It is important to understand your data before deciding to partition it. The key steps in my mind are - Understanding your data to identify partitionable dimensions. Can this data be partitioned by time, location, user, data specific attribute? - How are the partitions sized? For example, if you partition data by location, it's quite expected to have more data in New york partition vs Iowa partition. - Having different partitions means that different partitions can also be of slightly different scales. For example you can have more machines serving california partition vs neveda partition - Of course, you have to understand your data to make these judgements Yes. It is important to understand your data before deciding to partition it. The key steps in my mind are
- Understanding your data to identify partitionable dimensions. Can this data be partitioned by time, location, user, data specific attribute?
- How are the partitions sized? For example, if you partition data by location, it’s quite expected to have more data in New york partition vs Iowa partition.
- Having different partitions means that different partitions can also be of slightly different scales. For example you can have more machines serving california partition vs neveda partition
- Of course, you have to understand your data to make these judgements

]]> By: Jeffrey Friedl http://www.dronamraju.com/blog/2005/10/scaling.html#comment-54 Jeffrey Friedl Fri, 28 Oct 2005 11:47:39 +0000 http://www.dronamraju.com/blog/?p=64#comment-54 One problem with the many-partition DB is potential skew, with replication and with backups. You have to be very careful to either design the app so that such skew doesn't matter, or somehow ensure that it doesn't happen. The latter is probably impossible, though. One problem with the many-partition DB is potential skew, with replication and with backups. You have to be very careful to either design the app so that such skew doesn’t matter, or somehow ensure that it doesn’t happen. The latter is probably impossible, though.

]]> By: John Allspaw http://www.dronamraju.com/blog/2005/10/scaling.html#comment-52 John Allspaw Fri, 28 Oct 2005 04:05:37 +0000 http://www.dronamraju.com/blog/?p=64#comment-52 'many box' solution: yep, can't handle the writes, SOL indeed. :) a coupla other limitations of that layout: (without partitioning) - even if the Master *can* handle the writes, the Slaves may not be able to, and serve read traffic at the same time, no matter how many slaves you have. LiveJournal's presentation last year (http://www.danga.com/words/2004_mysqlcon/mysql-slides.pdf) covered this scenario really well. - having one Master is still a Single-Point-of-Failure Jeremy's book is chock full of great stuff, although I'm adamant about not using RAID5, whereas he still suggests it's ok. :) thanks for reading my slides! ‘many box’ solution: yep, can’t handle the writes, SOL indeed. :)

a coupla other limitations of that layout: (without partitioning)

- even if the Master *can* handle the writes, the Slaves may not be able to, and serve read traffic at the same time, no matter how many slaves you have. LiveJournal’s presentation last year (http://www.danga.com/words/2004_mysqlcon/mysql-slides.pdf) covered this scenario really well.

- having one Master is still a Single-Point-of-Failure

Jeremy’s book is chock full of great stuff, although I’m adamant about not using RAID5, whereas he still suggests it’s ok. :)

thanks for reading my slides!

]]>