technology

My Suggestions For Yahoo Eng – circa 2003

Yahoo Eng Platform. It all feels like way back. In 2003, I wrote up bunch of suggestions for structuring engineering teams for yahoo. A lot of these suggestions made it to a “committee”. I feel that these are still applicable to yahoo! and yahoo can benefit from there.
After these, I tried to pitched that Hack Yahoo team should build an internal sourceforge like product. I could never convince product/eng leaders to invest that either – then google code came along

Amazon EC2 Goodness

recently started working with folks at EveryTrail. EveryTrail is a cool gps travel story sharing site. With the proliferation of GPS enabled mobile phones (iphone 3g, bb curve, etc), people can capture great information of where they go and what they do. EveryTrail has cool apps to do so. I am excited about this space, as more and more people are buying these devices. It is a great opportunity to build services to help those users.

As you can guess, the website traffic for everytrail.com is growing at a healthy clip. When i came in to help with engineering, the first order of business was to tackle that. We were hosting a grid hosting company layered.com. The issue with this grid hosting company is that we had to pay for compute units we got “provisioned” and invariably it seemed that we had to provision a lot of compute and memory resources for our site. As we were growing we need to add more compute units. The idea seemed great but 3tera’s software which powered this grid sucked. We were ending up paying way too much money for the value we were getting.
I pressed early on to move to Amazon EC2, mainly because i knew how to set up things on there, and partly because 3tera’s system seemed like a lot of overhead. Somewhere the number of compute + memory resources and the performance we were getting from those were not adding up. It seemed ridiculously low.
The move to EC2 went way smoother than i thought it would. A lot of different software available, and easy to setup and upgrade! The results were even more amazing. We noticed significant improvement in the site performance as we measured it, and even just browsing to the site, but even more impressive are google bot stats from google webmaster site. As you can see, early december we moved amazon ec2 and the load time for each page is down and the number of pages downloaded is up
.

Needless to say, our servers have been humming since, and handling 60%+ month over month growth since december. All this at a cost saving of approximately 60%!! Moving to EC2 cut our hosting cost by 60% (this may quickly catch up to old number if we keep growing this way), and increased performance. It sounds like i am getting paid by amazon for writing this post, but really – no. I am just a satisfied customer.