Wednesday, April 1, 2009

Werner Vogels

Super cool guy.  He's one of those super geeks that's past the point where he feels it necessary to get dragged into details but is totally upbeat.  It's good to be the king, right?  Perhaps my biggest surprise was to learn just how much Amazon views itself as a technology company.  Amazon's retail experience is just a customer of AWS.  Riiiight.  I find that hard to believe, but he claims that the retail site is not even the biggest customer of AWS.  Whoa.

My big takeaway from his talk and comments afterward is that only companies that really, really must scale to incredible extremes should build their applications for AWS, i.e. bake in use of SimpleDB, SQS, etc.  What he said to me was, "run your Linux infrastructure on EC2 so you can leave if you don't like us."  So, really, build great SOA systems and use AWS as a virtualization provider.  He had a lot of great principles in his talk for building scalable systems, including:

individual components make decisions only based on local information
Make progress under all circumstances
Controlled Concurrency
operations are designed such that limmited or no concurrency control is required
Controlled Parallelism
use fully decentralized (p2p) techniques to remove bottlenecks
Remove dependencies
Break up into more granular services
Failure Tolerant
Failure is a normal part of operation
Local Responsibility for Consistency
Similar to Autonomy, don't look elsewhere for consistent state
pare away functionality until you cannot
all nodes can do any function in the system

Some other interesting bits were that the Amazon Homepage is produced by 200-300 services. Every service at Amazon is developed and operated by the same team. There are about 500-600 services/teams. Elasticity is both shrinking AND growing on demand.