Werner Vogels: Keynote at Structure 08

Werner Vogels is now giving a keynote. He is the CTO of Amazon.com.

At Structure 10, the whole discussion will be different. We will be talking about different business models. This is a snapshot at the beginning of the movement. He showed an Animoto video presentation created by him.

What's so special about Animoto? They have no server infrastructure, even though what they do is very compute intensive. When they had 25,000 customers they were hovering around 50 instances. They launched a Facebook app that allowed you to import photos, create video and post it back to Facebook. At that point, they started signing up 25,000 customers per hour. They had to increase their instances to more than 5,000. Imagine Animoto going to VC and asking for money for 5k servers.

Cloud computing is moving the world from capital expenditure (CAPEX) to operating expenditure (OPEX). We are moving to a variable cost model.

Amazon is now in 7 countries with more than 79 million active customer accounts.

Bandwidth used by AWS is way higher than Amazon store.

Amazon used to be a technology consumer. Now, there is no third party software left at Amazon because of scale. There are more than 1M+ sellers. They moved from single application to a platform.

First 5-6 years, uptil 2001, Amazon was like a traditional site. The challenge was how do they keep scaling? How will we make it to the next year. Around 2001, Target came to Amazon and asked to be integrated. At the same time, a number of architecture pieces broke. Then they wanted to move to a platform while working on integrating Target.

At times, Amazon was thinking of going back to mainframe. They wanted to create a very agile environment.

They created an infrastructure where no direct database connections were allowed. Everything must go through a business logic layer.

The gateway page on Amazon can use upto 200+ services to be created.

The 70/30 switch

• Companies now have to become experts in many areas not related to their business and answer questions like, why is BGP protocol not stable?, why do datacenters go down? etc. These companies are spending upto 70% of time, energy and dollars on undifferentiated heavy lifting.
  
• Only 30% of time, energy and dollars are spent on differentiated value creation.

He is showing a photo of destroyed datacenter. If your data was in that datacenter, it is gone!. Now talking about 365 main which did 'everything right.' 6 of their 8 diesel generators failed and brought Web 2.0 down. At Amazon, the thought is to survive an entire datacenter failure.

Don't depend on Raid-5 to protect your data.

Peak capacity management is a big issue for companies such as Walmart.com and Target.com that experience seasonal spikes in traffic.

They wanted to cover three areas: compute, messaging and storage. EC2 covers compute, S3, Simple DB and EC2 PS (Persistent storage) covers storage and SQS is the fabric that holds everything together. EC2-PS still doesn't has a name.

Most data at Amazon was key-value based. There were secondary key accesses. SimpleDB was a compliment to S3.

It's easy for companies to spend as much as 70% of their intellectual capacity in scaling.

Infrastructure Services Drivers

1. Security
2. Scalablity
3. Availability
   
4. Performance
5. Cost-effective

Next he is showing example of SmugMug who relies heavily on Amazon's EC2 and S3. They currently have 600TB of pictures stored in Amazon S3. In Amazon S3 there are more than 18 billion objects as of March 2008.

SmugMug is now venturing in different businesses where they provide interface to allow you to store anything. The product is called Smugmug Vault.

Addressing Uncertainty

• Acquire resources on demand and release them
• pay for what you use
• leverage other's core competencies
• turn fixed cost into variable

What sense does it makes to order a lot of hardware when you don't even have a product? You also aren't sure how many customers you'd get.

Get everything from http://aws.amazon.com, you only need a credit card.