[ This is my personal blog so all opinions expressed here are mine. I am a product, scalability, operations and monetization advisor and currently employed as Director of Business Operations & Technical Strategy for a top 50 website that delivers billions of page views per month. I was a keynote panelist for Scaling Up or Out keynote at MySQL Conference and speak regularly at conferences and user groups.
]
Community One East is happening this week and I for sure will be attending. The event is taking place at Marriott Marquis Hotel, New York, NY. I am especially looking forward to the announcements tomorrow which sound very interesting :). Unfortunately, I can't go into details about what Sun Microsystems is announcing.
The first day of Community One is a free event featuring. The second day of the event is focused on Deep Dives with two half-day sessions on MySQL and two full-day sessions on Java and Web development. I will be attending the session, "Using Java EE and SOA to Architect and Design Robust Enterprise Applications."
Since I have been busy developing our new startup on EC2, it would be great to catch up with Mårten Mickos and Dr. Werner Vogels whom I originally met at Structure 08.
Cloud Computing is probably the most abused buzz word, but if anyone rightly deserves to be in the cloud computing space, it's Amazon.
Speakers at the seminar include Dr. Werner Vogels (CTO Amazon), Mårten Mickos (ex-CEO of MySQL) and Michael Crandell (CEO RightScale). I am hoping to catch up with Mårten Mickos during the event. He is a great guy and probably the most favorite Open Source CEO.
I find it funny that the event site still shows Mårten's title as "SVP of Sun Microsystems’ Database Group," even though Marten resigned in February.
8:30am Registration and breakfast
9:00am Ahead in the Cloud — The Power of Infrastructure as a Service Dr. Werner Vogels, CTO, Amazon.com Werner Vogels will reveal the efficiency principles behind Amazon’s drive to develop Infrastructure as a Service, and discuss why the current economic climate is rapidly changing the IT perspective on cloud computing.
10:00am Growing Your Business in the Cloud with Open Source Marten Mickos, SVP Database Group, Sun Microsystems Marten Mickos will discuss why fast-growing businesses are taking advantage of the combined benefits of open source and cloud computing to accelerate the delivery of new applications, reducing overall risk and scaling infrastructure consumption up and down to meet demand.
10:45am Coffee Break
11:00am Smart Enterprises Moving to the Cloud Michael Crandell, CEO, RightScale Michael Crandell will talk about how enterprises are experimenting with the cloud today. You’ll learn methods for deploying applications to the cloud, and be guided through Right Scale’s best practices for designing, configuring, deploying and managing the lifecycle of multi-server cloud deployments.
11:45am Cloud Computing and the 2008 Beijing Olympics Lotta Latsuo, COO, Starcut Ltd. Lotta Latsuo will discuss how Starcut Ltd., a professional services firm that specializes in digital media and marketing, adopted cloud technology to build the NBC Olympics Mobile website for NBC’s coverage of the 2008 Beijing Olympics from Beijing.
12:15pm Closing remarks
In case you're attending and spot me, please say hi!
Amazon's S3 service went down today and more than 7 hours later, it is still down. The service initially went down around 12:00PM EST and my latest check shows troubles continuing for sites that depend on S3.
“Funny how Amazon doesn't use S3 to store any assets for amazon.com”tweet by @gruber
Smugmug, a popular photo sharing site with more than 600 TB of data stored on S3 has been accessible along with several "Web 2.0" startup sites. Because of the astounding amount of data stored by Smugmug on S3, it can definitely be considered a poster child for the service. Last time I checked Smugmug, at 11:10 PM EST, it was still inaccessible.
“As a consequence, a variety of businesses such as Twitter, digital photo sharing Web site, SmugMug and The Huffington Post all had issues. Twitterers were claiming their avatar images could not be displayed. The Huffington Post was also unable to display images to its stories, while SmugMug could not offer any service at all. ”ComputerWorld
This is not good for Amazon and for startups that are looking to count on Amazon's "redundant" S3 platform. Dr. Werner Vogels is probably pretty upset right now.
Next session is Clouds Are No Substitute for Competence by Javier Soltero of Hyperic.
The promise of cloud computing:
Cloud computing is the next big thing: Because it is green, easy, scalable, available and disposable.
Cloud computing adds complexity:
clouds allow you to run your applications, but mask the performance of the infrastructure powering them. NYT is not going to stop their own infrastructure just because they had success with one project on EC2.
when a problem happens, where is the source of the problem? cloud or your own app.
cloud, by definition is always available and the status is always green.
how quickly can I provision new servers?
what is the throughput in the regions I use?
what latency am I getting for my messages?
How do you answer?
'is it my application, or is it the cloud?"
Hyperic is introducing cloudstatus.com which shows performance, availability and health of Amazon's Web services. On CloudStatus.com, you can monitor EC2, S3, SQS, SDB and FPS (5 most popular and critical services of AWS). You can look at performance metrics such as deployment latency. They are firing Amazon instances and monitoring response times.
Now there is a panel, Working the Clouds: NextGen Infrastructure for New Entrepreneurs. Panelists include: * Geva Perry GigaSpaces * Jason Hoffman Joyent * Tony Lucas XCalibre * Lew Moorman Rackspace * Christophe Bisciglia Google * Joe Weinman AT&T
CB: Google App Engine can be called a cloud but it is a little different that allows you to only focus on very high levels.
Q: Are we selling our souls when moving to cloud? Or should we not worry about lock-in? There are no standards yet. To take advantage of cloud you need to embed it into code. It does create some locking issues. Hopefully in future there will be standards. If you're going to scale on EC2 and have scripts to automatically launch instances then moving to a different cloud will create issues. A lot of clouds that are gaining traction have very custom APIs. There is going to be a proprietary stack. On the other hand we will see convergence towards standards based platform. Then we end up with choice where you will then make decision based on your strengths.
There is still value in proprietary technologies. The more value you are going to build for your customers, the more proprietary the technology behind it would be. Both open and proprietary markets have their potential.
Even though App Engine is proprietary, the documentation is very open. If you are using Big Table, you are kind of selling your soul. Until Google open sources BigTable so users can export their data, it is locked-in.
API that Google provides isn't specific to BigTable. One of the criteria is how scalable the platform is. Christophe Bisciglia thinks that Google is a step ahead of Google when it comes to BigTable and their platform.
One shouldn't be locked into proprietary data store, period.
But, CB insists that BigTable performs better. There are things for which relational databases don't perform well. There is a sacrifice but it allows you to scale. What differentiates providers is how low their latency is and how scalable the infrastructure is and how protected the data is.
LM: A lot of companies are not right away ready for cloud.
The dirty little secret model for cloud computing is that if licensing models don't catch up, everything can drop dead.
Application development 5-10 years from now will be focused on the need to scale. The challenge is to move people up to cloud and provide them with tools to develop their applications the right way.
Thought shift is required for enterprises to move to cloud. CB gives an example of how people were hesitant to use banks when they came around as people were nervous of keeping their money with someone else. But then, as people realized the dangers of not keeping money at bank, banking became the norm.
AT&T will be spending $20 billion on their infrastructure.
Enterprises want to sign a contract. They don't want to just go and use credit card to start a relationship like this.
There is a big marketing problem. The word cloud means less and less everyday. Cloud is getting into the enterprise through the backdoor.
Werner Vogels is now giving a keynote. He is the CTO of Amazon.com.
At Structure 10, the whole discussion will be different. We will be talking about different business models. This is a snapshot at the beginning of the movement. He showed an Animoto video presentation created by him.
What's so special about Animoto? They have no server infrastructure, even though what they do is very compute intensive. When they had 25,000 customers they were hovering around 50 instances. They launched a Facebook app that allowed you to import photos, create video and post it back to Facebook. At that point, they started signing up 25,000 customers per hour. They had to increase their instances to more than 5,000. Imagine Animoto going to VC and asking for money for 5k servers.
Cloud computing is moving the world from capital expenditure (CAPEX) to operating expenditure (OPEX). We are moving to a variable cost model.
Amazon is now in 7 countries with more than 79 million active customer accounts.
Bandwidth used by AWS is way higher than Amazon store.
Amazon used to be a technology consumer. Now, there is no third party software left at Amazon because of scale. There are more than 1M+ sellers. They moved from single application to a platform.
First 5-6 years, uptil 2001, Amazon was like a traditional site. The challenge was how do they keep scaling? How will we make it to the next year. Around 2001, Target came to Amazon and asked to be integrated. At the same time, a number of architecture pieces broke. Then they wanted to move to a platform while working on integrating Target.
At times, Amazon was thinking of going back to mainframe. They wanted to create a very agile environment.
They created an infrastructure where no direct database connections were allowed. Everything must go through a business logic layer.
The gateway page on Amazon can use upto 200+ services to be created.
The 70/30 switch
Companies now have to become experts in many areas not related to their business and answer questions like, why is BGP protocol not stable?, why do datacenters go down? etc. These companies are spending upto 70% of time, energy and dollars on undifferentiated heavy lifting.
Only 30% of time, energy and dollars are spent on differentiated value creation.
He is showing a photo of destroyed datacenter. If your data was in that datacenter, it is gone!. Now talking about 365 main which did 'everything right.' 6 of their 8 diesel generators failed and brought Web 2.0 down. At Amazon, the thought is to survive an entire datacenter failure.
Don't depend on Raid-5 to protect your data.
Peak capacity management is a big issue for companies such as Walmart.com and Target.com that experience seasonal spikes in traffic.
They wanted to cover three areas: compute, messaging and storage. EC2 covers compute, S3, Simple DB and EC2 PS (Persistent storage) covers storage and SQS is the fabric that holds everything together. EC2-PS still doesn't has a name.
Most data at Amazon was key-value based. There were secondary key accesses. SimpleDB was a compliment to S3.
It's easy for companies to spend as much as 70% of their intellectual capacity in scaling.
Infrastructure Services Drivers
Security
Scalablity
Availability
Performance
Cost-effective
Next he is showing example of SmugMug who relies heavily on Amazon's EC2 and S3. They currently have 600TB of pictures stored in Amazon S3. In Amazon S3 there are more than 18 billion objects as of March 2008.
SmugMug is now venturing in different businesses where they provide interface to allow you to store anything. The product is called Smugmug Vault.
Addressing Uncertainty
Acquire resources on demand and release them
pay for what you use
leverage other's core competencies
turn fixed cost into variable
What sense does it makes to order a lot of hardware when you don't even have a product? You also aren't sure how many customers you'd get.
