Harnessing Explosive Growth: Infrastructure Strategies and Tactics

The session I am now waiting for is Harnessing Explosive Growth: Infrastructure Strategies and Tactics. The official description of the session is:

What worked in the garage can rarely be scaled effectively for the boardroom. This panel will bring together some of the biggest names in web infrastructure to share their thoughts, insights and tactics for harnessing explosive growth, with a focus that goes beyond simply which technologies are available but how to best deploy them. This panel is not to be missed

Panelists include:

• Sandy Jen Meebo
• Akash Garg hi5 Networks
• Jeremiah Robison: Slide
• Jonathan Heiliger Facebook
• James Barrese eBay

How much of scalability is architecture, and how much is throwing servers at the problem?

SJ: 99% architecture
JH: Product is what drives architecture. We have more than 10,000 servers. For chat, they built a separate network. www.facebook.com/eblog.

JR: The biggest consideration is how their servers work with Facebook.

When's it broken? When did you know your first architecture was broken?
AG: First million users. Then they started focusing on caching and sharding.

JB: eBay had a few catastrophic issues near 2000. People were very forgiving of availability issues. They give refunds when there is downtime.

JR: They moved a lot of their technology to EDGE.

SJ: Since Meebo is just one page, javascript delivery is a major issue for them. They are not generating pages. Dynamic loading and background computation is really important.

RP: Application and infrastructure is going to break. If you look hard enough you can find where are the scalability issues.

JR: They are very metric driven. It's not often they see outages. They have number of Facebook's ops folks on their IM lists and they talk continously.

JH: We work through problems together.

JH: They have had to turn off some applications because other applications were being affected.

JB: At eBay they have created a central application login system. They can flag and identify problems really quickly. If you don't have it, you're shooting in the dark.

Rolling your own stuff? Off-the-shelf vs. custom:
Did you roll your own? Do you regret it?

SJ: For cash restraint startup, off-the-shelf can work. But for scale, you'd want to build yourself. Open Source is awesome. No one can scale your system as well as you can. Off-the-Shelf can be bulky. You have to get your hands dirty.

JR: We built our own caching backend. Invest time in core stuff, anything that's not core, don't focus there.

What also needs to scale as you grow? What non-technology things you had to scale?

JB: Need to scale out your business as well as technology.

AG: Building anti-spam features into the product that are scalable.

JH: Make community part of the process in translations as you grow.

JR; They introduced user moderation for photos. Hard to find what's porn and what's not. It's about a dozen people looking at photos full-time to hunt down porn.

How should we handle the fallout? If you were Twitter what would you have done last month?
JB: You have to be transparent. Tell them what's going on. Setup message boards for communication. You've got to communicate.

AG: Setting realistic timelines is very important

JH: We do it often. We roll in small chunk and if things don't look right, we roll back.

JB: You cannot operate a large system without the ability to turn things on and off.

JR: If you have an aggressive competitor, you don't have the luxury of downtime.

RP: You can't roll out something that can't be rolled back.

The Race to the Next Database: Overclocking and Analytics Augment Your Data Layer

A star studded panel on The Race to the Next Database: Overclocking and Analytics Augment Your Data Layer is next. Panelists include

* Mayank Bawa Aster Data Systems
* Doug Judd Zvents
* Luke Lonergan Greenplum
* Damian Black SQLstream
* Dave Schrader Teradata
* Scott Wiener Cloud9Analytics

Nitin Borwankar of TagSchema is moderating the session. He has an impressive background in databases. He used to be a J2EE consultant. He is focusing on what kind of problems are pain points for large scale web applications? In social applications you have many to many relationships.

Mayank Bawa: (CEO and founder of Aster Data Systems)

• Scalable database for warehousing and anaytics that runs on a cluster of commodity nodes
• founded in 2005
  
• colleagues from Stanford
• Investors
• Sequoia Capital
• Cambrian Ventures
• First Round Capital
• Sample Customers
• MySpace (they are the backend data warehousing engine)
• In one day MySpace more than 1 billion impressions per day which is loaded into aster
• every hour resulting in over 1 TB of new data
• 100 node Aster cluster
  
• Aggregate Knowledge

Doug Judd (Zvents)

• Google is arguably the king of data. They capture more data and analyze more data than anything in the world
• Google has developed three key pieces of infrastructure
• GFS
• MapReduce: Computation framework that works closely with GFS
• BigTable: Somewhat analogous to a traditional db except it is massively scalable.
• Hypertable
• Open Source implementation of BigTable
• pulls common scaling logic into a general distribution layer

Luke: (Greenplum)

• Took a traditional db architecture and twisted it into a massively parallel infrastructure
• Customers
• 40% of business came from Asia last quarter
• Leveraging PostgreSQL developers worldwide
• helped them get to more places
• Data is growing tremendously
• All sectors need to understand their data

Damian Black (SQLStream)

• Manages high volume streams
  
• SQLStream eliminates latency via a pipelined approach.
• SQL example

Dave Schrader Teradata

• $1.7 billion a year company
• WellsFargo, PayPal and many big companies are clients
• Founded in 1979
• First MPP db engine

Scott Wiener Cloud9Analytics

• focused on different part of market
• using Internet to deliver BI to unserved
  
• $50B in 2008 spent on BI
• business users still unsatisfied
• $6B on SaaS, expectations gap wider and growing fast
• deliver end-user analytical applications
  
• BI moving from on-premise to on-demand model
• they don't use databases or reporting
• Anytime, anyplace analytics
• using Flex

How is web disrupting databases?
MB: Everytime scale changes, we have to rethink scalability. Web allows for interactivity so feedback loop has to finish very fast.

DJ:Traditional db technology designed to run on a single machine. Web has massive amount of data that must scale. The problem of scaling one machine is exponentially more difficult.

LL: Google is making billions of advertising. Importance of data is more now. They are building 10-20 petabyte warehouse for a customer. It's become more important to solve those problems. They had to re-invent the concept of databases to drive the workload

DB: Real time decision making is built into their business.

SW: Web is speeding up pace of the business. A lot of decision making is being automated. When you automate, you need to be able to detect exceptions. Now technology drives business.

LL One thing they have focused on is data loading. They have a subscription based pricing model.

Working the Clouds: NextGen Infrastructure for New Entrepreneurs

Now there is a panel, Working the Clouds: NextGen Infrastructure for New Entrepreneurs. Panelists include:
* Geva Perry GigaSpaces
* Jason Hoffman Joyent
* Tony Lucas XCalibre
* Lew Moorman Rackspace
* Christophe Bisciglia Google
* Joe Weinman AT&T


CB: Google App Engine can be called a cloud but it is a little different that allows you to only focus on very high levels.

Q: Are we selling our souls when moving to cloud? Or should we not worry about lock-in?
There are no standards yet. To take advantage of cloud you need to embed it into code. It does create some locking issues. Hopefully in future there will be standards. If you're going to scale on EC2 and have scripts to automatically launch instances then moving to a different cloud will create issues. A lot of clouds that are gaining traction have very custom APIs. There is going to be a proprietary stack. On the other hand we will see convergence towards standards based platform. Then we end up with choice where you will then make decision based on your strengths.


There is still value in proprietary technologies. The more value you are going to build for your customers, the more proprietary the technology behind it would be. Both open and proprietary markets have their potential.

Even though App Engine is proprietary, the documentation is very open. If you are using Big Table, you are kind of selling your soul. Until Google open sources BigTable so users can export their data, it is locked-in.

API that Google provides isn't specific to BigTable. One of the criteria is how scalable the platform is. Christophe Bisciglia thinks that Google is a step ahead of Google when it comes to BigTable and their platform.

One shouldn't be locked into proprietary data store, period.

But, CB insists that BigTable performs better. There are things for which relational databases don't perform well. There is a sacrifice but it allows you to scale. What differentiates providers is how low their latency is and how scalable the infrastructure is and how protected the data is.

LM: A lot of companies are not right away ready for cloud.

The dirty little secret model for cloud computing is that if licensing models don't catch up, everything can drop dead.

Cloud has become an overused term. duh!

Google has geographically distributed clusters.

Cloud computing is architecture 3.0.

There is a live stream going out at http://www.mogulus.com/structure08

Application development 5-10 years from now will be focused on the need to scale. The challenge is to move people up to cloud and provide them with tools to develop their applications the right way.

Thought shift is required for enterprises to move to cloud. CB gives an example of how people were hesitant to use banks when they came around as people were nervous of keeping their money with someone else. But then, as people realized the dangers of not keeping money at bank, banking became the norm.

AT&T will be spending $20 billion on their infrastructure.

Enterprises want to sign a contract. They don't want to just go and use credit card to start a relationship like this.

There is a big marketing problem. The word cloud means less and less everyday. Cloud is getting into the enterprise through the backdoor.

Werner Vogels: Keynote at Structure 08

Werner Vogels is now giving a keynote. He is the CTO of Amazon.com.

At Structure 10, the whole discussion will be different. We will be talking about different business models. This is a snapshot at the beginning of the movement. He showed an Animoto video presentation created by him.

What's so special about Animoto? They have no server infrastructure, even though what they do is very compute intensive. When they had 25,000 customers they were hovering around 50 instances. They launched a Facebook app that allowed you to import photos, create video and post it back to Facebook. At that point, they started signing up 25,000 customers per hour. They had to increase their instances to more than 5,000. Imagine Animoto going to VC and asking for money for 5k servers.

Cloud computing is moving the world from capital expenditure (CAPEX) to operating expenditure (OPEX). We are moving to a variable cost model.

Amazon is now in 7 countries with more than 79 million active customer accounts.

Bandwidth used by AWS is way higher than Amazon store.

Amazon used to be a technology consumer. Now, there is no third party software left at Amazon because of scale. There are more than 1M+ sellers. They moved from single application to a platform.

First 5-6 years, uptil 2001, Amazon was like a traditional site. The challenge was how do they keep scaling? How will we make it to the next year. Around 2001, Target came to Amazon and asked to be integrated. At the same time, a number of architecture pieces broke. Then they wanted to move to a platform while working on integrating Target.

At times, Amazon was thinking of going back to mainframe. They wanted to create a very agile environment.

They created an infrastructure where no direct database connections were allowed. Everything must go through a business logic layer.

The gateway page on Amazon can use upto 200+ services to be created.

The 70/30 switch

• Companies now have to become experts in many areas not related to their business and answer questions like, why is BGP protocol not stable?, why do datacenters go down? etc. These companies are spending upto 70% of time, energy and dollars on undifferentiated heavy lifting.
  
• Only 30% of time, energy and dollars are spent on differentiated value creation.

He is showing a photo of destroyed datacenter. If your data was in that datacenter, it is gone!. Now talking about 365 main which did 'everything right.' 6 of their 8 diesel generators failed and brought Web 2.0 down. At Amazon, the thought is to survive an entire datacenter failure.

Don't depend on Raid-5 to protect your data.

Peak capacity management is a big issue for companies such as Walmart.com and Target.com that experience seasonal spikes in traffic.

They wanted to cover three areas: compute, messaging and storage. EC2 covers compute, S3, Simple DB and EC2 PS (Persistent storage) covers storage and SQS is the fabric that holds everything together. EC2-PS still doesn't has a name.

Most data at Amazon was key-value based. There were secondary key accesses. SimpleDB was a compliment to S3.

It's easy for companies to spend as much as 70% of their intellectual capacity in scaling.

Infrastructure Services Drivers

1. Security
2. Scalablity
3. Availability
   
4. Performance
5. Cost-effective

Next he is showing example of SmugMug who relies heavily on Amazon's EC2 and S3. They currently have 600TB of pictures stored in Amazon S3. In Amazon S3 there are more than 18 billion objects as of March 2008.

SmugMug is now venturing in different businesses where they provide interface to allow you to store anything. The product is called Smugmug Vault.

Addressing Uncertainty

• Acquire resources on demand and release them
• pay for what you use
• leverage other's core competencies
• turn fixed cost into variable

What sense does it makes to order a lot of hardware when you don't even have a product? You also aren't sure how many customers you'd get.

Get everything from http://aws.amazon.com, you only need a credit card.

The Platform Revolution: A Look into disruptive technologies

Next up at Structure 08 is Jonathan Yarmis AMR Research (VP of Disruptive Technologies). The session is titled, The Platform Revolution: A Look into disruptive technologies.

He has been in technology for more than 30 years. He was a sports broadcaster early in his career.

• What is this wave of disruption we see coming?
• The industry sometimes forgets its all about the user
• Users are figuring out cloud computing faster than enterprise.
  

Technology Trends

• Independent phenomena
• Social networks, virtual worlds and other community-based solutions.
• Social networking is not a new phenomena, it started with Eve.
• Mobility
• Cloud computing and stream computing
• Alternative business models
• Most often advertising supported
• Different license, revenue models
• Mutually reinforcing
  

Social Networking:

• We improve individual processes but don't think in terms of tasks
• How many CRM implementations have failed?
• They solved problems of CEO but not of sales people.
• Sales people job is inherently social.
  

Mobility

• The platform for the next generation and emerging markets.

PC, Mobile

• Size: 200 million units / year, 1.4 billion units per year
• Growth: 10% in a good year, 20% in a bad year
• Useful life: 3-5 years, 21 months
  

• Intersecting with social networks and location-based services
• Reaching new users and new uses.
• Even OLPC is not going to be the dominant platform in developing countries.
• The growth will be 2 billion units this year.
  

Cloud Computing:

• interesting phenomenon and dominant trend
• it won't encourage SaaS but Eaas (everything as a service)
• From a PC based, desktop client to services in the cloud
• Zero deployment
• Click-to-run
• Not just Saas but Eaas
• Sotware/applications
• Storage
• Content
• Music/video
• Data
• A real paradigm shift
• Exploding the opportunity vs. protecting your base.
• Really significant change.
  

Stream Computing:

• Data creation is expanding beyond our ability to store it all
• Growth estimates range from 50%/year to an order of magnitude
• Everything communicates
• Communicates everything
• 50 million photo uploads per day @ Facebook
• Where do we process it?
• How do we determine what's interesting
  

Business Models

• What's the value in social networks? It's peers
• Advertising changes the customer/vendor/user relationship
• Who will pay for end-user software and services?
• Moving from a license model to blended advertising-supported model
• What des this mean for the market?
• Economic value of peer relationships is going to become driving force
• Facebook's beacon: the holy grail of adveriting. Giant leap then step back, then go beyond where you were going.
  

User 2.0

• 1.0, 2.0
• structured, ad-hoc
• systems oriented, socially oriented
• process defined, user defined
• personal, collaborative,
• intra-enterprise, interenterprise (enterprise unaware or agnostic)
  

The New Enterprise Reality

• Users have more computing power at home than at work
• users are embracing disruptive technologies
• social networking
• video
  

He is showing classic innovators dilemma chart from http://web2.wsj2.com.

An economic downturn can accelerate cloud computing and mobile usage. Companies run after protecting their existing revenue base and not after exploring new opportunities.

Google's Android delayed because they are trying to please everyone. Apple wins for moving fast.

Structure 08

I am sitting with Dan White waiting for Structure 08 to start. Om Malik of GigaOM will be opening the sold out event.

Om Malik is now on stage.

Cloud computing is a lot of fog, a little cold. A lot of activity going on in cloud computing world. Companies are spending a lot of money on infrastructure. Few top companies spent over $6 billion in infrastructure. There are so many issues with infrastrcture: scaling out, power etc.

There is a video message from Nick Carr.

- Structure is happening at the same time Bill Gates is retiring
- Fundamental unit of computing is shifting from individual computer to datacenter or grid computing.
- A lot of technical challenges such as reliability, energy efficiency are becoming center to competitive advantage.
- Management of infrastructure is becoming more and more important.
- Functioning of data center is becoming very important
- Moving from IT provided locally to grid/over the network
- We don't know what the ultimate structre of the industry will be.
- Structure will be crucial to success.
- Electric current didn't have ethical dimension to utility. It was ethically neutral.
- Computing involves information, personal information which has a very important ethical component.

Om takes the stage again.

twitter.com/structure08
Official tag for conference is structure08.