Harnessing Explosive Growth: Infrastructure Strategies and Tactics

The session I am now waiting for is Harnessing Explosive Growth: Infrastructure Strategies and Tactics. The official description of the session is:

What worked in the garage can rarely be scaled effectively for the boardroom. This panel will bring together some of the biggest names in web infrastructure to share their thoughts, insights and tactics for harnessing explosive growth, with a focus that goes beyond simply which technologies are available but how to best deploy them. This panel is not to be missed

Panelists include:

• Sandy Jen Meebo
• Akash Garg hi5 Networks
• Jeremiah Robison: Slide
• Jonathan Heiliger Facebook
• James Barrese eBay

How much of scalability is architecture, and how much is throwing servers at the problem?

SJ: 99% architecture
JH: Product is what drives architecture. We have more than 10,000 servers. For chat, they built a separate network. www.facebook.com/eblog.

JR: The biggest consideration is how their servers work with Facebook.

When's it broken? When did you know your first architecture was broken?
AG: First million users. Then they started focusing on caching and sharding.

JB: eBay had a few catastrophic issues near 2000. People were very forgiving of availability issues. They give refunds when there is downtime.

JR: They moved a lot of their technology to EDGE.

SJ: Since Meebo is just one page, javascript delivery is a major issue for them. They are not generating pages. Dynamic loading and background computation is really important.

RP: Application and infrastructure is going to break. If you look hard enough you can find where are the scalability issues.

JR: They are very metric driven. It's not often they see outages. They have number of Facebook's ops folks on their IM lists and they talk continously.

JH: We work through problems together.

JH: They have had to turn off some applications because other applications were being affected.

JB: At eBay they have created a central application login system. They can flag and identify problems really quickly. If you don't have it, you're shooting in the dark.

Rolling your own stuff? Off-the-shelf vs. custom:
Did you roll your own? Do you regret it?

SJ: For cash restraint startup, off-the-shelf can work. But for scale, you'd want to build yourself. Open Source is awesome. No one can scale your system as well as you can. Off-the-Shelf can be bulky. You have to get your hands dirty.

JR: We built our own caching backend. Invest time in core stuff, anything that's not core, don't focus there.

What also needs to scale as you grow? What non-technology things you had to scale?

JB: Need to scale out your business as well as technology.

AG: Building anti-spam features into the product that are scalable.

JH: Make community part of the process in translations as you grow.

JR; They introduced user moderation for photos. Hard to find what's porn and what's not. It's about a dozen people looking at photos full-time to hunt down porn.

How should we handle the fallout? If you were Twitter what would you have done last month?
JB: You have to be transparent. Tell them what's going on. Setup message boards for communication. You've got to communicate.

AG: Setting realistic timelines is very important

JH: We do it often. We roll in small chunk and if things don't look right, we roll back.

JB: You cannot operate a large system without the ability to turn things on and off.

JR: If you have an aggressive competitor, you don't have the luxury of downtime.

RP: You can't roll out something that can't be rolled back.

Open Social: Open for Business

Next up is the session Open Social: Open for Business. Panelists are Patrick Chanezon (Google), Paul Lindner (hi5), Max Newbould (MySpace) and Sachin Rekhi (imeem).

Open Social offers a standard for everyone. The IT rights of Open Social are owned by Open Social Foundation. Among Open Social members are hi5, imeem, bebo, ning, Oracle, Yahoo!, six apart, LinkedIn, viadeo, friendster, AOL and many more.

Open Social is 88 days old and reaches 275,000,000 users with 66 million installs of 2000+ apps developed by 20,000 developers. 10 million users use applications on Open Social.

Two client APIs : one for javascript and the other for REST.

Three areas:

• people and friends: access friends information programatically.
• activities: see what friends are doing
• persistence: provide state without a server and share data wth friends.
  

Some examples follow next.

An Open Social application has 80% code that can be readily implemented on other OS containers.

If you have a social site and want to implement OS you can use Apache Shindig which has PHP and Java version of it. Shindig makes it really easy to implement. There is a very active mailing list for it.

SocialSite by Sun: Open source project that utilizes Shindig and builds on top of it. Dave "Roller" Johnson announced it at JavaOne. Heavy potential at Enterprise.

iGoogle: 50% userbase in US!

Google Friend Connect:

• Users... more ways to do more things with my friends
• Site owners... more and engaged traffic for site.
  

MySpace Developer Platform:

• 7 months old
• Supporting REST APIs - over a year old
• #1 social network
  

Why Develop for MySpace?

• unique demographics: users you can't find elsewhere
• forthcoming metrics / analytics focused on small to medium developers
• user base is so large, you just need to get a small portion of MySpace users by popularity not virality.
  

Some Metrics of MySpace:

• 60K registered developers
• 1800+ apps
• 15 million installations (3 months)
  

MySpace Developer platform at developer.myspace.com

• IRC: irc.freenode.net #myspacedev
• Email: developerrelations at myspace.com
• Twitter: MySpaceDevTeam
• Dev Jams: range from 2-8 hours, devs bring their laptops and get first hand instruction and help from MDP team members
• Myspace offers free application press releases apply at myspace at spark.pr.com
  

MySpace Developer Platform future:

• application communication channel
• custom notifications
• invites - requesShare App
• Metrics/analytics to playing field
  

Sachin (Imeem):

• Imeem is social network focused on sharing and discovering all kinds of media.
• 24m unique users per month and third largest social network based on US traffic.
• http://imeem.com/developers
• they have made their content licenses to developers allowing them to utilize imeem's legally licensed media.
• Types of applications
• originally Adobe ActionScript 3 Flex Apps
• OpenSocial Javascript APIs (in May)
• External iFrame
• OpenSocial extensions
• immem-specific extensions that allow access to imeem media metadata, including music, videos and photos.
• Example:
• The Echo Chamber application example built on imeem.
  

My.Aol.com

• newest partner with OpenSocial
• 4 brands
• 16 locales
• 12 languages and more on the way
• 55 myAOL portals
• 2-3 new locales per month
  

My.AOL.com Gadgest

• myAOL is an AJAX basde web applciations (uses Dojo)
• moving to Google Gadgets
• this summer their platform will be completely transformed to work with Google Gadgets and Open Social.
• jennifer.consalvo at corp.aol.com
  

Social + Mobile = Sociable (Social Networks for SMS, IM & Mobile Devices

Last session of the day is a panel discussion. Panelists include Benjamin Joffe, Ben Keighran, Gregory Cypes, Craig Dalton and Chris Butler.

BJ: Spent last 8 years in Asia (4 years in Japan, 3 in Korea and 1 in China).

Information Arbitrage: Identifying most successful best practices.

Some history: Mobile SNS (Social network services) are not new. ImaHima ("are you free now"). In 2001, they had 250,000 users.

What changed? numbers and speed of data access. Also, what you can do with phone. Mindset of people using mobile has changed.

It works: Japan is #2 economy, #1 mobile society. 80% penetration of 3G. China is #1 mobile and Internet population. QQ has 500 million users. Cyworld has 35 million and mixi has 90 million compared to 200 million Facebook users.

Mobile users: 50mln for QQ, 3 mln for facebook, cyworld 45 mln,

Revenue: fb: (50 mln), QQ 250 mln

FB is focused on profile page, IM and groups. QQ has avatars, paid games also.

Sticky feature: FB (applications.
QQ: IM digital currency, pmt system.
Cyworld: real friends, digital currency , pmt systems
mixi (footprints)

Business models: FB: ads, QQ (digital goods, paid games, mobile VAS)
cyworld (brand pages, digital goods.

Revenue mix (ARFU: avg revenue from users):
fb: 0%?
QQ 87%
cyworld: 80%
mixi: 5%

Mixi has more page views on mobile than PC. Mobile CPM is half the price of PC.

Community and Social networks are turning into full-fledged media. Cyworld is the nicest mobile application in social networks.
These top mobile SNS in Asia won't "invade us" as they are busy in home markets and they lock cross cultural expertise. Cyworld started and closed operation in Europe.

Myspace has started operation in China, Japan and Korea but so far has failed to gain traction revenue wise.

How to make mobile social networks work:
1. high speed network and flat fee: (No one from large mobile network present in the audience.)
2. Proper payment systems: seems a low hanging fruit.
3. Girls: it's not rocket science.
4. Companies and Schools: Because employees and students spend so much time on it that their computer access is blocked and so they turn to using mobile services.

benjamin at plus8star.com

Panel Introduction:

Ben Keighran (BK) ben at bluepulse-inc.com
Launched in December 2006. built world's largest social messaging platform.

Gregory Cypes (GC): gregory.cypes at corp.aol.com
QQ is trying to break into US market. GC is tech lead for open AIM. Big component of open AIM is mobile. AIM does over a billion IMs a day with 80 million users.

Craig Dalton (CD) (craig.dalton at hookmobile.com)
Hookmobile in mobile development for last years. Now in multimedia messaging. Created a platform to integrate messaging with social networks that takes away complexities such as device recognition. Just launched platform in April.

Chris Butler (cbutler at dash.net):
They do crowd sourcing. People can do Yahoo! mobile search from their car. Twitter from the car :). Working on facebook apps.


What's view on social network on mobile in US?

CD: XHTML is going to die.

BK: Browser is rapidly getting faster and is faster than it was ever before. iPhones, Android and Safari are fast. Create it as a browser based application.

GC: Every application needs SMS strategy.

BK: SMS gives you a huge touch point and advantage but times are changing.

How do you think operators are positioned? Do they hinder or support initiatives.

Careers are changing, they are opening up more. Interest around the browser is very great as it allows companies to come together and create standards.

Google has made cost of information really low.