[ This is my personal blog so all opinions expressed here are mine. I am a product, scalability, operations and monetization advisor and currently employed as Director of Business Operations & Technical Strategy for a top 50 website that delivers billions of page views per month. I was a keynote panelist for Scaling Up or Out keynote at MySQL Conference and speak regularly at conferences and user groups.
]
Next session is Stress, Load and Performance Testing in Quality Assurance by Goranka Bjedov of Google.
I have been wanting to hear Goranka for some time now as her sessions usually end up becoming the highlight of the event. For record, she passionately hates Power Point (I don't blame her).
I couldn't find a video of her Velocity talk but here is a video from her previous talk that's equally interesting.
Goranka spends all her time doing performance testing at Google. She tests Adwords, AdSense and hates any kinds of presentation tools.
focus on the backend. Steve Souders is the client side performance guy.
she works on the servers.
what are the bottlenecks?
QA people should be able to tell you what to expect.
2 purposes: what is going on in application? and monitor application for changes.
a small code change can cause tremendous performance decline.
first thing is if a mistake is made, everyone should know right away. Finding it later can cost a lot
80% of the performance problems can be worked out with one front end and one backend as long as you have the right database.
figure out what is happenging with important transactions
if you don't know what important transactions are, make a guess. it's better than nothing. don't be paralyzed, then worry about perfection
there is nothing as perfection in performance testing. she cannot guarantee the exact results. All tests are run as statistical tests and run 5 times or so.
big proponent of open source tools: jmeter, grinder and funkload. In Windows environment look at Open SDA.
Vendor tools do reasonably job. (and solve the problem of having too much money)
Open Source tools do exactly the same job. they are not free completely as time is required.
if you're not willing to spend time on OS tool, then why even spend half a million dollars?
she is happy that OS tools don't have monitoring built into it.
monitoring is absolutely essential and must be done separately.
for majority of things you can troubleshoot and benchmark within 3 weeks.
check google blog for her posts: open source performance testing tools
Just found on Slashdot that Google is attempting to patent 'FriendRank', an idea that Jeremy Zawodny came up with in, get this, 2004!
“A computer-implemented method for displaying advertisements to members of a network comprises identifying one or more communities of members, identifying one or more influencers in the one or more communities, and placing one or more advertisements at the profiles of one or more members in the identified one or more communities.” Way to go Google!
Five minutes after entering my office after a week long trip to CA, I learned that Seth MacFarlane, the creator of Family Guy, and Google have struck a deal. Under the deal, Seth MacFarlane will create 50 two-minute episodes exclusively for the web. There will be a number of new characters introduced in this new web-only series which will be distributed through the AdSense network. There will be a four way revenue share.
Now there is a panel, Working the Clouds: NextGen Infrastructure for New Entrepreneurs. Panelists include: * Geva Perry GigaSpaces * Jason Hoffman Joyent * Tony Lucas XCalibre * Lew Moorman Rackspace * Christophe Bisciglia Google * Joe Weinman AT&T
CB: Google App Engine can be called a cloud but it is a little different that allows you to only focus on very high levels.
Q: Are we selling our souls when moving to cloud? Or should we not worry about lock-in? There are no standards yet. To take advantage of cloud you need to embed it into code. It does create some locking issues. Hopefully in future there will be standards. If you're going to scale on EC2 and have scripts to automatically launch instances then moving to a different cloud will create issues. A lot of clouds that are gaining traction have very custom APIs. There is going to be a proprietary stack. On the other hand we will see convergence towards standards based platform. Then we end up with choice where you will then make decision based on your strengths.
There is still value in proprietary technologies. The more value you are going to build for your customers, the more proprietary the technology behind it would be. Both open and proprietary markets have their potential.
Even though App Engine is proprietary, the documentation is very open. If you are using Big Table, you are kind of selling your soul. Until Google open sources BigTable so users can export their data, it is locked-in.
API that Google provides isn't specific to BigTable. One of the criteria is how scalable the platform is. Christophe Bisciglia thinks that Google is a step ahead of Google when it comes to BigTable and their platform.
One shouldn't be locked into proprietary data store, period.
But, CB insists that BigTable performs better. There are things for which relational databases don't perform well. There is a sacrifice but it allows you to scale. What differentiates providers is how low their latency is and how scalable the infrastructure is and how protected the data is.
LM: A lot of companies are not right away ready for cloud.
The dirty little secret model for cloud computing is that if licensing models don't catch up, everything can drop dead.
Application development 5-10 years from now will be focused on the need to scale. The challenge is to move people up to cloud and provide them with tools to develop their applications the right way.
Thought shift is required for enterprises to move to cloud. CB gives an example of how people were hesitant to use banks when they came around as people were nervous of keeping their money with someone else. But then, as people realized the dangers of not keeping money at bank, banking became the norm.
AT&T will be spending $20 billion on their infrastructure.
Enterprises want to sign a contract. They don't want to just go and use credit card to start a relationship like this.
There is a big marketing problem. The word cloud means less and less everyday. Cloud is getting into the enterprise through the backdoor.
Patrick Chanezon (API Evangelist) and Paul McDonald (Product Manager for Google App Engine) presented a technical overview of OpenSocial and Google App Engine at Graphing Social Patterns East. If you aren't familiar with OpenSocial see the second presentation below, first.
Adam Lovallo of Inside Facebook was live blogging the session. Introduction to Google OpenSocial
Here's a video of Patrick Chanezon:
If you're interested in building OpenSocial Applications using Google App Engine, you may want to check out the article Building an OpenSocial App with Google AppEngine by Lane LiaBraaten, Google Developer Programs.
Open Social offers a standard for everyone. The IT rights of Open Social are owned by Open Social Foundation. Among Open Social members are hi5, imeem, bebo, ning, Oracle, Yahoo!, six apart, LinkedIn, viadeo, friendster, AOL and many more.
Open Social is 88 days old and reaches 275,000,000 users with 66 million installs of 2000+ apps developed by 20,000 developers. 10 million users use applications on Open Social.
Two client APIs : one for javascript and the other for REST.
Three areas:
people and friends: access friends information programatically.
activities: see what friends are doing
persistence: provide state without a server and share data wth friends.
Some examples follow next.
An Open Social application has 80% code that can be readily implemented on other OS containers.
If you have a social site and want to implement OS you can use Apache Shindig which has PHP and Java version of it. Shindig makes it really easy to implement. There is a very active mailing list for it.
SocialSite by Sun: Open source project that utilizes Shindig and builds on top of it. Dave "Roller" Johnson announced it at JavaOne. Heavy potential at Enterprise.
iGoogle: 50% userbase in US!
Google Friend Connect:
Users... more ways to do more things with my friends
Site owners... more and engaged traffic for site.
MySpace Developer Platform:
7 months old
Supporting REST APIs - over a year old
#1 social network
Why Develop for MySpace?
unique demographics: users you can't find elsewhere
forthcoming metrics / analytics focused on small to medium developers
user base is so large, you just need to get a small portion of MySpace users by popularity not virality.
Some Metrics of MySpace:
60K registered developers
1800+ apps
15 million installations (3 months)
MySpace Developer platform at developer.myspace.com
IRC: irc.freenode.net #myspacedev
Email: developerrelations at myspace.com
Twitter: MySpaceDevTeam
Dev Jams: range from 2-8 hours, devs bring their laptops and get first hand instruction and help from MDP team members
Myspace offers free application press releases apply at myspace at spark.pr.com
MySpace Developer Platform future:
application communication channel
custom notifications
invites - requesShare App
Metrics/analytics to playing field
Sachin (Imeem):
Imeem is social network focused on sharing and discovering all kinds of media.
24m unique users per month and third largest social network based on US traffic.
http://imeem.com/developers
they have made their content licenses to developers allowing them to utilize imeem's legally licensed media.
Types of applications
originally Adobe ActionScript 3 Flex Apps
OpenSocial Javascript APIs (in May)
External iFrame
OpenSocial extensions
immem-specific extensions that allow access to imeem media metadata, including music, videos and photos.
Example:
The Echo Chamber application example built on imeem.
My.Aol.com
newest partner with OpenSocial
4 brands
16 locales
12 languages and more on the way
55 myAOL portals
2-3 new locales per month
My.AOL.com Gadgest
myAOL is an AJAX basde web applciations (uses Dojo)
moving to Google Gadgets
this summer their platform will be completely transformed to work with Google Gadgets and Open Social.
A Morgan Stanley team consisting of "Queen of the Net" Mary Meeker, David Joseph and Anand Thaker recently did a presenation on Internet Trends.
The presentation focused on the topics of usage patterns, social networking, widgetization and componentization, measurability and transparency, customer satisfaction, video, monetization, mobile, emerging markets and last but not least, recession.
According to the presentation, consumer IP traffic will surpass business IP traffic in 2008 as consumers use more than 5,000,000 TB of data per month. Since 2005E, there has been a 58% CAGR in IP traffic.
"The surge in online content is at the center of the most dramatic changes affecting the Internet today. In three years' time, 20 typical households will generate more traffic than the entire Internet today."
"Fifteen years ago, enterprise technology was higher-quality than consumer technology. That's not true anymore. It used to be that you used enterprise technology because you wanted uptime, security and speed. None of those things are as good in enterprise software anymore (as they are in some consumer software). The biggest thing to ask is, 'When consumer software is useful, how can I use it to get costs out of my environment?'"
Douglas originally gave this quote in a WSJ interview while answering the question, "What's driving the "consumerization" of tech in the enterprise, where companies are borrowing tech ideas from the consumer Internet?"
Massive Transition in Available Ad Units This part of the presentation shows massive decline in the number of page views Yahoo! used to enjoy since 2002. At the same time the Alexa graph comparing Yahoo!'s page views to Google, Facebook and Youtube shows interesting patterns. Specifically YouTube's rapid rise to become the #2 destination on the Internet. This slide also touches on an important point that Supply of ad units is now greater than demand. An important thing to note is that Alexa changed their ranking algorithm recently. The latest Alexa graph for these sites doesn't go back to 2002, however it tells a slightly different story:
- Page views wise, Yahoo! is still number 1, followed by YouTube, Google and Facebook. - Reach wise, Google is number 1 followed by Yahoo!
Social Networking Characteristics - Fast Growth and Low Penetration
Next slide has a graph from from comScore's "Digital World: State Of The Internet" report which highlights growth in Emerging Internet Markets. The graph shows very fast growth for social networking sites but at the cost of low penetration. In contrast, the online search sector showed very high penetration but low levels of growth. Online personals, retail movies and Retail music industries depicted significant decrease in growth. Multimedia, on the other hand, shows decent growth but much higher penetration than social networks. Visitors growth to community focused sites that aren't social networks also decreased although these sites still have a strong penetration (even stronger than social networks).
Some other interesting findings from the comScore report that weren't in the Morgan Stanley presentation:
more than 300 million Internet users from Asia Pacific region, 15 years of age or older, were online in January 2008. This shows an increase of 14 percent over 2007 numbers (compare this to 10.4% increase in worldwide users). According to comScore, this increase "makes Asia Pacific the largest of the five worldwide regions"
Latin America and Middle East-Africa are two other regions that have also experienced "above average audience growth" since 2007. Latin America experienced 16.6% growth where Middle East-Africa experienced 20.2% growth, the highest percentage of growth among five worldwide regions.
US online audience now only represents 21% of worldwide Internet users.
Visits to social networking sites by global Internet users increased 34%
approximately 2 out of every 3 Internet users now visit a social network site with total visitors to social networking sites exceeding 530 million
MySpace and Facebook both now attract more than 100 million visitors per month
YouTube leads the way in online entertainment. Video is now the "dominant online entertainment format." More than 250 million visitors visited YouTube in January alone.
Social Networking Sites Gaining Significant Share of Online Traffic The next presentation slide highlights new entrants in the top-10 list of Alexa as well as top-10 sites of 2005 that have lost significant traffic. The sites that lost their top-10 ranking since 2005 include ebay.com, amazon.com, microsoft.com (not counting Microsoft Passport), google.co.uk, aol.com and go.com.
The new entrants in the top-10 list as of 2008 (based on old Alexa ranking model) included youtube.com, live.com, facebook.com, hi5.com, wikipedia.org and orkut.com. If the ranking list was a top-15 list, my employer would have been included at number 13.
How People Worldwide Spend Their Time Online
22% time is spent in online communication
16% time is spent in social communication. The presentation makes a note that this category didn't exist 3 years ago.
8% time is spent in online shopping activities
14% time is spent in entertainment and leisure activities
6% time is spent in work, business and education activities
Another interesting point made by the presentation, although no surprise, is that younger users (aged 15-24) tend to communicate more via Facebook whereas older Internet users (aged 44+) tend to use Yahoo! Mail more to communicate. The presentation raises the question whether email is becoming more archaic.
Comparison of popular sites
Facebook (#4 in global minutes) has experienced most growth (305%) since last year reaching 101 million members according to comScore. YouTube (#3 in global minutes) had the second highest growth (94%) with a total of 258 million users. Other two sites mentioned are PayPal and Skype, both eBay properties.
Two interesting facts about YouTube that I didn't know: A very high number of YouTube visitors (51%) visit the site weekly and half of the users "watch all videos to the end"
What is the most important source of Information?
Citing a study titled "Online World As Important to Internet Users as Real World?" conducted by Annenberg School for Communication , the presentation makes an important point: personal and online sources are two most important sources of information, and together they are the "essecnce of a social network."
The original report by digitalcenter.org also says that "online communities are a catalyst for connection and activism" and that "involvement in online communities leads to offline actions. More than one-fifth of online community members (20.3 percent) take actions offline at least once a year that are related to their online community." The original report also states that online activism is making online users get involved "in causes that were new to them when they began participating on the Internet" and that "more than 40 percent (43.7 percent) of online community members participate more in social activism since they started participating in online communities."
Another interesting point is that according to the Digital Future Project, Internet users 17 years of age or older trust Internet more (80%) than personal source (73%)
Some other important findings from USC-Annenberg Digital Future Project's report:
64.9 percent are involved in new causes
43.7 percent now participate more in social activism
56.6 percent log in to their community once a day
70.4 percent of community members "sometimes or always interact with other members of their community while logged in"
7.4 percent of American Internet users maintain a blog
23.6 percent of Internet users now post photos online. Previously only 11 percent were posting photos online.
12.5 percent of users now maintain their own website.
Internet users meet an average of 4.65 "friends online whom they have never met in person" and 1.6 "friends met in person whom they originally met online"
Facebook growth
Next, there are some metrics from Facebook's growth. Some highlights
as of 03/08 there were more than 14 million photo uploads a day being uploaded to Facebook and 6 million active user groups on Facebook.
there are 55,000 total networks (partitions?/shards? :) ) and 50% of the networks are outside of college. I wonder how many networks are location based?
Segment of users who are 25 years or older are the fastest growing segment. No surprise for me as my mother in law just joined Facebook.
There are more than 250,000 new registrations per day since January of 2007.
There are more than 859 million installations of 20,000 applications
Super Wall by Rock You! is the most popular application with 28 million installs on Facebook followed by Top Friends (26 million installs) and Fun Wall (25 million), both by Slide
Both Slide and Rock You! have 3 applications each in top 10 applications.
Facebook's own Video application is ranked #7.
Reasons for Facebook growing faster than MySpace
advertising that is considered less intrusive
news feed that is personalized
UI that is cleaner
friends section that receives more prominence
ads that are more personalized
more applications
more mobile friendly
facebook focusing more on monetizing word-of-mouth and conversations
The presentation titled Internet Trends is available on Slide Share, thanks to Tech Crunch.
This is the first edition of a new series of blog posts that I will try to write regularly (no hardcore promises though). I will be calling it, for the lack of better imagination and because it's 3:09 in the morning, The Strategy Chronicles, or TSC. The primary purpose of this series is to help me keep track of what's happening each week. So whether you liked it or hated it, please let me know. So, let's get started.
Microsoft makes Yahoo! investors very unhappy: At least some of the Yahoo! investors are now worried about an increased Microsoft bid to acquire Yahoo!. The primary reason being that there are several Yahoo! investors who are also Microsoft investors. Separately, several share holders are blasting Yahoo! for rejecting Microsoft's bid.
Yahoo! losing another ground: Opera has now decided to dump Yahoo! in favor of Google for its mobile browser. - "every month, Opera Mini users browse more than 1.7 billion pages" - most of the page views were related to search. Yahoo! has a serious relevancy problem in addition to the problem of effectively monetizing its search traffic. I can see how easy it was for Opera decision makers to ditch Yahoo! in favor of Google. - Google has been Opera's choice for desktop browser for 7 years.
Even more troubles for Yahoo!: Remember the time when Yahoo! handed over the information about its Chinese users to the authorities? Well, that issue continues to haunt Yahoo! as yet another lawsuit was filed against the company by its Chinese dissidents. Yahoo!'s submitted evidence lead to one year imprisonment of a plaintiff.
Finally!: Meanwhile, Google has finally re-launched JotSpot as Google Sites. Exactly how that will hurt Microsoft's similar initiatives is yet to be seen, however the outlook according to the media, doesn't look so promising as far as Microsoft is concerned.
Clean console!: Consumerist reports about an incident where a passionate Xbox user lost his beloved collection of autographs and custom artwork drawn on the console by prominent members of the gaming community. Perhaps, the Microsoft employee responsible for this thought the collectible autographs were making the Xbox unit look dirty.
Bill Gates now LinkedIn!: Facebook's loss is LinkedIn's gain. Bill Gates recently quit Facebook despite Microsoft's recent investment in the social networking powerhouse, and decided to start a profile on LinkedIn. Once his profile was created, Gates then asked a question that received more than 1,000 answers. Interestingly enough the same day I noticed the featured question, Facebook changed their site's theme.
Sliding the Social way!: Slide, the media and gadget distribution powerhouse announced recently that it will create new applications to for MySpace users using MySpace Developer Platform. This could increase Slide.com's profile and reach significantly. Earlier this year, Slide announced that it raised $50 million in its latest funding round.
A new research study by SVM Group shows some very interesting statistics regarding the Internet population that generates a big percentage of overall clicks.
According to the study, only 6% of Internet users are responsible for generating 50% of "all display ad clicks." Interestingly enough, this 6% doesn't represent general Internet population. Most of the users represented in this population are from 25-44 age group and have household income of less than $40,000.
Could this be the reason for Google admitting the troubles they are having with being able to monetize on social networks such as MySpace?
What implications does this have as far as the click fraud issue is concerned?
If a majority of 50% of these clicks is made by those with household income of less than $40K, are they just clicking on ads for curiosity? Should the advertisers be paying for their curiosity?
Is Google's smart pricing justified then? Google slaps publishers with smart pricing when it determines that clicks generated on the publisher's site aren't resulting in conversions on the advertiser's site. This is what Google says about Smart Pricing (also see facts about smart pricing):
...if our data shows that a click is less likely to turn into business results (e.g. online sale, registration, phone call, newsletter sign-up), we may reduce the price you pay for that click. You may notice a reduction in the cost of clicks from content sites.
One site getting hit with smart pricing can affect all sites in the network of the publisher. It seems that if these statistics are in fact true, most sites will run the risk of being smart priced.
Today, I noticed something very interesting on my MySQL blog. Google has started using AJAX functionality in their AdSense ad units. Notice the two blue arrows on bottom-left? Clicking on these buttons does an in-page refresh of ads so you can see more Google's ads. An excellent addition from publisher's point of view. However, as far as I am concerned, Google's attempt will be of no use to me. Are they serious? Do I browse so I can go from site to site clicking on these buttons to keep seeing an endless supply of ads just so Google can make more money.
So, let me ask you a question. Do you think you will be clicking on these buttons the next time you see them? Will they help publishers monetize better?
Just received email from Google Press and damn, Google has done it again:
"Google reported revenues of $4.23 billion for the quarter endedSeptember 30, 2007, an increase of 57% compared to the third quarterof 2006 and an increase of 9% compared to the second quarter of 2007. "
Recently I found out Google has been sending people Nokia Internet Tablets. I guess they need new ways to spend money.
