High-performance Ajax Applications

Yet more notes from Velocity.

Next session is High-performance Ajax Applications by Julien Lecomte (Yahoo!).

• Plan for performance from day 1
• work closely with designers and product managers
• understand design rationale
• explain the tradeoffs between design and performance
• offer alternatives and shw what is possible (prototypes)
• as a last resort, simplify design
  

Engineering high performance: a few basic rules:

• don't do anything unnecessary
• less is more
• break the rules
• work on improving perceived performance
• users can generally deal with a little bit of discomfort if they can see something is happening.
• can't compromise security but someother things can be compromised
• in general avoid presentational markup
  

Measuring performance:

• test performance using a setup similar to your user's environment
• profile your code during development
• automate profiling/performance testing
• keep historical records of how features perform
• consider keeping some (small amount of ) profiling code in production
  

Yahoo!'s exceptional performance rules

• make fewer http requests
• use a content delivery network
  

Asset optimization:

• minify CSS and JS files
• combine CSS and JS files
• optimize image assets
  

Reduce unminified code size:

• loading and parsing HTML, CSS and JS code is costly
• be concise and write less code
• make good use of javascript features
• consider optimizing your large JS files into smaller files (bundles) when the parsing and compilation of the script takes an excessive amount of time
• load code (HTML, CSS and JS) on demand
  

Optimize initial rendering (1/4) misc tips

• consider rendering the first view on the server. (server should generate the markup)
• this will speed up the intial rendering.
• close your HTML tags to speed up parsing
• consider flushing the apache buffer very early on
• load only essential assets/load assets on a delay or on demand.
• don't always wait for onload
• most DOM operations can be accomplished before he onload event has fired
• post load script loading:
• a well designed site should be fully functional even without the JS enabled
• therefore you may be able to load scripts on a delay
• conditional preloading:
• preload assets that you know user is likely to need very shortly
• however, one must be smart about when the preloading takes place. Otherwise the preloading may actually worsen the user experience.
  

Part 3: High Performance Javascript:

• look up is performed in JS everytime a variable is accessed.
• declare with the var keyword and use variables in the same scope whenever possible and avoid global resources at all costs.
• never use the with keyword as it prevents the compiler from generating code for fast access to local variables.
• cache the results of expensive lookups in local variables.
  

The prototype chain:

• accessing member found in the primary object is about 25% faster
• optimize object instantiation:
• if you need to create many objects, consider adding members to the prototype instead.
  

Don't use eval

• eval is evil
• the string passed to eval (and its relatives, the function constructr and setTimeout and setInterval function needs to be compiled and interpreted (extremely slow).
  

optimize string concatenation:

• on IE concatentating two strings causes a new string to be allocated and the two original strings to be copied.
• therefore, it is mch faster on IE to append string to an array and then use Array.join
  

optimize regular expressions.

• don't use the RegExp constructure unless your regular expression is assembled at runtime. Instead, use regular expression literals.
• use the test mehod if all you want to do is test for a pattern. (the exec method carries a small performance penlty.
  

Caching

• caching can be justified when there is a high cost associated with getting/accessing the data and when data wouldn't change over time.- increases memory consumption (tradeoff)

memorization

long running javascript running process (longer than 300ms):

• the entire browser UI is frozen.
• to maintain a decent user experience make sure that JS threads never take more than 300 ms.
  

misc:

• function calls have overhead associated with them
• consider using primitive operations since they are often faster than the corresponding function cals
• if possible, avoid using try..catch in performance critical sections
• if possible, avoid for...in in perormance critical sections
• branch outside, not inside, whenever the branching condition does not change

Harnessing Explosive Growth: Infrastructure Strategies and Tactics

The session I am now waiting for is Harnessing Explosive Growth: Infrastructure Strategies and Tactics. The official description of the session is:

What worked in the garage can rarely be scaled effectively for the boardroom. This panel will bring together some of the biggest names in web infrastructure to share their thoughts, insights and tactics for harnessing explosive growth, with a focus that goes beyond simply which technologies are available but how to best deploy them. This panel is not to be missed

Panelists include:

• Sandy Jen Meebo
• Akash Garg hi5 Networks
• Jeremiah Robison: Slide
• Jonathan Heiliger Facebook
• James Barrese eBay

How much of scalability is architecture, and how much is throwing servers at the problem?

SJ: 99% architecture
JH: Product is what drives architecture. We have more than 10,000 servers. For chat, they built a separate network. www.facebook.com/eblog.

JR: The biggest consideration is how their servers work with Facebook.

When's it broken? When did you know your first architecture was broken?
AG: First million users. Then they started focusing on caching and sharding.

JB: eBay had a few catastrophic issues near 2000. People were very forgiving of availability issues. They give refunds when there is downtime.

JR: They moved a lot of their technology to EDGE.

SJ: Since Meebo is just one page, javascript delivery is a major issue for them. They are not generating pages. Dynamic loading and background computation is really important.

RP: Application and infrastructure is going to break. If you look hard enough you can find where are the scalability issues.

JR: They are very metric driven. It's not often they see outages. They have number of Facebook's ops folks on their IM lists and they talk continously.

JH: We work through problems together.

JH: They have had to turn off some applications because other applications were being affected.

JB: At eBay they have created a central application login system. They can flag and identify problems really quickly. If you don't have it, you're shooting in the dark.

Rolling your own stuff? Off-the-shelf vs. custom:
Did you roll your own? Do you regret it?

SJ: For cash restraint startup, off-the-shelf can work. But for scale, you'd want to build yourself. Open Source is awesome. No one can scale your system as well as you can. Off-the-Shelf can be bulky. You have to get your hands dirty.

JR: We built our own caching backend. Invest time in core stuff, anything that's not core, don't focus there.

What also needs to scale as you grow? What non-technology things you had to scale?

JB: Need to scale out your business as well as technology.

AG: Building anti-spam features into the product that are scalable.

JH: Make community part of the process in translations as you grow.

JR; They introduced user moderation for photos. Hard to find what's porn and what's not. It's about a dozen people looking at photos full-time to hunt down porn.

How should we handle the fallout? If you were Twitter what would you have done last month?
JB: You have to be transparent. Tell them what's going on. Setup message boards for communication. You've got to communicate.

AG: Setting realistic timelines is very important

JH: We do it often. We roll in small chunk and if things don't look right, we roll back.

JB: You cannot operate a large system without the ability to turn things on and off.

JR: If you have an aggressive competitor, you don't have the luxury of downtime.

RP: You can't roll out something that can't be rolled back.

Visualizing Multidimensional Data

Bernard Kerr, the designer for Delicious, talks about threads, tags and visualizations at Innovations Forum. It's a very interesting presentation covering, among other things, IBM's Remail, thread arcs, and Tag Orbitals.

Some lessons he wants the audience members to take home:
- When you are working on visualizing multidimensional data, you cannot use all dimensions, instead you must focus on one or two dimensions. If you focus on more than 1 or 2 dimensions, you'll end up with a spaghetti.
- Get real data into visualization as soon as you can.
- Try illustrator for quickly visualizing.

Watch Bernard's talk:

Bernhard Kerr // Yahoo! from Innovationsforum on Vimeo.

Geotagging with where On Earth ID (WOEID) API

Yahoo! has taken the next step in geotagging by providing developers with the ability to easily geo tag their content. Specifically Yahoo! has opened up their geotagging API using which developers can now get real-time WOEID (Where on Earth ID) information.

Flickr is now providing the following WOE API services:
* flickr.places.find
* flickr.places.findByLatLon
* flickr.places.resolvePlaceId
* flickr.places.resolvePlaceURL


Dan Catt of Flickr (Yahoo!) has more regarding WOEIDs on geobloggers.com. In his blog post he points out Flickr photos search by WOEID which allows you to enter WOEID and get all photos that are of that area. He also highlights the differences between Flickr API and Yahoo! geo API:

“Over at Flickr, we only use specific ‘levels’ of geo information, such as city, region, state, country, while the APIs over at Yahoo will spit out far more levels in-between the ones Flickr uses, as well as deeper down to neighborhood levels, which Flickr doesn’t do (yet).”

One thing to note is that WOEID isn't assigned at address level, only at town or neighborhood level.

The Strategy Chronicles #1

This is the first edition of a new series of blog posts that I will try to write regularly (no hardcore promises though). I will be calling it, for the lack of better imagination and because it's 3:09 in the morning, The Strategy Chronicles, or TSC. The primary purpose of this series is to help me keep track of what's happening each week. So whether you liked it or hated it, please let me know. So, let's get started.

Microsoft makes Yahoo! investors very unhappy: At least some of the Yahoo! investors are now worried about an increased Microsoft bid to acquire Yahoo!. The primary reason being that there are several Yahoo! investors who are also Microsoft investors. Separately, several share holders are blasting Yahoo! for rejecting Microsoft's bid.

Yahoo! losing another ground: Opera has now decided to dump Yahoo! in favor of Google for its mobile browser.
- "every month, Opera Mini users browse more than 1.7 billion pages"
- most of the page views were related to search. Yahoo! has a serious relevancy problem in addition to the problem of effectively monetizing its search traffic. I can see how easy it was for Opera decision makers to ditch Yahoo! in favor of Google.
- Google has been Opera's choice for desktop browser for 7 years.

Even more troubles for Yahoo!: Remember the time when Yahoo! handed over the information about its Chinese users to the authorities? Well, that issue continues to haunt Yahoo! as yet another lawsuit was filed against the company by its Chinese dissidents. Yahoo!'s submitted evidence lead to one year imprisonment of a plaintiff.

Finally!: Meanwhile, Google has finally re-launched JotSpot as Google Sites. Exactly how that will hurt Microsoft's similar initiatives is yet to be seen, however the outlook according to the media, doesn't look so promising as far as Microsoft is concerned.

Clean console!: Consumerist reports about an incident where a passionate Xbox user lost his beloved collection of autographs and custom artwork drawn on the console by prominent members of the gaming community. Perhaps, the Microsoft employee responsible for this thought the collectible autographs were making the Xbox unit look dirty.

Bill Gates now LinkedIn!: Facebook's loss is LinkedIn's gain. Bill Gates recently quit Facebook despite Microsoft's recent investment in the social networking powerhouse, and decided to start a profile on LinkedIn. Once his profile was created, Gates then asked a question that received more than 1,000 answers. Interestingly enough the same day I noticed the featured question, Facebook changed their site's theme.

Sliding the Social way!: Slide, the media and gadget distribution powerhouse announced recently that it will create new applications to for MySpace users using MySpace Developer Platform. This could increase Slide.com's profile and reach significantly. Earlier this year, Slide announced that it raised $50 million in its latest funding round.

Microsoft's hostile bid now jeopardizing Yahoo!'s future?

A lot of chatter in the news about the negative effects Yahoo! is experiencing because of Microsoft's hostile bid earlier this year. The bid left a lot of uncertainty as far as employees of Yahoo! are concerned.

In Yahoo!'s SEC filing, the company is not shy of explaining what Microsoft's bid is doing to the company. The filing

Microsoft’s unsolicited acquisition proposal has created a distraction for our management and uncertainty that may adversely affect our business.

Microsoft’s unsolicited acquisition proposal has also created uncertainty for our employees and this uncertainty may adversely affect our ability to retain key employees and to hire new talent. Microsoft’s unsolicited acquisition proposal may also create uncertainty for current and potential publishers, advertisers and other business partners, which may cause them to terminate, or not to renew or enter into, arrangements with us. Additionally, we and members of our Board of Directors have been named in seven purported stockholder class action complaints relating to the Microsoft proposal as more fully described in Part I, Item 3 “Legal Proceedings” of this Annual Report on Form 10-K. These lawsuits or any future lawsuits may become time consuming and expensive. These consequences, alone or in combination, may harm our business.

Regarding the volatility of Yahoo!'s stock:

We further believe that, as a result of Microsoft’s unsolicited acquisition proposal, and speculation concerning a potential acquisition, the future trading price of our common stock is likely to be volatile and could be subject to wide price fluctuations. There can be no assurance whether a transaction will occur or at what price. If a transaction does not occur, or the market perceives a transaction as unlikely to happen, our stock price may decline.

Yahoo!'s brain drain problem may have also intensified due to the company's planned layoffs announced earlier this year.

Yahoo's Daily Top Searches

On Yahoo! Buzz, you can see the top 20 searches being performed every day.