Stress, Load and Performance Testing in Quality Assurance

Next session is Stress, Load and Performance Testing in Quality Assurance by Goranka Bjedov of Google.

I have been wanting to hear Goranka for some time now as her sessions usually end up becoming the highlight of the event. For record, she passionately hates Power Point (I don't blame her).

I couldn't find a video of her Velocity talk but here is a video from her previous talk that's equally interesting.



Goranka spends all her time doing performance testing at Google. She tests Adwords, AdSense and hates any kinds of presentation tools.

• focus on the backend. Steve Souders is the client side performance guy.
• she works on the servers.
• what are the bottlenecks?
• QA people should be able to tell you what to expect.
• 2 purposes: what is going on in application? and monitor application for changes.
• a small code change can cause tremendous performance decline.
• first thing is if a mistake is made, everyone should know right away. Finding it later can cost a lot
• 80% of the performance problems can be worked out with one front end and one backend as long as you have the right database.
• figure out what is happenging with important transactions
• if you don't know what important transactions are, make a guess. it's better than nothing. don't be paralyzed, then worry about perfection
• there is nothing as perfection in performance testing. she cannot guarantee the exact results. All tests are run as statistical tests and run 5 times or so.
• big proponent of open source tools: jmeter, grinder and funkload. In Windows environment look at Open SDA.
• Vendor tools do reasonably job. (and solve the problem of having too much money)
• Open Source tools do exactly the same job. they are not free completely as time is required.
• if you're not willing to spend time on OS tool, then why even spend half a million dollars?
• she is happy that OS tools don't have monitoring built into it.
• monitoring is absolutely essential and must be done separately.
• for majority of things you can troubleshoot and benchmark within 3 weeks.
• check google blog for her posts: open source performance testing tools
  

Also see:

• Performance Testing: Blog post by Goranka Bjedov
• Google Testing Blog
• Optimizing Linux Performance by Ezolt
  

KITE: Keynote Internet Testing Environment Launch at Velocity

Better late than never. At Velocity Conference I took a lot of notes and didn't get to publish them earlier. Now that I find some time on my hand, I am going to go ahead and publish them.

At Velocity, there will be two product launches. Vik Chaudhary (Keynote Systems, Inc.) and Abelardo Gonzalez (Keynote Systems) are on stage now.

The first product is KITE (Keynote Internet Testing Environment). Keynote has been in business for 12 years. Providing a single performance testing environment for everyone (web developers, QA and system administrators) has always been a challenge for both startups and major players. This presentation sounds very promising.

• Keynote collects 200 million internet measurements everyday.
• With KITE, You can test from desktop to the internet cloud.
• Keynote built it to provide a single performance testing environment for web developers, QA and IT operations team.
• Sites are making upgrades every single day. Customers want to be able to test it from multiple locations
  

KITE is Ideal for 3 things

1. recording , scripting and playback of web transactions
2. instant web tests from desktop
3. scheduled testing for higher productivity
   

KITE Web Performance Engine allows for:

• performance analysis for multi-page transactions, not just single pages
• Javascript programmability for scripting and DOM analysis
• testing from the desktop, Last Mile and the Internet Cloud
• Native IE integration which in turn allows for easily analysis of AJAX, Flash and Javascript
• Recording of test scripts which can be played back in "burst" mode and share scripts
  

Next is a demo by Abelardo Gonzalez.

• You get a script repository that you can save for performance benchmarks.
• There is a record button and a free global test button.
• They will be testing iGoogle.
• You start a session then as you take actions, they are recorded in the console.
  

KITE helps with three main things:

1. user experience time
2. network time
3. cache network time (for repeat visitors)
   

You can download very deep into each page and each performance metric. Tests from Global Network are also possible where you can replay exactly the steps you made from your desktop from all supported areas (e.g. Atlanta).

KITE

• early adopter program
• starts in August 2008
• signup at http://kite.keynote.com
• KITE is FREE!

Great product. Definitely worth checking out.

Also see:

• KITE Presentation
• Velocity Product Launch: KITE by Steve Souders
• Announcing KITE for early adopters at Velocity by Keynote official blog