PerformanceEngineer.com

Performance Testing from the Cloud

I’ve seen a lot of activity lately about using “the cloud” for performance testing. Here’s the current run-down of load testing options “from the cloud:”

Open Source

• Pylot - Web Load Testing from Amazon Elastic Compute Cloud (EC2)
• Grinder In The Cloud - A cloud based load testing environment
• Load Testing From The Cloud (JMeter)

Pay to play

• SOASTA
• BrowserMob
• KeyNote
• LoadStorm

It’s exciting to see so many new options cropping up. I would not be surprised to see some of the other smaller players in the performance testing space start offering similar services this year, while the larger (HP, Borland) companies will be unable to move quickly enough away from their existing “consulting services.” The open source options appear to be well behind the current commercial offerings, but could be viable options if you’re willing to put in the time.

___________
UPDATE: 2/24/09 2:08 PM — Added LoadStorm
___________
UPDATE: 3/13/09 10:14 AM — Corrected URL for “Grinder In The Cloud …”

Posted in Test Tools | 3 Comments »

The Art of Application Performance Testing

Book Description

This practical book provides a step-by-step approach to testing mission-critical applications for scalability and performance before they’re deployed — a vital topic to which other books devote one chapter, if that.

Businesses today live and die by network applications and web services. Because of the increasing complexity of these programs, and the pressure to deploy them quickly, many professionals don’t take the time to ensure that they’ll perform well and scale effectively. The Art of Application Performance Testing explains the complete life cycle of the testing process, and demonstrates best practices to help you plan, gain approval for, coordinate, and conduct performance tests on your applications. With this book, you’ll learn to:

* Set realistic performance testing goals
* Implement an effective application performance testing strategy
* Interpret performance test results
* Cope with different application technologies and architectures
* Use automated performance testing tools
* Test traditional local applications, web-based applications, and web services (SOAs)
* Recognize and resolves issues that are often overlooked in performance tests

Written by a consultant with 30 years of experience in the IT industry and over 12 years experience with performance testing, this easy-to-read book is illustrated with real-world examples and packed with practical advice. The Art of Application Performance Testing thoroughly explains the pitfalls of an inadequate testing strategy and offers you a robust, structured approach for ensuring that your applications perform well and scale effectively when the need arises.

Posted in Books | No Comments »

The Art of Capacity Planning: Scaling Web Resources

Book Description

Success on the web is measured by usage and growth. Web-based companies live or die by the ability to scale their infrastructure to accommodate increasing demand. This book is a hands-on and practical guide to planning for such growth, with many techniques and considerations to help you plan, deploy, and manage web application infrastructure. The Art of Capacity Planning is written by the manager of data operations for the world-famous photo-sharing site Flickr.com, now owned by Yahoo! John Allspaw combines personal anecdotes from many phases of Flickr’s growth with insights from his colleagues in many other industries to give you solid guidelines for measuring your growth, predicting trends, and making cost-effective preparations. Topics include:

• Evaluating tools for measurement and deployment
• Capacity analysis and prediction for storage, database, and application servers
• Designing architectures to easily add and measure capacity
• Handling sudden spikes Predicting exponential and explosive growth
• How cloud services such as EC2 can fit into a capacity strategy

In this book, Allspaw draws on years of valuable experience, starting from the days when Flickr was relatively small and had to deal with the typical growth pains and cost/performance trade-offs of a typical company with a Web presence. The advice he offers in The Art of Capacity Planning will not only help you prepare for explosive growth, it will save you tons of grief.

Posted in Books | No Comments »

Performance Engineering Podcast

Software Engineering Radio has a podcast on Performance Engineering.

In this episode Martin talks with Chris Grindstaff about the fundamentals of performance engineering. The episode discusses when and how to work on performance of client- and server-side systems, what you should take into account during development to avoid performance issues, typical situations that cause performance problems, and some common pitfalls when analysing performance.

It’s worth a listen

Posted in General, Performance | No Comments »

New Site Feature: Careers

I’ve added a new feature to the site: Careers

If you’re in the market, keep an eye on this page and I’ll post job openings as I receive them. If you would like to list a job opening, please send an email to job-listings@performanceengineer.com

Posted in Careers | 1 Comment »

Orbitz Open Sources Monitoring Tools ERMA and Graphite

I saw this recent article about some new tools from Orbitz: InfoQ: Orbitz Open Sources Monitoring Tools ERMA and Graphite

Graphite looks particularly interesting as it has features overcoming some of the things I don’t like about RRDtool:

Graphite is a Python web application that has been developed to provide scalable storage and visualization for numeric time-series data. It receives the output from the CEP engine, which consists of data aggregated for over 70,000 metrics. A Graphite portlet was developed in order to integrate with a monitoring portal, which is based on JBoss Portal framework. The portal presents tabular and graphical views of vital system statistics and RSS feeds for alarms. Users can subscribe to feeds for particular alarm severities and/or affected applications. The Graphite web application itself is used to display visual graphs of monitoring statistics generated using RESTful URLs. Graphite data can be presented as line graphs, pie charts or raw CSV data. Graphite also provides a web-based command line interface, which power users can use to very quickly and easily create and share dashboards containing collections of related graphs.

I’ll hopefully find some time to try this out

Posted in Monitoring, tools | 2 Comments »

Extract Transaction Data From LoadRunner Analysis

I had a need to extract transaction response time data from a bunch of LoadRunner Analysis files, and I really didn’t want to do endless cut and paste operations from within the LoadRunner Analysis tool. I created this Python script to extract transaction response time data from the LoadRunner Analysis mdb file and output into CSV format, which can then be imported into MySQL, Excel, etc. The analysis file is a MS Access JET Database which allows the use of Python’s win32com module to access the data.

# lr_extract.py
# --------------------------------------------------------------------
# Extracts Transaction response time data from the MS Jet database file (.mdb)
# created by LoadRunner Analysis tool
#
# Requires DAO 3.6 library.
# --------------------------------------------------------------------
# Usage: python lr_extract.py lr_analysis.mdb
import sys
import string
import pythoncom
import win32com.client

const = win32com.client.constants

daoEngine = win32com.client.Dispatch('DAO.DBEngine.36')
db = daoEngine.OpenDatabase(sys.argv[1])

query = """
select
    [Result].[Start Time]+[Event_meter].[End Time] as t_stamp,
    [Event_map].[Event Name] as txn_name,
    [Event_meter].[Value] as resp_time,
    [Result].[Result Name] as result_name
from
    Result,
    Event_map,
    Event_meter
where
    [Event_map].[Event Type] = 'Transaction' and
    [Event_meter].[Event ID] = [Event_map].[Event ID]
order by [Result].[Start Time]+[Event_meter].[End Time]
"""
rs = db.OpenRecordset(query)

print("timestamp,transaction,response time,result name")
while not rs.EOF:
    print("%s,%s,%s,%s" % (rs.Fields[0],rs.Fields[1],rs.Fields[2],rs.Fields[3]) )
    rs.MoveNext()

Posted in Python, Tips & Tricks, loadrunner | 2 Comments »

Introduction to The Grinder

The Grinder is:

The Grinder is a Java^TM load testing framework that makes it easy to run a distributed test using many load injector machines.

Until recently, I had been reluctant to investigate The Grinder because I was unfamiliar with Jython/Python, and hesitant to dig into a new language. I was wrong.

Not only am I happy to have discovered that Python and Jython are not too difficult to pick up, but The Grinder is quite a flexible tool that has a lot of potential and I would say a nearly capable replacement for LoadRunner from a load generation perspective.

In this article I’ll review how to setup The Grinder and use Eclipse to edit your scripts. 

Read the rest of this entry »

Posted in Test Tools | 10 Comments »

Customize size of images in LoadRunner HTML reports

Quick tip: With prevalence of larger monitors these days, it is nice to have charts and graphs that take advantage of the extra space. If you want to customize the size of the charts in the LoadRunner HTML reports, change these settings in C:\Program Files\Mercury\LoadRunner\bin\dat\GeneralSettings.txt:

[HTML report]
ChartWidth=1024
ChartHeight=576

Posted in Tips & Tricks | No Comments »

New look

If you’ve been to this site before, you may have noticed the new look. For various reasons, I’ve converted the site from drupal to Wordpress. Everything seems to be working, and hopefully most of the old links will redirect to their new locations.

One thing that is no longer here are the forums. They weren’t too active, but if you have a question feel free to add a comment here or find me on Twitter at http://twitter.com/perfeng

Thanks for visiting, come back often!

Posted in General | 2 Comments »