Orbitz Open Sources Monitoring Tools ERMA and Graphite

July 1, 2008 – 12:36 pm

I saw this recent article about some new tools from Orbitz: InfoQ: Orbitz Open Sources Monitoring Tools ERMA and Graphite

Graphite looks particularly interesting as it has features overcoming some of the things I don’t like about RRDtool:

Graphite is a Python web application that has been developed to provide scalable storage and visualization for numeric time-series data. It receives the output from the CEP engine, which consists of data aggregated for over 70,000 metrics. A Graphite portlet was developed in order to integrate with a monitoring portal, which is based on JBoss Portal framework. The portal presents tabular and graphical views of vital system statistics and RSS feeds for alarms. Users can subscribe to feeds for particular alarm severities and/or affected applications. The Graphite web application itself is used to display visual graphs of monitoring statistics generated using RESTful URLs. Graphite data can be presented as line graphs, pie charts or raw CSV data. Graphite also provides a web-based command line interface, which power users can use to very quickly and easily create and share dashboards containing collections of related graphs.

I’ll hopefully find some time to try this out

Extract Transaction Data From LoadRunner Analysis

June 15, 2008 – 3:38 pm

I had a need to extract transaction response time data from a bunch of LoadRunner Analysis files, and I really didn’t want to do endless cut and paste operations from within the LoadRunner Analysis tool. I created this Python script to extract transaction response time data from the LoadRunner Analysis mdb file and output into CSV format, which can then be imported into MySQL, Excel, etc. The analysis file is a MS Access JET Database which allows the use of Python’s win32com module to access the data.


# lr_extract.py
# --------------------------------------------------------------------
# Extracts Transaction response time data from the MS Jet database file (.mdb)
# created by LoadRunner Analysis tool
#
# Requires DAO 3.6 library.
# --------------------------------------------------------------------
# Usage: python lr_extract.py lr_analysis.mdb
import sys
import string
import pythoncom
import win32com.client

const = win32com.client.constants

daoEngine = win32com.client.Dispatch('DAO.DBEngine.36')
db = daoEngine.OpenDatabase(sys.argv[1])

query = """
select
    [Result].[Start Time]+[Event_meter].[End Time] as t_stamp,
    [Event_map].[Event Name] as txn_name,
    [Event_meter].[Value] as resp_time,
    [Result].[Result Name] as result_name
from
    Result,
    Event_map,
    Event_meter
where
    [Event_map].[Event Type] = 'Transaction' and
    [Event_meter].[Event ID] = [Event_map].[Event ID]
order by [Result].[Start Time]+[Event_meter].[End Time]
"""
rs = db.OpenRecordset(query)

print("timestamp,transaction,response time,result name")
while not rs.EOF:
    print("%s,%s,%s,%s" % (rs.Fields[0],rs.Fields[1],rs.Fields[2],rs.Fields[3]) )
    rs.MoveNext()

Apache JMeter 2.3.2 Released

June 14, 2008 – 1:51 pm Version 2.3.2 of Apache JMeter has been released. For details of new features and fixes, please see the JMeter web site. Download JMeter

Tools to Develop Faster Web Pages

June 12, 2008 – 9:03 am Jacob Gube just published "15 Tools to Help You Develop Faster Web Pages" at sixrevisions.com. It gives a nice list and high level overview of free tools that are useful for web performance. It includes such tools as load generators, profilers, and debugging proxies.

Pylot (my tool) made his list.. so go have a look!

His description:
"Pylot is an open-source performance and scalability testing tool. It uses HTTP load tests so that you can plan, benchmark, analyze and tweak performance. Pylot requires that you have Python installed on the server - but you don’t need to know the language, you use XML to create your testing scenarios."

Introduction to The Grinder

June 4, 2008 – 3:32 pm

The Grinder is:

The Grinder is a JavaTM load testing framework that makes it easy to run a distributed test using many load injector machines.

Until recently, I had been reluctant to investigate The Grinder because I was unfamiliar with Jython/Python, and hesitant to dig into a new language. I was wrong.

Not only am I happy to have discovered that Python and Jython are not too difficult to pick up, but The Grinder is quite a flexible tool that has a lot of potential and I would say a nearly capable replacement for LoadRunner from a load generation perspective.

In this article I’ll review how to setup The Grinder and use Eclipse to edit your scripts. 

Read the rest of this entry »

Customize size of images in LoadRunner HTML reports

May 29, 2008 – 9:57 pm

Quick tip: With prevalence of larger monitors these days, it is nice to have charts and graphs that take advantage of the extra space. If you want to customize the size of the charts in the LoadRunner HTML reports, change these settings in C:\Program Files\Mercury\LoadRunner\bin\dat\GeneralSettings.txt:

[HTML report]
ChartWidth=1024
ChartHeight=576

New look

May 23, 2008 – 11:59 am

If you’ve been to this site before, you may have noticed the new look. For various reasons, I’ve converted the site from drupal to Wordpress. Everything seems to be working, and hopefully most of the old links will redirect to their new locations.

One thing that is no longer here are the forums. They weren’t too active, but if you have a question feel free to add a comment here or find me on Twitter at http://twitter.com/perfeng

Thanks for visiting, come back often!

Article: Scalability Principles

May 21, 2008 – 11:06 am At the simplest level, scalability is about doing more of something. This could be responding to more user requests, executing more work or handling more data. This article presents some principles and guidelines for building scalable software systems. By Floyd Marinescu

Post-Deployment Slowdown?

May 17, 2008 – 9:27 am In this month’s issue of Software Test and Performance, Michele Kennedy shares a great experience report on troubleshooting performance issues. It includes some great tips, recommends some free tools, and includes some real examples. It’s a good read. I wish more performance testing articles were written this way - real examples.

determyne Releases Open Source J2EE Performance Monitoring Tool

May 12, 2008 – 11:57 pm determyne Inc. has announced the technology preview of an open source transaction-level performance monitoring solution for J2EE applications. insideApps is an end-to-end transaction tracing and reporting system that enables you to centrally and automatically monitor the performance of your J2EE applications. In contrast to the traditional approach of collecting and displaying aggregated metrics for different system components, insideApps focuses on monitoring applications from a transactions perspective.

Open Source Troubleshooting for Java

May 12, 2008 – 3:56 pm VisualVM is an OpenJDK project from Sun to create an all-in-one troubleshooting tool for Java applications. The tool is a combination of several existing tools and newer profiling capabilities. By Craig Wickesser

Examples on LoadRunner Regular Expressions

April 30, 2008 – 11:11 pm I'm going to show and explain how to use Regular Expressions in LoadRunner.

Introduction:
The present article is a summarizing of the LoadRunner Regular Expressions challenge and its results. Also, I added code for RegExp patterns/subpatterns matching.
All LoadRunner Regular Expressions functions are shown with examples.


Outline:
  1. How to check - whether RegExp pattern matches against a text or not
  2. How to get a matched strings (RegExp patterns and subpatterns)

How to check - Whether RegExp pattern matches against a text or not


I thanks Charlie Weiblen and Tim Koopmans for the solution. I modified it slightly.
So, here it is:
Read the rest of this entry »

Python - RRDTool Utilities (module and scripts for RRDs)

April 29, 2008 – 8:37 am

I started a project on Google Code to create a set of Python tools to make dealing with Round Robin Databases (RRD) less painful.  Setting up RRD's can be tough if you don't know what you are doing.

anyone interested can check it out here:  rrdpy

I used it to create a simple HTTP monitoring script (included in source) to graph web response latency like this:

Multiple Dimensions of Performance Testing

April 24, 2008 – 6:18 pm Almost all experts agree that pre-deployment "waterfall" performance testing (which, with the record/playback method, confused by many as the performance testing itself) is not enough - too little, too late. Actually it is just one very specific way of performance testing - with a full spectrum of other approaches, which are used so infrequently (at least as intentional performance testing techniques) that I don't recall finding any good classification. Thinking about it, I see several dimensions of performance testing which, although definitely correlated, probably might be considered somewhat independently - of course, just a raw idea for the moment, just an effort to order thoughts a little.

Solaris Performance Primer: Introduction

April 23, 2008 – 9:38 am

This web log will is the start of a small series of Solaris performance monitoring tools. We are going to present freely available tools from Solaris, the Tools CD and the DTrace toolkit.

This blog will help the upatient ones who want to solve 90% of the day to day performance issues without long studies.

Being able to scale quickly is pivotal in the web age where the load can grow by an order of magnitude overnight. Our little primer will release smaller chapters which are going to apply the following methodology:

  • Chapter 2: Document your performance issue: How to instrument my data center with dimSTAT to get an idea when the performance worries happened
  • Chapter 3:What are the features and limits of my system: How to analyze your Solaris installation, patches etc.. Quantify the abilities of your hardware: How many processors, disks, memory etc.do I have?
  • Chapter 4: What is my system actually doing? Checking out your processes, top users, resource consumption. Understand your system utilization.
  • Chapter 5: What is my process doing? Tools for process introspection. Learn which files are being used, which libraries are being loaded, which call stacks you're currebtly using etc.
  • Chapter 6: Monitoring the processes and threads in projects and zones with prstat
  • Chapter 7:Understand your IO: Who is writing how to where? How the the disk sub system doing?
  • Chapter 8: Understanding network traffic: Which network interface is working how hard? Is my interface overloaded etc.
  • Chapter 9: Tracing at large: How to get information about current system call. How to use DTrace to answer common questions like: Who is writing to which file right now?

This upcoming sequence of blogs will hopefully answer the most common performance questions with standard tools available for Solaris.

Other sites which have Solaris Performance related information are: 

  • The one's who are interested in a more complete (and professional) resource may want to consider Darryl Gove's book "Solaris Application Programming". A great chapter detailing the system tools is available for free online.
  • solarisinternal.com: A comprehensive wiki with available tools and best practise documents.
  • Solaris Performance and Tools: a book by Richard McDougall, Jim Mauro and Brendan Gregg from July 2006.
     

    The content of this document has been created by Thomas Bastian, a performance tuning specialist who gained his knowledge through many performance projects with the key software partners in EMEA from Sun Microsystems