Building Performance Testing Teams

November 4, 2007 – 8:31 am

Introduction

There have been many books, articles and web sites dedicated to various aspects of performance testing: how to use load testing tools, how to create automated scripts, how to monitor servers, etc. etc. What I haven’t seen, is any discussion about how to create a performance testing team: what processes to use, what works in a team context and what doesn’t. In this article, I will discuss some of the issues I have faced in building performance testing teams.

Division of Labor

As soon as you have more than one person responsible for performance testing, you will need to determine how the performance testing tasks are divided. Division of labor can be done by application, by project, by activity (creating test plans, creating test scripts, testing, monitoring, analyzing, etc.) There are many factors in play when determining the best way to including: size of team, size of application, release schedules, team abilities, etc.

By Application By Project By Testing Activity
Pros:
  • Test engineers gain depth of knowledge for their application, become Subject Matter Experts (SMEs)
  • Test engineers utilize skills in entire performance testing life cycle
  • Test engineers are exposed to full performance testing life-cycle
  • Potential to make the best use of individual skills
Cons:
  • Harder for the more skilled or senior test engineers to share or apply their skills and expertise to the other applications
  • Knowledge sharing and continuity can be a challenge if using contract employees or have high turnover
  • Harder to do with large-scale applications or long-term projects
  • Employee satisfaction can take a hit. Who wants to be stuck doing the same thing over and over?
  • Additional communication overhead
Best for:
  • Larger teams with multiple applications
  • Small-scale applications and/or projects
  • Companies with few applications, but many projects or regular application updates requiring frequent testing.
  • Smaller teams (less than about 3)
  • High percentage of short-term contract employees and enough work or projects to keep them busy
  • Team members lacking skills in some areas (test plans or analysis, for example)

Hopefully this table provides some insight for you to consider for your testing teams.

What To Do With All This Stuff?!

Performance testing generates a lot of “stuff”. A month of testing could easily generate hundreds of gigabytes of result data. In addition to test results, there are other test assets that may need to be saved and managed- test plans, test scripts, meeting notes. What to do with all of this stuff? It is critical for effective and efficient performance testing teams to have processes and standards about how to deal with all of the test assets and result data. Here are some ideas to manage your test assets.

Where to save? How long to keep?
Test Plans
  • Document Repository
    • Sharepoint
    • Confluence
    • Documentum
  • Corporate wiki
  • Ideally, forever. Test plans, even out-dated ones, are useful reference materials
Test Scripts

Ideally test scripts are versioned along with the actual application code under test. That way, it is easier to back and test previous versions of the application.

  • Source Code Management
    • CVS
    • Subversion
    • ClearCase
  • Central File-Server
  • They don’t take a lot of space, so could potentially be forever
  • If using SCM, managed in smae manner as application code
  • Otherwise, keep scripts as long as for supported versions.
Test Results
  • Raw Data
    • File Server
  • Summary Data
    • File Server
    • Database
  • Raw data should be discarded after summary data and reports are generated
  • After summary reports created, raw data is rarely looked at again
  • Can be kept for duration of project if needed for comparison purposes
Test Reports
  • Document Repository
    • Sharepoint
    • Confluence
    • Documentum
  • Corporate wiki
  • File Server
    		•
    • Should be able to keep forever

    Building Expertise and Efficiency

    Building expertise and efficiency is critical to the ongoing health of your performance testing team and can lead to better accuracy of results, more efficient use of resources and lower employee turnover. Achieving the two Es (expertise and efficiency) can be done by establishing standard processes and templates.

    Processes

    Well defined processes are critical to efficienct teams. Having a well-defined process doesn’t guarantee success, but having the wrong processes can guarantee that your performance test results will not be as useful as they should be. What is a good process? A good process is rigid and flexible. Rigid in the sense that the process enforces that the proper performance testing methodology is followed, but flexible in the sense that the process is not heavy, and can be used for small and large projects, or in the areas required for your company. Additionally, the process should be repeatable, such that the same results and conclusions are reached no matter who the test engineer is.

    Templates

    Having templates is critical for consistency and efficiency. Templates for test plans, test scripts and test reports can be a tremendous time-saver. Not only will they lead to more efficient use of time, but the consistency in appearance and content of the test plans, scripts and reports will reinforce your test methodology and make your reports more readable and understood by the target audience. If all of your test reports use the same template, then the target audience will learn how to read them, get more value from them, and, therefore, increase the value of the performance testing team.

    Choosing the Right Tools

    Post a Comment

    PerformanceEngineer.com is proudly powered by WordPress Entries (RSS) and Comments (RSS). Theme by Bob