40 BETTER SOFTWARE FEBRUARY 2006 www.StickyMinds.com

The Last Word

Stress Testing—What a Load!by Scott G. Ames

Over the years, I have seen many stresstesting projects. One question I am oftenasked is how to ramp up the amount ofstress in a project. Now, I know thatwhat is really being asked is how to increase the amount of load—notstress—in a test. But I am a quality engineerby trade and I like specifics, so I will attempt to answer the question—not as itwas intended, but rather as it was asked.If you really want to increase the overallstress on your people and in your testingproject, these are some of the best practicesto follow:

Skip “Unnecessary” StepsDefining goals, requirements, and test

specifications takes a long time. Thesesteps not only cause test engineers to create tests that will execute properlywith something other than just simpletest data, but also seriously cut into theirfunctional testing of Bejeweled and loadtesting of Internet Web servers. If a testworks with any possible type of data, itsignificantly reduces the amount of stressin a testing project. Not taking the timeto do this right increases stress by forcingpeople to do it over—and possibly overagain—under deadline pressure.

Defined goals, requirements, andspecifications remove any possibility of“scapegoating” outsourced resources forfailures that occur in the test project. Thisis especially important when validatingthat a system works is more importantthan actually fixing any problems thatmight arise.

Have a MeetingBetter yet, have lots of meetings.

Require that everyone connected to theproject attend, no matter how small hisrole and without regard to his other responsibilities. The meetings should require preparation and follow-up tasksthat take at least twice as long as themeetings themselves. To increase stress, itis far more important that everyoneknows everyone else’s responsibilities

impossible for the consultants to deliver asuccessful project, it may even be possibleto avoid paying them for their work. Thisis best accomplished when used in combination with the previous practices.

Don’t Bother to Validate theTest Environment

This is a very important step for increasing stress in a testing project. It’seven better to ignore any piddling detailslike test tool environmental requirements,such as the operating system, networkingprotocols, and any other necessary softwareand system configuration settings. Justmake blanket statements like:

“All of the machines have exactlyidentical configurations.”“The test system is an exact duplicateof the system we have here.”“Everything has been set up as you requested.”Later, when these statements are

proven inaccurate, the project stress willgreatly increase.

On one occasion, the third statementabove was combined with “Our peoplespent the whole weekend working on it.” (Of course, that was before an automatic system restore early the

rather than actually accomplishing any-thing productive. This process creates stressin two ways: it increases micromanagement,and it aids in choosing potential scapegoats.

Delegate, Delegate, DelegateYou probably think this actually

reduces stress, but proper delegation toincrease stress is almost an art form. Delegate critical tasks to people who arecompletely inappropriate for those tasks.For example, make your testing tool consultants, who will have little or noknowledge of your business processes,responsible for collecting the data necessary to execute the tests. Give themno direction as to how to collect the dataand no authority to obtain assistance fromthe subject matter experts, who do knowhow to collect and validate the data.

Later on, when the project is approaching its deadline, you can re-delegate this task to people who are capable of actually accomplishing it. Inthis way, you can increase stress on multiple fronts and still keep the projectfrom failing. If the project does fail, you canalways scapegoat any of the people towhom these tasks were originally delegated.

In the case mentioned, since it is

Scott Ames offers tips for a different sort of stress testing.

(Continued on page 38)

38 BETTER SOFTWARE FEBRUARY 2006 www.StickyMinds.com

A Critical Line of Defense(Continued from page 28)

quality of the software you produce. Over time, the process can be aug-

mented with secure code reviews. Thisgradual addition of security into theprocess can have a drastic impact onpost-release metrics, such as the number ofsecurity vulnerabilities found in the field.These results will reinforce the value of thoseprocess changes and encourage managementto support further process enhancements.

SummaryBuilding secure software requires

changing the way designers, developers,and testers think about how their applications will be used and abused. It’sabout creating a culture of concern thatbegins in requirements and design, ispropagated through development andtesting, and continues into deploymentand support. Achieving security requirescommitment from management and astructure that rewards developers andtesters for delivering software that’s secure as well as timely, robust, usable,and feature-rich. Our industry has a desperate need for more secure softwareand the processes, techniques, tools, andtechnologies to produce it. The need isacute, and many are moving toward security-aware development processes.Don’t let your applications be left behind!

Herbert H. Thompson, Ph.D., is chief security strategist at Security InnovationInc. He earned his Ph.D. in mathematicsfrom the Florida Institute of Technology.Herbert is co-author (with James Whittaker) of How to Break SoftwareSecurity: Effective Techniques for SecurityTesting (Addison-Wesley, 2003) and(with Scott Chase) The Software Vulner-ability Guide (Charles River, 2005). Heis also the author of numerous papers onsoftware security and testing.

StickyNotes

For more on the following topics, go towww.StickyMinds.com/bettersoftware

■ Microsoft’s “Trustworthy Computing Security Development Lifecycle”

■ Washington Post video ■ Resources for security information

next morning undid all of the systemconfiguration work that had just been completed.) Rebuilding an environmenttwo or three times while working on aproject deadline is a wonderful way toincrease stress. You will be amazed bythe amount of stress added to a projectby not having a properly configured envi-ronment and by making people scrambleto create inadequate work-arounds in ashort amount of time.

Better yet, don’t bother with a testsystem at all. Simply use the productionsystem as the test bed, and don’t backup anything. Remember: Backups arefor people who make mistakes.

Ignore the Stupid Questionsand Just Make Assumptions

What assumptions, made at all levels,keep the project from running smoothlyand, at the same time, greatly increasethe amount of stress on the project?With today’s complex and detailed soft-ware-testing projects, there are no stupid questions—only bad assumptions.

On second thought, just do it right

the first time. You may not even bearound to do it over, and who needs allthat extra stress?

Scott G. Ames is an American Societyfor Quality Certified Software QualityEngineer with nine years of experiencein software quality. He is a quality assurance manager for NEC SolutionsAmerica and previously was the qualityassurance and software testing managerfor AutoTester, Inc. Scott specializes in functional, stress, and load test automation. He can be reached atscott.ames@necsam.com or sames@att.net.

!Have the Last Word!

If you have a point to make or a side to take on issues and trends

that affect the industry, we want to hear from you.

We are looking for insightful, thought-provoking commentary

for possible use as a Last Word column.Please send your submission to editors@bettersoftware.com.

You will be notified if you are being considered for publication.

The Last Word(Continued from page 40)