View
296
Download
2
Category
Tags:
Preview:
Citation preview
Software Testing inSmall Teams
Don R. Gilman, P.E.: Energy Systems Lab – TEESAdvanced Gaming Systems Inc
The Presenters
Don• Professionally programming/managing IT applications since 1984• Contributed to over one million units of various computer games• Licensed Professional Software Engineer (one of about 35 in the entire US)• Member of the Software Engineering Task Force for the Texas PE Board• Manager, Software Engineering Group – Energy Systems Laboratory – TEES
Presentation Overview
1. What is testing?2. What are the levels of testing?3. Types of testing:4. Who tests?5. Case Studies
1. Unit Testing2. eCalc Automated Testing3. A Computer Game
6. Summary7. Questions
What IS Testing?
• IEEE: The process of operating a system or component under specification conditions, observing and recording the results, and making an evaluation of some aspect of the system or component.
• Boehm: Verification and ValidationVerification – are we building it right?Validation – are we building the right
thing?
Testing is NOT
• Code inspections• Design reviews• Configuration management• Bug tracking
These are, along with testing, are part of Software Quality Assurance (SQA). Together these improve the quality of
the product
Types of Testing:
ScopeUnitModule Integration InterfaceCoverage
PurposeFunctionalStress
ExecutionManualAutomated
Phases/TriggersNightly BuildsDefect RemovalChange Order
CompletionAlphaBeta
Who Tests?
DevelopersUnit testing
QA TeamModule testing, automated testing
SoftwareAutomated testing
DesignersEngineers, Analysts
UsersUnpleasant but true
Who … some thoughts
• All New Members– To train– To get fresh test paths– In Industry – not a good model– In Gaming – a great way to recruit
• Release/QA Folks– Typically Engineers or Accountants– CompSci typically make lousy testers
Case Study: Unit TestingWhat is Unit Testing?
• Lowest Level of Testing• Programmer Safety Net• Instant Feedback• Documentation
Case Study: Unit Testing Good Unit Tests:
• Tests functionality not edge cases– GOOD: 100 points damage to demon ==
death– BAD: referencing null demon throws
exception
• Capture the meaning behind the code in tests
Case Study: Unit Testing xUnit Tool:
• Unit testing tools in many languages– .NET: NUnit– Java: junit– C++: CppUnit
NUnit
Case Study: e2Calc
Case Study: e2CalcWeb Server
User Interface
Emissions Service
BDI Service
Web Server
Database Server
Reporting Service
Que M
gmt
ProjectRecording
Service
XMLSQL
Server 2000
eGrid
DB Server
CalcEngine(s)
DOE-2 v2.1EFchart
PVFChartIMT
Calc EngineWrappers
Calc Servers
WebUsers
Optional Email w results
Browser
Emissions
Jobs
BDI User
EmissionsUser
Notification Email
BTIWeather
Lookups
Project and User Data
Report
Project and User XML
Case Study: e2Calc Quick QA Information
Use this for quickly checking the emails.
Scenario Annual kWh Annual ThermsSNGFAM Express Calc 12162 158.96SNGFAM Detailed Calc 12162 158.96
MULTIFAM Express Calc 13142 221MULTIFAM Detailed Calc 13142 221
Office Express Calc 966150 7245Office Detailed Calc 966150 7245Retail Express Calc 205002 1248Retail Detailed Calc 205002 1248
Municipal 966150 7245Utility Bill* -- --
Street Lights – Design Mode* 136875(post-retrofit) --Steet Lights – Utility Bill* -- --
Traffic Lights – Design Mode* 1621(post-retrofit) --Traffic – Utility Bill* -- --
Water Supply** N/A N/AWaste Water** N/A N/A
PVFCHART High End* 805 --PVFCHART Low End* 379 --
FCHART DHW* -- 3,553FCHART POOL* -- 2827
Wind** N/A N/A
All these agree on the energy savings with Andrews old tests!!
Do not match or not found in Andrews testsClose to Andrews testsMatches with previous tests by AndrewNo previous tests to validate against
Case Study:Games
Case Study: Games
Manual TestingCoderCore team
Public Alpha/Beta TestingSometimes process andfocus areas listed on
websitesTools
Command lines for auto loggingScreen dumpsSaved gamesDebug compilesGame vs game – with new log analysis tools
Case Study: Games
Case Study: Games
Case Study: Games
Summary
• Writing automated tests is one of the most difficult programming assignments
• Design to Test• Let the computer do the dirty work• Not everyone is wired to do this – only the
successful succeed.
Maintainable Code Correct Code
Reliable Production Accurate deliveries
It’s done means it is DONE!
Resources
Software Development Magazine (www.sdmagazine.com)
CrossTalk (www.stsc.hill.af.mil/crosstalk) Better Software (www.stickyminds.com)
The Rational Edge (www.rational.com) SEI (www.seir.sei.cmu.edu) TDD, an Introduction: (
www.agiledata.org/essays/tdd.html#Figure1StepsOfTDD)
Object Mentor – TDD: (www.objectmentor.com/writeUps/TestDrivenDevelopment)
SE Books we’ve read that matter
• How to Win Friends and Influence People, Carnegie• Software Requirements, Wiegers• Agile Project Management, Jim Highsmith• Planning Extreme Programming, Beck/Fowler• Agile Project Management w Scrum Schwaber• Mythical Man Month, Brooks• Test Driven Development: By Example, Kent Beck• MS Solutions Framework Papers• Decline and Fall of the American Programmer, Yourdan• The E-Myth Revisited, The E-Myth Manager, Gerber
Questions?
Recommended