Upload
charles-schultz
View
831
Download
0
Embed Size (px)
DESCRIPTION
Slides from today's Indie Games Summit at FIU
Citation preview
REAL GOODUNREAL TESTING
CHARLES P. SCHULTZAuthor “Game Testing All In One”
INDIE GAMES SUMMIT
TEST SUBJECTS
WHAT CAN I TEST?• UNREAL ENGINE VERSIONS, PORTS• MODS• LEVELS• WEAPONS, VEHICLES• PC, CONSOLE or MOBILE
INDIE GAMES SUMMIT
TEST OBJECTIVES
• FIND BUGS• PINPOINT CAUSES
- Environment- Repeatability- Frequency
• ESTABLISH WHAT WORKS• VERIFY FIXED BUGS• GIVE DEVS FEEDBACK
INDIE GAMES SUMMIT
TESTER’S WEAPONS
• PARAMETERS
• MODELS
• USE CASES
• METHODS
• TOOLS
• PLAYER STYLES- Achiever, Explorer, Killer, Socializer
INDIE GAMES SUMMIT
BOUNDARY VALUES
• 0• MINIMUM• MAXIMUM• MAXIMUM + 1• SPECIAL VALUES
- High Scores- February 29th
- Zone Boundaries: KOTOR example
INDIE GAMES SUMMIT
GAME PHASES
PRE-GAMEPRE-GAME
ACTIVATE
GAME START
Initialize system componentsLoad maps and game assetsEstablish network communicationGame controllers detectedetc.
IN-GAME POST-GAME
DEACTIVATE
INDIE GAMES SUMMIT
DEFECT TRIGGERS
• NORMAL BEHAVIOR
• CONFIGURATION
• STARTUP
• EXCEPTIONS
• STRESS
• RESTART
INDIE GAMES SUMMIT
DEFECT TRIGGERS
• NORMAL BEHAVIOR
- typically 80 to 100% of tests
- most costly to find
- relatively low cost to repair
- less effective as project progresses
- serious issues removed before ship
INDIE GAMES SUMMIT
DEFECT TRIGGERS
• CONFIGURATION• STARTUP• EXCEPTIONS• STRESS• RESTART
- up to 20% of tests- higher rate of defects found per test- requirements may be lacking
INDIE GAMES SUMMIT
TRIGGER GOALS
• NORMAL BEHAVIOR: 40%• CONFIGURATION: 20%• STARTUP : 10%• EXCEPTIONS: 20%• STRESS: 5%• RESTART: 5%
- grow non-Normal test cases- elicit requirements from developers
INDIE GAMES SUMMIT
RESTART
“You can fast swap weapons to reload the first one to make it fire faster, for example you can shoot the sniper, quickly swap to another weapon lets say hammer, swap back and shoot the sniper again as the reload time has been wiped”
BUG EXAMPLE
INDIE GAMES SUMMIT
COMBINATORIAL TABLES
• SELECT PARAMETERS TO TEST- GUN, AMMO, RANGE
• SELECT TEST VALUES FOREACH PARAMETER- AMMO: 0, 1, MAXIMUM
• PAIR EACH VALUE WITH ANOTHERPARAMETER’S VALUES
- AMMO=1 withGUN=ROCKET, LINK GUN, SNIPER
INDIE GAMES SUMMIT
COMBINATORIAL TABLES
• START WITH FIRST VALUE OF ONE PARAMETER
GUN AMMO RANGE
ROCKETROCKETROCKET
01
MAX
1MAX
MAX+1
INDIE GAMES SUMMIT
COMBINATORIAL TABLES
• FOR THE NEXT PARAMETER, CREATE NEW PAIRS
GUN AMMO RANGE
ROCKETROCKETROCKET
01
MAX
1MAX
MAX+1LINK GUNLINK GUNLINK GUN
01
MAX
MAXMAX+1
1
INDIE GAMES SUMMIT
COMBINATORIAL TABLES
• CONTINUE UNTIL ALL PAIRS ARE SATISFIED
GUN AMMO RANGE
ROCKETROCKETROCKET
01
MAX
1MAX
MAX+1LINK GUNLINK GUNLINK GUN
01
MAX
MAXMAX+1
1SNIPERSNIPERSNIPER
0
MAX1
MAX+11
MAX
INDIE GAMES SUMMIT
COMBINATORIALNOMICS
• For our 3x3 matrix, we covered all pair combinations with only 9 tests. A set of 13 parameters with 3 values would require 1.5 million tests for all combinations, but only requires 15 tests for pairs
• FOR LARGE OR COMPLEX SETS OF PARAMETERS, USE A TOOL
ALLPAIRShttp://www.satisfice.com/tools.shtml
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Graphic model of a function or use case•Take the player’s point of view• Use loops• Add reverse flows• Can double as a design tool
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Start with an entry point and initial “state”
IN
AMMOFULL
1. Start
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Do something to change to another state
IN
AMMOFULL
NOAMMO
2. Fire All Ammo
1. Start
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Can you go back to where you started?
IN
AMMOFULL
NOAMMO
2. Fire All Ammo
3.Reload
1. Start
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Can you repeat anything?
IN
AMMOFULL
NOAMMO
4. Fire Weapon
5.Reload
2. Fire All Ammo
3.Reload
1. Start
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Can you do something a different way?
IN
AMMOFULL
NOAMMO
4. Fire Weapon
5.Reload
2. Fire All Ammo
3.Reload
1. Start
6. Pick up fresh weapon
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Provide an Exit point
IN
AMMOFULL
NOAMMO
4. Fire Weapon
5.Reload
2. Fire All Ammo
3.Reload
1. Start
6. Pick up fresh weapon OUT
7. Exit
INDIE GAMES SUMMIT
TEST FLOW DIAGRAMS
• Create tests from various paths
IN
AMMOFULL
NOAMMO
4. Fire Weapon
5.Reload
2. Fire All Ammo
3.Reload
1. Start
6. Pick up fresh weapon OUT
7. Exit
1,2,4,71,2,3,2,71,2,6,2,71,2,3,5,2,71,5,2,3,5,2,4,6,2,7
INDIE GAMES SUMMIT
ANY QUESTIONS?
EXTRAS
ESCAPED PROBLEMS
CONFIGURATION
Unreal Tournament 3 flickers after playing for some time / UT3 Flickering screenSolution A : Make sure that you are using the latest drivers.Solution B : Turn down / Turn-ff Anisotropic Filtering in UT3′s menu
Unreal Tournament 3 models are the same for all players after playing the first map.Make sure that the ‘Allow custom characters’ option is checked in the options. This is because there are times that the config dont get saved or some of the items in the options menu gets randomly checked. Make sure that the UT3 config is changed after ticking the “allow custom characters” option by checking the options menu again after closing.
CONFIGURATION
Unreal Tournament 3 Configs dont get saved / UT3 settings and cd-key not saved / cd-key problem Make sure that the MY DOCUMENTS or and UT3 folder is not set as read-only
Cannot Play Unreal Tournament 3 Campaign Open the UTEngine.ini (My Documents\My Games\Unreal Tournament 3\UTGame\Config) and change PumpWindowMessagesWhenRenderThreadStalled=False to PumpWindowMessagesWhenRenderThreadStalled=True
ESCAPED PROBLEMS