“Capable of great and terrible things”

“leads the Decepticons”

“knows rights from wrong” “protect the things he cares

about”

Thomas Hulvershorn Operations Manager

Olly Hart QA Manager

…getting it right

Thomas Hulvershorn

Olly Hart

Outplay Entertainment

Established 2011 in Dundee Outplay Entertainment is a fast growing, venture-backed Developer and Publisher of social and mobile Games.

‘Classic Producer’

> Deadline >> Quality

> QA as excuse

> Quality afterthought

‘Classic QA’

> Non committal

> Over cautious

> All bugs are ‘A’ bugs

The Game has changed!

Sample Group A Sample Group B Control Group

Project Stakeholders Real Time Analytics

ETL Layer

Analysts Tracker

Store Data

Analytics Events

Game on Store

Download Deploy new Version

F2P Data Feedback

Driven Deployment

Game Life cycle

Player

Lau

nch

Amazon Cloud

Game Settings Outplay Server

Mo

dif

y Se

ttin

gs

Set

up

Exp

erim

ents

Game Settings

Producer

Feature Backlog (Jira)

Targets Designer

Game in Vault

Product Team

New Version

Sign off and Store Submission

Game Development

Client Update

#1 Priority is Player Satisfaction

“If your game sucks, stability won’t save you”

> Customer Support support

> Soft Launches

Impact on

Managing Quality

“The Fear” &

& Know your Coder

QA responsible for Customer Support > Makes sense – we know the projects

> Creates a direct line between problem and solution

Use Crash Logs and User Feedback

Test is committed to the Project

Producer is overall responsible

Live issues are Team issues

New Feature has terrible Bug

Classic Reaction

QA

Our take on this:

Why did the team miss this?

What will the team do better next time?

Production at Outplay

> Quality && Deadline

> QA right hand ‘man’

> Quality from the start

QA at Outplay

> provides insights

> is Embedded

> tests the right thing

> Three options

> External Recruitment

> Internal Appointment

> Internal Interviews

For many people this is a SCARY thought!

> Common Objections

> “We know who should get the job” -> Speed

> “No one is capable” -> Innovation

> “Team will be disgruntled” -> Acceptance

> “Losers work less after” -> Motivation

Speed Cost Innovation Acceptance Risk Motivation

~

~

~

~

External Recruitment

Internal Appointment

Internal Interviews

Test Management

People

Management

Team

Management + Personality + Games + Experience + Delegation + Stress Resistance

> Make it comparable!

> Same questions for all candidates

> Pair up with non QA Manager (e.g. Producer)

> Make it count!

> Practical Interview with Exercises

> Real life problems

> Test all areas with examples

Postkorbübung Priorities

Delegation

Pressure

Information

Communication

Results form the basis for the Interview

> Situation

> You are the QA Team lead. It is Tuesday 09:30 and you have a full day meeting starting at 10:00.

> You have not been in the office yesterday and your Inbox contains 25 mails. You got 25 minutes to read and sort out your mails.

There are 3 tester in your team

Adam Supertester He has just started to conduct a test on the Kindle Fire

Bernd Normaltester He has nothing to do at the moment and is looking for work when he comes in at 10:00.

Callum Problemtester He started a Smoke Test on the iPad3 handset the day before at lunchtime, but hasn't finished it. After that he is supposed to do a full test on the iPhone 4s.

Example for a mail

From

When

Subject

Content

Actions

The Inbox

Spam

Connections

Underperformance

Misleading

Different Time zones

Delegation

Example for a mail with actions

> Interview Structure

> Introduction to process // 0.25h

> “Postkorbübung” // 0.5h

> Review of results // 0.5h

> Role Play Staff Feedback // 0.5h

> Traditional Interview // 1.5h

>Team Management

>Testing Know How

>CV and experience

> Total time: ~3 hours per candidate

> Process has been regarded as fair

> We made the right choice

> No grumblings in the team

> Clear goals for the non successful candidates

> Signal to the team there is room and a fair chance for progression

Thanks for reading! Any Questions?

@DerHUL @ILOVETHISOZZ