Embed Size (px)
Citation preview
Making usability testing agile
How our approach to testing in the EdWeb CMS development project
might work for you too…
Neil AllisonUniversity Website ProgrammeUX Interest Group 17 Dec 2014
This presentation isn’t about Agile
• It’s about regular, rapid, inclusive usability testing with minimal overheads
• It just so happens that the agile process we’re using to develop the new University CMS (EdWeb) forced me to work this way
– It can work for you regardless
Usability testing
It’s easy, right?
What’s challenging
• Getting the go ahead to use your time on usability testing
• Getting colleagues to take on board what you uncover
• Getting fixes to problems implemented
(Why usability problems go unfixed: http://bit.ly/LvrGoq)
My challenges as UX Lead on EdWeb
• It’s not a formal role in Information Services
• Misconception that it’s the UX Lead’s job to ‘decide what’s usable’
• Team is too close to the product, with not enough exposure to CMS users
• Striking a balance between delivering new functionality and improving what we have
So what do we do?
1. Get the right people in a room
2. Watch a small number of short sessions with users doing something
3. Prioritise the issues they see
4. Collaboratively consolidate their priority lists
5. Agree actions for usability issues
6. Repeat every few weeks
Who are the right people?
• Everyone with a stake in the product
– No exceptions
http://bit.ly/1I1lZfQ
“Have you had your recommended dose
of research?”
What do they watch?
• Real CMS users doing real tasks
• Facilitated usability testing sessions
• Focus of testing agreed collaboratively in team
“Research shows that teams make better services when everyone on a project team observes users first hand.”
http://bit.ly/1I1rlYI
How many do they watch?
“The most striking truth of the curve is thatzero users give zero insights.”
• As many as you can fit into the time you have (so probably not very many)
http://bit.ly/1vQ7eHD
How do they prioritise?
“Running a usability test has been compared with taking a drink from a fire hydrant…”
• Rocket Surgery template:
1. Individual notes while observing
2. Distil to 3 issues after each participant
How do we consolidate?
“If you prioritise usability problems using 'gut feel' or intuition, you run the risk
of being exposed as a fraud…”
http://bit.ly/1I1mCWW
Then what?
• Usability issues prioritised, not solutions
• Agree actions based on:
– Is the solution “obvious”?
– Is there an easy development solution?
– Is there an alternative to development?
Recap: our process for EdWeb
In advance
• Agree test focus with team
• Write and pilot test script
• Recruit 3 participants to turn up on the day
On the day
• 3 sessions:– 30 minutes max
– 15 minutes between
• Observers prioritise their notes between sessions
• Final 30(ish) minutes spent prioritising top observations & agreeing actions
Top tips• Participants
– A pool of volunteers really helps recruitment– Krug – “Recruit loosely, grade on a curve”– Reminders the day before– Have an emergency stand-in prepared
• Do whatever it takes to get observers in the room– Start over lunch break– Supply refreshments– Bribery, favours, threats…
• Be well organised so you don’t waste anyone’s time– Test everything before hand
• Stick to the process and schedule (particularly in the final recap)– It’s easy to digress when you’ve all seen so much
• Encourage collective reflection on the session– Admitting usefulness is first step to getting observers to turn up next time– Have the next one scheduled ASAP
What’s good about our process
For the team
• Closer to our CMS users –immediate impact
• Shared insight & experience
• Ownership of the priority issues– What to fix immediately
– What we can live with that we thought was a problem
For me
• Process keeps set up and organisation of session to a minimum
• No report writing
• Moves the culture of the team on, emphasising CMS usability on the agenda
What we need to do better
• How we reduce usability problems occurring in the system in the first place
– Developer time at a premium
– Limited time for collaborative forward planning
• Getting more of the right people in the room
– For longer and more frequently
Everything you need
• Steve Krug’s Rocket Surgery resources:http://bit.ly/1I1muXo
• David Travis’ prioritisation flowcharthttp://bit.ly/1I1mCWW
Now let’s try it together…
• 3 participants– 5-10 minute break between each to review notes and
prioritise top 3 issues
• On your table consolidate your issues into a master list– Use flowchart to propose severity
• Once round the room to feed issues & priority back to the EdWeb development team
Thank you
Questions?
UX Manager
University Website Programme
Twitter: @usabilityed
