Embed Size (px)
DESCRIPTION
Changing product features, introducing a new on boarding process or a redesign of your entire website; making a large scale change and not fully understanding the impact of design decisions can be a real challenge for both start-ups and established businesses. This can lead to a negative impact with no clear way of understanding what specific change is to blame and then the challenge of finding the 1 or 2 alterations out the hundreds that have been implemented that is causing the issue. The key to avoiding the “redesign” problem is to work towards large scale change through a managed program of iterative experiments. Testing the the effects of each change through a fast, comprehensive measurement approach that aims to understand both the “what” through quantitative measurement of experiments; as well as the “why” through simultaneous user research testing and validation. Learning what works and taking ideas further through iteration, building on top of winning experiments to release large changes in a controlled way that minimises risk. The practical outcomes of this talk that I would like the audience to take away are: The benefits of iterative design and how it differs from incremental change and optimisation. How to structure experiments, test plans and create learning focused hypotheses with the goal of iterative change in mind. How designers can take redesigns and break them down into testable parts. How to pick the the right method to validate your assumptions. How to manage simultaneous quantitative and qualitative experiments. Examples of the methodology in use. (All examples with be evidenced with real world cases studies from experiments run on live websites) How iterative experiments can be used in large organisations and start-ups.
Citation preview
Redesign
@kmullr
WeExperiments
More challengers in a month than
competitors do in a year
Focused on learning and
fail-fast methodology
Redesign
Global Maximun
Simulated annealing
Local Maximun
It's
Slow, and really expensive
>1 Year£40 Million
1.5 Years£60 Million
2 Years£150 Million
Full Redesign
New Functionality
Page Redesign
1 Year +
4 Months +
2 Month +
Designs Fail,
A Lot
67% Fail66% Fail
90% Fail
90% Fail
Nearly Everything Fails
Difficult to predict
Results
Current DesignNew Design
Hidden Potential
C R
Fixed Header
New Gallery Layout
Alternative Images
Persuasive Messaging
Removed Video Icons
No Reviews Icons
Price Typography
Fixed Header
New Gallery Layout
Alternative Images
Persuasive Messaging
Removed Video Icons
No Reviews Icons
Price Typography
⨯
⨯
✓
✓
✓
-
⨯
Iterative product
Design
Incremental Iterative Discontinuous
Small Leaps
Run Hypothesis Experiment
ObservationsAccept What Works,Reject
The Rest
Iterate From
Learning
Experiments
Metrics
Research
Vision
Accept✓
Reject⨯
Inconclusive-
⨯ Hypothesis Rejected ✓ Hypothesis Accepted⨯ Hypothesis Rejected ✓ Hypothesis Accepted
Navigating through
uncertainty
Iterating towards
Redesign
Have a
Vision (but be flexible)
Prioritise
Learning
Prove / Disprove Assumptions
Continuous
& Discontinuous
Con
tinuo
usD
isco
ntin
uous
Live Site
2 Weeks 2 Weeks 2 Weeks 2 Weeks
Iteration 1 Iteration 2 Iteration 3 Iteration 4
Champion vs
Control
2 weeks
Iteration 1
⨯ ✓ ⨯✓ ✓
2 weeks
Iteration 2
⨯ ⨯ ✓
2 weeks
Iteration 3
✓ ⨯2 weeks
Iteration 4
⨯ ✓ ⨯ ✓⨯ ⨯ ✓ ⨯ ⨯ ⨯
C I
I R
Large scale change is possible through
iteration
Understanding the implications of change
Making measured leaps to better performance.
Kyle Muller @kmullrQuestions
