Real World Lessons Using Lean UX (Workshop)

  • Published on

  • View

  • Download

Embed Size (px)


Half Day Workshop given 5/22/2013 at WebVisions Portland. In this workshop Bill will explore the mindset of LeanUX and how it relates to bring products to life in the midst of big organizations that don't normally think "Lean". He will look at how teams can create a strong partnership between product, design & engineering in a way that tears down the walls and instead focuses on three key principles: Shared understanding Deep collaboration Continuous customer feedback The workshop will take a look at how Bill has been able to apply Lean UX at PayPal a place that in recent years has been the total antithesis of the lean startup idea. With very specific examples, he will share lessons learned applying lean to the full product life cycle as well as how it relates to agile development. Finally, the workshop looks at the technology stack. In the last few years there has been an explosion of open source technology stacks that can support rapidly creating products, launching them to scale and rapidly iterating on them when live. While startups embrace these stacks from the get-go, large organizations struggle with how to embrace this change. This workshop will also look at the shift that has happened, what is driving this change, and how organizations can embrace this stack and how to marry Lean Tech with Lean UX.


  • real world lessonsmoving to leanux at paypal

    bill scott (@billwscott)sr. director, user interface engineering, paypal

    webvisions workshopmay 22, 2012

  • the schedule2:00-2:30 the problem (30)2:30-3:00 the pattern (30)3:00-3:15 q & a (15)3:15-3:30 break (15)3:30-3:45 the solution (15)3:45-5:15 the lessons learned (90)5:15-5:30 wrap up + q & a (15)

  • the purposehelp you identify the common problems encountered

    put before you a clear pattern of how it should work

    talk specific ways we solved our problems

    leave you with a number of lessons learned you can apply

  • the problema look at where paypal has been. can you relate?

  • team breakdown

    standard process creates distinct work phases

    boundaries are the hand-off points between roles

    product(business) design engineering

  • productproduct manager, business analyst

    owns the features. doesnt do design. drives by hypothesis.

    typically produces PRD

  • designUED, UX, ID, IA, VizDe, Content, designer

    not responsible for engineering the experience, but designing the experience.

    typically consumes PRD and produces design specs.

  • engineeringfront-end engineer, user interface engineer, web developer.

    not the designer, but the engineer that creates the experience.

    typically consumes UI specs and PRDs (for context).

  • typical product life cycle

    product(business) design

    engineering(agile team)

    PRD UX spec

    (wall) (wall)



    upon delivery, team disbands and forms into new teams

  • what was broken in design?late 2011/early 2012

  • deep silositeration planning done by developers without designers involved

    designers hand off specs without prior involvement of developers

    developer days (dev days) valued over design time

    frequent WFH days created low energy and less collaboration time

    hyper-segmentation of products

  • broad team distributiongeographic distribution created wedges, duplications and blocked collaboration

    lack of alignment with UED partners (not uncommon to have designers & engineers in same region to be working on different products)

  • lack of agile understandingwhile UED interfaced with agile teams they did not participate directly in agile planning, retrospectives, etc.

    agile machinery also did not account for experience design

  • no sense of ownershipUED staff in a pooled/studio model instead of a dedicated model

    once delivery happened the designers moved to another project

    often engineers did not know who the designer was for a product to ask questions to

    teams not empowered to make decisions as a gauntlet of other teams had to approve to go live

  • what was broken in product?late 2011/early 2012

  • no measurement/learn culturein several products there were no key performance indicators to measure & learn against

    since a/b testing was hard to do, there was no concept of an MVP (minimal viable product)

  • feature-itussince the organization rallied around projects instead of products, product tended to try to put as much on the train as possible

    without kpis you guess more and more (F.O.G.)

    without measurement you never get rid of features

  • too many silosproduct was divided over 9 different organizations!

    mobile was also a separate business, product and engineering silo

  • what was broken in engineering?late 2011/early 2012

  • too many silosjust like our counterparts, we were broken into many different organizations

    mobile was a separate organization

  • too hard to go live37 tickets just to get a bug fixed and pushed to live

    every organization was set up to say no to anything that might be innovative for fear of failure, risk, security issues, etc.

  • technology brokenno modern services architecture

    all solutions were built as silos

    ui logic and business logic intertwined

    technology platform assumed developers were not to be trusted

  • agile way too granularone product had 20+ agile streams. 12 of these were experience streams. each stream was responsible for one small part of the experience

    created nightmares of integration

    created a fractured experience for users

  • paypal circa 2011

    roll your own. disconnected delivery experience. culture of long shelf life. inward

    focus. risk averse.

  • the patternfollowing a build/measure/learn mindset

  • a good pattern continuous customer feedback (get out of the building - GOOB)

    customer metrics drive everything

    think it. build it. ship it. tweak it

    fail fast. learn fast.

    lots of experimentation... build/measure/learn

  • fourdifferent PS3 experiences

    launched on same day

    launching the ps3 experience16 different test cells

    2 different tech blogs were simultaneously reviewing different experiences

    focus was on build/measure/learn

  • the epiphanydesign for volatility

  • you have to engineer for volatilitychange is the norm

    the ui layer is the experimentation layer

    experimentation is not a one time event

    launching a product is giving birth to the product. the products life just begins.

    design for throwaway-ability

    majority of the experience code written is thrown away in a year

  • lean startupfounded on build/measure/learn

    get out of the building (GOOB)

    invalidate your risky assumptions

    go for the minimal viable product (MVP)

    fail fast, learn fast

    get to the pivot

  • lean uxdesigning products for build/measure/learn (lean startup)requires 3 rules to be followed at all times

    get to & maintain a shared understandingform deep collaboration across disciplineskeep continuous customer feedback flowing

  • engineering focused on learningengineering the build/measure/learn cycle shift the focus to minimal viable everything (MV*)

    follows the key rules of lean ux:shared understanding deep collaboration continuous customer feedback


    Applying Lean Startup Principles to

    Bring Design to Life

  • shared understandingthe more understanding the less documentation

    but this doesnt mean NO documentation

    you need whatever is needed to gain a shared understanding

  • deep collaborationstrong belief that ideas come from many different voices

    trust is essential

    all efforts never stray far from collaborative efforts

  • continuous customer feedbackthis is the lifeblood of the teamgets rid of politicsturns a team outside-in

  • getting to mvpmvp is a key tenant of lean startupapplied to engineering we should think: minimal viable everything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?

  • getting to mvpmvp is a key tenant of lean startupapplied to engineering we should think: minimal viable everything. mv*minimal viable processminimal viable team sizeminimal viable technologyminimal viable toolswhat are startups using?

    zucks rule: how w

    ill the startup dow

    n the

    street do this tom

    orrow? do it like t

    hat today.

  • healthy product life cycle

    Discover Customer Insights

    Define Customer Problems

    Define Solution


    Deliver & Test


  • the solutionrefactoring your way to the pattern

  • paypal vs netflix

  • new dna @paypaljan 2012fleshed out ui layer that could support rapid experimentation

    march 2012david Marcus becomes president of PayPal

  • in the midst of transformation

  • hermes projectre-inventing checkout with lean ux

  • hermes project

    whiteboardto code code to usability

    product/design/engineering teams usability/customers

    lean ux in action

  • free to iterate independent of agile

    user interface engineering - agile scrum team

    lean ux - lean team track

    engineering - agile scrum teamsprint 0

    usability usability usability usability usability

    release release release release


    lean ux can provide a brain for agile

  • the lessons learnedwhat you can apply to your teams

  • create a sandboxIMVU allows every engineer to put a test out to 1% of users

    at netflix we often created additional tests that designers or engineers independently wanted to try as a solution to a hypothesis


  • hotwire case study

    how do you protect the parent organization from the internal startup? create a sandbox

    Source: Lean Startup in the Hotwire Enterprise by Kristen Mirenda & Karl Shultz

  • hotwire case study: feedback

    hate it - can't even sort anymore

    I don't like it because you cannot filter the results or

    even sort them.. What were you thinking?

    absolutely blows...pure garbage. need to be able to sort asap. i'll come work for you and

    help you figure it out. wtf.

    Source: Lean Startup in the Hotwire Enterprise by Kristen Mirenda & Karl Shultz

  • hotwire case study: dataSource: Lean Startup in the Hotwire Enterprise by Kristen Mirenda & Karl Shultz

  • move to a living specbreak down barriers between prototyping and production

    use developers for prototyping as forcing function

    embrace RITE

    avoid tools/process that get away from collaboration


  • make the spec realthere are many, many prototyping tools available now

    you can create a living spec with these

    however the fidelity is never the same as real code

    recommend HTMLprototyping(more on this later)

  • but what about docs?watch out for predictive documentation

    watch out for documentation that replaces collaboration or is a band-aid for bad process

    good documentation will enhance collaboration, shared understanding and disseminate learnings

  • use a prototype stack

    whiteboardto code code to usability

    product/design team

    user interfaceengineers


    to enable learning

  • use a prototype stack

    whiteboardto code code to usability

    product/design team

    user interfaceengineers


    to enable learning


    JS librariesJS Templating(dustjs)less -> CSS images

  • enables sketch to codeforcing function

    it brings about a close collaboration between engineering and designit creates a bridge for shared understanding

    requires a lot of confidence and transparency

  • how lean & agile can play together

    fuller integration with services, unhappy paths & hardening error handling

    prototyping happens in rapid succession (lean ux track)

    services agile scrum teams

    usability usability usability usability usability

    release release release release


    lean ux can provide a brain for agile

    stories & code shared

    lean & agile teams should blend together

  • code prototypes vs tools?use the right tool at the right time

    as you get closer to agile

    axure,, POP and a host of other prototyping tools are amazing -- especially early in the learning cycle

    code prototypes important once you get close into the actual agile sprintsprovide high fidelity to the user testingfaster cycle from learning to live

  • suggestions for code prototypingbootstrap is one of the quickest to get going with

    we use it on our production stack as well

    jetstrap allows you to drag and drop a bootstrap page to get a quick start

    nodejs is really powerful for prototyping your full application (web, tablet, desktop)

  • prototyping toolssee:

    list of prototyping tools on my blog:

    few that we also use:Axure RPInVisionPOP

  • engineer for experimentation

    long shelf life to rapid experimentationfocus on learning not on deliverydesign for volatilityrefactor the tech stack with learning in mind


  • experiences must learnAll buildings are predictions. All predictions are wrong.

    There's no escape from this grim syllogism, but it can be softened.Stewart Brand

    Our software is always tearing itself apart (or should be)

    Recognize that different layers change at different velocities

  • velocity changes by layerrecognize that different parts of tech stack change at different velocities

    any building is actually a hierarchy of pieces, each of which inherently changes at different rates - Stewart Brand. How Buildings Learn.

    design for throwaway-ability (volatility)!

    use before you reuse (optimize for change)

    utilize packaging or paths to capture experiments

  • why start with experience?stay honest & pure by having experience be the driver

    (not what your boss thinks or what looks good on your resume or what the loudest one in the room thinks)

    rememberuse before you reuselet the experience drive the engineeringreuse is an engineering optimization. use is what users do. reuse is what engineers do.

  • experience vs components

  • experience vs components

  • build in rapid experimentationthink of the UI layer as the experimentation layer

    early rapid prototyping leads to learnings to get into the right ballpark

    follow with live A/B Testing. Lots of it.creates a healthy environment with constant customer feedback loopscontrast this with long shelf life culture

  • requirements for lean Stackindependent of the backend language

    flexible enough to run in either the server or in the client

    equally good at building web sites as it is building web applications

    pushable outside of the application stack (publish model)

    cleanly separated from the backend/app code (ideally by JSON)

    utilize what is common to developers

    quick & easy to build & tear down components

  • tangled up technologybig problem. technology and processes not geared to build/test/learn.

    refactor your way out of technical and experience debt

  • technical debtwe have to be on modern tech stack to continuously innovate

    we have to be on a continuously available stack

    continuously integrating

    continuously deploying

  • stack circa 2011/early 2012

    simple change could take minutes to see

    follows an enterprise application model. ui gets built into the appjava


    restricted capabilities*

    prototyping was hard

    ui bits could only

    live here

    * assumed client developers were low-skill* required server side java eng for simple client changes** java server pages. server-side java templating solution

    server side components**



  • a tale of two stacks (c++ & java)

    two non-standard stacks

    new stack tied to Java

    one word change could take 6 weeks to fix on c++ stack

    c++ java

    xml jsp

    proprietary ui

    proprietary ui

    old newish

    long release cycles

  • decision was to move to java

    migration was already in place to leave the c++/xml stack behind

    some odd choices preceded this: write everything in java (html, css & js!!)

    c++ java

    xml jsp

    proprietary ui

    proprietary uiXold newish

  • old stack not designed for learning

    this new stack was not conducive to prototyping

    followed an enterprise application model. ui gets built into the app

    ajax/components all done server-side (subclass java controller)



    proprietary ui

    prototyping was hard

    ui bits could only

    live here

  • separate the ui bitscode = JS

    (backbone)templates = JS

    {dust}style = CSS


    re-engineered the user interface stack so that the only artifacts are: javascript css images

    ditched the server-side mentality to creating UIs no more server-side only

    templates no more server-side

    components no more server-side

    managing the ui

  • use javascript templating

    templates get converted to javascriptHello {name}

    we use dust.js

    code = JS(backbone)

    templates = JS{dust}

    style = CSS(less)


    JavaScriptcompiles to...

    javascript executedto generate ui

  • ui bits now just natural web artifacts

    server-side language independent

    server/client agnostic

    CDN ready


    rapid to create

    code = JS(backbone)

    templates = JS{dust}

    style = CSS(less)


  • portable in all directions

    JS templating can be run in client browser or server on the production stack

    we can drag & drop the ui bits from prototyping stack to the production stack