User Centered Design Experience Report Visual Studio Team System Sam Guckenheimer

User Centered DesignUser Centered DesignExperience ReportExperience Report

Visual Studio Team SystemVisual Studio Team System

Sam GuckenheimerSam Guckenheimerhttp://lab.msdn.microsoft.com/vs2005/teamsystem/http://lab.msdn.microsoft.com/vs2005/teamsystem/

[email protected]@microsoft.com

http://lab.msdn.microsoft.com/vs2005/teamsystem/

mailto:[email protected]

AgendaAgenda

ContextContext Techniques & ExamplesTechniques & Examples Lessons LearnedLessons Learned Hands On ExerciseHands On Exercise

AgendaAgenda


InfrastructureInfrastructureArchitectArchitect

Project Project ManagerManager DeveloperDeveloper

TesterTester SolutionSolutionArchitectArchitect

Business Business StakeholderStakeholder

Context: The Business ProblemContext: The Business Problem

Starting Situation (Apr 03):Starting Situation (Apr 03):Bottom UpBottom Up Disjunct teams, chartered by roleDisjunct teams, chartered by role

Abstract view of usersAbstract view of users Committee leadershipCommittee leadership

Classical requirements managementClassical requirements management From “market research”From “market research” Long tables of featuresLong tables of features

Strong in-house Microsoft toolsStrong in-house Microsoft tools Limited Ux involvementLimited Ux involvement

Minimal value perceivedMinimal value perceived

Starting Situation (Apr 03):Starting Situation (Apr 03):Top DownTop Down Management mandateManagement mandate Bring innovation to stagnant marketBring innovation to stagnant market Three driversThree drivers

ProductivityProductivity IntegrationIntegration ExtensibilityExtensibility

AgendaAgenda


TechniquesTechniquesResearchResearch Contextual InterviewContextual Interview Kano AnalysisKano Analysis Dev LabDev Lab SDRSDR Community Community

DiscussionDiscussion Longitudinal Longitudinal

observationobservation

DesignDesign Persona/ScenarioPersona/Scenario StoryboardStoryboard Composite Team Composite Team

Persona/ScenarioPersona/Scenario Look for WOWsLook for WOWs Design standardsDesign standards Design walkthruDesign walkthruTestTest Cognitive walkthruCognitive walkthru Usability labUsability lab Scenario testScenario test

TimelineTimelineMgmt DirectiveMarket ResearchLeadership Team

Partial ScenarioPersonasStoryboard

Contextual Interviews

Storyboard #2X-team influence

Cognitive walkthrusUsability labs

Storyboard #3 - e2eDefines iteration exitHard deadline

AnnouncementStoryboard #3 demo’d live

Storyboard #4 completes pathsAgain, exit criteria

Dev LabsCommunity Tech PreviewsDiscussion Groups

SDR SDR SDRApr-03 Jan-04 Sep-04

Hiring Envisioning

ReorgDev iterations begin

Tech EdConferences

Beta

Research Example: Unit TestingResearch Example: Unit Testing

Visit to major bankVisit to major bank In conference, managers confirmed that In conference, managers confirmed that

unit testing was standard practiceunit testing was standard practice Then, Contextual InterviewThen, Contextual Interview

4 hours sitting with developer4 hours sitting with developer At end, asked about unit testingAt end, asked about unit testing Answer:Answer:

Haven’t you seen me? I’ve been doing it all along -- Haven’t you seen me? I’ve been doing it all along -- every time I press F5, that’s unit testing.every time I press F5, that’s unit testing.

Personas and Scenarios ExamplePersonas and Scenarios Example

PM Starts New Portfolio Project

PM Enumerates Requirements in Excel

PM Schedules Work in MS Project

PM Monitors Project Status

PM Reviews Project Status

PM Promotes For Deployment

CEO Signs Contract

Architect Updates Design

Architect Adds Tasks & Checks In

Dev Writes Code

Dev Writes & Runts Unit Tests

Dev Reviews Work

Dev Runs Code Analysis

Dev Writes Load Tests

Dev Checks In Work

Dev Diagnoses & Fixes

Dev Checks In Work

Tester Checks Build Status

Tester Runs Load Test

Tester Reports Bug

Jacqui AckermanJacqui AckermanProject ManagerProject Manager

Art BensonArt BensonArchitectArchitect

Martin GainesMartin GainesDeveloperDeveloper

Renee DavisRenee Davis TesterTester


Dev Checks In Work

PM Reviews Project StatusPM Reviews Project Status




CEO Signs Contract





Tester Reports Bug

PROJECT MANAGEMENT ARCHITECT DEVELOPER TEST




Dev Writes Code


Dev Reviews Work



Dev Checks In WorkTester Checks Build Status


Tester Reports Bug


PROJECT MANAGEMENT

JacquiJacqui

In MS Project, Sees Task In MS Project, Sees Task StatusStatus

In Phase View, Sees Exit In Phase View, Sees Exit Criteria StateCriteria State

Reviews Find / Fix Rates…Reviews Find / Fix Rates…

..and Daily Trends..and Daily Trends

……and Bugs Against Code and Bugs Against Code Churn and Testing ActivityChurn and Testing Activity

Files WIs For Dev And Files WIs For Dev And Tester To InvestigateTester To Investigate

PM reviews ProjectPM reviews Project


CEO Signs Contract





Tester Reports Bug

PROJECT MANAGEMENT ARCHITECT DEVELOPER TEST




Dev Writes Code


Dev Reviews Work



Dev Checks In WorkTester Checks Build Status


Tester Reports Bug


Dev Checks In Work




PROJECT MANAGEMENT

Dev Writes Code


Dev Reviews Work



Dev Checks In WorkDEVELOPER



Tester Reports Bug

TEST

PM Reviews Project StatusPM Promotes For Deployment

PROJECT MANAGEMENT

JacquiJacqui

Design Problem ExampleDesign Problem Example

ProblemProblemWork Item – Work Item – Build Build TrackingTracking

Build report Build report to showto showwork items work items resolved in resolved in buildbuild

Design Problem ExampleDesign Problem Example

Work Item – Build TrackingWork Item – Build Tracking Rarely practiced in the wildRarely practiced in the wild

Mandated at MicrosoftMandated at Microsoft Requires 17+ gestures, bi-di cut&pasteRequires 17+ gestures, bi-di cut&paste Very error-prone, unauditableVery error-prone, unauditable

Money - Microsoft Visual C#.NET [design] - Money.cs

Run Test

Test Explorer

Scope By

Browse By

Solution

All Tests by Area

Server Explorer

Test Explorer

Suggested Tests

| Suggested T… | Reason | Test Run...

Suggested Tests

Start Page BugID26005

Review Bug Details

View Source History

Select Workspace

Walk Through Trace

Deploy and Run Tests

Submit Changes

Title

Status

Build

Reported By

Assigned To

Description

Exchange Rate Mismatch

Active

2.1.SP1

Renee

See Attached Screenshots

Melvis

Properties Dynamic Help

Properties

testPlus1(Money) Start Page BugID26005

Money.cs

Money.Money Plus(Money)

BugID26005 - Microsoft Visual WITS

Check In

Class View

Class ViewSolution Explorer

Task List OutputTest Results

! | | | Description | File

Task List

BugID 26005

BugID 26008

Bug: Exchange Rates don’t match …

Bug: Conversion Rate error in calc ...

Test Results Task List | Output

Test Results

| Name | Result | Test Log | Test Output

View Results All Selected Tests By Type

Executed 7 of 7, 1 Error, 0 skipped

Create Report

!

testdollar(int)

testEquals(object)

testfranc(int)

Pass

Pass

Pass

UnitTest1

UnitTest2

UnitTest3

Pointer

Pointer

Pointer



Test Results


Executed 7 of 7, 0 Errors, 0 skipped

Create Report

testdollar(int)

testEquals(object)

testfranc(int)

testMoney(int,string)

testPlus(Money)

testToString()

testPlus1(Money)

Pass

Pass

Pass

Pass

Pass

Pass

Pass

UnitTest1

UnitTest2

UnitTest3

UnitTest4

UnitTest5

UnitTest6

UnitTest7

Pointer

Pointer

Pointer

Pointer

Pointer

Pointer

Pointer

testPlus1(Money)Money.cs Start Page BugID26005

Review Bug Details

View Source History

Select Workspace

Walk Through Trace


Submit Changes

S29 30 01 02 03 04 05 S06 07 08 09 10 11 12 S13 14 15 16 17 18 19 S20 21 22 23 24 25 26 S2

May 2003

April 2003

Show Details

Latest Build 3.0 Beta

BugID26005 found inBuild 2.1 SP1

Fixed and changesdelivered on branch.

View Source History


Test Results



Executed 7 of 7, 0 Errors, 0 skipped

Create Report

testdollar(int)

testEquals(object)

testfranc(int)

Pass

Pass

Pass

UnitTest1

UnitTest2

UnitTest3

Pointer

Pointer

Pointer

testPlus1(Money)Money.cs Start Page BugID26005

Review Bug Details

View Source History

Select Workspace

Walk Through Trace


Submit Changes Submit Changes

Files to Deliver:

Money.cs Add...

Remove

Source Files to Deliver:The following source and test files have changed and will be checked in...

Add...

Remove

Tests to Deliver:

<Add comment here>

Comments:

Save and SubmitSave Changes

testdollar(int)testEquals(object)testfranc(int)

BugID 26005: Exchange Rates don’t match …

BugID 26008: Conversion Rate error in calc ...

Work Items Closed by Changes

Task List OutputTest Results

! | | | Description | File

Task List

BugID 26005

BugID 26008

Bug: Exchange Rates don’t match …

Bug: Conversion Rate error in calc ...

Initial SolutionInitial Solution

Second AttemptSecond Attempt

Created Usability ProblemsCreated Usability Problems

Wizard too constrainingWizard too constraining Which work items to show?Which work items to show?

No filter: too many WIsNo filter: too many WIs Tried adding state “Ready for checkin” Tried adding state “Ready for checkin”

with automatic transition ruleswith automatic transition rules Required workflowRequired workflow Undiscoverable!Undiscoverable!

Needed to rethinkNeeded to rethink

Third SolutionThird SolutionTabbed DialogTabbed Dialog

Single buttonSingle buttonto acceptto acceptPolicy warnings Policy warnings catch errorscatch errors

Work items Work items surfaced by surfaced by queryqueryUser specifies User specifies check-in actioncheck-in action

Retested After Third AttemptRetested After Third Attempt

Making Labs EffectiveMaking Labs Effective

Thinking out loudThinking out loud User not being tested; product isUser not being tested; product is No wrong answersNo wrong answers

Sampling profileSampling profile Tasks defined in user’s domainTasks defined in user’s domain

Not in product vernacularNot in product vernacular Don’t lead the witnessDon’t lead the witness

Analyze results carefullyAnalyze results carefully

CounterexampleCounterexample

Work Item Type Definition LanguageWork Item Type Definition Language Complex XML SchemaComplex XML Schema State Machine, Validations, Rules, UI State Machine, Validations, Rules, UI

Assumed unusableAssumed unusable Tried tasks in Dev LabTried tasks in Dev Lab Users accomplished!Users accomplished!

AgendaAgenda


ValuesValues

UsefulnessUsefulness UsabilityUsability AttractivenessAttractiveness

Communicating these uniformly is Communicating these uniformly is hard!hard!

People Apply UCD DifferentlyPeople Apply UCD Differently See Kirton Adaptation-Innovation InventorySee Kirton Adaptation-Innovation Inventory

Applied by Scott Isaksen in various writingsApplied by Scott Isaksen in various writings

InnovationInnovation EngineeringEngineeringDesirabilityDesirability Do-abilityDo-ability

User CenteredUser Centered Product CenteredProduct Centered

Pays attention to wholePays attention to whole Pays attention to DetailPays attention to Detail

OpportunisticOpportunistic SystematicSystematic

DisruptiveDisruptive IncrementalIncremental

Proliferation of ideasProliferation of ideas Prefer sufficient optionsPrefer sufficient options

Values unique pathwaysValues unique pathways Values established normsValues established norms

Discontinuous leapsDiscontinuous leaps Continuous improvementContinuous improvement

ReflectionReflection ActionAction

Multidisciplinary TeamsMultidisciplinary Teams

Produce user experiences that are:Produce user experiences that are: More innovative due to diversity of ideasMore innovative due to diversity of ideas More cohesive due to synergy of More cohesive due to synergy of

perspectivesperspectives More robust due to checked assumptionsMore robust due to checked assumptions More achievable due to multi-role More achievable due to multi-role

involvementinvolvement More cost effective due to reduced churnMore cost effective due to reduced churn More satisfying to users due to continual More satisfying to users due to continual

user focususer focus

AgendaAgenda


User Centered DesignUser Centered DesignExperience ReportExperience Report

Visual Studio Team SystemVisual Studio Team System

Sam GuckenheimerSam Guckenheimerhttp://lab.msdn.microsoft.com/vs2005/teamsystem/http://lab.msdn.microsoft.com/vs2005/teamsystem/

[email protected]@microsoft.com

http://lab.msdn.microsoft.com/vs2005/teamsystem/

mailto:[email protected]

Documents

User Centered Design Experience Report Visual Studio Team System Sam Guckenheimer