Copyright © 2005 – Brad A.

Myers

End Users in End-User Software Engineering:

Where HCI Cross Cuts SE

First Workshop onEnd-User Software Engineering

(WEUSE)

End Users in End-User Software Engineering:

Where HCI Cross Cuts SE

First Workshop onEnd-User Software Engineering

(WEUSE)

Brad A. MyersHuman Computer Interaction

InstituteSchool of Computer ScienceCarnegie Mellon University

Brad A. MyersHuman Computer Interaction

InstituteSchool of Computer ScienceCarnegie Mellon University

Brad A. Myers, CMU 2May 21, 2005 2

Human-Computer InteractionHuman-Computer Interaction

HCI (at least at CMU) = CS Social Sciences Designers

CS contributes New tools and techniques

Social Sciences contribute Evaluation & processes

Designers contribute Better appearance and interaction

HCI (at least at CMU) = CS Social Sciences Designers

CS contributes New tools and techniques

Social Sciences contribute Evaluation & processes

Designers contribute Better appearance and interaction

Brad A. Myers, CMU 3May 21, 2005 3

HCI Especially Important for WEUSE

HCI Especially Important for WEUSE

Even more so than “regular” SE All programmers are still Human Still must Interact with Computers But end-user programmers are

especially “not like me”

Even more so than “regular” SE All programmers are still Human Still must Interact with Computers But end-user programmers are

especially “not like me”

Brad A. Myers, CMU 4May 21, 2005 4

WhyWhy

Why not “Just don’t” for EUP? SE attitude?

How to let the millions of computer users take full advantage of computers

Computers are malleable and customizable But only if you know how

Why not “Just don’t” for EUP? SE attitude?

How to let the millions of computer users take full advantage of computers

Computers are malleable and customizable But only if you know how

Brad A. Myers, CMU 5May 21, 2005 5

How Can HCI Methods Be Helpful?

How Can HCI Methods Be Helpful?

What is “known” by the HCI Community that might not be known by SE Community

Knowledge about what works and doesn’t Methods for approaching design and

evaluation Appropriate statistical methods Techniques for interaction & visualization Toolkits for implementation

(nothing on this topic here?)

What is “known” by the HCI Community that might not be known by SE Community

Knowledge about what works and doesn’t Methods for approaching design and

evaluation Appropriate statistical methods Techniques for interaction & visualization Toolkits for implementation

(nothing on this topic here?)

Brad A. Myers, CMU 6May 21, 2005 6

HCI Issues that Span Many Papers

HCI Issues that Span Many Papers

Many of these issues appear in many of this workshop’s papers

Topics I noticed: Need to Understand Users New Interaction Techniques and

Visualizations for the Systems Evaluating the Resulting Systems

Maps to the phases of HCI development

Many of these issues appear in many of this workshop’s papers

Topics I noticed: Need to Understand Users New Interaction Techniques and

Visualizations for the Systems Evaluating the Resulting Systems

Maps to the phases of HCI development

Brad A. Myers, CMU 7May 21, 2005 7

Need to Understand UsersNeed to Understand Users

Seen in Scaffidi, Prechelt, Pickard, Segal, Sutcliffe, Ko, …

Sub-topics Properties of users Motivation of users Task analysis

“Formative” studies To find out what to do (As opposed to “summative” or evaluation

studies done after the system is built or prototyped)

Seen in Scaffidi, Prechelt, Pickard, Segal, Sutcliffe, Ko, …

Sub-topics Properties of users Motivation of users Task analysis

“Formative” studies To find out what to do (As opposed to “summative” or evaluation

studies done after the system is built or prototyped)

Brad A. Myers, CMU 8May 21, 2005 8

Properties of UsersProperties of Users HCI principles:

“Know the user” “The user is not like me” “Don’t blame the users”

Lots of different kinds of users, and lots of kinds of “programming” Can’t generalize from what is appropriate for

programming for a bank to what is needed for home or scientific use

Professional programmers Professional End-User Developers “Casual” EUDs Judith Segal’s review of scientists and financial

consultants Also, users of the abaXX system

HCI principles: “Know the user” “The user is not like me” “Don’t blame the users”

Lots of different kinds of users, and lots of kinds of “programming” Can’t generalize from what is appropriate for

programming for a bank to what is needed for home or scientific use

Professional programmers Professional End-User Developers “Casual” EUDs Judith Segal’s review of scientists and financial

consultants Also, users of the abaXX system

Brad A. Myers, CMU 9May 21, 2005 9

Properties of Users by SurveyProperties of Users by Survey

Scaffidi, Shaw and Myers Don’t use the 55 million number anymore!

Analyzed Boehm’s paper, BLS and other data Real numbers (2001): 72 million users, 45 million

spreadsheet users, 11 million “do programming,”3 million professional programmers

What do people use in their EUP environments

Try to generalize to what they know in terms of abstractions Variables, functions, data structures

Scaffidi, Shaw and Myers Don’t use the 55 million number anymore!

Analyzed Boehm’s paper, BLS and other data Real numbers (2001): 72 million users, 45 million

spreadsheet users, 11 million “do programming,”3 million professional programmers

What do people use in their EUP environments

Try to generalize to what they know in terms of abstractions Variables, functions, data structures

Brad A. Myers, CMU 10May 21, 2005 10

Motivation of UsersMotivation of Users Blackwell’s model of “Attention Investment”

A. F. Blackwell. “First steps in programming: a rationale for attention investment models,” IEEE HCC-VL’02, Arlington, VA. pp.2-10

Costs: learning time & actual programming time Time away from the “real work”

Risks: won’t work & be a waste of time Benefits: future savings if task done again

Also utility Is it really useful? Segal, Sutcliff sometimes!

Attention Investment [Joseph Ruthruff & Margaret Burnett]

“If we built it, will they come?” How make benefits more real and more apparent? How make risks and costs less?

Blackwell’s model of “Attention Investment” A. F. Blackwell. “First steps in programming: a rationale for

attention investment models,” IEEE HCC-VL’02, Arlington, VA. pp.2-10

Costs: learning time & actual programming time Time away from the “real work”

Risks: won’t work & be a waste of time Benefits: future savings if task done again

Also utility Is it really useful? Segal, Sutcliff sometimes!

Attention Investment [Joseph Ruthruff & Margaret Burnett]

“If we built it, will they come?” How make benefits more real and more apparent? How make risks and costs less?

Brad A. Myers, CMU 11May 21, 2005 11

Classic “Gentle Slope Systems”Classic “Gentle Slope Systems”

Brad A. Myers, David Canfield Smith and Bruce Horn. “Report of the `End-User Programming’ Working Group,” Languages for Developing User Interfaces, Boston, MA, Jones and Bartlett. 1992. pp. 343-366.

Brad A. Myers, David Canfield Smith and Bruce Horn. “Report of the `End-User Programming’ Working Group,” Languages for Developing User Interfaces, Boston, MA, Jones and Bartlett. 1992. pp. 343-366.

Brad A. Myers, CMU 12May 21, 2005 12

Task AnalysisTask Analysis Find out what users are really doing or need

to do Might ask users (surveys, questionnaires) or

introspect, but: Users are not like us Users tell what they think they want, not what

they need Surveys, etc. get at marketing data, not usage

Self reports not valid Dramatic bad events are the most salient People over-estimate time for mentally challenging

activities Under-estimate time on routine tasks

So need to see what people actually do

Find out what users are really doing or need to do

Might ask users (surveys, questionnaires) or introspect, but: Users are not like us Users tell what they think they want, not what

they need Surveys, etc. get at marketing data, not usage

Self reports not valid Dramatic bad events are the most salient People over-estimate time for mentally challenging

activities Under-estimate time on routine tasks

So need to see what people actually do

Brad A. Myers, CMU 13May 21, 2005 13

Evidence from abaXXEvidence from abaXX

Lutz Prechelt and Daniel Hutzel Found that many of the EUP facilities

were not appropriate Not useful, not allowed

Eventually, useful for understanding rather than creating

Lutz Prechelt and Daniel Hutzel Found that many of the EUP facilities

were not appropriate Not useful, not allowed

Eventually, useful for understanding rather than creating

Brad A. Myers, CMU 14May 21, 2005 14

“Contextual Inquiry”“Contextual Inquiry” In HCI at CMU, we teach the “Contextual

Inquiry” technique Most understandable, teachable and usable

method for task analysis Find user’s real tasks But much more: Context!!! Cultural influences (corporate, social, etc.)

Might have identified some of abaXX’s issues before design

Reference: H. Beyer and K. Holtzblatt. Contextual Design: Defining Custom-Centered Systems. San Francisco, CA, Morgan Kaufmann Publishers, Inc. 1998.

http://www.incent.com/cd/cdhow.html

In HCI at CMU, we teach the “Contextual Inquiry” technique Most understandable, teachable and usable

method for task analysis Find user’s real tasks But much more: Context!!! Cultural influences (corporate, social, etc.)

Might have identified some of abaXX’s issues before design

Reference: H. Beyer and K. Holtzblatt. Contextual Design: Defining Custom-Centered Systems. San Francisco, CA, Morgan Kaufmann Publishers, Inc. 1998.

http://www.incent.com/cd/cdhow.html

Brad A. Myers, CMU 15May 21, 2005 15

How Contextual Inquiry WorksHow Contextual Inquiry Works

Interpretive field research method Depends on conversations with users in the context

of their work Recommends “direct observation” when possible When not possible

Cued recall of past experience, or Recreation of related experience

Used to define requirements, plans and designs. Develop diagrams to communicate results and

show breakdowns Data flow Sequence of steps performed Cultural influences Physical environment of work Artifacts used and created

Interpretive field research method Depends on conversations with users in the context

of their work Recommends “direct observation” when possible When not possible

Cued recall of past experience, or Recreation of related experience

Used to define requirements, plans and designs. Develop diagrams to communicate results and

show breakdowns Data flow Sequence of steps performed Cultural influences Physical environment of work Artifacts used and created

Brad A. Myers, CMU 16May 21, 2005 16

Why Context?Why Context?

Design complete work process Fits into “fabric” of entire operations Not just “point solutions” to specific

problems Integration!

Consistency, effectiveness, efficiency, coherent

Design from data Not just opinions, negotiation Not just a list of features

Design complete work process Fits into “fabric” of entire operations Not just “point solutions” to specific

problems Integration!

Consistency, effectiveness, efficiency, coherent

Design from data Not just opinions, negotiation Not just a list of features

Brad A. Myers, CMU 17May 21, 2005 17

ExamplesExamples Andy Ko and our uses of CI

Revealed need for why not questions and support for false assumptions during debugging WhyLine

Revealed patterns of activity during maintenance this year’s ICSE paper

Revealed how people edit programs future work on structure editors

Earlier work: John Pane’s PhD on “Hands” How people think about algorithms, etc. E.g., “Men and women raise your hands”;

“Move all the chairs into a circle.”

Andy Ko and our uses of CI Revealed need for why not questions and

support for false assumptions during debugging WhyLine

Revealed patterns of activity during maintenance this year’s ICSE paper

Revealed how people edit programs future work on structure editors

Earlier work: John Pane’s PhD on “Hands” How people think about algorithms, etc. E.g., “Men and women raise your hands”;

“Move all the chairs into a circle.”

Brad A. Myers, CMU 18May 21, 2005 18

New Interaction Techniques and Visualizations

New Interaction Techniques and Visualizations

How can users control the system? How convey information to users? Many innovations in EUP areas:

Earlier work: PBD techniques; Rich McDaniel’s “Do Something” button, “hint highlighting”, “deck of cards”

Andy Ko’s why button and “WhyLine” visualization New interaction techniques planned for text editing of code

WYSIWYT & Markus Clermont: highlighting, “x?”, etc.

HCI can help with design so attractive, effective, professional-looking Color choice, icon design, layout, etc.

How can users control the system? How convey information to users? Many innovations in EUP areas:

Earlier work: PBD techniques; Rich McDaniel’s “Do Something” button, “hint highlighting”, “deck of cards”

Andy Ko’s why button and “WhyLine” visualization New interaction techniques planned for text editing of code

WYSIWYT & Markus Clermont: highlighting, “x?”, etc.

HCI can help with design so attractive, effective, professional-looking Color choice, icon design, layout, etc.

Brad A. Myers, CMU 19May 21, 2005 19

Design by Programmer vs. Designer

Design by Programmer vs. Designer

Brad A. Myers, CMU 20May 21, 2005 20

Evaluating the Resulting SystemEvaluating the Resulting System

Many HCI methods for evaluation Heuristic Evaluation Method Usability Testing

Lab or Field studies Cognitive Walkthrough Analytic methods Keystroke model, GOMS model Questionnaires, Interviews, Focus Groups,

Logging, Feedback Effectiveness in real deployment

Statistics methods for proper evaluation Will be covered more in Gregg’s section

Many HCI methods for evaluation Heuristic Evaluation Method Usability Testing

Lab or Field studies Cognitive Walkthrough Analytic methods Keystroke model, GOMS model Questionnaires, Interviews, Focus Groups,

Logging, Feedback Effectiveness in real deployment

Statistics methods for proper evaluation Will be covered more in Gregg’s section

Brad A. Myers, CMU 21May 21, 2005 21

Many Insights Revealed in Studies

Many Insights Revealed in Studies

Studies not only to show improvement percentages for the papers!

Reveals interesting new research problems and designs

E.g., Ruthruff: misconceptions: end-user programmers

make errors when testing (up to 25%), and affects 60% of success rates

Ko: WhyLine: people didn’t use reverse execution, instead used events; originally had why “did/didn’t” and “is/isn’t” but confusing

Might not have thought about these issues w/o information from users

Studies not only to show improvement percentages for the papers!

Reveals interesting new research problems and designs

E.g., Ruthruff: misconceptions: end-user programmers

make errors when testing (up to 25%), and affects 60% of success rates

Ko: WhyLine: people didn’t use reverse execution, instead used events; originally had why “did/didn’t” and “is/isn’t” but confusing

Might not have thought about these issues w/o information from users

Brad A. Myers, CMU 22May 21, 2005 22

HCI + SEHCI + SE

Real opportunities for innovation Renewed interest from ICSE, industry

and community in general

Award winning ideas possible!

Real opportunities for innovation Renewed interest from ICSE, industry

and community in general

Award winning ideas possible!