- 1 - End User Software Engineering CHI2008 Special Interest Group Meeting Tue, April 8, 2008, 9:00 AM - 10:30 AM Brad Myers, Margaret Burnett, Mary Beth

- 1 -

End User Software Engineering

CHI’2008 Special Interest Group MeetingTue, April 8, 2008, 9:00 AM - 10:30 AM

End User Software Engineering

CHI’2008 Special Interest Group MeetingTue, April 8, 2008, 9:00 AM - 10:30 AM

Brad Myers, Margaret Burnett, Mary Beth Rosson, Andrew Ko, Alan

Blackwell

End-User Software Engineering SIG at CHI’2008

- 2 -

Definitions

• (Note: may be controversial—defer discussion until later)

• Program– ‘‘A set of statements that can be submitted as a unit to some computer system and used to direct the behavior of that system’’

– Oxford Dictionary of Computing

• Programming– ‘‘The process of transforming a mental plan of desired actions for a computer into a representation that can be understood by the computer’’ – Jean-Michel Hoc and Anh Nguyen-Xuan


- 3 -

Definitions, cont.

• Professional Programmer

– Someone whose primary job function is to write or maintain software

– Typically have significant training in programming (e.g., BS in CS)

• Novice Programmer

– Someone who is learning to be a professional programmer


- 4 -

Definitions, cont.

• End-User Programmer (EUP)

– People who write programs, but not as their primary job function

– Instead, they must write programs in support of achieving their main goal, which is something else

– Covers a wide range of programming expertise

• Business executives and secretaries

• Physicists


- 5 -

Examples of EUP• Accounting (spreadsheets)

• Analysts using MatLab

• Creating a web page

• Recording Macros in Word

• Automating office tasks

• Business software (SAP programming)

• “Programming” VCRs, Microwaves

• Scientific research

• Authoring educational software

• Creating email filters

• Musicians configuring synthesizers

• Entertainment (e.g., behaviors in The Sims)

• Web 2.0: Mashups, end-user created content


- 6 -

Other Names

• Also called “End User Development” (EUD)

• Some “Domain-Specific Languages” (DSL)

– Often created for end-user programmers

• Visual (Graphical) Programs

– Sometimes created for EUP

• “Scripting” languages, “Macros”

• Rapid Application Development (RAD)


- 7 -

Definitions, cont.

• End-User Software Engineering (EUSE)

– End-User Programming with the addition of systematic and disciplined activities that address quality issues (“attributes”)

• E.g., analyzing code, understanding unfamiliar code, testing code, checking code against a model, sharing code with coworkers, maintaining code, and deploying code, to name a few


- 8 -

0

20,000,000

40,000,000

60,000,000

80,000,000

100,000,000

Users Spreadsheetsand DBs

Self-DescribedProgrammers

ProfessionalProgrammers

End User Programmers• A very large group

• In 2012: — Scaffidi, Shaw and Myers 2005

– 90 million computer users at work in US

– 55 million will use spreadsheets or databases at work (and therefore may potentially program)

– 13 million will describe themselves as programmers

– 3 million professional programmers

• We should make better tools for all of these people!


- 9 -

Evidence of Need for EUSE

• Why need Software Engineering for EUP?• Lots of errors attributed to End-User Programming of spreadsheets:– Columbia Housing Authority admitted to overpaying by $118,387 due to a spreadsheet data-entry error (February 22, 2006)

– New York Times, Oct 30th, 2003 - $1.2 Billion Spreadsheet Error at Fannie Mae

– TransAlta Corp. took $24 million charge to earnings due to cut-and-paste error in an Excel spreadsheet (June 3rd, 2003)

– Auditor, major accounting firm:

“...in 6 years work, checking literally hundreds of business-critical models, ... my team have never failed to find errors.”

– …. (many more!)

– See http://eusesconsortium.org/euperrors/


- 10 -

Consequences, 2• Also, errors in:

– Web pages

– Email filtering rules

– … (what else? Need more examples!)

• From the WEUSE II workshop at CHI’2006:– Clinical customization package used by medical personnel reports the need for better reuse and debugging support

– SysAdmins need better testability of database and other sorts of scripts

– Issues with reuse of MATLAB applications


- 11 -

So What is Happening?

• Growing group of researchers in EUSE

• The EUSES Consortium– End Users Shaping Effective Software

– http://eusesconsortium.org

– 7 sites, 13 researchers + ~15 students

• EUDNet– http://giove.isti.cnr.it/eud-net.htm

– European Commission

– 2002-3

– 16 partners


- 12 -

Addressing the EUSE Problem• A multi-(sub)disciplinary problem needing:

– Software engineering research

– Programming language research

– Education research

– End-user programming research

– HCI research of all types

• This is a big job, needs a whole community!

– Hence, this SIG, to encourage interested people to work together with us and with each other

– Also, connect researchers and EUP vendors


- 13 -

Many Sessions at CHI’08

• This kind of work is represented in many sessions

– Tues, 9:00-10:30am (Now ): “Aesthetics, Awareness, and Sketching” paper session

– Tues, 11:30-1pm: “Beyond End-User Programming” paper session

– Tues, 4:30-6pm: “Invited Session: The Next Challenge: from Easy-to-Use to Easy-to-Develop. Are You Ready?”

– Wed, 4:30-6pm: “Activity-Based Prototyping and Software” paper session

– Thurs, 2:30-4pm: “End-Users Sharing and Tailoring Software” papers session


- 14 -

Previous Meetings on This Topic• Dagstuhl Conference on End-User Software Engineering (WEUSE-III) on Feb, 2007

• 2nd Workshop on End-User Software Engineering (WEUSE-II) at CHI 2006

• 1st Workshop on End-User Software Engineering (WEUSE-I) at ICSE 2005

• CHI 2007, 2005, 2004 SIGs on End Users Creating Effective Software

• Many proceedings and notes available off of EUSES web page: http://eusesconsortium.org


- 15 -

Future Meetings• Fourth Workshop on End-User Software Engineering (WEUSE

IV) at ICSE'08 on May 12, 2008 in Leipzig, Germany– http://eusesconsortium.org/weuse/

• Cooperative and Human Aspects of Software Engineering (CHASE) at ICSE’2008 on May 13, 2008 in Leipzig, Germany – http://softwareresearch.ca/seg/CHASE/

• ICSE’2008, May 10-18, 2008 in Leipzig, Germany

• PPIG’2008, Sept 10-12, 2008, Lancaster, UK– http://www.comp.lancs.ac.uk/ppig08

• 2008 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2008), Sept. 16-20, 2008, Herrsching am Ammersee, Germany

• Intelligent User Interfaces (IUI’2009), January ??, 2009

• Second Symposium on EUD, March 2-4, 2009, Siegen, Germany– http://www.eud2009.org

• CHI’2009


- 16 -

Ways to Get Involved

• Attend upcoming events.

• Get newsletter on end-user programming/software engineering:

– (Low-volume). Sign-up sheet going around.

• Tell us today what you’re working on.

• Help shape future events (starting with today’s discussion).


- 17 -

Rest of This SIG

• Alan Blackwell: short introduction to EUDNet

• Andy Ko: overview of dimensions of organizing EUP and EUSE systems and concerns

• What companies are working on that is relevant to EUP and EUSE

• Break up into groups by Software Engineering Issue

• Reconvene and report (15 min – 3 min each)


- 18 -

EUDNet

• Research contrasts between (EU)EUD and (US)EUP/EUSE

• European projects are more likely to:

– target users in specific technical domains

– study organisational issues

– address adults, not children / students.


- 19 -

Breakout Groups• Groups:

– Design & Requirements (Alan Blackwell)

– "Create" (Brad Myers)

– Reuse (Mary Beth Rosson)

– Testing & Verification (Margaret Burnett)

– Debugging (Andy Ko)

• Focus on the S/W Eng issue and how it applies to each of the different domains

– Domain-driven compare/contrast

– What has been done relevant to that domain for this S/W Eng issue

– Which domains are most challenging and easiest?

– How similar and different

• Take notes in PowerPoint for 3 minute summary


- 20 -

Industry Attendees at CHI’2008 SIG• Paula Dimaio – from media, consultant: integrated

methodology S/W+UE for practitioners. Ontology engineering

• Marcus Stolze – IBM TJ Watson research lab. Business analysts, enterprise applications, dynamic access, business

• Jack Zaintz – SOAR Technology, modeling and simulation for military, let end users do their own modeling, training operators – adaptive scripting of models

• John Richards – IBM TJ Watson – support business analysts and architects when doing sense-making. Also, petaflop with 13-fold increase in programmer-productivity

• Claude Knaus – indep. developer: interaction design to create s/w dev. Tools

• Laura Beckwith – Microsoft Visual Studio team: middle between professional and EUP, e.g., consultants who want to make money off of programming

• Steve Clarke – also with MS VS

• Tessa Lau & Allen Cypher – IBM Almaden – Co-Scripter: EUP for web

• Steve Pemberton – Designer of XForms


- 21 -

Domains• Web Applications

• Education, pen-based application: primary school (6-10years old)

• Coaches of sports teams

• Enterprise users (all employees)

• Ambient user interfaces – in home, consumer electronics

• Visualization for air traffic controllers (ATC), operators monitoring computer systems, UAV path has to be designed – has a language w/conditionals but want a UI

• Modeling & simulation for military exercises for military operations, NASA telemetry engineers

• Professional programmers who act like EUP

• Analysts who design and build enterprise applications, how can they provide more of the implementation – let them design their own UIs. As part of WebSphere & Rational

• Personal Information Management


- 22 -

“Create” group results• Jack: Lots of different tools that have to be used

together– Large Simulation System (JSAP) – config. files, UIs for

scripting movement, intelligent entity control IDE (SoarTAC) but not for EU.

– Simplified state machines – IDEs but not very good, FSM editor but pretends to not be a graph

– Better tools for Intelligent Entity Control – textual IDEs w/some graphics for FSM, graphic forms

• Flash, Frontpage, – Make the web sites conform to accessibility guidelines

• Educators– How to learn Chinese, Geometry, Mathematics

– Low-level platform “UI Widgets”: Text, Ink, Math formulas, Geometry, Chemistry

– Pen-based design tool for designing functions they like

– Interactive, drag-and-drop tool, set properties

• Sport fields– Different coaches have similar needs, but different formats

– Different kinds of spreadsheets


- 23 -

“Create” group results• Olivier Liechti – simple for interactive remote control for consumer electronics using Visual Programming, on handheld display

• CoScripter – IDE for scripting on the web, mostly by PBE

• Enterprise Applications, start in an unorganized fashion – how turn spreadsheet into a real enterprise applications?– Describe their process with an XML language

– Structured applications around the data objects

– State machines, specifying operations to change states, describe choices, process and focusing on data and states of the data. Data flow.

– State of data objects, focus on access control questions

– Additional aspect: input validation – separate framework needed


“Create” group results• Max: PIM: scripting language (in JavaScript)

– Specifying conditions under which things are triggered by example – go into past and select a moment when would want something to happen

– Select things that are relevant in the real user interface of Web-based PIM

– Triggered on what documents are open, what music listening to, Twitter status (location)

– Actions: call any web-based service, desktop services (turn on music, etc.)

• Entry-level JavaScript programmer


- 25 -

Categories of Creating• Drag-and-drop

• Visual programming

• PBE

– Textual model that can read but not edit

– Hidden Markov Models

– Creates conventional script

• Textual

– Copy-and-paste: different style of programming

– IDE:

• Code completion, automatic error finding and fix suggestions makes them much better for EUP

– Exploration tool: PIG tool from Berkeley simulates as you go

• Form-filling – mail filters, Jack’s IFOR (intelligent forces)


- 26 -

Testing and Verification Group• Haven’t seen different work styles.

• Steven studied concept of a tester. How can I break? Study of people whose job responsibitilities were testing.

– Some were “testers”. Some were “business analysts”, “programmers”. All did the same, namely black-box.

• Piero: 1. Medical, 2. Mech engr, 3. Software people. They have different views on testing, have different procedures for testing.

– Eg, people from physics and medicine -- they have experience testing physical machines but not software. They do very “black box” approaches. (Often, their software is part of the machines they buy.)

• Steven has seen a lot of black-box.

• Sometimes end users don’t think about testing at all, at least in spreadsheets.

– Maybe because of the immediate visual feedback -- they may in some sensre regard it as “already tested”.

• Oracle problem -- making errors in judgments of failed/successful tests.

– Overconfidence is a problem with end-user programmers.

– With images it can be harder to evaluate the answer.

• Under the hood: software engr. talks about the underlying logic.

• Code inspection.

– Gender differences (see 11:30 session).

• Assertions.

• Scenario-based approach: “When a user does this, then that, the following happens”. A storyboard.

– A semi-quote: “Developers take the happy path. I take all the other paths.”

– We wonder: is this true mostly in imperative paradigms, or would it also apply to more declarative ones?

• Interactive systems have important testing issues in their own right,

– because end-user “HCI” work to customize these could lead to terrible errors.


Testing and Verification Group Results• Haven’t seen different work styles.

• Steven studied concept of a tester. How can I break? Study of people whose job responsibitilities were testing.– Some were “testers”. Some were “business analysts”,

“programmers”. All did the same, namely black-box.

• Piero: 1. Medical, 2. Mech engr, 3. Software people. They have different views on testing, have different procedures for testing.– Eg, people from physics and medicine -- they have

experience testing physical machines but not software. They do very “black box” approaches. (Often, their software is part of the machines they buy.)

• Steven has seen a lot of black-box.

• Sometimes end users don’t think about testing at all, at least in spreadsheets. – Maybe because of the immediate visual feedback -- they

may in some sense regard it as “already tested”.


Testing and Verification Group Results• Oracle problem -- making errors in judgments of

failed/successful tests.– Overconfidence is a problem with end-user programmers.

– With images it can be harder to evaluate the answer.

• Under the hood: software engr. talks about the underlying logic.

• Code inspection.– Gender differences (see 11:30 session).

• Assertions.

• Scenario-based approach: “When a user does this, then that, the following happens”. A storyboard. – A semi-quote: “Developers take the happy path. I take all the

other paths.”

– We wonder: is this true mostly in imperative paradigms, or would it also apply to more declarative ones?

• Interactive systems have important testing issues in their own right, – because end-user “HCI” work to customize these could lead to

terrible errors.


Design & Requirements Group Results

• Who “interprets” the requirement?

• Normally a professional translates from messy/concrete to the computer abstraction.

• Supply side: But without professional? Can a tool provide the trust

• Demand side: Two kinds of attitude to abstraction:

– Some expert communities have good domain abstractions

– Some communities share concrete examples, don’t explicitly discuss abstraction

• Alternative: Focus on community (content, analogy, sharing)


Reuse Group Results

• Gerhard Fischer (U Colorado), Dane Bertram (U Calgary), Allen Cypher (IBM Research), Craig Anslow (Victoria Univ. Wellington), Laura Beckwith (Microsoft), Tom LaTosa (CMU), Chris Scaffidi (CMU), Volkmar Pipek (U Ziegen)

• Domain-based comparisons

– Content domains versus paradigm domains

• E.g. web, scripting, DB versus business, science, etc.

– May also be something about where in lifecycle

• Find & adapt

• Create and organize

• Inherit and learn

• Collaborative/mutual discovery


Domain contrasts

• Business processes (scripting)– Oriented toward large enterprises that have shared practices and

thus built-in motivation for sharing

• Information providers, e.g. admin who has content to post– Example might be using Lotus Notes / Dominoes;

– Observed lots of example/edit based reuse

– This could include engineers

• Consultants, integration including some programming– Just want a solution demonstrated ASAP, find and connect code

together

• Java professional programmers– Reuse within maintenance paradigm, finding model in others’ code

• Meta level, building tools that are situated within a domain– Human problem domain interaction --> domain-oriented design

environments

– Professional programmer builds the blocks to enable domain specialist

– Assumes goal of “reflective communities” that include collaborations


Debugging Group

• Debugging across domains• Examples of end-user programming debugging:

– large scale defense systems

– multi-sensor data fusion

– mashups of web resources

– programming by example

– natural language programming

– debugging intelligent user interfaces

- 32 -


Debugging Group• One problem with debugging mashups is that the tools haven't caught up with the state of the art commercial tools for more popular debugging platforms.

• Debugging multi-sensor systems is difficult because they often involve complex AI and machine learning algorithms.

• Perhaps the debate between static and dynamic testing would be moot if we had better tools for understanding the dynamic behavior of program behavior.

• Almost always trust the sensors, but almost never trust data sources in web mashups.

• Henry's waterlanguage.com is trying to create a unified platform for web development.

- 33 -

Documents

- 1 - End User Software Engineering CHI2008 Special Interest Group Meeting Tue, April 8, 2008, 9:00 AM - 10:30 AM Brad Myers, Margaret Burnett, Mary Beth