The Semiotic Engineering of Human-Computer

InteractionSection I Foundation

Chapter 1 Introduction

“How do I send picture by email?”“Click on Attach button, or paper clip icon, select the picture and click attach”

The instructions above are clear and well understood by people that use computer everyday.

Conversation between two people interacting with a computer

1. The user has to navigate with mouse to the button with word “Attach”

2. Left clicking on mouse when the arrow on the computer screen is over the correct button.

A new dialog window will appear.

3. The user then has to navigate to the correct folder where the picture in the form of file is stored, by opening (click or double click with mouse) one folder at the time.

4. To select a picture the user has to click on the picture file and the file will be highlighted in dark blue.

5. Then the user has to click on “Open” button and it should attach the file to the email.

But the actual dialog that is happening between user and computer is more complex:

User Centered Design vs Semiotic Design

Section 1.1 Semiotic Theories of Human Computer InteractionSemiotic Study of:

1. Signs2. Signification processes, and3. How signs and signification take part in

communicationHCI artifacts are intellectual constructs

-result of choices and decisions rather than predictable natural laws.

Semiotic central themes of investigation: • meaning assignment, • meaning codification, and • the forms, ways, and effects of meaning in

communication – date to Greek classics

Graphical user interfaces Visual languages

Sign System◦ Produced and perpetuated by culturesThus.. A theory of culture

Semiotic in HCI

Bring together semiotic and HCI in a concise and consistent way to:

Support new knowledge organization and discovery Establishment of useful research methods for

analysis and synthesis Derivation of tools for training and practice

Semiotic Engineering

Signification and meaning-related processes between:

Semiotics involves

Support◦ Design◦ Construction

of artifacts

HCI artifactsresult of choices and decisions guided by reasoning, sense making, and technical skills

Engineering

We must be able to◦ Interpret,◦ Learn,◦ Use, and◦ Adapt to various contexts.

Goal produce meaningful interactive computer system discourse

HCI artifacts are communicated as signs

The branch of philosophy that studies the nature of knowledge, its presuppositions and foundations, and its extent and validity

In HCI◦ How knowledge is gained, analyzed, tested, and

used or rejected

Epistemological

Attitude Intent

content

Concrete objective stance◦ Psychological, social, and cultural contexts.

Encoded in signification systems◦ To communicate attitudes, intents, and contents

Signs

HCI artifact

Clients Owners

UsersDevelopers

Designers Technical Support

Interface signs◦ Meaning

Resulting signs◦ From

interaction Incorporates a

complex signification system

SmartFTP

Designers must tell users what they mean by the artifact they have created,

And users are expected to understand and respond to what they are being told….

Via the artifact’s interface◦ Words,◦ Graphics,◦ Behavior,◦ Online help,◦ And explanations.

1.2 The Semiotic Engineering Framework

Interlocutors in the communication process

Explicitly brings designers to HCI processes & assigns then an ontological position

Ontology -a rigorous and exhaustive organization of some knowledge domain that is usually hierarchical and contains all the relevant entities and their relations

Reflective Theory

User-Centered design compared to semiotic engineering

What is an intellectual artifact?◦ Artifact nonnatural objects created

Concrete = forks, knife (material artifacts) Abstract = safety measures (procedural artifacts

preventing accidents)◦ Physical purpose = chairs◦ Mental purpose = truth tables

Intellectual Artifacts are:◦ It encodes a particular understanding or interpretation

of a problem situation.◦ It also encodes a particular set of solutions for the

perceived problem situation.◦ The encoding of both the problem situation and the

corresponding solutions is fundamentally linguistic (based on a system)

1.3 Theorizing about Software as an Intellectual Artifact

Of symbols-verbal, visual, aural or other- that can be interpreted by consistent semantic rules); and

The artifact’s ultimate purpose can only be completely achieved by its users if they can formulate it within the linguistic system in which the artifacts is encoded.

Producer----> Intellectual artifacts----> Consumer

Same Language (genuine system of symbols)

Intellectual Artifacts (cont…)

System of Symbols Definition of Vocabulary Grammar Set of semantic rules.

We will have an entire lecture on this on Thursday! See next lecture.

What is a Language?