Learning, technology and collaboration in mobile environments

Zurich July 2006

Josie Taylor, Professor of Learning TechnologyCentre for Research in Computing

Institute of Educational Technology The Open University

Methods for studying learning, collaboration and technology use in mobile environments

Zurich July 2006

Aim of talk

• Key issue: – methodological challenges of trying to satisfy various

stakeholders when evaluating learning and technology use in informal settings.

• A method – for representing user activity (practices, strategies and

conflicts) that emerge when interacting with technological systems in an informal mobile learning setting (learning?)

– semiotic and technological space.

– cultural historical activity theory, and develops Engestrom's (1987) extended model of human activity.

– Task Models

Zurich July 2006

Acknowledgements

• Mike Sharples• Patrick McAndrew• Giasemi Vavoula• Claire O’Malley

Zurich July 2006

Model of Mobile Learning:Research from MOBIlearn

• Sharples, Taylor & Vavoula (in press) propose a theory of learning in the mobile age as:

‘the processes of coming to know through conversations across multiple contexts amongst people and personal interactive technologies’

Zurich July 2006

A theory of learning in the mobile age

• “Conversation and context are essential constructs for understanding mobile learning, and offering implications for the ownership of learning and the integration of mobile learning with conventional education.”

• Draws on Pask’s (1976) Conversation Theory, as does Laurillard (2002)

Zurich July 2006

Partner provides facility for practical model building and problem solving

Partner acts to build models and solve problems

Learner acts to build models and solve problems

Sets goalsAdjusts model

ActsModifies actions

How questions and responses

Partner demonstrates or elicits models and problem solutions

Learner demonstrates understanding of models and

problem solutions

Offers theories and ideasRe-describes theories

Offers conceptions and explanationsRe-describes conceptions

Why questions and responses

Partner provides facility for mediating agreements

Level of Actions

Level of Descriptions

Conversational Framework, Laurillard 2002

Zurich July 2006

A theory of learning in the mobile age

• The focus of our investigation is not the learner, nor their technology, but the communicative interaction between these to advance knowing.

• Conversation is the driving process of learning. It is the means by which we negotiate differences, understand each other’s experiences and form transiently stable interpretations of the world.

• Sharples, M., Taylor, J., and Vavoula, G., (in press) A Theory of Learning for the Mobile Age, in R. Andrews and C. Haythornthwaite, Handbook of e-learning Research, Sage Publications.

Zurich July 2006

Research in progress

Ways of capturing and analysing this learning– Based in activity theory– Use scenarios to develop mobile learning

activities– Run trials to test both the activity and the

technology

Zurich July 2006

Additional Issues

• Communication in large multi-site, multi-national projects building systems (Carroll: 1995 – Scenarios; Taylor & Evans, 2005)

• Evaluators need to move between stakeholders, responding appropriately to each, as well as serving the needs of the end users

• Stakeholders: funders, system designers, system builders, educators, domain specialists, curators, teachers, companies, end users…

Zurich July 2006

MOBIlearn Project

• Project Info:– 33 month EU-funded with 15+ partners– Circa 8m Euros– 200+ members of SIG

• Goals:– Designing an architecture for pedagogically sound

mobile learning environments– Implementing an instantiation of the architecture with

current technologies– Evaluate

Zurich July 2006

3 MOBIlearn Scenarios:3 types of learning

• Museum – visitors to Art Gallery (informal)

• MBA – professionals in full time work engaged in study

(formal curriculum, work based)

• First Aid Training – volunteers in full time work needing training in

situ (voluntary, curriculum, work-based)

Zurich July 2006

Internally initiated

Internally structured

Formal learning - not within scope for MOBIlearn

Externally structured

MuseumInformal learning

Externally initiated

First Aid Voluntary learning

MBA Resource

based learning

Adapted from Livingstone 2001

Zurich July 2006

Scenario Development Process

Zurich July 2006

Scenarios

Guidelines

Literature review

Functional Requirements

•Pilot studies

•Qnr studies

•Interviews

•Diary studies

UML

Non functional requirements

System Design

Trials

Software Development Process

Zurich July 2006

Development of Task Model• Based in AT – a tool to:

– Represent learning in mobile settings– Capture the complexity of the setting and

situation– Capture aspects of the dialectical process of

appropriation both technical and semiotic– Communicate with various stakeholders as

part of evaluation

Zurich July 2006

Activity Theory (AT)

• Cultural Historical Activity Theory (Leont’ev, 1978)

• Grounded in Vygotsky’s (1978) theorising about the social-cultural development of human mind (developmental studies of higher mental functioning)

Zurich July 2006

Extended model of human activity (Engeström, 1987)

Subject Object

CommunityRules Division of

Labour

Tools

Outcome

Zurich July 2006

Rules Community Division of labour

Control Context Communication

Reflecting social rules and control in technological sense

Not just human community but also technological setting

Negotiating both with other humans and technology

Zurich July 2006

Semiotic Layer – Human/task focused

Zurich July 2006

Technological Layer – Implementation focused

Zurich July 2006

Interaction between semiotic and technological layers

Zurich July 2006

A pretty version

Zurich July 2006

Purpose:

• To hold a very complex situation in tension– Looking for conflicts between layers– Looking for support between layers– Identifying relationships and interactions– Allowing systematic manipulation– Allowing multiple views to be represented

Zurich July 2006

Example: First Aid Training

• First aiders at the Open University• Need to keep training up to date and

refreshed, but not many opportunities to practise skills

• All are full-time workers, so training could take place in workplace

• Communication/collaboration is as important as access to content of training

Zurich July 2006

First Aid Training Scenario: players

• Task leader:–Configures task–Interacts with participants–Provides guidance

• Task participants –act singly and in pairs to complete task–Briefed and de-briefed as a whole group

Zurich July 2006

Task: Participant view

• Initial on-line chat (group)• On-line Quiz (individual)• Challenge is set (pairs)• G1: On-line Brainstorming (pairs)• G2: On-line Brainstorming (pairs)• Get together on-line and vote (group)

Zurich July 2006

Agenda for the First Aid Trial

Zurich July 2006

Task: Task Leader view

• Set up content and direct participants to it• Construct on-line agenda• Oversee activity during chats• Set tools to appropriate states (e.g. close

the vote at the end of voting)

Zurich July 2006

Technological Tool (PDA and mobile phones)

Semiotic Tool(First Aid Training)

Semiotic Subject (First Aider)

Semiotic Object ( knowledge & skills)

Changed Object (revised knowledge & skills)

Control

Technological

(usability of device)

Semiotic

(constraints & protocols in first aid)

ContextTechnological

(network facilities, wireless)Semiotic

(community of first aiders)

Communication

Technological

(texting, uploading pictures & text)

Semiotic

(conversation; support of co-workers)

Task Model for First Aid Scenario

Technological Subject (User)

Technological Object (access to information)

Zurich July 2006

Task: Evaluator view

• Developing scenario, constructing task models and feeding into requirements or design

• Designing evaluation instruments (e.g. observation schedules, questionnaires etc)

Zurich July 2006

Iterative feedback

GenericRequirements

Semiotic Model

Technical Model

DesignFeedback

ScenarioTrials

Zurich July 2006

Semiotic Tool(Collaborative: Working together to

assess situations)


Semiotic Object (G1: online brainstorming)

Changed Object (improved performance leading to G2)

Control

Semiotic

(Share; don’t diss

other person’s pov; be

constructive)

ContextSemiotic

(partner for activity; larger group)

Communication

Semiotic

(conversation; discussion;

Agreement; resolution of argument)

Semiotic model for First Aid Scenario: brainstorming

Zurich July 2006

Technological Tool Workspaces: agenda, brainstorm, chat, vote

Control

Moderated, user selections, Many choices, different levels,

task-based

Context

Wireless, fixed location, group work, divided groups

Communication

Guided, group and moderator, chats

Technical Model for First Aid Trial

Technological Subject

User at work

Technological Object

Work through agenda

Zurich July 2006

Post Trial Findings

• Technical:– Tool choice allows unified working– Usability issues are reduced by providing a

task focus– Performance on MOBIlearn system is

adequate when working on real tasks– Task-based control is available but does not

ensure shared views

Zurich July 2006

Post Trial Findings

• Semiotic:– Moderating requires control information to and from

other users– Alerting is needed to bring synchronicity– Mobility needs the user to be mobile not just the

tools

Zurich July 2006

Technological Tool Workspaces: agenda,brainstorm, chat, vote

Control


task-based

Context


Communication


From trial: Support from technology


User at work


Work through agenda

Zurich July 2006


Control


task-based

Context

Wireless, fixed location, groupwork, divided groups

Communication


From trial: Conflicts in technology


User at work


Work through agenda

Zurich July 2006


Control


task-based

Context


Communication


From trial: Conflicts in technology


User at work


Work through agenda

Zurich July 2006

Semiotic Tool(First Aid Training)




Control

Alerting

Context

Mobility

Communication

Moderator role

Task Issues for First Aid Scenario

Zurich July 2006

40

Museum Scenario

• Uffizi museum in Florence• Groups of users looking at Botticelli

paintings• Variety of handheld devices and phones

– Tablet PCs; pocket PCs; phones; PDAs

• Data collected by Italian partners– Observation studies– Videotape of movement through gallery– Pre/post questionnaires (satisfaction) 40

Zurich July 2006

Museum Scenario data

• Good multimedia facilities and high levels of usability in the tablet computers facilitated the semiotic level:– associations with other personal ‘devices’ (i.e.

diaries and calendars)

• Semiotic context has within it relatively large numbers of young women – levels of communication and sharing

increased

Zurich July 2006

Task Model Example: Museum Scenario

Technological Tool:(PDA and mobile phones, pocket PCs, tablet PCs)

Semiotic Tool(Learning about paintings)

Semiotic Subject (Museum Visitor)



Control

Technological

(usability of device)

Semiotic

(constraints & protocols in

museum)


(network facilities, wireless)Semiotic

(community of visitors)

Communication

Technological

(texting, uploading pictures & text)

Semiotic

(conversation; support of co-visitors)



Zurich July 2006

Museum Scenario data

• Younger participants were keen on the idea of using the Chat service to save, download and print conversations so that they had a textual photograph to remind them of their visit to the museum.

• Technological aspects of the scenario were supporting and augmenting the semiotic activities, providing a

much richer experience for visitors.

Zurich July 2006

Support in the Museum Scenario

Technological Tool (Pocket PC/ Notebook)

Semiotic Tool(Learning about

paintings)

Semiotic Subject (museum visitor)



ControlTechnological

(usability of device - good)Semiotic

(Social Rules: diary functions/increased sharing)


(interesting content well presented)Semiotic

(Young people/young women)

CommunicationTechnological

(good comms facilities; chat)Semiotic

(exchange of information/opinion)



Zurich July 2006

Conflict in the Museum Scenario

• In the case of the other technologies (mobile telephones and PDA’s) we find that the relative lack of usability in the technological domain inhibits all these

developments in the semiotic.

Zurich July 2006

Conflict in the Museum Scenario

Technological Tool (mobile phone/PDA)

Semiotic Tool(Learning about

paintings)

Semiotic Subject (museum visitor)



ControlTechnological

(usability of device -poor)Semiotic

(Social Rules: little interaction with exhibits, system or other

visitors)


(difficult interface, poor search)Semiotic

(community: no engagement)

CommunicationTechnological

(good comms; chat)Semiotic

(communications not used)



Zurich July 2006

Conclusions • Work-in-progress

• The task model helps to provide a structured space within which to investigate success or failure

• Enables evaluators :– to disambiguate specific technology from

functionality– to separate human issues from technological

issues, and put them back together again– to try several different implementation

solutions in comparable circumstances

Zurich July 2006

Conclusions (2)

• Understanding the dialectic is very important– Process level – appropriation and its impact

on behaviour– Specific technology level – what the tools are

offering in terms of functionalities and how the user makes the best use of them to achieve their goal (or not!)

• Focus on the activity keeps the learner to the fore

Zurich July 2006

Conclusions (3)

• So far the task models are proving useful for evaluators

• Enables us to say more about what might be going on in a given activity

• Enables us to pinpoint potential sources of conflict within and between levels

• Provides a way of representing learning and interaction with technology

Zurich July 2006

Future Work

• Computer support for diagrams• Drilling deeper into scenarios

– How deep is deep enough?– Is more useful data yielded?– Are triangles the best representation?– If so, why exactly?