Play and learn: getting empowered by interaction and motion: case Kinect

w w w . h a m k . f i

Play and learn: getting empowered by

interaction and motion: case KinectLeena Koskimäki, Timo Niemelä and Merja Salminen 30.10.2014

Openness Accelerating Learning Networks


The agenda of the workshop

14:45-15:30

15:45-16:45

Introduction to the

project

Hands on: Kinect

games and

applications


THE PROJECT


"Openness Accelerating Learning

Networks“ project (2012-2014)

peer education collegial support mobile services

effective learning networks new learning environments

continous learning distance education sharing

new tools for participation and learning

augmented reality communality open source

co-creating virtual worlds open decision-making

Collaborative, participatory and networked work culture in

educational institutions and organizations and among their stakeholders


Subproject: 3D and mobile environments for

participation and learning

In cooperation with the

University of Tampere, the

Salpaus further education

unit and the Educational

Association of Citizens'

Forum.

The goal: to develop

augmented and virtual reality

for educational and

participational use.


The goals in HAMK University of

Applied Sciences

To implement pilots to study

the possibilities of new

technologies and services as

tools for learning and

participation.

HAMK pilots first implemented

in an authentic learning

environment - Virvelinranta

Development and Resource

Centre for the Disabled,

Finland.


KINECT


Kinect – motion sensing input device

• Launched by Microsoft in late 2010 as a bodily

control for the Xbox game console

• A version for Windows on early 2012

• Key features:

– Skeletal and facial tracking

– Gesture recognition

– Voice recognition

| http://www.microsoft.com/en-us/kinectforwindows/discover/features.aspx


Why Kinect?

Visual, experimental environments for learning and

participation in 3D

The approachable technology facilitates the welfare sector

students', mostly new to Kinect, orientation.

All interaction at the user interface takes place in ways

natural to human beings: gestures, facial expressions and

speech.

Suitable for various user groups who may not be able to use

technically demanding user interfaces at all due to

disabilities relating to their cognitive, motion, motor or

sensory functions.


Kinect as the control unit in the augmented and

virtual reality applications

• A Virvelinranta client bowling virtually.

• The user sees digital data and content combined to user’s surrounding

environment.


PILOTS IN THE VIRVELINRANTA SERVICE

CENTRE FOR DISABLED PEOPLE


The pilots on year 2012

• Welfare students in work placement in Virvelinranta

directed game sessions for disabled clients.

– In the first pilot commercially available games were used.

– Knowledge and feedback was gathered about how the technology

works and supports learning, participation and interaction.– HAMK student on her experience of taking part in pilot:http://youtu.be/o5-H-YO1vUU

http://youtu.be/o5-H-YO1vUU

https://www.youtube.com/watch?v=DXY6SMhZPvg&index=24&list=UUUAu9cFZ30g3qnaodxUt_Sw

https://www.youtube.com/watch?v=DXY6SMhZPvg&index=24&list=UUUAu9cFZ30g3qnaodxUt_Sw


The pilots on year 2013• Analyzing the feedback and observations from the participated students,

Virvelinranta staff and clients. – Utilizing this feedback on developing accessible applications

• The ICT students planned and developed two game apps as their work

placement during summer. – Accessibility and usability were especially taken into account in designing phase of the

applications.

• Testing applications with users in Virvelinranta as a learning exercise on

Games and plays course– Practising directing acitivities and interaction with the game sessions.


The pilots on year 2014

• New user groups in different

environments were involved:

– Students with special educational needs

– The aged in day care centers and service

homes for elderly people

• The software development projects

continued till late autumn.

– Applications available for everybody!

• Pilot in course of nursing the aged

continues till December.• Video: Staff member and clients of Virvelinranta on

their gaming experience:

https://www.youtube.com/watch?v=3ScxZk9Orvg

https://www.youtube.com/watch?v=3ScxZk9Orvg


The progress of the pilots

Preparation

Directingacitivities,

observation

Surveys, feedback,

discussionsEvaluation

Development


The goals of the IT-students'

software development project• To design and develop accessible, tailor-made apps, based on the

feedback collected during the pilots - to create more functional apps

to focus on the directing process

• To understand the needs of the app users (students, staff,

customers)

• To understand the possibilities and possible restrictions of creating

apps used with Kinect.


THE EXPERIENCE FROM THE PILOTS


The natural user

interface (NUI)

Gesture controlled virtual and augmented reality

The possibilities for

various user groups not

able to use technically

demanding interfaces?

No mouse, keyboard

or handheld devices

needed

A technology easy to

approach and learn

Augmented

reality

Digital content

brought to one’s

surrounding

environment

How to support the user

by bringing audio and

visual elements to the

app?

Visual, experimental 3D environments

Virtual reality

Simulation

New experiences,

memorizing the past

events

The possibilities of NUI

simulations?What other elements

could be brought in?

Interaction within the

simulation


The gaming

experience

Supporting the participation

To whom gaming is

suitable?

Was found as a

refreshing and

different kind of an

acitivity

Ability to

function

Excercising ”without

noticing”

How to to improve

accessibility?

New experiences together and by oneselves.

The game

situation

A social event

Competitiviness

How does the social

interaction affect on

clients?

Cheering each other,

success

”I can do this!”

Influence on a longer

term?


The change

• New information about new possibilities of learning and

participation for the target groups (teachers and students as well as service

providers and their customers)

• Playing was seen as a way of participating together and

social interaction but also as a goal-directed acitivity

(exergaming?)

• Welfare sector actors familiarized with new technologies

• IT students learned about the interconnection between

Kinect and the promotion of wellbeing but also the

educational prospects.

• More possibilities for citizens' participation have emerged

during the project – including those for special groups.


Read more:

• The project website at HAMK: http://www.hamk.fi/avo2/en

• The brochure of the project at HAMK: http://bit.ly/18OM0AQ

Contact us: [email protected]

[email protected]

[email protected]

http://www.hamk.fi/avo2/en

http://bit.ly/18OM0AQ