Wearable Experience: New Educational Media for Knowledge Intensive Training

Wearable Experiences for Knowledge Intensive Training

Mikhai l Fominykh, PhD

Europlan UK Molde Univers i ty Co l lege

28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 1

June 28-30, 2016 • Vancouver, BC, Canada

Wearable Experience: New Educational Media for Knowledge Intensive Training

Invited speech


June 28-30, 2016 • Vancouver, BC, Canada

#WEKIT#edmedia#edmediaconf

About me


Project manager in workplace training with

Augmented Reality and Wearables

Associate Professor and researcher in

Emergency Management training

Adjunct Professor and teacherin Virtual Reality

Software developer in psychological treatment training

Software developer in welfare personnel training

Software developer in cooperation training for medical students

Background:Experiential learning


Mikhail Fominykh, Peter Leong, and Brenda Cartwright (2016/2017) A Study on Experiential Learning in a 3D Virtual Environment within a Professional Counseling Distance Course, in progress.

Background:Virtual Reality Recording


Real time class R0

R1

R1

R2

Recording

Replay

Recording

Mikhail Morozov, Alexey Gerasimov, Mikhail Fominykh, and Andrey Smorkalov (2013) Asynchronous Immersive Classes in a 3D Virtual World: Extended Description of vAcademia, DOI: 10.1007/978-3-642-38803-3_5

http://dx.doi.org/10.1007/978-3-642-38803-3_5





Photo by Mikhail Fominykh

Wearable ComputingAugmented Reality

Experience and KnowledgeWEKIT methodology

Information communication


Experience-communication

Images: http://www.unmuseum.org/, http://en.wikipedia.org/, http://digitalxtrememedia.com/; https://www.youtube.com/watch?v=7d59O6cfaM0, https://www.microsoft.com/en-us/research/project/holoportation/

https://www.youtube.com/watch?v=qAsu-RoVgWc

AR?

https://www.youtube.com/watch?v=7d59O6cfaM0

AR!

Object-centric


Action-centric

Performance Augmentationwith wearables

sensor-sensory loop:

super-real experience010000100001000010001010001111110

The guestenacts experiences

The stagerproduces narratives

WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 13

Photo by Ravensbourne

Wearable Computing

Wearable computing pursues an interface ideal of a continuously worn, intelligent assistant that augments memory, intellect, creativity, communication, and physical senses and abilities*.

* Starner (2011) The challenges of wearable computing: Part 1, IEEE Micro, Volume 21, Issue 4, DOI: 10.1109/40.946681

Why Wearable Computing?


Better access and management of information

Portability during operation

Enable hands free or hands-limited use

Can run continuously; and attempt to sense the user’s current context (Rhodes, 2000)

Focuses the user’s attention and presents information in an unobtrusive, context-dependent manner (Kortuem et al.)

Wearability


Head mounted (headsets, glasses,

ear attachments)

Hand, wrist and arm mounted (gloves, watches, rings, bands)Chest, neck back

mounted (belts, bands, wests, shirts)

Leg, feet mounted (bands, shoes)


Augmented Reality

Augmented Reality provides an enriched view onto the physical world, adding layers with contextually useful information, delivered visually or by stimulating other senses.



+80 /+100%

28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20https://www.youtube.com/watch?v=YJg02ivYzSs


Reality is more rough


Working example

Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning


Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning


Working example


Working example

What is WEKIT?


Funding: EU Horizon 2020, ICT-20 2015: Technologies for better human learning and teachingBudget: EUR 2.753.143,75WEKIT Community https://wekit-community.org/WEKIT project website http://wekit.eu/

https://wekit-community.org/

http://wekit.eu/

Experience and knowledge

Learning = converting

experience to knowledge


Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

http://dx.doi.org/10.3233/978-1-61499-530-2-298

Experience and knowledgeLearning =

= converting experience to knowledge

immediate

experience

(aka ‘practice’)

information for a

master level

(aka ‘theory’)

separated




immediate

experience


information for a

master level

(aka ‘theory’)




immediate

experience


information for a

master level

(aka ‘theory’)




immediate

experience


information for a

master level

(aka ‘theory’)




immediate

experience


information for a

master level

(aka ‘theory’)




immediate

experience


information for a

master level

(aka ‘theory’)




immediate

experience


information for a

master level

(aka ‘theory’)

Exp

erie

nc

ed

lea

rne

r


MethodologyWhat is Wearable Experience?


The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and enabling master-apprentice knowledge sharing.

Capture Re-enact

Evaluate



The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturingthe experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.

Giuseppe Scavo, Fridolin Wild and Peter Scott(2015) The GhostHands UX: telementoring with hands-on augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236

Captureexample

http://ebooks.iospress.nl/publication/39872



The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.

Wild, Perey, Helin, Davies and Ryan (2014) TELLME’s Learning Experience Model in action on the shop floor of a textile weaving mill

Re-enact example


Jan Schneider, Dirk Börner, Peter van Rosmalen and Marcus Specht (2015) Presentation Trainer: Polishing your communication skills, DOI: 10.1007/978-3-319-24258-3_52

Evaluate example

http://link.springer.com/chapter/10.1007/978-3-319-24258-3_52

Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

Capture | Re-Enact


http://dx.doi.org/10.3233/978-1-61499-530-2-298

TechnologyWhat is Wearable Experience?

An experience that is facilitated by devices located on or in the body that can include:

tracking user activity, biometric and environmental data

matching and mapping user performance

augmenting user’s activity with another user’s data

adapting user’s activity and behavior


# 41

Maintenance workers in Aeronautics

# 42

Imaging and diagnostic workers in Healthcare IT

# 43

Sub-system integration, operation & maintenance in Space

Photo by European Space Agency


WEKIT methodologyPhase ACapturing

WEKIT methodology Phase BRe-enactment

WEKIT methodology Phase CEvaluation

GoalsCapture of Expertise

What are relevant aspects of expertise and how to capture them?

Enable enactment of expertiseHow to actively support the transfer of expertise to a learner?

Design of learning taskHow should learning activities be designed to support / complementexpertise transfer?

Requirement generationHow should technology be designed to foster all of the above?

48WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016


Domain limited (Fernand and Herbert, 1996)

Failure to recall surface features (Schmidt and Boshuizen, 1993)

Inaccurate prediction judgement and advice (Hinds, 1999)

Inflexibility and bias (Sternberg and Frensch, 1992)

Qualities of ExpertsSuperior Mental Knowledge (De Groot, 1978)

Perceptual abilities (Lesgold et al., 1988)

Qualitative analysis (Klein, 1998)

Better strategies (Lemaire & Siegler, 1995)

Cognitive effort (Patricia 2003)

Shortcomings of Experts


Top-down approachIdentify tangible and intangible aspects relevant in all tasks

Find abstract key factors and rating scheme

Assess tasks and mechanisms according to factors and schemes

WEKIT methodology

Bottom-up approachStart to analyse concrete tasks in industrial settings

Classify them according to common scheme identifying aspects of expertise and learning

Try to find mechanism suitable to support expertise transfer


Task Classifications

Transfer mechanism H

igh

pre

cisi

on

Hig

h s

pee

d

Dec

isio

n t

akin

gC

olla

bo

rati

ve

task

sH

igh

mem

ory

lo

ad

Per

cep

tual

tas

k

Ass

erti

ve t

ask

Du

al t

ask

Per

form

ance

Man

agin

g U

nce

rtai

nty

Hig

h S

pat

ial

Task

Slow motion

Zoom 3 13

Think aloud protocol 1 8

Object enrichment 2

Contextualization 3, 9 3

Self awareness of physical state 14

Directed focus 15

Case identification 16

Spatial Simulation 18

In Situ Real-time Feedback 4 17

Mobile control

Virtual Post its 5 5

Haptic hints5,

12

Virtual/Tangible Manipulation 63,

10, 11

Remote symmetrical tele-assistance

7 7

Literature1 Schraagen, 1993

2 Tang et al., 2003

3 Quarles et al., 2008

4 BMW AR system, 2010

5 Sabine et al., 2011

6 Lahanas V et al., 2014

7 Chinthammit et al., 2014

8 Lundgrén-Laine et al, 2010

9 Curtis et al., 1998

10 Seichter, 2004

11 Wang, Chen, Gong, & Hsieh, 2007

12 Khademi et al., 2012

13 Mulloni, Dünser &Schmalstieg 2010

14 Larkin, 1983

15 Jarodzka et al., 2013

16 Mulzoff, 1990

17 Kotranza, Lind, Pugh & Lok 2009

18 Martin-Gutierrez et al., 2010


Tasks

Cognitive

Perceptual

Developmental

Human

Machine

Environment

Space

Time

Target

Domains Dimensions

Top-down approach

What is WEKIT?


TEL community – driven project



jTEL summer schooler







28/06/2016 https://wekit-community.org/ideas/

28/06/2016

http://bit.ly/wekit-facebookhttp://bit.ly/wekit-twitterhttp://bit.ly/wekit-linkedinhttp://bit.ly/wekit-plushttp://bit.ly/wekit-youtube

WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 59

D i s c l a i m e rT h i s p r o j e c t h a s r e c e i v e d f u n d i n g f r o m t h e E u r o p e a n U n i o n ’s H o r i z o n 2 0 2 0 r e s e a r c h a n d i n n o v a t i o n p r o g r a m m e u n d e r g r a nt a g r e e m e n t N o 6 8 7 6 6 9 . ht t p : / / w e k i t . e u /

Q & A#WEKIT

Presented by Mikhail Fominykh

mikhail.fominykh@europlan -uk.eu

http://www.mikhailfominykh.com/http://www.linkedin.com/in/fominykh/ https://www.facebook.com/mikhail.fominykh/ http://slideshare.net/mfominykh/


http://wekit.eu/

mailto:[email protected]

http://www.mikhailfominykh.com/

http://www.linkedin.com/in/fominykh/

https://www.facebook.com/mikhail.fominykh/

http://slideshare.net/mfominykh/