Wearable Experience: New Educational Media for Knowledge Intensive Training

Wearable Experiences for Knowledge Intensive Training

Mikhai l Fominykh, PhD

Europlan UK Molde Univers i ty Co l lege

28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 1

June 28-30, 2016 • Vancouver, BC, Canada

Wearable Experience: New Educational Media for Knowledge Intensive Training

Invited speech

June 28-30, 2016 • Vancouver, BC, Canada

#WEKIT#edmedia#edmediaconf

About me

Project manager in workplace training with

Augmented Reality and Wearables

Associate Professor and researcher in

Emergency Management training

Adjunct Professor and teacherin Virtual Reality

Software developer in psychological treatment training

Software developer in welfare personnel training

Software developer in cooperation training for medical students

Background:Experiential learning

Mikhail Fominykh, Peter Leong, and Brenda Cartwright (2016/2017) A Study on Experiential Learning in a 3D Virtual Environment within a Professional Counseling Distance Course, in progress.

Background:Virtual Reality Recording

Real time class R0

Recording

Replay

Recording

Mikhail Morozov, Alexey Gerasimov, Mikhail Fominykh, and Andrey Smorkalov (2013) Asynchronous Immersive Classes in a 3D Virtual World: Extended Description of vAcademia, DOI: 10.1007/978-3-642-38803-3_5

Photo by Mikhail Fominykh

Wearable ComputingAugmented Reality

Experience and KnowledgeWEKIT methodology

Information communication

Experience-communication

Images: http://www.unmuseum.org/, http://en.wikipedia.org/, http://digitalxtrememedia.com/; https://www.youtube.com/watch?v=7d59O6cfaM0, https://www.microsoft.com/en-us/research/project/holoportation/

https://www.youtube.com/watch?v=qAsu-RoVgWc

https://www.youtube.com/watch?v=7d59O6cfaM0

Object-centric

Action-centric

Performance Augmentationwith wearables

sensor-sensory loop:

super-real experience010000100001000010001010001111110

The guestenacts experiences

The stagerproduces narratives

WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 13

Photo by Ravensbourne

Wearable Computing

Wearable computing pursues an interface ideal of a continuously worn, intelligent assistant that augments memory, intellect, creativity, communication, and physical senses and abilities*.

* Starner (2011) The challenges of wearable computing: Part 1, IEEE Micro, Volume 21, Issue 4, DOI: 10.1109/40.946681

Why Wearable Computing?

Better access and management of information

Portability during operation

Enable hands free or hands-limited use

Can run continuously; and attempt to sense the user’s current context (Rhodes, 2000)

Focuses the user’s attention and presents information in an unobtrusive, context-dependent manner (Kortuem et al.)

Wearability

Head mounted (headsets, glasses,

ear attachments)

Hand, wrist and arm mounted (gloves, watches, rings, bands)Chest, neck back

mounted (belts, bands, wests, shirts)

Leg, feet mounted (bands, shoes)

Augmented Reality

Augmented Reality provides an enriched view onto the physical world, adding layers with contextually useful information, delivered visually or by stimulating other senses.

+80 /+100%

28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20https://www.youtube.com/watch?v=YJg02ivYzSs

Reality is more rough

Working example

Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning

Working example

What is WEKIT?

Funding: EU Horizon 2020, ICT-20 2015: Technologies for better human learning and teachingBudget: EUR 2.753.143,75WEKIT Community https://wekit-community.org/WEKIT project website http://wekit.eu/

Experience and knowledge

Learning = converting

experience to knowledge

Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

Experience and knowledgeLearning =

= converting experience to knowledge

immediate

experience

(aka ‘practice’)

information for a

master level

(aka ‘theory’)

separated

immediate

experience

information for a

master level

(aka ‘theory’)

immediate

experience

information for a

master level

(aka ‘theory’)

immediate

experience

information for a

master level

(aka ‘theory’)

immediate

experience

information for a

master level

(aka ‘theory’)

immediate

experience

information for a

master level

(aka ‘theory’)

immediate

experience

information for a

master level

(aka ‘theory’)

MethodologyWhat is Wearable Experience?

The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and enabling master-apprentice knowledge sharing.

Capture Re-enact

Evaluate

The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturingthe experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.

Giuseppe Scavo, Fridolin Wild and Peter Scott(2015) The GhostHands UX: telementoring with hands-on augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236

Captureexample

The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing.

Wild, Perey, Helin, Davies and Ryan (2014) TELLME’s Learning Experience Model in action on the shop floor of a textile weaving mill

Re-enact example

Jan Schneider, Dirk Börner, Peter van Rosmalen and Marcus Specht (2015) Presentation Trainer: Polishing your communication skills, DOI: 10.1007/978-3-319-24258-3_52

Evaluate example

Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

Capture | Re-Enact

TechnologyWhat is Wearable Experience?

An experience that is facilitated by devices located on or in the body that can include:

tracking user activity, biometric and environmental data

matching and mapping user performance

augmenting user’s activity with another user’s data

adapting user’s activity and behavior

Maintenance workers in Aeronautics

Imaging and diagnostic workers in Healthcare IT

Sub-system integration, operation & maintenance in Space

Photo by European Space Agency

WEKIT methodologyPhase ACapturing

WEKIT methodology Phase BRe-enactment

WEKIT methodology Phase CEvaluation

GoalsCapture of Expertise

What are relevant aspects of expertise and how to capture them?

Enable enactment of expertiseHow to actively support the transfer of expertise to a learner?

Design of learning taskHow should learning activities be designed to support / complementexpertise transfer?

Requirement generationHow should technology be designed to foster all of the above?

48WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016

Domain limited (Fernand and Herbert, 1996)

Failure to recall surface features (Schmidt and Boshuizen, 1993)

Inaccurate prediction judgement and advice (Hinds, 1999)

Inflexibility and bias (Sternberg and Frensch, 1992)

Qualities of ExpertsSuperior Mental Knowledge (De Groot, 1978)

Perceptual abilities (Lesgold et al., 1988)

Qualitative analysis (Klein, 1998)

Better strategies (Lemaire & Siegler, 1995)

Cognitive effort (Patricia 2003)

Shortcomings of Experts

Top-down approachIdentify tangible and intangible aspects relevant in all tasks

Find abstract key factors and rating scheme

Assess tasks and mechanisms according to factors and schemes

WEKIT methodology

Bottom-up approachStart to analyse concrete tasks in industrial settings

Classify them according to common scheme identifying aspects of expertise and learning

Try to find mechanism suitable to support expertise transfer

Task Classifications

Transfer mechanism H

Slow motion

Zoom 3 13

Think aloud protocol 1 8

Object enrichment 2

Contextualization 3, 9 3

Self awareness of physical state 14

Directed focus 15

Case identification 16

Spatial Simulation 18

In Situ Real-time Feedback 4 17

Mobile control

Virtual Post its 5 5

Haptic hints5,

Virtual/Tangible Manipulation 63,

10, 11

Remote symmetrical tele-assistance

Literature1 Schraagen, 1993

2 Tang et al., 2003

3 Quarles et al., 2008

4 BMW AR system, 2010

5 Sabine et al., 2011

6 Lahanas V et al., 2014

7 Chinthammit et al., 2014

8 Lundgrén-Laine et al, 2010

9 Curtis et al., 1998

10 Seichter, 2004

11 Wang, Chen, Gong, & Hsieh, 2007

12 Khademi et al., 2012

13 Mulloni, Dünser &Schmalstieg 2010

14 Larkin, 1983

15 Jarodzka et al., 2013

16 Mulzoff, 1990

17 Kotranza, Lind, Pugh & Lok 2009

18 Martin-Gutierrez et al., 2010

Cognitive

Perceptual

Developmental

Machine

Environment

Target

Domains Dimensions

Top-down approach

What is WEKIT?

TEL community – driven project

jTEL summer schooler

28/06/2016 https://wekit-community.org/ideas/

28/06/2016

http://bit.ly/wekit-facebookhttp://bit.ly/wekit-twitterhttp://bit.ly/wekit-linkedinhttp://bit.ly/wekit-plushttp://bit.ly/wekit-youtube

WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 59

D i s c l a i m e rT h i s p r o j e c t h a s r e c e i v e d f u n d i n g f r o m t h e E u r o p e a n U n i o n ’s H o r i z o n 2 0 2 0 r e s e a r c h a n d i n n o v a t i o n p r o g r a m m e u n d e r g r a nt a g r e e m e n t N o 6 8 7 6 6 9 . ht t p : / / w e k i t . e u /

Q & A#WEKIT

Presented by Mikhail Fominykh

mikhail.fominykh@europlan -uk.eu

http://www.mikhailfominykh.com/http://www.linkedin.com/in/fominykh/ https://www.facebook.com/mikhail.fominykh/ http://slideshare.net/mfominykh/

Wearable Experience: New Educational Media for Knowledge Intensive Training

Education

Wearable Experience for Knowledge-Intensive Training WEKIT lecture

Wearable Computers

Wearable Computing - Study Mafiastudymafia.org/wp-content/uploads/2015/01/CSE-wearable-computing... · Wearable Computing Submitted in partial ... A wearable computer is a computing

ROMANIAN ERASMUS INTENSIVE LANGUAGE EDUCATIONAL … · 2016-08-24 · Erasmus Intensive Language Courses for beginners: § 1-21 September 2012 (application deadline: June 1 st, 2012)

A Wearable Gesture Recognition Device for Detecting ... wearable gesture recognition device for... · A Wearable Gesture Recognition Device for Detecting Muscular ... WEARABLE GESTURE

Wearable Technology-Does It Fit Your Injured Workermageerehab.org/wp-content/uploads/2018/03/Wearable-Technology-Do… · The wearable device has ... Wearable Health Monitors Health

Computer Vision for Wearable Computing - University of … · Computer Vision for Wearable Computing ... Wearable devices promise to be less disruptive, ... eras may enable wearable

Doubling Up: Intensive Math Education and Educational ... · Intensive Math Education and Educational Attainment ... Of particular interest is the nature of remedial ... most common

Wearable Computers for the Educational Arena International Conference on Wearable Computers March 5, 2002 Chicago, Illinois

Wearable Computers for the Educational Arena

Wearable technologies

Wearable Tech

Wearable Computing - projects-web.engr.colostate.eduprojects-web.engr.colostate.edu/.../wearable/WearableComputing.pdfWearable Computing First Semester ... Our wearable computer allows

An Introduction to Augmented Reality - ISWC 2018 · Fundamentals of Wearable Computers and Augmented Reality. ... •Computer vision is computationally-intensive ... An Introduction

Table of Contents · After-school homework assistance, outreach educational services, summer-intensive literacy program, educational and recreational field trips, teen dialogue groups,

Will Wearable Technology mean Wearable Web Design?

Wearable Computing

Wearable Art

Intensive Math Instruction and Educational Attainment

Wearable Technologies for Health Care Applications - an ... · Wearable Technologies for Health Care Applications - an Overview ... Create a wearable device that could ... Wearable