Interactive Technologies And Games Conference. 16-17th October 2014

Remote operation of robots via mobile devices to help people with intellectual disabilities

María José Gálvez Trigo

School of Computer Engineering

University of Castilla-la Mancha

David J. Brown Computing and Technology Team

Nottingham Trent University

Contents

❖ INTRODUCTION

❖ Intellectual disabilities and assistive technologies

❖ Humanoid robot NAO

❖ Problem and justification

❖ GOALS

❖ THE PROPOSED SYSTEM

❖ EVOLUTION OF THE DEVELOPMENT

❖ RESULTS AND FUTURE WORK

❖ CONCLUSIONS

2

INTRODUCTION

3

Intellectual disabilities and assistive technologies

❖ Intellectual disability Learning disability:

❖ Impaired social functioning

❖ Reduced ability to

understand new information

❖ Assistive technology:

Product or service used to increase, maintain, or improve

the functional capabilities of a person with a disability

4

❖ Before adulthood

❖ Lasting effect on

development

º

5

❖ Mobile devices:

❖ Can be carried everywhere

❖ Can be used in multiple contexts

❖ Mainstream technology - Social acceptance

❖ Robotics:

❖ Increases engagement (Standen et. al 2014, Roscoe

2014)

❖ Predictable and reliable - Minimally threatening

❖ Holds interest more than some traditional methods

Humanoid robot NAO

❖ Manufactured by Aldebaran

Robotics

❖ 57 cm tall

❖ 25 Degrees of Freedom (DOF)

❖ Multiple tactile sensors

❖ Several microphones and

speakers

❖ Two cameras

❖ Fully programmable

❖ Toy-like appearance

6

Problem and justification

❖ Robotics and mobile devices need to be adapted for their

use within special education

❖ Use of robotics - Requires specific skills and knowledge

that educators do not possess

❖ Two possible solutions:

1. Specific training for educators

2. Development of tools that allow educators to use

robotics without specific training

7

Problem and justification

❖ Robotics and mobile devices need to be adapted for their

use within special education

❖ Use of robotics - Requires specific skills and knowledge

that educators do not possess

❖ Two possible solutions:

1. Specific training for educators

2. Development of tools that allow educators to use

robotics without specific training

8

GOALS

9

Two main goals

❖Develop an application for mobile devices that enables

the use of robotics by children with Special Educational

Needs (SEN) and their educators

❖Demonstrate how the application works by means of the

use of the humanoid robot NAO

❖ Development of specific modules and behaviours for the

robot

10

THE PROPOSED SYSTEM

11

The system

12

Main functionalities - On the robot

❖ Tethering mode

❖Wi-Fi - Touching front

sensor on NAO’s head

❖Bluetooth - Touching rear

sensor on NAO’s head

❖Stop the robot - Pressing

bumper on NAO’s right foot

13

14

Tethering mode and stop the robot

❖ Remote control

❖ Walk

15

Main functionalities - On the application

❖ Talk

❖ Change Posture

16

❖ Dance and music

❖ The robot performs a dance with music

❖ The robot teaches movements of a dance

17

❖ Simon says - Well-known game in which the robot tells the children

to do something

❖ Cause and effect - Two or three buttons to check the understanding

of the ‘cause and effect’ concept

18

❖ Questions and answers

❖ The educator sets a question and three possible answers

❖ Can be used with speech recognition

19

❖ Give orders to NAO

❖ React to voice commands

❖ React to a sound

❖ React to buttons

❖Settings

❖ IP address of the robot

❖Volume of the robot

❖Send to rest

❖Children mode

❖Security distance

❖Angle to turn and

distance to walk

20

❖Settings

❖Choose reward

❖Speech recognition

❖Sensibility of the sound

recognition system

❖Confidence of the

speech recognition

system

21

22

❖ Help - Shows help depending on the screen in which the

user is at that moment

23

Examples of screens using ‘children mode’

EVOLUTION OF THE DEVELOPMENT

24

25

❖ Co-design and testing by the developer (M. J. G. Trigo)

and the target group of users (children with SEN and

their educators)

❖ Meetings with educators from the Oak Field School and

Sports College in Nottingham to define the

requirements of the system

❖ Prototypes tested and improved taking into account the

feedback from the users in a process of iterative

prototyping

❖ Final version tested at the Oak Field School and Sports

College in Nottingham

RESULTS AND FUTURE WORK

26

Results

❖ Think-Aloud protocol

❖ A questionnaire was given to 7 educators:

❖ They had to give a value from 0 to 5 to aspects such

as the design, the implemented behaviours, the ease

of use, the tolerance to errors, etc.

❖ Mean over 4 in all questions - High degree of

satisfaction

❖ Low standard deviation - High degree of agreement

27

28

❖ The main goals have been accomplished

❖ Overall the system:

❖ Facilitates the integration of robotics in the

classroom of special education schools, being useful

for children with SEN

❖ Is consistent, clear, and easy to use and learn

❖ Offers an adequate and of good quality range of

behaviours and activities that the robot can perform

❖ Is robust and tolerant to errors, and giving

appropriate and relevant feedback when they occur

Future work

❖ RoboMobID will be used within the context of the

EDUROB project.

❖ EDUROB aims to explore the use of robotics in education

for people with learning disabilities and offer a solution for

its integration in the education of children with SEN.

29

The EDUROB project (543577-LLP-1-2013-1-UK-KA3-KA3MP) has been funded with support from the European

Commission.

Future improvements

❖ More dances and animations

❖ Activities using ‘Makaton‘ symbols for illiterate students

❖ User profiles for each child

❖ Translation to several languages

❖ Robot mimicking daily activities

❖ Support for creation of new animations within the

application

❖ Use along with physical input devices like micro-switches

❖ Support for more platforms (OS and robots)

30

CONCLUSIONS

31

32

❖ The use of mainstream technologies in the classrooms of

schools for students with SEN is of great help

❖ This is an emerging field with some promising early

studies to show that engagement in students is higher

using the NAO vs. some traditional methods (Standen et.

al 2014, Roscoe 2014)

❖ It is very important to help teachers to integrate new

technologies in the classroom

Thank you for your attention

Do you have any

questions?

33