1
AbstractINSCAPE aims at enabling non computer scientist people to use and master the latest Information Society Technologies for interactively conceiving, authoring, publishing and experiencing interactive applications and particularly in the field of virtual rehabilitation. It integrates a wide range of virtual reality concepts and offers the user with the possibility to bring interactivity to multimedia content thanks to an intuitive and user friendly interface. IRTUAL REALITY poses a new challenge for psychologists and therapists who want to use virtual rehabilitation technology. Indeed, the production of interactive applications requires computer and programming skills, especially if it comes to implement device interactions with real-time rendering. Thus, therapists do not control the complete workflow for implementing the application and are required to hire computer scientists and programmers, which represents a substantial cost. INSCAPE is an authoring platform designed to create interactive applications and stories for use in simulation & training, edutainment, cartoons, marketing, live performance, etc. It provides a unique approach for the implementation of real-time 3D and 2D interactive stories and simulations, and it supports a wide range of virtual reality devices and concepts (HMD, 6 DOF trackers, 2D stroke recognition, etc.). The Graphical User Interface of INSCAPE offers several modules that visually represent the authored story, the interactivity of the story elements and the possible user interactions. The level of abstraction of these modules prevents the author of being involved in the programming aspect, while an advanced user will write Lua scripts to personalize the application. A therapist from the Neuropsychology and Cognitive Rehabilitation Center (Sinapsy) in Roma in collaboration with the project consortium has developed an Augmentative and Alternative Communication (AAC) tool into INSCAPE. The aim is to provide a young boy having multiple disabilities (severe mental retardation, motor coordination disorder, absence of speech) with a communicative and interactive tool to overcome the gap and difficulties induced by his situation. In order to reach this target, most of the environment and social situations that surround the boy (house, school, restaurant, supermarket, transport, etc.) have been reproduced inside the software. Objects, people and Manuscript received July 1, 2008. This work was supported in part by the European Commission within the FP6 research program. CS Communication et Systèmes – France ; phone: +33 561 176 652; fax: +33561176578; e-mail: [email protected] . Sinapsy, centro di neuropsicologia, riabilitazione cognitiva, comunicazione, linguaggio - Italia; phone +393283781869, e-mail [email protected] places have been linked following context-based and idiosyncratic connections. Compared to the classical tools used in AAC, INSCAPE enhances a natural and context-based strategy to represent contents that belong to real life (memories and desires through image, video and sound). Using an analogical representation of the world inside a portable device (e.g. a laptop with a touch screen) instead of semantics based on linguistic categories, is the best way to let all the people communicate intuitively, especially those with cognitive disability in the area of language and communication. Based on an intuitive program language, INSCAPE also offers the ability to build and refresh customized rehabilitation programs. Moreover, INSCAPE provides the possibility to easily create simulations of reality in order to support specific training. Stories developed with INSCAPE could especially help people with disabilities in communication and socialization area (e.g. autistic children) to train and implement their emergent skills beside real life situation or to learn particular scripts (regularities and praxis) contained in social situations. On this particular issue, INSCAPE could be used as a video-based and interactive tutorial. In a more general context, it can be considered as a generic tool for prototyping, authoring and experiencing simulations in the field of virtual rehabilitation. INSCAPE – Interactive Storytelling & Augmentative-Alternative Communication Matthieu Dautricourt and Francesco Toninelli V Fig. 1. The Stage Editor. Food inside the fridge is linked to different situations of real life: eating at home and buying at the supermarket. User can drag and drop single or multiple elements to both destinations. Fig. 2. The Story Editor. Transitions and interactions in the story are represented by arrows. 978-1-4244-2701-7/08/$25.00 ©2008 IEEE 67

[IEEE 2008 Virtual Rehabilitation - Vancouver, BC (2008.08.25-2008.08.27)] 2008 Virtual Rehabilitation - INSCAPE - Interactive storytelling & augmentative-alternative communication

Embed Size (px)

Citation preview

Page 1: [IEEE 2008 Virtual Rehabilitation - Vancouver, BC (2008.08.25-2008.08.27)] 2008 Virtual Rehabilitation - INSCAPE - Interactive storytelling & augmentative-alternative communication

Abstract—INSCAPE aims at enabling non computer scientist people to use and master the latest Information Society Technologies for interactively conceiving, authoring, publishing and experiencing interactive applications and particularly in the field of virtual rehabilitation. It integrates a wide range of virtual reality concepts and offers the user with the possibility to bring interactivity to multimedia content thanks to an intuitive and user friendly interface.

IRTUAL REALITY poses a new challenge for psychologists and therapists who want to use virtual

rehabilitation technology. Indeed, the production of interactive applications requires computer and programming skills, especially if it comes to implement device interactions with real-time rendering. Thus, therapists do not control the complete workflow for implementing the application and are required to hire computer scientists and programmers, which represents a substantial cost.

INSCAPE is an authoring platform designed to create interactive applications and stories for use in simulation & training, edutainment, cartoons, marketing, live performance, etc. It provides a unique approach for the implementation of real-time 3D and 2D interactive stories and simulations, and it supports a wide range of virtual reality devices and concepts (HMD, 6 DOF trackers, 2D stroke recognition, etc.). The Graphical User Interface of INSCAPE offers several modules that visually represent the authored story, the interactivity of the story elements and the possible user interactions. The level of abstraction of these modules prevents the author of being involved in the programming aspect, while an advanced user will write Lua scripts to personalize the application. A therapist from the Neuropsychology and Cognitive Rehabilitation Center (Sinapsy) in Roma in collaboration with the project consortium has developed an Augmentative and Alternative Communication (AAC) tool into INSCAPE. The aim is to provide a young boy having multiple disabilities (severe mental retardation, motor coordination disorder, absence of speech) with a communicative and interactive tool to overcome the gap and difficulties induced by his situation. In order to reach this target, most of the environment and social situations that surround the boy (house, school, restaurant, supermarket, transport, etc.) have been reproduced inside the software. Objects, people and

Manuscript received July 1, 2008. This work was supported in part by the European Commission within the FP6 research program. CS Communication et Systèmes – France ; phone: +33 561 176 652; fax: +33561176578; e-mail: [email protected]. Sinapsy, centro di neuropsicologia, riabilitazione cognitiva, comunicazione, linguaggio - Italia; phone +393283781869, e-mail [email protected]

places have been linked following context-based and idiosyncratic connections.

Compared to the classical tools used in AAC, INSCAPE enhances a natural and context-based strategy to represent contents that belong to real life (memories and desires through image, video and sound). Using an analogical representation of the world inside a portable device (e.g. a laptop with a touch screen) instead of semantics based on linguistic categories, is the best way to let all the people communicate intuitively, especially those with cognitive disability in the area of language and communication. Based on an intuitive program language, INSCAPE also offers the ability to build and refresh customized rehabilitation programs. Moreover, INSCAPE provides the possibility to easily create simulations of reality in order to support specific training. Stories developed with INSCAPE could especially help people with disabilities in communication and socialization area (e.g. autistic children) to train and implement their emergent skills beside real life situation or to learn particular scripts (regularities and praxis) contained in social situations. On this particular issue, INSCAPE could be used as a video-based and interactive tutorial. In a more general context, it can be considered as a generic tool for prototyping, authoring and experiencing simulations in the field of virtual rehabilitation.

INSCAPE – Interactive Storytelling & Augmentative-Alternative Communication

Matthieu Dautricourt and Francesco Toninelli

V Fig. 1. The Stage Editor. Food inside the fridge is linked to different situations of real life: eating at home and buying at the supermarket. User can drag and drop single or multiple elements to both destinations.

Fig. 2. The Story Editor. Transitions and interactions in the story are represented by arrows.

978-1-4244-2701-7/08/$25.00 ©2008 IEEE 67