Upload
emily-fifield
View
218
Download
3
Embed Size (px)
Citation preview
AISN
http://www.cs.bgu.ac.il/~royif/AISN
Auditory Imaging for Sightless Navigation
ARD PresentationDecember, 2010
Academic Advisor:Dr. Yuval Elovici
Technical Advisor:Dr. Rami Puzis
Team Members:Yakir DahanRoyi Freifeld
Vitali Sepetnitsky
Project Team
2
Introduction
3
The Problem Domain
Most of our navigation in the everyday life heavily depends on visual feedback that we get from our environment
When the ability to see the surroundings is missing due to visual impairments, the ability to navigate is also damaged
4
Existing Solutions
Physical sense:› White Cane› Guide Dog
Sensory substitution:› Warning of obstacles (e.g.
Optical Radar)› Sonar-like images scanning (e.g. The
vOICe)
5
Sightless navigation by sensory
substitution
› Development of an application that allows a
person to navigate, relying primarily on the
sense of hearing
Integration with a spatial auditory
environment
Providing a flexible environment for future
research
Vision and Main Goals
6
Our SolutionA Combination of visual information processing and 3D sound creation and positioning: Taking a stream of frames from a
web-camera Processing the frames and retrieving
visual information relevant to the user Creating appropriate sounds according
the recognized information Performing an auditory spatialization of
the sound and informing the user about the locations of the detected information 7
Our Solution (cont.)
8
External Interfaces Hardware and Software
OpenCV
OpenAL
MATLAB engine library
9
System Users End Users
› Visually impaired (or even blind) people who use the system for the purpose of hearing their physical environment
Configuration Users› The system installation and initial tuning,
such as user profiles creation, will be done by configuration users having the ability to see the operations they perform
Researchers› Cognitive science researchers who wish
to conduct experiments regarding 3D sound
10
Functional Requirements
11
Functional Requirements Core Functionality For all users (especially the
researcher):1. Support several types of computer vision
and image processing algorithms for extraction of the following information:
Feature points (points of interest) Contours BLOBS (regions that are either darker or brighter
than the surrounding)
2. Provide a utility to add new implementations of the above algorithms according to a predefined API
3. Support specific configurability options for each algorithm type 12
Functional Requirements (cont.) Core Functionality (cont.) For all users:
4. Create appropriate sounds according to the following features:
Location Brightness Color
5. Support sound spatialization using OpenAL API implementations and HRTF datasets conforming with a predefined format
6. Allow to install new HRTF datasets and OpenAL implementations for improving the quality of sound localization and research purposes 13
Functional Requirements Operational Functionality For the configuration user:
1. Ability to install the system along with all the peripheral software and initial set of HRTF datasets
2. User profiles managing: Support creation of user profiles, which store
the system settings optimized to the user preferences
Support the ability to view the settings stored in a user profile
Support the ability to modify and delete profiles Supply a set of predefined (default) profiles
used for initial system configuring Ability to initialize the system according to a
given user profile and switch between profiles14
Functional Requirements Operational Functionality (cont.) For the blind user:
1. Support an extensive training mechanism for:
3D sound perception Environment understanding
2. Support the following training types: Visualizing random shapes Visualizing pre-defined image files Fully immersive use of the system by emphasis
of some feature
For the researcher: Support defining a training experiment task Support recording of the task results and
retrieve them later 15
Non FunctionalRequirements
16
Non Functional Requirements Performance Constraints (partial) Speed requirements:
1. Response time:The system will produce a 3D sound according to a frame taken by the camera within 0.1 seconds at most (we will strive to 0.03 seconds – 30 fps)
2. Training Speed:• A simple training in order to reach 50% accuracy of
recognition should take no more than 30 minutes for a blind user .
• A blind user should pass at least 80% of the accuracy tests after 2 days of extensive system usage.
• A regular user should pass at least 80% of the accuracy tests after 3 days of extensive system usage.
17
Portability requirements:1. Currently the system is designed to be
deployed on Microsoft Windows (XP / Vista / 7 and later) operating systems only
2. The system will be compatible with 32 / 64 bit machines having web-camera and audio drivers installed
Capacity requirements:1. The system should work on machines with
at least 1 GB of RAM2. The system will support many different
OpenAL implementations and HRTF datasets, the limit is the hard disk capacity only
Non Functional Requirements (cont.) Performance Constraints (cont.)
18
Non Functional Requirements (cont.) Look, Feel and Use Constraints
User Interface requirements:1. The UI should be easy to use even for
users that are not well familiar to the computers technology
2. User interface will be in English
Documentation and Help:1. A extensive documentation will be
supported along with an installation guide
2. Operations will be implemented as wizards
3. Error messages heard via headphones19
Non Functional Requirements (cont.) SE Project + Platform Constraints
1. The application core and the UI will be written in C++ language using .NET 3.5 Framework and Visual Studio 10.0 IDE.
2. MATLAB will be used as a computational engine
3. During the development stage of the system a home-made simple device will be used (a PC web-camera strapped to a top of headphones)
4. For the demo and testing purposes, a real device will be supplied by DT labs which are spy-sunglasses (sunglasses with a tiny camera hidden in the nose bridge of glasses) 20
Usage Scenarios
21
Usage Scenarios Use Cases Diagram
22
Usage Scenarios (cont.) Use Cases: UC-1: Visualize Environment
A blind user starts the visualization process
23
Usage Scenarios (cont.) Use Cases: UC-2: Train
A blind user performs a training process
24
Usage Scenarios (cont.) Use Cases: UC-3: Choose a user profile
A blind user chooses an existing user profile for the purpose of performing a training or in order to use the system
25
Usage Scenarios (cont.) Use Cases: UC-4: Visualize Image
The core of the visualization process
26
Usage Scenarios (cont.) UC-4: Visualize Image
27
QUESTIONS?THANK YOU
!
28