Programming by Voice with Scratch:Teaching the Cat to Obey a Bird
Ramaraju Rudraraju, Srinivasa Datla, Avishek Banerjee, Mandar Sudame Univ. of Alabama at Birmingham, Department of Computer and Information Sciences
Jeff Gray University of Alabama, Department of Computer Science
Gary Edwards United Cerebral Palsy of Greater Birmingham
This research sponsored by a Google Research Award
Overview of Presentation
Scratch Script Video
Motivation
Architecture & Working of Myna
Dynamic Grammars
Myna Features
Scratch Script
Get your popcorn, its Movie Time!!
The WIMP Metaphor
Motivation
Window
Menu
Icon
s
Pointing Device
Motivation
The WIMP assumes dexterity
The MYNA Solution for Scratch
Drag and Drop Navigation Continuous Navigation Navigation using Transparent Frames
Features
Types of Navigation
7
Features
(An extension of Transparent Frames)
Drop After Drop In Drop Before
Macro Commands
Myna Architecture
ComponentMappings
Sprite State Storage
Component Hierarchy
Scratch UI
Transparent Frames
Speech Recognizer
Command Executor
Grammar
Java Robot
MYNA
Model
View Controller
Cloud Garden
Grammar
Speech Recognizer
Under the Hood of Myna
Speech Recognizer
GrammarCommand Executor
MappingsMappings
Move
Right Controller Model
Command Executor
Java RobotJava
RobotComponent Hierarchy
Component Hierarchy
Dynamic Grammar
Default Grammar
Navigation Grammar
Speech Pause
Grammar
File Grammar
File Grammar
Control Grammar
Looks Grammar
Default Grammar
Navigation Grammar
FilePauseResume
Limitations Scratch needs to be in maximized mode Not compatible with different screen resolutions Myna is not scroll-enabled Large vocabulary and still growing!
Future Work Integration with UCP - Deep human subjects testing with
United Cerebral Palsy of Greater Birmingham Portability on other platforms Configurable screen resolutions
Looking Ahead
Myna is our preliminary attempt towards voice-driven programming applied to Initial Learning Environments (ILEs) to overcome the physical challenges of using the traditional WIMP metaphor
Our long-term goal is to assist children with disabilities by giving them a learning tool that enables them to encounter the joys and excitement of programming, while also improving their logic skills
Summary
Thank you
Questions and
comments?
Additional questions?teammyna@googlegroups.
com
https://sites.google.com/site/projectmyna2010/