Embed Size (px)
Citation preview
© 2003 Gina Joue & Brian Duffy
A Flexible Framework for All
Socially engage each other on an equal footing by encouraging and facilitating minority groups to seamlessly integrate with everyone through universal design and a modular peripherals approach
The Multimodal Approach
Flexible integration between sensor modalities:
Fusion
Flexible integration of actuator modalities:
Fission
© 2003 Gina Joue & Brian Duffy
Multimodal Fusion
TypingHandwriting
Pushing and clickingGloved hand gestures
Speaking
Body movementHead movement
Free hand gesturesFacial expression
Eye movement
Hand pressure
Muscle activity
Body stress levels
Brain activity
Position / Motion
Audio
Video
Tactile / Force
Skin Conductance
Neural
Voltage
keyboardtabletmouseglove
microphone
camera
haptic device
EEG
EMG
GSR
© 2003 Gina Joue & Brian Duffy
Plug′n PLAY
Flexible Hardware ::
Social Interfaces ::
Play with Reality ::
Active Chair
Robotics
Virtual &Augmented
Reality
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Design
Air System3 pneumatic cylinders Variable air speed controllersPressure controllersSolenoid valves
Control SystemUSB system using a PIC controller (PIC16F87x) with Serial Communications Interface in full duplex mode (asynchronous) & a USB interface circuit
DesignHolds the occupant in a minimal-stress posture between the neutral body posture of zero gravitation and a normal posture of relaxation on a flat, level surface (based on NASA experimentation).
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Control
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Vicarious Adrenaline
Can you really know how I feel?
… where the OBSERVER
becomes the PARTICIPANT
OBJECTIVE: the transference of adrenaline-based motor sport experiences to an observer through a complex inter-modal interaction.
© 2003 Gina Joue & Brian Duffy
The Active Chair :: Vicarious Adrenaline
“I used the term ‘rodeo-chair’. It is embedded in therapeutic connotations that are attached to the medical viewpoint of disability. However, the haptic
chair uses a reality that involves a playful atmosphere. This maze allows the mind to be in tune with an exterior force. It can be set in a virtual
environment or ultimately in a social context. If the body is paralysed, it does not mean it has to curl up. It needs exercise.
The mind is in the driving seat.” JAMES BROSNAN
© 2003 Gina Joue & Brian Duffy
Immersive Play for All :: TRUST
Installed at the KK Women & Children’s Hospital in Singapore in 2005
© 2003 Gina Joue & Brian Duffy
Social Interfaces :: JoeRobot
OBJECTIVE: To develop a robust modular robot for explicit social interaction between a robot and one or more other robots, virtual avatars or humans
The ideal is a balance between developing a system that can realistically meet the expectations of the people it interacts with while still embracing its machine-like qualities that make it ultimately useful
© 2003 Gina Joue & Brian Duffy
Social Interfaces :: JoeRobot
Articulation When it moves it comes “alive”
Perception Vision: Colour tracking, feature recognitionLocalisationBiosensors
Communication Speech recognition & synthesisGestures
ControlModular code structureBehaviour librariesVision algorithmsCommunication protocolsSoftware interfaces (presentation) Embedded systems (e.g. speech)
© 2003 Gina Joue & Brian Duffy
Play with Reality :: Seamless Worlds
Seamless integration of realities
Judicious synthesis between the real & the virtual
Manage how Virtual & Augmented Reality can become a powerful medium in our social play
fuse the REALGAME
© 2003 Gina Joue & Brian Duffy
Make it real …
A strong need to develop flexible modular designs to promote real interaction regardless of abilities, age, or situation.
SENSE – FEEL –EXPERIENCE
The integration of multi-sensory stimulation creates a highly immersive environment that helps facilitate motor control rehabilitation.
“… allows the mind to be in tune with an exterior force.”
James Brosnan
