Utilizing Kinect Control for a More

Immersive Interaction with 3D

Environments

Supervised byDr. Noor Shaker Dr. Ammar Joukhadar Eng. Mohammad Shaker

Designed, Implemented and Tested by

Mohammed Saed Haj Ali Marah Halawa Kinda Tarboush

F.I.T.E of Damascus, Syria – AI Department 2014

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

“Why using a

stylus when God

gave us ten

fingers!”

Steve Jobs when introducing

the first iPhone, 2010

Good design looks great, yes,

Good design looks great, yes,

but

smell great and sound great?

Good design looks great, yes,

but

why shouldn't it also feel great,

smell great and sound great?

Good design looks great, yes,

but

why shouldn't it also feel great,

Jinsop Lee

“Design for all 5 senses”

The Concept of

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

The player mostly has only one method to interact with the game (most games has the same playing peripherals) like

joystick or mouse

Most games don’t let the player to create their own content\ worlds and share it with others.

For most games there’s no direct interaction between the game and the player (only one way from the player to the game)

For most games there’s no direct interaction between the game and the player (only one way from the player to the game)

A Game For You to Use

Your Senses

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

Sketching a Level

Sketch Your Level

Using Kinect

Interaction with the environment

Hand Gesture

Interaction with the environment

Voice Command

Our Game

Put here a picture or a short video of the game. Just so that people know what are you talking about

The Immersion Process

Player

Sketch

Generating

basic Level

Canvas

Allow Edit and

Generation of

Content

The Immersion Process

Player

Sketch

Generating

basic Level

Canvas

Allow Edit and

Generation of

Content

Hand Gesture

Interaction with

Dynamic

Hand Gesture

Interaction with

Static

Hand Gesture

Dynamic Hand Gesture

• Animated gestures

Static Hand Gesture

• Comparison with reference images

• Using machine learning

• Rule based system

Select

Rotate

Delete

Move

T he ge s tu re s

that we will use

Hand gesture in our application

Dynamic

Hand Gesture

Static

Hand Gesture

Hand gesture in our application

Dynamic Hand Gesture

Discrete Markov

Processes

Static Hand Gesture

Rule based

system

Hand gesture in our application

Interaction via Voice Commands

Voice Command

• Audio to text conversion

Voice Command

• Speech Recognition Hidden Markov Model

Voice Command

• Words Meaning Conceptual Grammar

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

Our ApplicationGenerate 3D environment and edit it using most of your senses!

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

Future Perspective

• Enhance the social aspect of the game

• Multiple player playing at the same time at the same level

• Assess the game immersions metrics

• Allow for more immersive and enhanced gestures

• Analyze the applicability of the approach on other hardware (like Oculus Rift)

• Analyze the applicability of the approach on other game genres

Content

• Motivation

• The Problem with Today’s Games

• Modeling

• Sketching a Level

• Generating the Basic Level Structure

• Allow Edit and Creation on Content through in-game Interactions

• Hand Gesture

• Voice Command

• Our Application

• Future Perspectives

• Demo

Thanx for listening!