Connect with your

Wow!

Motorized TiltRGB Camera

3D Depth SensorsMicrophone Array

openkinect.org

www.openni.org

kinectforwindows.org

Kinect for Windows

That’s not USB

You’ll need this

Xbox Kinect

What you Need

SDK

Demo 0Hello Tilt

BGR32 Format

width in pixels

heig

ht in

pix

els

Stride = Bytes Per Row

= bytes per pixel * width = 4 * width

4 Bytes Per pixel

Byte Array:

Demo 1Hello Colorful World

Depth data500mm 500mm500mm 500mm

2000mm2030mm2010mm2000mm

500mm500mm500mm2500mm

Depth dataMax resolution for raw depth data is 640 x 480

2 bytes per pixel of depth data (short)

11011110 pixel pixel pixel

00001 1110101000000 1110101000001

depthInMM = pixel >> DepthImageFrame.PlayerIndexBitmaskWidth

000 000 000

Demo 2Depth

Depth + Player Data500mm 500mm500mm 500mm

2000mm2030mm2010mm2000mm

500mm500mm500mm2500mm

player 1

player 1

player 1 player 1player 1

player 1 player 1

SkeletonStream.Enable();

var player = pixel & DepthImageFrame.PlayerIndexBitmask; 11011110

00001010

Demo 3Depth + Player Data

Skeletal Tracking

skeletonFrame.Skeletons                                where skeleton.TrackingState == SkeletonTrackingState.Tracked

skeletonFrame.Skeletons

from skeleton in skeletonFrame.Skeletons                where skeleton.TrackingState == SkeletonTrackingState.PositionOnly

hand_left

wrist_left

elbow_left

shoulder_left

hand_right

wrist_right

elbow_right

shoulder_right

hip_left

head

hip_right

knee_left knee_right

ankle_rightankle_left

foot_left foot_right

hip center

spine

shoulder_center

Skeleton.Joints[JointType]Joint.Position (X,Y,Z) Skeletons are mirroredX, Y, Z in metersOrigin(0,0,0) = center of camera.

+Z is out +X is to the right+Y is up

Demo 3Skeletal Tracking

Composites

+Color +

CompositesColor

Color

+

+

+

MapDepthFrameToColorFrameMapDepthToColorImagePoint

MapDepthToSkeletonPoint

MapSkeletonPointToColor

MapSkeletonPointToDepth

Demo 4Green Screen

Kinect Audio

Echo Cancellation

Echo CancellationEchoCancellationMode: None, CancellationOnly, CancellationAndSuppression

int EchoCancellationSpeakerIndex: Index of speaker to cancel

Noise Suppression

Noise Supressionbool NoiseSuppression – true, false

Automatic Gain Control

Automatic Gain Control

bool AutomaticGainControl – true, false

Beam Forming

Beam Forming-0.875 radians-50 degrees

0.875 radians 50 degrees

.0175 radians

10 degrees

Demo 5Speech Recognition

Audio @!%!#!! Gotchas

Use Microsoft.Speech not System.Speech

Must run in an MTA

AEC SystemModes crash if no speaker output.

Must plug Kinect into Wall

DevicePairGuid / Qualitymetrics = crash

Should run in ThreadMode Priority Highest?

Demo 6Beam Forming

Kinect SDK 1.5!Seated Skeletal Tracking

Face Tracking Joint Rotation

Kinect Studio

Creative Commons Acknowledgments• Flickr by Ian Lewis• Flickr by major nelson• Flickr by markodesign• Flickr by stephantom• Flickr by Abstract Machine• Flickr By Aixo.co.kr• Flickr By Andresmh• Flickr by Antony Bennison• Flickr by Ars Electronica• Flickr By bm.iphone• Flickr by campuspartymexico• Flickr by Chris Jagers• Flickr By ci_polla Paola Sucato• Flickr by creepyhalloweenimages• Flickr by cvander• Flickr by IMAL.org• Flickr by Institut Douglas• Flickr by Joshua and AmberFlickr by Logon Sakai• Flickr by openexhibits• Flickr by Paul_Clarke

• Flickr by petyr.rahl• Flickr by pt• Flickr by Stilldavid• Flickr by y3rdua• Flickr by yesyesnono• Flickr by yum9me• Flickr by Jeff Howard• Flickr by jhscreenwm• Flickr by Kyle McDonald• Flickr by Morten Rand-Hendriksen• Flickr by Stuart Herbert• Flickr by Visiophone• Flickr by orkomedix• Michael Zöllner and Stephan Huber at the University of Konstanz• Openclip by J_Alves• Openclip by jetxee• Openclip by molumen• Openclip by shokunin• Openclip by Jhnri• Openclip by Machovka• Adafruit image of Hector MartinThanks!!