Upload
ashley-heath
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
ALFRED THOMPSONMICROSOFT ACADEMIC TEAM
HTTP://BIT.LY/ALFREDTH
Kinect for FRC 2012
Outline
2
IntroductionKinect HardwareKinect for WindowsSkeleton, depth, image and audio dataKinect for FRC 2012Default FRC gesturesProgramming your own gesturesAdditional Resources
Kinect Hardware
3
Multi-Array Mic Motorized Tilt
IR Projector
RGB Camera
IR Camera
Kinect and FRC 2012
The Kinect has multiple powerful sensors for depth, skeleton, video and audio input
The Kinect is likely intended for robot control via the drivers station for the 2012 FRC game
4
Kinect for Windows Architecture5
Skeleton Data
Kinect Audio
Four microphone array with hardware-based audio processing– Multichannel echo cancellation (MEC)– Sound position tracking– Other digital signal processing (noise
suppression and reduction)
7
Kinect Sensor Data8
FIRST Kinect Architecture
Driver Station Robot
FRC Driver Station
Kinect for Windows SDKKinect for Windows SDK
WPI Kinect Server
cRIO
WPI Lib (Kinect
Enabled)
KinectKinect
Joystick Position and Skeleton points
9
Kinect for FRC 2012
10
Driver Station PCKinect SDK
FRC Kinect Server (C#)
Driver Station (Kinect Diags)
Dashboard(Skeletal Display)
Robot Gesture Code (LabVIEW, C/C++ or Java)
Scenario 1 Scenario 2
Raw Skeletal Data, Diagnostic Data,
Default Gesture Data (Kinect Sticks),Optional Data
Default Gesture Data (Kinect Sticks)
Raw Skeletal Data, Diagnostic Data,
Optional Data
Programming
Teams using C# can change/modify the WPI Kinect Server on the Driver Station
Teams using LabView, Java or C++ can process joystick AND skeleton point data on the cRIO robot controller
11
Default FRC Gestures
12
90
-90
0
RIGHT
Right KinectStick2 Y valuesMap to Wrist/Shoulder Angle
Left KinectStick1 Y valuesMap to Wrist/Shoulder Angle
Y= 1.00
Y= -1.00
Y= 0.00
Y= 0.50
Y=- 0.50
Fig 1 Human Player facing Kinect Y joystick value / angle interpretation
90
-90
0
Y= 1.00
Y= -1.00
Y= 0.00
Y= 0.50
Y=- 0.50
LEFT
70
Y= -1.00
70
Y= -1.00
70
Y= 1.00
35
-35
35
35
-70
Y= 1.0090
-90
0
-45
45NAN / Zero
90
-90
0
-45
45
RIGHT LEFT
NAN / Zero
Facing Kinect
Fig 2 Wrist/hand to Elbow angle definition of Zero
Buttons:1) Head to the Left2) Head to the Right3) Right Leg Out
4) Left Leg Out5) Right Leg Forward6) Right Leg Back
7) Left Leg Forward8) Left Leg Back9) Kinect Control “Enabled”
Control Considerations
Robot control is achieved via skeletal position
Gestures/poses relate to skeletal position and inform which control is desired
Consider your gestures carefully!
Response times will be different with Kinect compared to traditional joysticks
13
Using the FRC Gestures
14
Labview
KinectStick leftArm(1);KinectStick rightArm(2);
leftArm.getY()leftArm.getRawButton(#)
C++
KinectStick leftArm;KinectStick rightArm;…leftArm = new KinectStick(1);rightArm = new KinectStick(2);…leftArm.getY()leftArm.getRawButton(#)
Java
Gesture Programming
15
Teams may modify the Kinect Server code (C#) using Visual C# Express. Allows access to depth/RGB data Easier to display feedback on dashboard Keeps cRIO processing burden low Gesture examples available from general Kinect
development community Teams may use the raw skeleton data sent to the robot to
process their own gestures in their chosen language (LabVIEW, C/C++ or Java)
Language familiarity
Getting Help
1. FRC Forums
2. Public Kinect for Windows Forums
3. Escalate to Microsoft via official FIRST contacts
16
Demo and Questions
Go Teams, go!
17
Additional Resources
18
Official Kinect for Windows Website FRC Beta Kinect Forum
Kinect SDK Quickstart Video Tutorials Coding4Fun Kinect Toolkit Coding4Fun Kinect Projects Visual C# Express Kinect for Windows SDK Beta Forums
Getting Started with Microsoft Kinect for FRC document – released at Kickoff
Kinect Server Code Walkthough – released at Kickoff