Transcript Kinect for FRC 2012

Kinect for FRC 2012
ALFRED THOMPSON
MICROSOFT ACADEMIC TEAM
HTTP://BIT.LY/ALFREDTH
Outline
2
 Introduction
 Kinect Hardware
 Kinect for Windows
 Skeleton, depth, image and audio data
 Kinect for FRC 2012
 Default FRC gestures
 Programming your own gestures
 Additional Resources
Kinect Hardware
3
IR Projector
RGB Camera
IR Camera
Multi-Array Mic
Motorized Tilt
Kinect and FRC 2012
4
 The Kinect has multiple powerful sensors for depth,
skeleton, video and audio input
 The Kinect is likely intended for robot
control via the drivers station for the 2012
FRC game
Kinect for Windows Architecture
5
Skeleton Data
Kinect Audio
7
 Four microphone array with hardware-based
audio processing
– Multichannel echo cancellation (MEC)
– Sound position tracking
– Other digital signal processing (noise suppression
and reduction)
Kinect Sensor Data
8
FIRST Kinect Architecture
9
FRC Driver Station
WPI Kinect Server
WPI Lib
(Kinect Enabled)
Kinect for Windows SDK
cRIO
Kinect
Driver Station
Robot
Joystick Position and Skeleton points
Kinect for FRC 2012
10
Kinect
SDK
FRC Kinect
Server (C#)
Driver Station PC
Raw Skeletal Data,
Diagnostic Data,
Default Gesture Data
(Kinect Sticks),
Optional Data
Dashboard
(Skeletal Display)
Driver Station
(Kinect Diags)
Default Gesture Data
(Kinect Sticks)
Raw Skeletal Data,
Diagnostic Data,
Optional Data
Robot Gesture
Code (LabVIEW,
C/C++ or Java)
Scenario 1
Scenario 2
Programming
11
 Teams using C# can change/modify the WPI Kinect
Server on the Driver Station
 Teams using LabView, Java or C++ can process
joystick AND skeleton point data on the cRIO robot
controller
Default FRC Gestures
12
Y= -1.00
Y= -1.00
90
70
Y=- 0.50
Y= 0.00
Y= -1.00 Y= -1.00
90
35
-90
Y=- 0.50
35
RIGHTight
R
0
-90
70
-45
LEFT
0 Y= 0.00
-45
RIGHT
0
LEFT
0
35
-35
Y= 0.50
Y= 0.50
-70
Y= 1.00
-90
Y= 1.00
Right KinectStick2 Y values
Map to Wrist/Shoulder Angle
-90
70
45
45
NAN / Zero
NAN / Zero
Y= 1.00
90
Y= 1.00
Left KinectStick1 Y values
Map to Wrist/Shoulder Angle
Facing Kinect
90
Fig 2 Wrist/hand to Elbow angle definition of Zero
Fig 1 Human Player facing Kinect Y joystick value / angle interpretation
Buttons:
1) Head to the Left
2) Head to the Right
3) Right Leg Out
4) Left Leg Out
5) Right Leg Forward
6) Right Leg Back
7) Left Leg Forward
8) Left Leg Back
9) Kinect Control “Enabled”
Control Considerations
13
 Robot control is achieved via skeletal position
 Gestures/poses relate to skeletal position and
inform which control is desired
 Consider your gestures carefully!
 Response times will be different with Kinect
compared to traditional joysticks
Using the FRC Gestures
14
KinectStick leftArm(1);
KinectStick rightArm(2);
Labview
C++
Java
KinectStick leftArm;
KinectStick rightArm;
…
leftArm = new KinectStick(1);
rightArm = new KinectStick(2);
…
leftArm.getY()
leftArm.getRawButton(#)
leftArm.getY()
leftArm.getRawButton(#)
Gesture Programming
15
 Teams may modify the Kinect Server code (C#) using
Visual C# Express.





Allows access to depth/RGB data
Easier to display feedback on dashboard
Keeps cRIO processing burden low
Gesture examples available from general Kinect development
community
Teams may use the raw skeleton data sent to the robot to process their
own gestures in their chosen language (LabVIEW, C/C++ or Java)

Language familiarity
Getting Help
16
1.
FRC Forums
2. Public Kinect for Windows Forums
3. Escalate to Microsoft via official FIRST contacts
Demo and Questions
17
 Go Teams, go!
Additional Resources
18
 Official Kinect for Windows Website
 FRC Beta Kinect Forum
 Kinect SDK Quickstart Video Tutorials
 Coding4Fun Kinect Toolkit
 Coding4Fun Kinect Projects
 Visual C# Express
 Kinect for Windows SDK Beta Forums
 Getting Started with Microsoft Kinect for FRC document – released at Kickoff
 Kinect Server Code Walkthough – released at Kickoff