Wrist Recognition and the Center of the Palm Estimation Based
on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual
Reality and Visualization (ICVRV), 2013 International Conference on
1
Slide 2
Outline Introduction Related work Proposed method Experimental
results Conclusion 2
Slide 3
Introduction 3
Slide 4
Problem: Can not separate a hand from a forearm using color and
depth information Solution: Find wrist to recognize hand 4
Slide 5
Related Work 5
Slide 6
Hand segmentation and extraction Color [11,12] Depth threshold
[13,14] The location of other body parts [15~17] Wrist Wear
wristband[14] Palm detection[18] 6
Slide 7
Reference [13] D. Uebersax, J. Gall, and M. Van den Bergh, and
L. Van Gool, Realtime sign language letter and word recognition
from depth data. International Conference on Computer Vision
Workshops (ICCV Workshops), 2011 [14] Z. Ren, J. Yuan, and Z.
Zhang, Robust hand gesture recognition based on finger- earth
mover's distance with a commodity depth camera. ACM international
conference on Multimedia, 2011 [15] T. I. Cerlinca and S. P.
Pentiuc, Robust 3D Hand Detection for Gestures Recognition. Proc.
the 5th International Symposium on Intelligent Distributed
Computing, Delft, 2012 [16] M. Van den Bergh and L. Van Gool,
Combining RGB and ToF cameras for real-time 3D hand gesture
interaction. Workshop on Applications of Computer Vision (WACV),
Kona, 2011 [17] K. Fujimura and L. Xia, Sign recognition using
depth image streams. Automatic Face and Gesture Recognition, 2006
[18] U. Lee and J. Tanaka, Hand Controller: Image Manipulation
Interface Using Fingertips and Palm Tracking with Kinect Depth
Data. Proc. of 10th Asia Pacific Conference on Computer Human
Interaction(APCHI), Matsue, 2012 7
Slide 8
Related Work Candescent NUI(Natural User Interface) project
Hand and finger tracking Develop by Stefan Stegmueller, Swiss Open
source: Use the OpenNI framework with the Kinect sensor
http://blog.candescent.ch/ http://candescentnui.codeplex.com/
Finger direction detection Blue : cluster centroid Green : palm
center Red : fingertips Yellow : hand contour Long lines : finger
directions A Robust Method of Detecting Hand Gestures Using Depth
Sensors, Yan Wen; Chuanyan Hu; Guanghui Yu; Changbo Wang, 2012 IEEE
International Workshop on HAVE
http://www.camdemy.com/media/11513http://www.camdemy.com/media/11513
8
Slide 9
Proposed Method 9
Slide 10
Hand segmentation and palm estimation Wrist recognition The
center of the palm estimation 10
Slide 11
Hand Segmentation and Palm Estimation (1/5) a. Cluster the hand
data K-means clustering algorithm Specify the depth range: 0.5~0.8m
11
Slide 12
Hand Segmentation and Palm Estimation (2/5) b. Compute the
Convex hull of the hands The Graham scan algorithm 12
Slide 13
Hand Segmentation and Palm Estimation (3/5) c. Detect the hand
contours Moor-Neighbor tracking algorithm 13
Slide 14
Hand Segmentation and Palm Estimation (4/5) d. Detect the
fingertips Find all candidate points that are both on the convex
hull and the contour The distance of P 0 and P> threshold
14
Slide 15
Hand Segmentation and Palm Estimation (5/5) e. Estimate the
center of the palm The biggest circle inside the hand contour
15
Slide 16
Wrist Recognition Wrist: pit points Find an obvious pit point
in the contour of hand Create an appropriate to find another wrist
point. inscribed rectangle 16
Slide 17
Wrist Recognition (1/4) Step 1: Find candidate lines of wrist
The ends of the candidate line should not be both fingertips. The
distance of the candidate line should not be less than a specific
value. 17
Slide 18
Wrist Recognition (2/4) Step 2: Find the corresponding
candidate contours whose ends are the ends of the candidate lines.
18
Slide 19
Wrist Recognition (3/4) Step 3: Find one of the wrist points
Calculate the maximum distance between the candidate line and the
corresponding candidate contour. The largest distance from these
maximum distances. The point with the largest distance is one of
the wrist points 19
Slide 20
Wrist Recognition (4/4) Step 4: Find another wrist point
Connect this wrist point to each point in the hand contour, and
take these connecting lines as the diagonals of rectangles. If the
rectangle is not inside the hand contour, the corresponding point
in the contour is not another wrist point. Find out the point with
the shortest rectangle diagonal as another wrist point. 20
Slide 21
Wrist Recognition Candidate lines Corresponding contour Find
one of the wrist points Find another wrist point 21
Slide 22
Estimating the Center of the Palm (1/4) Step 1: Select three
points from the hand contour The three points (P 1, P 2, P 3 ) form
an acute triangle. 22
Slide 23
Estimating the Center of the Palm (2/4) Step 2: Find
circumcenter O j of the triangle The O j coordinate The radius of
circle : 23
Slide 24
Step 3: Determine the center Calculate the distances from each
point in the hand contour to the center Formula. Condition A: The
number of distance R ji > R j is bigger than the threshold
Condition B: R j >minR (minR=> the minimum radius of the palm
) If A or B is not satisfied, Step 4 Estimating the Center of the
Palm (3/4) 24
Slide 25
Step 4: Find another appropriate palm center One end-point of
these two intersectant chords is replaced by point P min Repeat
step 2 to step 4 until the ending condition is true Estimating the
Center of the Palm (3/4) 25
Slide 26
Proposed Method Hand segmentation and palm estimation Wrist
recognition The center of the palm estimation Fingertips detection
26
Slide 27
Experimental Results 27
Slide 28
Experimental Results Device: AMD Athlon(tm)Formula Dual Core
Processor Formula CPU, 4GB RAM, NVIDIA GeForce 9600GT Graphics card
and Window7 32bit OS Threshold setting Depth: 0.5-0.8m Minimum
distance of line: 50 Minimum radius of the palm: 33 #hand contour
inside circle: 25~50 28
Slide 29
Experimental Results Before After 29
Slide 30
Experimental Results Divide into three groups based on the
number of the points inside the hand contour. #contour#inside
contour a6257085 b4705919 c5057335 d107411205 e135612362 f115513047
g140716716 h147817961 i102418990 30
Slide 31
Experimental Results 31
Slide 32
Experimental Results Improved original algorithm: every 8 th
point The new algorithm: proposed method 32
Slide 33
Conclusion 33
Slide 34
Conclusion Propose the wrist recognition algorithm to separate
the hand from the forearm, Propose a new algorithm of estimating
the center of the palm to reduce the computing time. Without Kinect
SDK 34