Eye Detection and Tracking: A Platform For Novel Computer Interfaces

Bryant Tan (arctanb@stanford.edu) Max Praglin (mpraglin@stanford.edu)

Our long­term goal for this project is to develop a robust eye­detection and tracking platform that would allow unique interfaces for a computer equipped with a webcam. For our EE368 end of quarter project, however, we plan to develop a MATLAB application that can perform the following tasks on a fixed high­resolution image with controlled lighting conditions:

1. Accurately recognize locations of important facial features, namely pupils/irises, nose, lips. 2. Estimate gaze angle based on the relative positions and shapes of the above facial

features. Our aim is to detect gaze with sufficient accuracy to be able to control a computer interface.

3. Recognize whether each eye is open. Once the core functionality above is accomplished, we would attempt to make improvements that will allow for a more robust and performant platform, including:

Low resolution images Uneven or low lighting Motion and Gaussian blur Obstructions, such as glasses Extracting information about user intent from gestures and blinking Operation on low­power computing platforms (which could enable this to run quickly on a

mobile device’s interface) Prior work (much of which is described in the below references) includes techniques such as:

Determining eye position relative to other facial features, thus estimating gaze. This could be a promising technique for us. [1]

Inspect the shape of the iris to estimate gaze angle based on the assumption that the iris underwent a rotational transform. This technique requires high resolution images that a webcam might not be able to provide. [2]

Use of the Hough transform to detect circular irises. [3] Use of a neural network to localize eyes, followed by a mean­shift algorithm to detect

gaze changes. [4]

Android Device? No; we think the applications for our software are better­suited to large displays with webcam­style cameras. References [1] A. C. Varchmin, R. Rae, and H. Ritter: “Image Based Recognition of Gaze Direction Using Adaptive Methods.” In AG Neuroinformatik, 1997. [Link] [2] J. G. Wang, E. Sung, R. Venkateswarlu: “Eye Gaze Estimation from a Single Image of One Eye.” In IEEE Computer Vision, 2003. [Link] [3] Y. Ito, W. Ohyama, T. Wakabayashi, and F. Kimura: “Detection of Eyes by Circular Hough Transform and Histogram of Gradient.” In 21st International Conference on Pattern Recognition. Tsukuba, Japan, 2012. [Link] [4] E. Y. im and S. K. Kang: “Eye Tracking Using Neural Network and Mean­Shift.” Springer­Verlag, Berlin, Heidelberg, 2006. [Link]