Gaze Prediction for Recommender Systems

Qian Zhao, Shuo Chang, F. Max Harper, Joseph A. Konstan

1

Why gaze?

2

3

Why gaze: users are thinking a lot!

Go beyond normally logged ratings and actions

Understand what users are thinking Choice/decision making research Cognitive modeling

4

Why gaze?

Understand user inaction Do users see the displayed items?

Problem with machine learning assumptions Are positive training instances really

positive? Are negative training instances really

negative?

5

However

Eye tracking is not widely used (and in the near future)

Eye tracking may not be widely used.

6

Lets model and predict gaze

7

Aggregated Fixation Prediction

Consider browsing one page in a grid-based interface (r rows * c columns)

Aggregating entire page browsing, predict each displayed items fixation probability fixation time

Given user browsing data item positions, page dwell time, user

actions (top-down vs. bottom-up)

8

Two scenarios

Training models only based user browsing data

Training models with both user browsing data eye tracking data from a small number of

users Evaluation Extrapolation (across users)

9

Data Sets - MovieLens

November 2015, user browsing data 102K page views

17 subjects eye tracking data, each recorded for ~30 mins 452 page views, 10K data points Tasks: free using, rating, finding movies

etc.

10

Building Linear Models

Logistic regression for fixation probability

Hurdle linear models for fixation time Features

Position: row index and column index Dwell time 1/minActionDist

11

Building HMM

12

Fixation (latent or observable): F r * c possible values

Action (latent or observable): A r * c + 1 possible values

Building HMM

13

Estimation with eye tracking data (MLE) with only browsing data (EM or Appr.)

Prediction based on posterior of F

Evaluation

Randomly pick 20% of the 17 subjects for testing, others for training

Repeat for 100 times but always using a different set of testing subjects (independence)

14

Fixation Probability AUC

Collecting eye tracking data greatly

helps and it extrapolates across users. HMM is better than linear models

15

Action stats Linear models HMM Training with only browsing

data 0.580 N.A. 0.693

Training with eye tracking and

browsing data - 0.757 0.823

Fixation Time MAE

16

Collecting eye tracking data significantly helps.

Hurdle linear model is better than HMM R-squared: 21%

Action stats Linear models HMM Training with only browsing

data 0.466 N.A. 0.520

Training with eye tracking and

browsing data - 0.332 0.488

F-pattern (vs. center effect)

17

Messages from this talk

Gaze prediction extrapolates across users! Collecting eye tracking data from a small

number of users greatly help. Applying the right models makes a

significant difference. HMMs for fixation probability. Hurdle linear models for fixation time.

F-pattern instead of center effect

18

Thanks! Questions?

Title: Gaze Prediction for Recommender Systems See the paper for more results on other HMM

models and prediction for different user tasks!

Authors: Qian Zhao, Shuo Chang, F. Max Harper, Joseph A. Konstan

Contact zhaox331@umn.edu http://www-users.cs.umn.edu/~qian/

19

mailto:zhaox331@umn.edu

Gaze Prediction for Recommender SystemsWhy gaze? 3Why gaze: users are thinking a lot!Why gaze?HoweverLets model and predict gazeAggregated Fixation PredictionTwo scenariosData Sets - MovieLensBuilding Linear ModelsBuilding HMMBuilding HMMEvaluationFixation Probability AUCFixation Time MAEF-pattern (vs. center effect)Messages from this talkThanks! Questions?