Slide 1

Making Touchscreen Keyboards Adaptive to Keys, Hand Postures, and Individuals A Hierarchical Spatial Backoff Model Approach Ying Yin 1,2, Tom Ouyang 1, Kurt Partridge 1, and Shumin Zhai 1 1 Google Logo here 2 MIT Logo here Slide 2 Foundations to current methods Language modeling vocabulary 1-gram, 2-gram N gram frequencies Spatial models converting input touch points into probabilities of letters Edit distance correction assigning cost to insertion, deletion, and other spelling errors User and posture independent Slide 3 Research questions One promising area for improvement is by making them adapt to the user What types of adaption are possible? How do they affect performance? Slide 4 Contributions A novel hierarchical adaptive model Show benefits of posture and user adaptation Online posture classification method 13.2% reduction in character error rate compared to base model without language model Slide 5 Types of adaptation Individual differences (cf. Findlater & Wobbrock, 2012) Furthermore, people use different hand postures to type (cf. Azenkot and Zhai, 2012) Slide 6 Different typing postures: two thumbs, one finger, or one thumb Slide 7 Types of adaptation Different postures different touch patterns Touch patterns also depend on letter keys (Azenkot & Zhai, 2012) Need adaptation Slide 8 Challenges of adaptation Complexity three adaptive factors: key, posture, individual large number of submodels need sufficient data to build each submodel Model selection wrong selection may hurt keyboard quality uncertainty in posture classification Slide 9 Hierarchical spatial backoff model (SBM) Slide 10 Combinatorial and fine grained adaptation Conservative Does not require an extra training phase Updates the model continuously online Slide 11 Research method Pepper dataset (Azenkot & Zhai, 2012) 30 right-handed participants given random phrases to type between-subject: each person uses one posture 84,292 touch points in total 10-fold cross validation Slide 12 Comparison of spatial models Slide 13 Two-thumb One-finger Effective key areas Slide 14 Posture classification SVM-based classifier Based on correlation between time and distance between consecutive touch points no additional sensors required speed independent 86.4% accuracy Real-time Slide 15 Posture adaptation Slide 16 Individual adaptation Slide 17 Prototype implementation of SBM 13.2% reduction in character error rate compared to base model without language model Integrated with real keyboard combined with language model runs on Android phone in real-time Slide 18 Future work Weighted average of submodels instead of making binary decisions More data: real-use logging and game playing User studies validate the accuracy and speed improvement how users adapt their behavior to SBM Combine spatial and language models Slide 19 Contributions A novel hierarchical adaptive model Show benefits of posture and user adaptation Online posture classification method Opens up many more interesting HCI questions Slide 20 Q & A Slide 21 Slide 22 Prototype implementation of SBM Posture & key adaptation models supervised and batch learning Individual adaptation models unsupervised and online learning Backs-off to more basic models when posture estimation is uncertain (conf. < 0.94) there is insufficient user data (< 50 data points)