GymSkill - A Personal Trainer for Physical Exercises

Technische Universität München Fachgebiet Verteilte Multimodale Informationsverarbeitung Prof. Dr. Matthias Kranz

GymSkill: A Personal Trainer for Physical Exercises

Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr,

Matthias Kranz

Technische Universität München, Germany

Nils Hammerla, Patrick Olivier, Thomas Plötz

Newcastle University,

United Kingdom

March 22 PerCom 2012, Lugano, Switzerland

Outline

•  Motivation

•  Automatic Assessment of Physical Exercises

•  Case Study

•  Conclusion

04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 2

Motivation

•  Physical activity is required for healthy lifestyle •  Problem: people do not exercise enough

–  Lack of knowledge of correct exercise execution for fast improvement –  High level of long-term motivation needed

•  Advantages of a personal trainer: –  Continuous supervision and professional feedback –  Individualized advice and motivation –  Minimization of injury risk

•  Disadvantages of a personal trainer: –  Expensive –  Privacy

•  Scientific focus on –  Activity recognition –  Wearable sensors

•  Health and Fitness Systems –  Heart rate monitor, foot pod, GPS watch, …

•  No skill assessment

–  Fixed activity tracking (Wii balance board, Kinect) •  Gaming aspect •  Dedicated hardware

Related Work

GymSkill

•  Smartphone-based personal trainer •  Monitoring and assessment of physical exercises

–  Based on phone sensor data –  No need for additional sensors

Outline

•  Motivation

•  Personal Health and Fitness Systems

•  Case Study

•  Conclusion

GymSkill: Automatic Assessment of Physical Exercises

•  We look at rocker board exercises •  Example use case for equipment-based training •  Defined quality criteria:

tilt angles, speed, smoothness of movement, touching the ground… •  Phone is attached to the board •  During exercise: Basic situated feedback •  After exercise: Fine-grained analysis of exercise quality and reasons for

quality differences

move back and forth move left and right balance on the center

Skill Assessment Principle

Sensor Data Recording

Sensor Data Processing PCBA Analysis

Skill Assessment

Detailed Skill Report

Real-Time Feedback

Skill Level (Score)

Output Rendering

Log File

Smartphone Server“Cloud“

Simple Analysis

User Feedback

Direct Feedback While Training

•  Repetition count •  Visual feedback of board movement •  Warning when tilted too far •  Optional acoustic feedback

Post-Exercise Analysis

•  Performed for each exercise run (e.g.: 10 repetitions of tilting back and forth) •  Global Analysis

–  Smoothness and continuity of movement –  Global motion quality –  Usage of board‘s degrees of freedom

•  Local Analysis –  Identify „interesting portions“ of sensor data –  What is „interesting“?

In recurrent data, this means unusual data compared to the rest –  E.g. participant hesitates or gets stuck

Global Analysis

•  Estimation of motion axis (providing the dominant signal) •  Comparison of the empirical distribution

to ideal distribution function („gold standard“) •  Usage of normalized and un-normalized functions to determine

smoothness and utilization of board‘s degrees of freedom •  Transformation into a performance quality metric between 0 and 1

��

��! � "��!�

��#

��

��#

��

�� $�

��

%&��'��

Local Analysis

•  Assumption: sensor data of a movement shares (unknown) statistical properties

•  Unusual portions of a sequence violate this assumption and can be identified •  Extension to PCA: Principal Component Breakdown Analysis •  PCA model is learned from local neighborhood

(using sliding window technique) •  Frames are projected to lower-dimensional subspace using PCA •  Reconstruction errors used as a measure for motion quality •  Problem: ideal window size not known •  Solution: multi-scale comparison (iteratively growing window)

User Feedback

•  Visual feedback –  PCA-based assessment diagram

(red and yellow parts contain irregularities)

•  Textual feedback –  Based on global and local metrics –  Rule-based combination of aspects trigger

textual cues

•  “Thumb” feedback –  Overall assessment at a glance

Try to be more continuous in your motion!

Outline

•  Motivation

•  Case Study

•  Conclusion

Case Study

•  Set of 20 exercises developed by sports medicine specialist

•  6 participants, 5 days of training (20 different exercises twice a day)

•  1200 exercise records •  Goal 1: Collection of training data

–  Identification of criteria and assessment by physician

•  Goal 2: Qualitative evaluation of prototype –  Questionnaire study

Case Study: Trial Assessment Example 1

PCBA: Continuity

Time [s]5 10 15 20

!2 !1 0 1 20

displacement [std]

frequency

General motion

observedideal

!max 0 +max0

0.25Angle usage

displacement [°]

frequency

Try to be more continuous in your motion!

You touched the ground 3 times.

Your movement is not ideal.

! Move back and forth in a continuous motion.

Try to move similarly to both sides of the board.

! You do not utilise the full range of angles!

! You lean towards the front!

Case Study: Trial Assessment Example 2

PCBA: Continuity

Time [s]5 10 15

!2 !1 0 1 20

displacement [std]

frequency

General motion

observedideal

!max 0 +max0

0.2Angle usage

displacement [°]

frequency

Your movement is continuous, nice!

You did not touch the ground!

Overall you perform the movement nicely!

Try to move similarly to both sides of the board.

! You do not utilise the full range of angles!

! You lean towards the front!

Qualitative Feedback

Fully agree

Not agree at all

Qualitative Feedback

•  Feature wishlist

Fully agree

Not agree at all

Outline

•  Motivation

•  Case Study

•  Conclusion

Summary

•  Mobile skill assessment of overall exercise quality •  Identification of typical exercising errors •  “Personal trainer” •  Participant feedback indicates potential for long-term exercising motivation

Further work: •  Long-term study on training progress •  Assessment and feedback entirely on mobile phone •  Generalization of assessment model

GymSkill in Google Play

Thank you for your attention! Questions?

? ? 04.01.13 Andreas Möller et al. - GymSkill: A Personal Trainer for Physical Exercises 23

andreas.moeller@tum.de www.vmi.ei.tum.de/team/andreas-moeller.html

Paper Reference

•  Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2012/percom2012-preprint.pdf

•  Please cite this work as follows: •  Andreas Möller, Luis Roalter, Stefan Diewald, Johannes Scherr, Matthias

Kranz, Nils Hammerla, Patrick Olivier, Thomas Plötz GymSkill: A Personal Trainer for Physical Exercises In: 2012 IEEE International Conference on Pervasive Computing and Communications (PerCom2012), Lugano, Switzerland, March 2012, pp. 213-220

If you use BibTex, please use the following entry to cite this work:

@INPROCEEDINGS{6199869, author={M\"{o}ller, Andreas and Roalter, Luis and Diewald, Stefan and Scherr, Johannes and Kranz, Matthias and Hammerla, Nils and Olivier, Patrick and Pl\"{o}tz, Thomas}, booktitle={Pervasive Computing and Communications (PerCom), 2012 IEEE International Conference on}, title={GymSkill: A personal trainer for physical exercises}, year={2012}, month={march}, volume={}, number={}, pages={213 -220}, doi={10.1109/PerCom.2012.6199869}, ISSN={},}

Image Sources

•  Slide 5 –  gpsreview.net –  amazon.com –  spieleradar.de –  golem.de

GymSkill - A Personal Trainer for Physical Exercises

Health & Medicine

Aquatic Exercises BY: Dr. Gehan Shaalan Physical Therapy Lecturer

stretching exercises parc-ph - Physical Activity Resource Center for

4 Jump Rope Exercises Every Cross Trainer Should Know

Tower Trainer - Costco · The multifunctional Tower Trainer™ can be used for Pilates, dance and traditional fitness training, facilitating hundreds of exercises, from simple to

Workbook of Mental, Physical, and Spiritual Exercises

Research Paper Effect of Kyphosis Exercises on Physical ...salmandj.uswr.ac.ir/article-1-1150-fa.pdfKyphosis exercises including stretching exercises, strengthening exercises and spinal

Lower Extremity Physical Therapy Exercises

MHealthy Physical Activity: DESK EXERCISES › sites › default › files › desk-exercises_0.pdf · MHEALTHY PHYSICAL ACTIVITY: DESK EXERCISES STRENGTH TRAINING SAFELY • Frequency:

Corrective Exercises Post-Pregnancy Physical Issues Lisbeth Geertsen, CMT, CES

Mediflo Duo breathing trainer. Core Competences MEDIFLO duo breathing trainer Breathing trainer for inspiratory (SMI) and expiratory exercises (PEP) SMI

Why is this book called ‘Trainer’?€¦ · Why is this book called ‘Trainer’? This book is called ‘Trainer’ because it is full of exercises to help you get better and

Checklist for trainer in teambuilding exercises

Wim Langenbick...WIM LANGENBICK SPORTKINESITHERAPEUT –PHYSICAL TRAINER PHYSICAL TRAINER ACADEMY HOEVEEL UUR MOET JE LOPEN OM 1 CM VAN JE TAILLE AF TE HALEN? 1,8 KG VET KANS OP

GymSkill: A Personal Trainer for Physical Exercises · 2014-04-29 · practical case-study, in which 6 trainees over a period of 5 consecutive days, with different levels of experience

PART 5 – VASA TRAINER INDIVIDUAL EXERCISES Vasa Trainer Instruction Manual | PART 5 - VASA TRAINER INDIVIDUAL EXERCISES 81 V A S A GETTING ON AND OFF SAFELY Getting safely on and

A WORKOUT SET - Kompernass · 2019. 12. 17. · Abdomen/Back trainer exercises (KH 6003/4) 7 Flex band expander exercises (KH 6003/3) 14 Push-up handle exercises (KH 6003/1) 16 Exercise

The Course Exercises - IMO · 2016-01-07 · TRAIN THE TRAINER COURSE Energy Efficient Ship Operation Course Exercises Exercises - Page 3 Module 1 Exercises Exercise 1.1 – Shipping

Why is this book called ‘Trainer’? - Cambridge University Pressassets.cambridge.org/97811076/30529/excerpt/...This book is called ‘Trainer’ because it is full of exercises

Physical Therapy Exercises & Stretches for Acute Neck

Upper Extremity Physical Therapy Exercises