1. 1. Laplacian Vision: Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays and Projectors
2. 2. Photo: Steve-65 Nave Physics (Hayes et al. 1978): our untrained perception of basic physical phenomena
3. 3. In daily life Photo: AEQWF
4. 4. Nave physics is, however, imperfect http://www.failepicfail.com/
5. 5. In Science Fiction, a precog sees his short-term future Next, Nicholas Cage, TM & Paramount (2007)
6. 6. or brain implants enable you to calculate it Jump 225 Trilogy, David Louis Edelman, 2006-2010 Wizards brain, Reiichi Saegusa, 2001-present
7. 7. Laplace's demon: a causal determinism If the Demon knows the precise location and momentum of every atom in the universe, Essai philosophique sur les probabilits, Pierre-Simon Laplace, 1814 their past and future values for any given time are entailed; they can be calculated from the laws of classical mechanics.
8. 8. Laplacian Vision: a vision augmentation system to assist motion prediction. Optical See-Through HMDShot directly through the system
9. 9. Related work:Aftermath(AR + physics prediction) Leigh & Maes CHI15
10. 10. Related work: projection mapping Koike & Yamaguchi, AH15 Projector Charette et al., ICCP12 Custom DLP projector Okumura et al., ICME12 Saccade Mirror
11. 11. Related work: human vision enhancement Orlosky et al. TVCG15 Field of view Itoh & Klinker AH15 Eyesight X-ray vision Avey et al. VR09
12. 12. Our goal is to assist our nave physics ability by augmenting our vision with an Optical See-Through HMD
13. 13. We built a proof-of-concept system, and track a flying ball Tracking system User OST-HMD Virtual Image
14. 14. We built a proof-of-concept system, and track a flying ball Tracking system
15. 15. Projection mapping add-on by a projector
16. 16. Projection mapping add-on by a projector
17. 17. Our system estimates the future path of a flying ball in real time,
18. 18. then visualizes an estimated path on the OST-HMD
19. 19. Some views from the system when placed different positions
20. 20. Some views from the system when placed different positions
21. 21. Summary Assisting our nave physics skill Optical See-Through HMDs with a real-time AR rendering Simulating the physical behavior of the users environment
22. 22. x Physics Human x OST-HMD Laplacian Vision: Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays and Projectors
23. 23. Appendix
24. 24. Example 1
25. 25. Example 2
26. 26. Example 3
27. 27. User study: a fall-point prediction game With Laplacian vision Without (i.e. the nave vision) 29 subjects * 5 launches * with/without
28. 28. 80% 90% Impact Video Length conditions 29 subjects * 5 launches * with/without * 3 lengths
29. 29. 0 9045 View Angle conditions 29 subjects * 5 launches * with/without * 3 lengths * 3 angles
30. 30. Result: 3x accuracy improvement 0 9045 Birds-eye view
31. 31. Proposed Baseline 45 0 90 80% 90% 100% View Angle Video Length ** ** ** ** ** ** ** **
32. 32. IR image (60Hz) Tracking (2x ARTTRACK2) 3D position of the ball (60Hz) Target ball System B Prediction & Rendering (Unity 3D) Real world System A OST-HMD (nVisor ST60) ~19 ms latency ~80 ms latency (User-view camera) Observer Displayed image (60Hz) Rendered image (60Hz) Latency diagram
33. 33. Forward prediction 100 ms forward prediction No prediction Real ball carrying
34. 34. HMD Display Calibration World Eye (a user-view camera)
35. 35. Single Point Active Alignment Method (SPAAM) [Tuceryan 00] [Genc 02] Align a 3D point in the scene to 2D screen points A.R.T tracking system
36. 36. Hardware