Upload
ryley-patchin
View
222
Download
3
Embed Size (px)
Citation preview
HTML5 Haptics Mohamed Mansour 2
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 3
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 4
Vision
• Haptics Integration into HTML5• Rich JavaScript API to communicate to Haptics• Cross platform • Haptic device independent• Standardized in W3C
HTML5 Haptics Mohamed Mansour 5
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 6
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 7
HTML51991 HTML
1994 HTML 2
1996 CSS 1 JavaScript
1997 HTML 4
1998 CSS 2
2000 XHTML
2002 Tableless
2005 AJAX
2009 HTML 5
INNOVATION
HTML5 Haptics Mohamed Mansour 8
What is HTML5?
HTML5 Haptics Mohamed Mansour 9
HTML5 New features
• Storage• Application Cache• Web Workers• Web Sockets• Notifications• Drag and Drop• Geolocation• Semantic Tags• Link Relations, and Microdata• ARIA
• New form fields• Audio and Video• Canvas and SVG• Font Text Support, • Opacity, Rounded Corners• Shadows, Gradients• Backgrounds, Transitions• Transformations, Animations• Device• A lot more!
HTML5 Haptics Mohamed Mansour 10
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 11
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 12
Haptics
• Technology that interfaces with the user through the sense of touch [definition]
• The process of recognizing objects through touch [another definition]
HTML5 Haptics Mohamed Mansour 13
Haptic Devices
• Omni
• Novint
• Gloves
• Wrist band
• Jacket
• Way more …
HTML5 Haptics Mohamed Mansour 14
How they work?
• Robotic• Sensors• Feedback
• Ex:– You send forces to the device, and it magically
moves. – You request positional data from the device.
HTML5 Haptics Mohamed Mansour 15
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 16
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 17
Plugins
• Browser plugins is the only way you can allow the user to communicate to native code.
• Current state of HTML is still limited … look at Flash, why did it exist?
• Can do almost anything!• But very very very Dangerous!
HTML5 Haptics Mohamed Mansour 18
Plugins
• Active X– Microsoft Internet Explorer
• NPAPI (very low-level)– Google Chrome– Firefox– Opera– Safari
HTML5 Haptics Mohamed Mansour 19
NPAPI
• Very long name …. Netscape Plugin Application Programming Interface.
• Very powerful!• It is just a single library (.dll, .so, .plugin) file
that gets loaded.• You just need at least 3 Library Exports:
– NP_GetEntryPoints– NP_Initialize– NP_Shutdown
HTML5 Haptics Mohamed Mansour 20
NPAPI Architecture
<embed>
Scriptableto
NPObject*
Plugin
DLL EXPORTS
LOGIC
Browser APINPN_*
5
4
1
2
3
HTML5 Haptics Mohamed Mansour 21
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 22
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 23
Google Chrome
• Multi Process Browser– Tab, Plugins, Renders, Extensions, Browser
• Very fast JavaScript and Rendering• GPU Accelerated Canvas and WebGL• Secure Sandbox, for Plugins• Very fast at implementing HTML5 features.• Open Source browser
HTML5 Haptics Mohamed Mansour 24
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 25
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 26
Google Chrome Extensions
• Small software that can modify and enhance the functionality of Google Chrome.
• Using web technologies, HTML, JavaScript, and CSS.
• Easily expose NPAPI functionality.• Separate Process
HTML5 Haptics Mohamed Mansour 27
Chrome Extension Basics
• Background page– Lengthy process, long lived, singleton.
• Browser Actions / Page Actions• Desktop Notifications• Override Pages• Browser Interaction (Bookmarks, Cookies,
History, Events, Management, Windows, Tabs)• Plugin Interaction.
HTML5 Haptics Mohamed Mansour 28
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 29
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 30
Prototype
• HTML5 Device Tag <device>
• Interface
• Type
HTML5 Haptics Mohamed Mansour 31
Device Tag
• Still in implementation (research phase)• It can change! Add a Haptics “type”Dreamy Ex:<p> To start, Select a haptic device: <device type=haptic onchange="update(this.data)"></p><script> function update(stream) { console.log(stream.data); } </script>
HTML5 Haptics Mohamed Mansour 32
Prototype , Needed?
• Proof of concept needed to prove that Haptics needs love! And the future of computer human interactions!
• Rescue …
HTML5 Haptics Mohamed Mansour 33
Prototype, Chrome Extension
• How do we do it right now!– HTML5 (Web Workers)– CSS3, Canvas (WebGL Context Graphics)– Chrome Extension
• Dedicated Thread (Background Page)• Haptics Controller (Browser Action)• Page Interactions (Page Action)• Accessibility (ARIA)
– NPAPI • Haptics Plugin communicates to Chrome Extension• C++ extreme (scene graph, triangles) collision detection algorithms
HTML5 Haptics Mohamed Mansour 34
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 35
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 36
Architecture
• High Level
HTML5
C Plus Plus
Novint Device
HTML5 Haptics Mohamed Mansour 37
Architecture Plugin
Plugin Module
Scripting
Bridge
Plugin Gate
Utils
Haptics Service
Haptics Device
Haptics Signal
NP_GetEntryPoints
NP_Initialize
NP_Shutdown
HTML5 Haptics Mohamed Mansour 38
Glimpse of Plugin Code
HTML5 Haptics Mohamed Mansour 39
Architecture, Extension
• Browser Action to act as the controller
• Background Page, to allow the plugin to live forever.
• JavaScript for Some Collision Detection, Haptic Loop, and Rendering Loop.
HTML5 Haptics Mohamed Mansour 40
Framework
• O3D JavaScript API for creating rich and interactive 3D application in the broswer using Canvas, WebGL context.
• Haptics API– void startDevice();– void stopDevice();– void sendForce(double[3]);– double[3] position;– addEventListener(‘servo’, function(Haptic data));
HTML5 Haptics Mohamed Mansour 41
Open Source’dRelease it to the public! That is how I learned how to code, give back to the community! You can fork it from GitHub from here:http://github.com/mohamedmansour/haptics-chrome-extension
HTML5 Haptics Mohamed Mansour 42
Working Prototype
HTML5 Haptics Mohamed Mansour 43
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 44
HTML5 Haptics Standardization
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7
HTML5 Haptics Mohamed Mansour 45
Future
• Present this to the W3C Developer Group• Help Google Chrome complete <device> tag
implementation.• Implement the Prototype again using the
<device> tag.• Implement webpage interactions, like
annotated YouTube Videos to feel video touch.
HTML5 Haptics Mohamed Mansour 46
Done! Thank you, questions?
Vision HTML5 Haptics
Plugins Google Chrome Extensions
Prototype Architecture Future
1 2 3
4 5
6 7