HTML5 Haptics 1 HTML5 Haptics Standardization Mohamed Mansour hello@mohamedmansour.com Haptics...

HTML5 Haptics 1

HTML5 Haptics Standardization

Mohamed Mansourhello@mohamedmansour.com

HTML5 Haptics Mohamed Mansour 2

Vision HTML5 Haptics

Plugins Google Chrome Extensions

Prototype Architecture Future

Vision

• Haptics Integration into HTML5• Rich JavaScript API to communicate to Haptics• Cross platform • Haptic device independent• Standardized in W3C

HTML51991 HTML

1994 HTML 2

1996 CSS 1 JavaScript

1997 HTML 4

1998 CSS 2

2000 XHTML

2002 Tableless

2005 AJAX

2009 HTML 5

INNOVATION

What is HTML5?

HTML5 New features

• Storage• Application Cache• Web Workers• Web Sockets• Notifications• Drag and Drop• Geolocation• Semantic Tags• Link Relations, and Microdata• ARIA

• New form fields• Audio and Video• Canvas and SVG• Font Text Support, • Opacity, Rounded Corners• Shadows, Gradients• Backgrounds, Transitions• Transformations, Animations• Device• A lot more!

Haptics

• Technology that interfaces with the user through the sense of touch [definition]

• The process of recognizing objects through touch [another definition]

Haptic Devices

• Omni

• Novint

• Gloves

• Wrist band

• Jacket

• Way more …

How they work?

• Robotic• Sensors• Feedback

• Ex:– You send forces to the device, and it magically

moves. – You request positional data from the device.

Plugins

• Browser plugins is the only way you can allow the user to communicate to native code.

• Current state of HTML is still limited … look at Flash, why did it exist?

• Can do almost anything!• But very very very Dangerous!

Plugins

• Active X– Microsoft Internet Explorer

• NPAPI (very low-level)– Google Chrome– Firefox– Opera– Safari

• Very long name …. Netscape Plugin Application Programming Interface.

• Very powerful!• It is just a single library (.dll, .so, .plugin) file

that gets loaded.• You just need at least 3 Library Exports:

– NP_GetEntryPoints– NP_Initialize– NP_Shutdown

NPAPI Architecture

<embed>

Scriptableto

NPObject*

Plugin

DLL EXPORTS

Browser APINPN_*

Google Chrome

• Multi Process Browser– Tab, Plugins, Renders, Extensions, Browser

• Very fast JavaScript and Rendering• GPU Accelerated Canvas and WebGL• Secure Sandbox, for Plugins• Very fast at implementing HTML5 features.• Open Source browser

Google Chrome Extensions

• Small software that can modify and enhance the functionality of Google Chrome.

• Using web technologies, HTML, JavaScript, and CSS.

• Easily expose NPAPI functionality.• Separate Process

Chrome Extension Basics

• Background page– Lengthy process, long lived, singleton.

• Browser Actions / Page Actions• Desktop Notifications• Override Pages• Browser Interaction (Bookmarks, Cookies,

History, Events, Management, Windows, Tabs)• Plugin Interaction.

Prototype

• HTML5 Device Tag <device>

• Interface

• Type

Device Tag

• Still in implementation (research phase)• It can change! Add a Haptics “type”Dreamy Ex:<p> To start, Select a haptic device: <device type=haptic onchange="update(this.data)"></p><script> function update(stream) { console.log(stream.data); } </script>

Prototype , Needed?

• Proof of concept needed to prove that Haptics needs love! And the future of computer human interactions!

• Rescue …

Prototype, Chrome Extension

• How do we do it right now!– HTML5 (Web Workers)– CSS3, Canvas (WebGL Context Graphics)– Chrome Extension

• Dedicated Thread (Background Page)• Haptics Controller (Browser Action)• Page Interactions (Page Action)• Accessibility (ARIA)

– NPAPI • Haptics Plugin communicates to Chrome Extension• C++ extreme (scene graph, triangles) collision detection algorithms

Architecture

• High Level

C Plus Plus

Novint Device

Architecture Plugin

Plugin Module

Scripting

Bridge

Plugin Gate

Haptics Service

Haptics Device

Haptics Signal

NP_GetEntryPoints

NP_Initialize

NP_Shutdown

Glimpse of Plugin Code

Architecture, Extension

• Browser Action to act as the controller

• Background Page, to allow the plugin to live forever.

• JavaScript for Some Collision Detection, Haptic Loop, and Rendering Loop.

Framework

• O3D JavaScript API for creating rich and interactive 3D application in the broswer using Canvas, WebGL context.

• Haptics API– void startDevice();– void stopDevice();– void sendForce(double[3]);– double[3] position;– addEventListener(‘servo’, function(Haptic data));

Open Source’dRelease it to the public! That is how I learned how to code, give back to the community! You can fork it from GitHub from here:http://github.com/mohamedmansour/haptics-chrome-extension

Working Prototype

Future

• Present this to the W3C Developer Group• Help Google Chrome complete <device> tag

implementation.• Implement the Prototype again using the

<device> tag.• Implement webpage interactions, like

annotated YouTube Videos to feel video touch.

Done! Thank you, questions?

HTML5 Haptics 1 HTML5 Haptics Standardization Mohamed Mansour hello@mohamedmansour.com Haptics...

Documents

Haptics Device Falleaf

Design of Class Components for Laparoscopy Surgery ... · which is human haptics, computer haptics, multimedia haptics and machine haptics [21]-[23]. Research related to haptics has

Human Computer Interaction - HAPTICS

Haptics for Mobile Devices

HAPTICS TECHNOLOGY.docx

Haptics touch the virtual

HAPTICS Seminar Report

Final Report on Haptics

Mansour alfala

Technical Committee on Haptics

haptics document

Haptics Presentation

Haptics Technology

HapticsA. García-Alonso1 >> Haptics

Computer Networking Yishay Mansour (mansour@cs.tau.ac.il) Nir Andelman (andelmni)

23424402 HAPTICS Seminar Report

Introduction to Haptics

Teaching Nanotechnology Using Haptics

Haptics ppt

Chris Hosea - HCI - Haptics