17

Clientside/Offline (onefile) Lecture Player in a Web Browser

Embed Size (px)

Citation preview

Page 1: Clientside/Offline (onefile) Lecture Player in a Web Browser
Page 2: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

YALMS: Yet Another LMS

V A

S

Step 1: Record

C

Long-Term Storage Lecturer

Files

YALMS Robot

Viewers HTML5

Step 2: Annotate

Lecturer S YALMS

Robot

01 myself+1 "YALMS: A Data Structure for Storage and Replay of Multimedia Content in Digital Classes" IEICE・LOIS研 (2014)

06 myself+0 "Homepage of the YALMS.format Subproject" https://github.com/maratishe/yalms.format (2015)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 2/17...

2/17

Page 3: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

The MultiFile Player (trivial)

• traditional method• HTML + MP4 + PNG + ... files• user needs to download andunpack (ZIP, RAR, TBZ, ...)

• no way to have a fullybrowser-based interface

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 3/17...

3/17

Page 4: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Dev: Components

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 4/17...

4/17

Page 5: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

HTML5 Features

• native binary: no need toconvert, used to be BSON,JSON, etc.

• WebSockets: streaming• Workers: multicore• FileReader: read access tolocal filesystem

• saveAs: trigger file download(binary OK)

08 "HTML5: W3C Working Draft" http://www.w3.org/TR/html5/ (2015)

09 "File API: W3C Working Draft" www.w3.org/TR/FileAPI (2015)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 5/17...

5/17

Page 6: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Binary + saveAs (auto download)

• saveAs() is not part of HTML5 standard! -- but too good to give up

• can use standard INPUT type="file" dialogs to get filepath/name• the other side of FileReader

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 6/17...

6/17

Page 7: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

VIDEO: Access to Raw Buffer

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 7/17...

7/17

Page 8: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

WebSocket Aggregator

Network

Web Server Chrome Browser

Control

…….

Buffer

User

WebSocket

WebWorker

WebSocket

WebWorker

WebSocket Server

WebSocket request

fork() Handshake

02 myself+0 "A Practical Software Model for Content Aggregation in Browsers...." 2nd COMPSAC/MVDA (2013)

03 myself+0 "Experiments with application throughput in a browser with full HTML5 support" IEICE ComEx (2013)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 8/17...

8/17

Page 9: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

WebSocket Aggregator in Action

03 myself+0 "Experiments with application throughput in a browser with full HTML5 support" IEICE ComEx (2013)

07 myself+0 "WebSockets, WebWorkers, and Raw Binary in Chrome" https://github.com/maratishe/msaplayer (2015)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 9/17...

9/17

Page 10: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

The Offline/SingleFile Player

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 10/17...

10/17

Page 11: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

The Offline/SingleFile Objectives

PNG

Replay Screen

VIDEO

S A V MISC. C

Browser

Webapp

Internet

Local filesystem (FileReader )

PNG

VIDEO

Down-loads

• with/without Internet,both OK

• when with Internet, can alsouse Cloud APIs (Dropbox,GoogleDrive)

• 3 functions1. PNG -- screen, slide pages2. VIDEO -- raw buffer

access3. Download -- saveAs

04 Z.Pan+5 "Technologies for E-Learning and Digital Entertainment" 1st Edutainment (2006)

05 A.Ram+1 "Video Analysis and Repackaging for Distance Education" Springer (2012)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 11/17...

11/17

Page 12: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

YALMS: Encoding Design

VA

S

Step 1: Record

CLecturer

YALMS.format

YALMSRobot

Watchfilesystem

Watchscenechange

Step 2: Add PDF

Lecturer

S

AV

SE

CEPE

Annotation

SEX

• no need to be online, offline/cloud processing OK

• encoding format --headers, payload, etc. at 01

• in the end, becomes one file

01 myself+1 "YALMS: A Data Structure for Storage and Replay of Multimedia Content in Digital Classes" IEICE・LOIS研 (2014)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 12/17...

12/17

Page 13: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Screen Encoding: Effective?

0 10 20 30 40 50 60 70 80 90Time sequence of pages/frames

0.8

1.6

2.4

3.2

4

4.8lo

g( b

inar

y d

iff)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 13/17...

13/17

Page 14: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Binary Diffs for Screen Encoding

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 14/17...

14/17

Page 15: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Binary Diff + Multicore (workers)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 15/17...

15/17

Page 16: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

Wrapup

• Offline/SingleFile player is already a reality, based on:

1. FileReader 08 and saveAs()2. access to raw VIDEO buffer 033. binary diff and related compression/encoding 01

08 "HTML5: W3C Working Draft" http://www.w3.org/TR/html5/ (2015)

03 myself+0 "Experiments with application throughput in a browser with full HTML5 support" IEICE ComEx (2013)

01 myself+1 "YALMS: A Data Structure for Storage and Replay of Multimedia Content in Digital Classes" IEICE・LOIS研 (2014)

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 16/17...

16/17

Page 17: Clientside/Offline (onefile) Lecture Player in a Web Browser

.

That’s all, thank you ...

M.Zhanikeev -- [email protected] -- Clientside/Offline (onefile) Lecture Player in a Web Browser -- http://bit.do/marat150523 17/17...

17/17