J Manoj Kumar (M.Tech IIT Bombay)Guided by: Prof. Sridhar Iyer

ADAPTATION OF APPLETS AND LIVE VIDEOS TO MOBILE

DEVICES

Problem Definition1. Adapt CDEEP Live Videos to Mobile Devices.

Adapt to low bandwidth networks.Problems in video streaming to mobile devices

Limited Memory in Mobile devices. Low Bandwidth of Mobile Network like GPRS.

2. Adapt Java Applets to Mobile DevicesJava Applets are built with J2SE technology that

do not work on mobile devices.Limited Memory in Mobile devices.Mobile devices work on J2ME technology.

Adaptation Of CDEEP Live videos to Mobile Devices

Existing System - Study Element Based System

Works only for “stored CDEEP videos”

Study Element Based SystemThree types of Study Elements are

categorized

i. Presentation Element -- portion of video

that shows one slide of presentation.

ii. White paper Element – portion of video

that shows white paper.

iii. Instructor Element – portion of video that

shows instructor.

Classifying ImagesClassification -- Feature based TaggingFeature -- a unique property

Type of Element

Feature Used

Instructor Element

Face Recognition

White paper Element

CDEEP Logo Image

Presentation Element

Heading Color

Proposed System

Live Video Adaptation Module

Capture sets of 300 images from Live stream.Classify each set of images study elements.Copy images and audio to server.

Image Extraction Methodology

Approach 1:Using Video LAN(VLC)

Save or Record for five minutes.Extract Images from saved video.

DisadvantagesHigh CPU utilization because of Video

transcoding.Delay between actual video and seeing video

increases for each iteration.

Convert to MJPEG StreamMJPEG - Motion JPEG ( Images are shown one

after the other). Ex: surveillance camera.Images are separated by delimiter.

DisadvantagesHigh CPU Utilization because of Video

Transcoding.Unable to display images.

Approach 2:Using FFmpeg

Images extracted directly from live stream.ffmpeg -i url -an -r 1 -f image -s 320x240 video

%d.jpgGroup images to sets of 300.

Using Mplayer:Images extracted and grouped directly from

live stream.mplayer -nosound -vf framestep=25 -vo

jpeg:subdirs=img:maxfiles=300 url

Audio ExtractionUsing FFmpeg

Audio is saved in the form of mp3 chunks.

ffmpeg -i url -t 00:00:20 -acodec libmp3lame -ab 24 -ar 8000 audio.mp3

Generate XML

System Implementation

Sequence Diagram

Design ConsiderationsPort blocking in GSM networks

Direct TCP connections not possible due to port blocking.

Allows only HTTP connection to ports 80 and 8080.

Audio StreamingAudio streaming to mobile device is not

supported.Hence, need to do progressive download.

Usage

1.Enter URL 2. select video 3. slide show

Comparison to Existing System

Supports automatic creation of offline

copy of video.

Live video can be viewed as soon as it

starts.

Misclassifications of images cannot be

corrected.

Adaptation of Applets to Mobile Devices

Project OSCAR

Project OSCAR – Open Source Courseware Animations Repository.

Repository for web-based, interactive animations for various teaching concepts.

Animation is typically an Java Applet with brief description of concept.

Overview of J2MEJava Micro Edition J2ME – small devices like

mobiles, PDA’s with low RAM and speed.

J2ME has 3 principles

Configuration – JVM + set of API’s

Profile – set of API’s designed for specific

configuration

Optional package –technology specific API’s

to extend the capabilities of JVM

Architecture of J2ME

J2MEConfiguration: specifies JVM and core set of

API’s for a specific family of devices.Two types of configuration

Connected Device Configuration CDC – minimum of 512KB ROM, 256KB of RAM, full JVM must be supported.High end PDA’s, TV setup boxes, car navigation

systems.Connected Limited Device Configuration CLDC –

minimum of 160KB of ROM, 32KB RAM, KVM Mobiles, pagers, PDA’s.

Cannot add native methods.

J2SE Applets to J2ME Conversion Difficulties1. Incompatibility of Architectures of J2SE and

J2ME

MIDlet architecture Applet Architecture Only one form per screen in MIDlet, Multiple

panels in applet. Difficult to convert from J2ME architecture to J2SE

architecture.

1. .

2. Canvas size and Co-ordinates of Image Object Designed for Desktop systems Co-ordinates have to be set manually – cannot

be automated

Sample OSCAR paint codepublic void paint(Graphics G){g.drawImage(Image img1, 480, 600,this);g.drawImage(Image img2, 530, 400,this);g,drawImage(Image img3, 60, 370, this);}

1. Amount of Text on Screen Designed for Desktop Resolutions – cannot be fit

mobile screen, preplanning cannot be automated.Sample OSCAR animation with text field.

public void init(){ta = new TextArea("", 31, 47);ta.setText("\n\t Energy Transaction in Biological

Systems\n"+"\n In this animation you will learn the concepts of ATP

Metabolism……….……………… 10 lines };

Other General DifferencesNo floating point support in J2ME.

No Java Native Interface (JNI) in J2ME.

No finalize method in J2ME.

No Thread Groups.

J2SE to J2ME – Conversion Possibilities1. Copy of Paint Method

J2ME uses J2SE’s Graphics class with minor changes -- enables copy paint method as it is during conversion

Sample applet with paint methodpublic void paint( Graphics g){

g.drawString("Sending",150,75);

g.drawRect("40,50,10,20);

g.drawImg(Image img,20,30,center);

g.drawArc(30,40,10,30,60,120);}

g.drawString, g.drawRect, g.drawImg, g.drawArc are common in both j2se and j2me

Buttons to Key mappingJ2ME doesn’t display buttons, it uses mobile

device keys take input OSCAR animations -- buttons like start, stop. Buttons to key mapping is possible.

LWUITLight Weight User Interface Toolkit is a UI

library developed by Sun.Offers advanced UI capabilities and API

inspired by Swing.LWUIT allows multiple panels on screen.

Overview of Flash and Action Script

Flash – multimedia platform language for creating

animations

Action Script can be embedded. – ECMA script.

AS is Object Oriented similar to Java Script.

AS + Flash – more interactive animations.

Used to build Rich Internet Applications.

Flash Lite – highly optimized flash runtime for mobiles.

Provides same functionality similar to desktop.

Applet to Flash – Conversion DifficultiesNo Multi-Threading in Flash

Multi-Threading – two or more parts in a program to run concurrently ( Multitasking ).

Essential ability to have code do something while UI is doing something else.

Single Thread – Having heavy computation, cannot update UI.

Program becomes un-responsive and crashes after 60 seconds.

Sample OSCAR program with threadclass Timer extends Thread {//code//void Settime();void sleep(); }

Conversion difficultiesNo method(function) overloading

Difficult for multiple methods – if constructor overloading

Sample java programpublic void defaultSetter(int iValue) {

setter(iValue, false); }

public void setter(int iValue, boolean bForce) {

// whatever }

In action scriptpublic function setter(iValue:int, bForce:Boolean =

false):void {

// whatever }

Other General Difficulties and Possibilities1. Limited Data Structures

Action script supports only Arrays and Dictionary. No queues, no linked List, no stack compared to Java. Need to re-implement data structures which are

wrapped.

2. No Blocking Next code of line is supposed to run, user cannot

prevent it from run – Alert.show(); does not stop exexctuing next line.

Possibilities• Java and Action Script have same syntax with a little

change.• Syntax change automated by Java to Action script

converter (J2AS3)

ConclusionSuccessfully integrated to existing Study

element based adaptation.Live video transmission of CDEEP videos

can be adapted to networks with low bandwidth.

Automated porting Java applets to mobile devices as J2ME MIDlets and flash files is difficult and infeasible.

Future Work1. Decrease Misclassifications of images

Study element based system is prone to errors. Better feature based classification.

2. Synchronization between audio and images

Synchronizing audio with the image slideshow.

3. Text Improvement Identify text in study elements and improve text

quality.

References1. Fabrice Bellard. Ffmpeg codec manual.

http://www.ffmpeg.org/ffmpeg-doc.html .

2. Free Software Foundation. Mplayer manual. http://www.mplayerhq.hu/DOCS/HTML/en/index.html.

3. G.N. Murthy and S. Iyer. Study element based adaptation of lecture videos to mobile devices. In Communications (NCC), 2010 National Conférence on, pages 1 - 5, 29-31 2010.

4. Herbert Schildt . Java 2ME the complete reference. Osborne/McGraw-Hill, fourth edition, 2001.

Questions?