Upload
eshe
View
37
Download
0
Embed Size (px)
DESCRIPTION
Study-Element Based Adaptation of Lecture Videos to Mobile Devices. Ganesh Narayana Murthy (M.Tech IIT Bombay) Sridhar Iyer (Associate Professor, IIT Bombay). Problem Definition. Adapt CDEEP videos to be viewable on mobile devices: Viewable at low network bandwidths (like GPRS) - PowerPoint PPT Presentation
Citation preview
Study-Element Based Adaptation of Lecture Videos to Mobile Devices
Ganesh Narayana Murthy (M.Tech IIT Bombay)Sridhar Iyer (Associate Professor, IIT Bombay)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 1
Problem Definition• Adapt CDEEP videos to be viewable on mobile devices:
– Viewable at low network bandwidths (like GPRS)– Viewable at low cost
• Video bit-rate – Size of video stream over time– Total size = bit-rate * total time– CDEEP video bit-rate: 1150kbps– GPRS bit-rate: 40kbps
• The problem:– Video playing incurs delays if available network bandwidth is less than
video-bit rate
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 2
Video Transcoding
• Converting from one video format to another– Changing video bit rate– Changing other parameters like frame rate, screen
resolution
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 3
Format Name Typical Bit Rate ApplicationMPEG-1 1.5Mbps or less CD-ROM
MPEG-2 5-8Mbps DVD, HDTV
H.263 Typically low bit rates Low bit-rate video conferencing
MPEG-4 / H.264 40Kbps to 10Mbps and above
Internet Streaming, Video Telephony
Flash Video (FLV) Typically low bit rates Embedded video in websites
Video Quality at low-bit rates
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 4
(a) MPEG-1 (b) MPEG-2
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Video Quality at low bit-rates (contd.)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 5
(c) H.264 (mp4) (d) H.263 (3gpp)
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 6
(e) Flash Video (flv)
Images from transcoded videos (Target bit rate : 40kbps, No audio)
Comparison of Video CodecsFormat Name
Original VideoSize
Converted VideoSize
Video Quality at low-bit rates
Remarks
MPEG-1 432MB 26MB Poor Cannot be used at low bit-rates
MPEG-2 432MB 29.12MB Poor Cannot be used at low bit rates
H.263 432MB 38.3MB Poor Cannot be used at low bit rates
H.264 432MB 16.9MB Good Processing power / Decoding complexity is high.[1]
Flash 432MB 20.5MB Good Can be used, but cost is still high.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 7
(Note: Video bit rate = 1150kbps, No audio, Target bit rate = 40kbps, No audio)
Video Sizes are still high forviewing over GPRS
Study-Element Based Adaptation
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 8
Motivation• CDEEP video usually consists of
– Presentation slides– Instructor explaining on white paper– Video of instructor talking
• Presentation slide is usually not changing– Video of slide is not required. One image is sufficient
• Idea– Extract one image every ‘n’ seconds and send to client.– This would reduce amount of data sent for showing one
slide.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 9
Method-1
• Send one image every ‘n’ seconds– Server sends one image every ‘n’ seconds to client– Audio is simultaneously streamed
• Network bandwidth and Size – Network Overhead (NO) = Image Size / n– Size Overhead (SO) = Total size of images
• What is the user experience?
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 10
User Experience Basis• Presentation Study Element
– Portion of video showing one slide
• White Paper Study Element– Portion of video showing instructor writing on
white paper
• Instructor Study Element– Portion of video showing instructor talking
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 11
…………..
0 5 10 15
Presentation Slide
Delay in start of slide
3
………..
25 30 35
White Paper
Video Time(secs)
User Experience• Presentation Element
– Delay Experienced (D2) = • Delay in start of slide as compared to audio
• White Paper Element– Delay Experienced (D1) =
• Delay between any two consecutive images = Sending Rate
• Instructor Element– Only audio important. No image need be sent.
• User Experience is assumed to be one
• User Experience (Ui) = 1 sec / Di
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 12
Method-2• Trade-off for user experience
• Cost incurred in terms of number of images sent
• Same sending interval for all elements, cannot balance user experience and cost.
• Choose different sending interval for each study element
• Probably:– Higher user experience for white paper
element– Lower user experience for presentation
element
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 13
User Experience
Cost
Sending Interval
Trade-Off Relation
System Overview
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 14
Building the index• Corpus of 10 videos
– Representative of various departments
• Consider different sending intervals ‘r’– For each ‘r’ find NO,SO and U for every study element in a
video. – Repeat for all videos and take average.
• This relation can be used backwards:– For calculating sending interval, given network bandwidth
and user experience.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 15
Graphs of User Experience
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 16
Graphs of overheads
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 17
Results
Original Video Size(MB)
Images Size (MB)
Reduction (%)
U1 U2 SupportedNetwork Bandwidth
432 2.85 97% 0.2 0.38 20kbps and above
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 18
Achieved Size Reduction
Fig: Video stream size reduction (note: Original video bit-rate = 1150kbps, No audio)
Results (contd.)
Sending Interval U1 U2 NO1(kbps)
S01(MB)
NO2(kbps)
SO2(MB)
Total Size (SO)
White Paper Element
Presentation Element
5 5 0.2 0.337 15.12 1.25 23.43 1.6 5.46
5 15 0.2 0.115 15.12 1.25 7.81 0.53 1.78
15 15 0.067 0.115 5.254 0.781 7.63 1.03 1.81
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 19
Balance User Experience and Cost
Required Network Bandwidth =max(NO1,NO2) is reduced
Reduction in size user experience for white paper element remaining same
Conclusion
• Large size reduction can be achieved by using the concept of slideshows
• Identifiying study-elements within the video helps define user-experience of the slideshow.
• CDEEP Lecture videos can be adapted to low network bandwidths and in a cost-controlled manner.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 20
Future Work
Automated tagging– Identifying study element boundaries – Shot detection techniques
User Experience Correlation– Identifying relation between obtained user
experience and actual user values
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 21
References1. H.264 white paper.
http://ati.amd.com/products/pdf/h264_whitepaper.pdf.2. Real-time Content-Based Adaptive Streaming of Sports Videos.
Shih-Fu Chang, Di Zhong, and Raj Kumar. In CBAIVL '01: Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), page 139, Washington, DC, USA, 2001. IEEE Computer Society.
3. Content-aware video adaptation under low-bitrate constraint. Ming-Ho Hsiao, Yi-Wen Chen, Hua-Tsung Chen, Kuan-Hung Chou, and Suh-Yin Lee. EURASIP J. Adv. Signal Process,2007(2):27-27, 2007.
4. A Characteristics-Based Bandwidth Reduction Technique for Pre-recorded Videos. Wallapak Tavanapong and Srikanth Krishnamohan. In IEEE International Conference on Multimedia and Expo (III), pages 1751-1754, 2000.
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 22
Questions?
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 23
Content-Aware Adaptation
Ganesh Narayana Murthy NCC 2010, Chennai, 30/01/2010 24
Method Name Adaption Mechanism Video Quality Remarks
Hsiao et.al.[2] Identify visual attention regions in a frame. Encode them at high quality.
Poor Quality of important objects still depends on network bandwidth
Chang.et.al [3] Identify events in sports videos at high quality. Other regions as slideshows.
Good Slideshow of images reduces network bandwidth and size
Tavanapong. et. Al. [4]
Identify non-changing portions of lecture video and extract one image from them
Good Exploits redundancy in lecture videos