1LYU0703 Electronic Advertisement Guide on PS3

1

LYU0703Electronic Advertisement Guide on PS3

Huang Hiu Fung 05700512Wong Chung Hoi 05596742

Supervised by Prof. Michael R. Lyu

Department of Computer Science and Engineering, CUHK2007-2008 Final Year Project Presentation (2nd term)

2

Agenda

• Background Information

• Project Motivation and Objectives

• Implementation

• Result Analysis

• Performance and Cost Comparison

• Remote Demo

• Q&A

3LYU0703 Electronic Advertisement Guide on PS3

3

Background Information

Market of Commercial Monitoring

• Verify Ad. broadcast as stated in contract(No. of broadcast, broadcast time, duration)

• E.g. Large scale Ad. Campaign of HKD$10,000,000, spend 5% (HKD$ 500,000)for commercial monitoring

Current Solution• Monitor manually• inefficient

4LYU0703 Electronic Advertisement Guide on PS3

4

Project Motivation

• Hundreds of TV channels

• Increasing need for TV commercial monitoring

• PlayStation®3, a cheap parallel machine

5

Project Objectives

• accurate algorithm for commercial monitoring

• Parallelize better performance

• Generate an Electronic Advertisement Guide (EAG)

6LYU0703 Electronic Advertisement Guide on PS3

6

Developing Environment

• PlayStation®3

• A multi-core machine produced by Sony with Cell Broadband Engine processor

• Strong Computation Power

• Linux run on it

• Open platform for different applications and development

7

Implementation of Commercial Monitoring Algorithm

8

High Level Description of the Solution1. Converting raw video data into series of Hl3 files 2. Processing Hl3 files to become the final

EAG

9

Hl3 Files

• In form of a 32 x 32 array (1024 integers)

• Frame capture from analog TV card

• Digest of a frame 352 x 288 pixels

• Time information on file name

Hl3

2008.03.12.00.10004415.T.1.Hl3

10

Obtaining Hl3 Files from Raw Video Data

• At first, frames captured in constant frequency

• For example, 25fps, 1 hr video

• 3600 seconds X 25 = 90,000 frames

High frequency Low frequency

11

Obtaining Hl3 Files from Raw Video Data

• Frames captured by scene change

12

Processing Hl3 Files to Become EAG

• Minimum difference algorithm

• m “Target” files P (p1, p2, …, pk)

• n “Repository” files Q (q1, q2, …, qk)

• Match P to Q such that is minimum

• O (m x n x k)

13

Weaknesses of Minimum Difference Algorithm

1. Many to one

matching

2. Out of phrase

matching

3. Force matching

14

Longest Common Subsequence (LCS)

• ACGGT

• AGCTC

• LCS = AGT or ACT

• Dynamic programming

i\j A G C T C0 0 0 0 0 0 0

A 0 1 1 1 1 1 1

C 0 1 1 2 2 2 2

G 0 1 2 2 2 2 3

G 0 1 2 2 2 2 4

T 0 1 2 2 3 3 5

0 1 2 3 4 5

i\j A G C T C0 0 0 0 0 0 0

A 0 1 1 1 1 1 1

C 0 1 1 2 2 2 2

G 0 1 2 2 2 2 3

G 0 1 2 2 2 2 4

T 0 1 2 2 3 3 5

0 1 2 3 4 5

15

LCS on Character Strings to Hl3 Files

• Alphabet = Hl3 file• Subsequence = Adv.

1.One to one

matching

2. In phrase matching

3.No force matching

16

Modification of LCS algorithm

• R frame not match with T frame– T-R combination

• Out of phrase advertisement– Multiple passes LCS

17

Original LCS algorithm

• Allows R frame match with T frame

• Computation = m x n x ki\j R T R T R T

0 0 0 0 0 0 0 0

R 0 11　

1　

1　

1　

1　

1

T 01　

2　

2　

2　

2　

2　

2

R 01　

2　

2　

3　

3　

3　

3

T 01　

2　

3　

3　

3　

3　

4

R 01　

2　

3　

4　

4　

4　

5

T 01　

2　

3　

4　

4　

5　

6

0 1 2 3 4 5 6

i\j R T R T R T0 0 0 0 0 0 0 0

R 0 1 　 　 　 　 　 1

T 0 　 　 　 　 　 　 2

R 0 　 　 　 　 　 　 3

T 0 　 　 　 　 　 　 4

R 0 　 　 　 　 　 　 5

T 0 　 　 　 　 　 　 6

0 1 2 3 4 5 6

18

T-R Combination

19

T-R Combination

• Save the comparisons between T and R frame (half the computation)

• Table size reduce

to m/2 x n/2

• Computation:

m/2 x n/2 x 2k

= ½ x m x n x k

i\j T-R T-R T-R

0 0 0 0 0

T-R 0 1 1 1 1

T-R 0 1 2 2 2

T-R 0 1 2 2 3

0 1 2 3

20

Single Pass LCS

• Cannot recognize crossed advertisement segments

• The longer advertisement is recognized

21

Multiple Pass LCS

• Number of passes depends

on relative complexity of

2 videos

• Around 7-8 passes for 2

1 hour long video

• Speed up by caching

“Equality” comparison

result in 1st pass

22

Program Flow

1. Compute result table2. Backtrack LCS result3. Analyze LCS result4. Synchronize analysis result5. Propagates result back to input data stream6. Print output7. Loop back to 1 until no more new

advertisements detected

23

Analyze LCS Result

• LCS result has no information on start time, end time of advertisement

• An LCS may contain more than one advertisements or some mismatch frame

24

Printing Out Result

25

Speeding up with PlayStation®3

1. Compute result table Parallelization, SIMD2. Backtrack LCS result Double buffering3. Analyze LCS result Loop unrolling4. Synchronize analysis result5. Propagates result back to input data stream6. Print output7. Loop back to 1 until no more new Caching

advertisements detected comparison result

Step 1: O(m x n x k) Step 2-6: O(m + n)

26

Parallelization on Computing Result Table

1 2 3 4 52 3 4 5 63 4 5 6 74 5 6 7 85 6 7 8 9

• When filling the cell (i, j), information from (i-1, j-1), (i, j-1) and (i-1, j)

27

Parallelization on Computing Result Table

• Example, 3 cores, 5x5 table

• Not fully utilize in first 2 steps and last 3 steps

• Example, 6 cores,

4000 x 4000 table

• Only first 5 and

last 5 steps not fully

utilize

Core 1

1 2 3 4 5

Core 2

2 3 4 5 6

Core 3

3 4 5 6 7

Core 1

6 7 8 910

Core 2

7 8 910

11

28

Caching “Equality” Comparison Result

• 1st pass done all the “Equality” comparison between all T-R pairs

• Memory requirement m/2 x n/2 = ¼ x m x n• 1st pass m/2 x n/2 x 2k = ½ x m x n x k• 2nd pass and onward m/2 x n/2 = ¼ x m x n• In our case, k = 1024 (no. of values in Hl3)• 2nd pass and onward are speeded up by 2048X

29

Result Analysis

30

Experiment of Cross-Comparison of 7 Videos

• “TV Easy(宣傳易 )” from 7 different days

• about 5 minutes• only Ad. available

Aim:• To show all Ad. appeared

twice within 7 days can be found by cross-comparison

• times2172 C

31

Data Information

32

Flow of Comparison

33

Flow of Comparison

34

One of the Detail Result

miss in one day, still found in other day

35

Overall Result

36

Experiment of Comparing 2 One-Hour-Long Videos

• two one-hour-long videos• including commercials, news reports and

drama programs

Aim:• Show the matching rate • Show the performance

37

Overall Result

38

False Positive

• Blank frame• Low min difference

Example of a False Positive Advertisement

39

Performance and Cost Comparison

40

Performance Comparison on PC and PlayStation®3

41

Performance Comparison on PC and PlayStation®3

2376

27899

562

0

500

1000

1500

2000

2500

Time (sec)

PPU 1 SPU 6 SPU PC

Running Conditions

Runing LCS algorithm in Different Conditions

Time (sec)

42

Performance Comparison on PC and PlayStation®3

• 24 times faster than using only PPU on PlayStation®3

• 2.8 times faster than using 1 SPU on PlayStation®3

• 5.7 times faster than running on a PC

43

Cost Comparison

• 1 hr compare to 6 hr

• 6 x 99 sec = 10 mins on PlayStation®3

• 6 x 562 sec = 56 mins on PC

• Cost for PS3 $3000

• Cost for 5 PCs $20000

• Cost for 3 staff $20000 per month

Elapsed Time for1hr compare 1hr

PS3 PC

sec 99 562

44

Conclusion

• Implement LCS algorithm for commercial monitoring, better than 1st term approach

• Parallize on PS3, 5.7 times faster than PC

• Achieve:accuracy ~= 95%performance ~= 99 sec for 1hr video comparing 1hr video

• Generate a EAG from the result

45LYU0703 Electronic Advertisement Guide on PS3

45

Q&A

46LYU0703 Electronic Advertisement Guide on PS3

46

The End

47

Supplementary

48

Limitation on Direct Memory Access (DMA)

• SPU transfer data to and from main memory to local store via DMA

• At least 16 bytes or multiple of 16 bytes• Each element in result table is integer (4bytes)• Compute 4 elements at a time

Core 1

1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4

Core 2

2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5

Core 3

3 3 3 3 4 4 4 4 5 5 5 5 6 6 6 6

Core 1

5 5 5 5 6 6 6 6 7 7 7 7 8 8 8 8

Core 2

6 6 6 6 7 7 7 7 8 8 8 8 9 9 9 9

49

Double buffering

• Memory Flow Controller (MFC) operate in parallel SPU

• Fetch data from main memory to local store and compute result at same time

• Using extra buffer to store data that are pre-fetched

50

SIMD intrinsic function

• Operate on multiple data at the same time

• Mostly work on float or double

• 128 bits register as input

• 4 float value at a time

• Speed up 4X

51

Loop unrolling

• Get rid of computation time for loop counter in a loop

• Contribute to a lot computation time if little statements within loop

• Reduce run time from 2mins 1mins 35 secs

52

Analyze LCS result

• Assign a flag for each LCS unit (T-R pairs)• START_FLAG 'S' indicating it is a start point of

an advertisement• END_FLAG ‘E’ indicating it is an end point

of an advertisement• MIDDLE_FLAG ‘>’ indicating it is part of the

advertisement• SINGLE_FLAG ‘A’ indicating it is an

advertisement itself (Start + End)

• ISOLATE_FLAG ‘X’ indicating it does not belong to any advertisement

53

Analyze LCS result

• Split the LCS result into different segments by using a threshold

• Recognized a segments as an advertisement if duration of the segments > 5 seconds

• Assign a character string to the analysis result

54

Synchronizing flags

• Analysis of LCS on 2 video stream may give different result

• Synchronizing is needed

55

Synchronizing flags

• Similar to logical AND operation

• Recognized as advertisement if both analysis string agree with each other

56

Propagates flags to input data stream

57

Advantage of Comparing to 6 Other Video

miss in one day, still found in other day

58

False Negative

Comparing 0304 to 0305 Comparing 0305 to 0304

False-negative condition cause by using same scenes in 2 Ad.

59

Future Development

60

Supplementary – Future Development

• Remove TV Promo

• Logo recognition

• Use Knowledge of Electronic Program Guide

• Identifying New Commercials

61

“Equality” Comparison

• Computing result table requires “Equality” comparison between 2 symbols

62

Simple Logic System

• Difficult to decide “best” logic or scoring system

• As simple as possible

• Give high accuracy

result