Detection of Illicit Content in Detection of Illicit Content in Video StreamsVideo Streams

Niall Rea & Rozenn DahyotNiall Rea & Rozenn Dahyot

http://www.pixalert.com/

What? Why?What? Why?

By illicit content, we mean By illicit content, we mean pornographic materialpornographic material

Applications: Applications: Kid protection (Parental control)Kid protection (Parental control) Company protection (company computers Company protection (company computers

scanning)scanning) Pedophilia (cyber surveillance)Pedophilia (cyber surveillance)

State of the ArtState of the Art Text Analysis (Internet Filtering) Text Analysis (Internet Filtering)

FilenamesFilenames Text surrounding imagesText surrounding images Known URLs Known URLs

Image analysisImage analysis Skin detectionSkin detection Geometrical constraints + OrientationGeometrical constraints + Orientation Face localisationFace localisation

Video analysisVideo analysis Current approach: keyframe extraction at regular time intervals and Current approach: keyframe extraction at regular time intervals and

still image analysisstill image analysis

Extension to Illicit Video Extension to Illicit Video DetectionDetection

Using Video informationUsing Video information ColorColor TextureTexture MotionMotion

Using Audio informationUsing Audio information Audio energy (loudness)Audio energy (loudness)

Video AnalysisVideo Analysis

Considered 2 approaches exploiting Considered 2 approaches exploiting features from the partially decoded MPEG features from the partially decoded MPEG video streamvideo stream Smart keyframe selection for real time Smart keyframe selection for real time

performance based on macroblock typeperformance based on macroblock type Exploiting motion vectors for periodic motion Exploiting motion vectors for periodic motion

detectiondetection Optimised open-source ffdshow decoder (extraction Optimised open-source ffdshow decoder (extraction

of compressed domain motion features from MPEG-of compressed domain motion features from MPEG-1/2/4)1/2/4)

Poesia filter for skin colour detectionPoesia filter for skin colour detection

Shot cut detection based on Shot cut detection based on ratio of ratio of macroblock types of consecutive inter-coded macroblock types of consecutive inter-coded frames in a sub-GOP frames in a sub-GOP (e.g. shot cut occurs between the first (e.g. shot cut occurs between the first

reference frame and the first B-frame)reference frame and the first B-frame) Macroblocks in both B-frames will be heavily Macroblocks in both B-frames will be heavily

backward predicted (indicated by the heavier backward predicted (indicated by the heavier arrows). A shot cut is deemed to have occurred arrows). A shot cut is deemed to have occurred if and if and

Shot Cut detectionShot Cut detection

Shot cut

n-1n-3 n-2 n

PBB I

min2 Tbfbn

min1 5.0 Tbfbn

Motion extractionMotion extraction Compressed (MPEG motion vectors)Compressed (MPEG motion vectors) Background/ global motion compensation (Coudray 2004)Background/ global motion compensation (Coudray 2004)

4 parameter motion model4 parameter motion model

Calculate zoomCalculate zoom

Calculate translationCalculate translation

Assume global motion only occurs in non-skin and Assume global motion only occurs in non-skin and reasonably high texture areasreasonably high texture areas

Compute a 2D histogram of those motion vectorsCompute a 2D histogram of those motion vectors Global translation is the mode of the histogramGlobal translation is the mode of the histogram

Motion fieldsMotion fields

When Harry met Sally Illicit video

Motion and color segmentationMotion and color segmentation

Assume a simple local homogeneous motion field and global homogeneous motion

k-means clustering (2 clusters) of motion field

Likelihoods computed from Poesia 32^3 bin RGB skin/non skin histograms

Priors set empirically

Audio stream Audio stream

Content recognition using Audio dataContent recognition using Audio data SportSport

Specificity of illicit: (pseudo) periodicity Specificity of illicit: (pseudo) periodicity Simple feature used: loudness (Audio energy)Simple feature used: loudness (Audio energy)

Does not discriminate between different sources of Does not discriminate between different sources of noise (ie voices, specific sounds, etc.)noise (ie voices, specific sounds, etc.)

Capture the dominant pattern of the audio dataCapture the dominant pattern of the audio data

Audio stream Audio stream

Illicit Audio:

Scene of When Harry met Sally

LoudnessLoudness

Harry and Sally are talking (5s) Sally is faking it (5s)

Audio energy computed over a 40ms (duration of an video frame at 25 fps)

PeriodicityPeriodicitySignal: s(t)=1+sin(t) Signal: s(t)=rand(t)

Autocorrelation: sscorr(s)

Periodicity of audio energyPeriodicity of audio energy

Harry and Sally are talking Sally is faking it

Correlation of the loudness computed over 5s

Measure of periodicity in audio Measure of periodicity in audio datadata

Harry and Sally are talking Sally is faking it

Measure of periodicity: difference between the surface defined by the maxima and the minima

Measure of periodicity in audio Measure of periodicity in audio datadata

Measure of periodicity over the sequence ‘When Harry met Sally’

Other resultsOther results

False alarm rate assessed on False alarm rate assessed on 20minutes of non-illicit material 20minutes of non-illicit material (scenes from movies and music (scenes from movies and music videos)videos)

Detection rate assessed on 10 Detection rate assessed on 10 minutes of 8 illicit materialsminutes of 8 illicit materials

FA: 2%

DR: 5 extracts (~9minutes of recordings) are flagged as illicit3 (~minutes recordings) are missed

Future work: Motion and Future work: Motion and periodicityperiodicity

Periodic Motion on P-Periodic Motion on P-framesframes Mean motion vector Mean motion vector

over skin regionsover skin regions

Correlation between Correlation between periodic motion and periodic motion and audio?audio?

Any question