Video ProcessingBY: Maria, Ben, Jason

• Video basics

• Video compression

• Manipulation of video

• Stitching multiple videos

• Reducing errors in stitching

• The future of processing

Overview:

How Is Video Captured

The CMOS Sensor Captures the light and converts it to an electrical signal. This happens numerous times a

second to compose a video.

Basics of Video

A video is simply a sequence of images that changes rapidly.

The human eye can distinguish the difference in images that change about 15-20 times per second (fps).

When the images change at a rate quicker than 15-20 fps our eye can not distinguish them and therefore we are fooled into seeing a continuous stream.

Video CompressionUncompressed video files are Extremely Large

- 1 minute of 1280 x 960 uncompressed video = 4 GB

Compressed files can be almost infinitely times smaller depending on how much video quality you are willing to sacrifice.

Simply put, compressing a video will tell the computer to ignore everything that isn't moving to save processing power.

Video Compression

Over Compression

Over Compression

VIDEO TRAILS

Video EncodingCodec - Compression Decompression algorithms

- H.264, Divx, FFMpeg - These are the algorithms that reduce the file size of a video

Container - File type that is output after compression - MOV, AVI, MP4 - Containers can hold more than one codec - Most common container on the web is Flash which is compatible with H.264, VP6, and Spark Codecs

(x,y)

Geometric Transformations

Translation

(x',y')

ΔXΔY

Geometric Transformations

Scale

X

Y

X'

Y'

Geometric Transformations

Rotation

Ө

(x,y)

(x',y')

Alpha BlendingStandard Video has 3 Channels: Red, Green, and Blue,

totaling 24 bits (8 bits per channel).

A 4th channel, called the alpha channel is sometimes available in 32 bit video sources, and if not, can be created within the computer.

The alpha channel is represented by transparency and is understood by the computer on a scale of 0 to 255.

Alpha Blending

Do we want the alpha channel of every video to be the same, or individually adjustable?

StitchingConventional stitching algorithms leave a

visible seam in the overlapped region (due to accuracy limitations, vignetting, difference in exposure times, and white balance)

Most are also too complicated to be practical in real-time. They require too much computational energy.

Choosing the right endpoints is important when aligning the images, so that there are no obvious seams. Picture (b) shows an alignment with poorly chosen endpoints, so there are visible seams shown in the circled regions.

• Start with videos that share an overlapping region

• Use a Block Matching Algorithm (BMA) to find corresponding points of the images to line them up

• You need to account for motion in the frames, using phase correlation.o Phase Correlation: a method to estimate how far

offset two images are

• Finally, blend the images together at the overlapping region.

Process:

Determining where to overlap the images in each frame individually creates a more seamless image than if the overlap remains the same for all the frames.

Ghost Images

Some steps for eliminating ghost images:

Calculate the size in pixels of the overlapping region, then find the center point of this region, shown as P in the diagram. This gives you a focal point for the overlap. The article gives an equation to create a gradual change in the overlapping region, blending the images gradually from one to the next.

*Be sure to account for differences in color and position of moving objects

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5541517

Current Video Processing Research

• Researchers in india are creating algorithms to improve face recognition technology.

ContinueIn the medical field, there has been a big focus

on improving quality in every image that a piece of equipment makes.

MRI

Ultrasound

Bronchoscopy

Reconstruction from Brain Activity

Measuring the blood flow and other actvitity in the brain, researchers have been able to project what the person sees in their head

<iframe width="420" height="315" src="http://www.youtube.com/embed/KMA23JJ1M1o" frameborder="0" allowfullscreen></iframe>

Questions?