Upload
mayra-swires
View
215
Download
0
Embed Size (px)
Citation preview
Video Object Tracking and Replacement for Post TV Production
LYU0303 Final Year Project
Spring 2004
Outline
Brief project overview and targets achieved New components added to the proposed
system Working principles of individual parts Future work Q & A
Overview
Post-TV production processes and changes the content of a video clip
Difficult for computers to process video in a global view
Step-by-step programming enables the computer to “think” more thoroughly
Overview
We have introduced the following parts in the last semester: Bitmap I/O RGB <-> HSV converter Edge detector Edge equation finder Translation detector Texture mapper
New parts have been added in order to increase the functionality of the system
Additional Basic Components
The following parts have been added Graphical User Interface (GUI) Corner point finder Video file reader and writer Video player processor (frame-based) Improved texture mapper
The following part has been modified Bitmap I/O Edge finder Texture mapper Removed Equation processor
Graphical User Interface
Although UI is not a main project part, a good GUI can significantly decrease the time needed for processing and program maintenance
Since C++ language is being used, the Microsoft Foundation Classes are adopted to develop the user interface
Provide most of the basic functions needed for information input and output
Graphical User Interface
Corner Point Finder
As we know that processing dots are much faster than processing lines, we may wish to try to find the corner points directly before finding the edge equations
Stick some brightly coloured labels at the corners of the rectangular surface
Can apply on cylindrical objects as well to indicated the curve control points
A simple K-Means method is used to group the points together.
Corner Point Finder In order to know the orientation of the mapped
texture, we need to know which is the “first” point For simplicity we just make the first point larger so
that it contains more corner points than the others
First point
Corner Point Finder
We repeat the process until all the corner points has been processed
Points which are too far away from the main groups are discarded
Find out the centroids of the groups and move them out to give an estimation of corner positions
Group No. of Member
1 1
2 1
3 1
1 2
4 1
1 5
2 5
3 5
4 5
Corner Point Finder
Video File I/O
As we are talking about video processing, it is inevitable to handle video reading and writing
DirectShow interfaces are being used in the project
Able to process any type of AVI video as long as the Window Media Player can play them
Video Player
We need to choose a video renderer that is suitable for our application
To reserve the function of real-time processing and video texture in the future, the Video Mixing Renderer 9 (VMR9) is used
Allows the rendering of more than one video or bitmap on the screen at the same time
Video Processor
Video are sequence of bitmap frames We capture each video frame and process t
hem in the same way as processing the bitmaps
After a video frame is being processed, we add them to an AVI file
Default video compressor is DivX Mpeg4 encoder (FourCC code “divx”)
Video RendererVideo System Structure
Texture Mapper
A graphics design technique used to wrap a surface of a 3-D object with a texture map
The 3-D object acquires a surface texture similar to the texture map.
Colors, brightness values or altitudes
Texture Mapper
Mapping
function
(u,v)(r,c)
Texture coordinates
Image coordinates
line by line process each pixel on every line last semester, we mapped cubes. How about this
semester?
Scan-line conversion
Scanline yk
Scanline yk+1
scanning order
for every line
Cylindrical mapping
How to map this cylinder with different textures?
Cylindrical mapping
Mapping cylinders
3 parts: Surface detection Line scanning Texture lookup
+ Orientation determination
Surface detection
1. Hough algorithm
to detect the two straight edges
four points are detected
Surface detection
2. Scanning from top to bottom
the scanning slope is orthogonal
yellow points are detected
by color transition
Surface detection
3. Model the curved edges by ellipses
x2 / a2 + y2 / b2= 1 2 ellipses are
needed rotational factor
x
y
z
Scan-line mapping4. Scan-line from left t
o right
Problem:
How to determine the corresponding texture for each pixel, say p=(x,y) ?
x
y
z
(x,y)
Texture lookup5. Rotate the cylinder t
o the upright position along z-axis
6. Translate the cylinder to (0,0) and determine the z coordinate p=(x’,y’,z’)
7. Rotate the along x-axis upright from viewpoint
x
y
z
(0,0)h
Texture lookup8. Detect the marker w
hich indicates where the texture starts
9. Look up the texture coordinate (u,v) by
(y’+h/2)/h = v
tan-1(x’/z’) / 2 = u
x
y
z
(u,v)
Cylindrical mappingAfter mapping the top we have :
TEXTURES
MAPPING
MAPPED CYLINDER
Spherical mapping
How to map this sphere with different textures?
Spherical mapping
Mapping spheres
Also 3 parts: Surface detection Line scanning Texture lookup
+ Orientation determination
Surface detection
1. Scan from 4 directions and get the boundary points
2. Determine center
Line scanning
3. Scan from left to right.
4. For each pixel p=(x,y), look up the texture by transforming it to the upright position
x
y
z
(0,0)
Upright
Texture lookup
4. Translate to origin
5. Look up z-coordinate
6. Rotate the ball along z-axis
7. Rotate the ball along x-axis (x’,y’,z’)
x
y
z
(0,0)
(5)(6)
Texture lookup8. Lookup the texture by
v = cos-1(y’/R) / u = (cos-1(x’/(R sin(v))) ) / (2)
Spherical mapping After mapping:
Mapped Sphere
Texture
Shadow mapping
Mapping of surface brightness Method:
Covert the image from RGB HSV Modify the V (intensity) value
5 types of shadow mapping
Shadow mapping
1. Mapping with texture brightness
often unrealistic
2. Mapping with image brightness
Shadow mapping
3. Mapping with average brightness of the image
more realistic
4. Mapping with other shadow maps (e.g. wood)
Shadow mapping
5. Transparency mapping
(background information has to be obtained)
Problem with sphere mapping
The texture is “squeezed” at the pole
Solution
Modify the texture to this:
Mapped sphere becomes:
No distortion at
pole
Q & A