Upload
marjory-hines
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Computer Vision in Graphics Computer Vision in Graphics ProductionProduction
Adrian Hilton
Visual Media Research GroupCentre for Vision, Speech and Signal Processing
University of Surrey
http://www.ee.surrey.ac.uk/CVSSP/VMRG
Overview
Where is computer vision useful in content production?
Where is vision used?
What can and can’t current computer vision do?
Computer Vision in Content Production
Computer vision: analysis & interpretation of real images/video
• Match moving• Matting • Model building• Human motion capture• Studio Production• Facial animation• Image-based illumination
Match Moving
Reconstruction of camera movement to composite CG
Products: 2d3, RealViz … (semi-automatic camera tracking)
… standard tool in film production
Oxford University/2d3
Matting
Separation of foreground and background objects - actor/background separation
- wire/set removal Studio: chroma-key (solved) Post-production: Imagineer, RealViz, …. (open-problem)
Model Building
Capture of real shape for CG models • objects• environments• characters
Shape capture• active sensors (laser/structured-light) - accurate 3D surface measurements - static objects or environments• structure from images - low-accuracy - static objects
Problem: unstructured surface measurements
Human Motion Capture
Capture of real motion
Marker based systems• widely used in performance animation whole-body/face• character animation (Golum)• ‘realistic’ motion characteristics• accurate • real-time?
widely used in film production (with post-production)
Human Motion Capture(2)
Markerless human motion capture• advantages: unintrusive; simultaneous appearance capture• model-based visual tracking• low-accuracy• visual ambiguity (uniform apperance, non-rigid shape)
Studio Production
Modelling actors• 3D animated models• animation • re-illumination• loss of visual realism to captured images
Free-viewpoint video• post-production of arbitrary camera views/paths• ‘matrix’ flowmo shots• aim: quality equivalent to captured video
Facial AnimationProduction of photo-realistic faces
2D video-based approaches• video rewrite [Bregler’97]• concatenative synthesis [Cossato’98,Ezzat’02]• photo-realistic• limited viewpoint, illumination
3D markers• Performance animation [Phigin’98]• shape only
3D video (shape+appearance)• concatenative synthesis• photo-realistic • control of viewpoint, illumination
Image-based illumination
Illumination of CG scenes with real illumination• high-dynamic range capture of illumination• environment mapping illumination to CG scene
widely used in production
Debevec SIGGRAPH’00 FiatLux
Computer Vision in Content Production
• Match moving - yes, automatic camera tracking available
• Matting - partial, many unsolved problems
• Model capture – yes, tools for semi-automatic restructuring
• Human motion capture – no, inaccurate
• Studio Production – partial, free viewpoint video
• Facial animation – yes, 3D video
• Image-based illumination – yes, widely used
• Other applications of computer vision….
Future Directions
Video-based rendering• photo-realistic, free-viewpoint rendering of dynamic scenes• relighting required
Video-based animation• animation from captured video (face,whole-body…)• control of motion, viewpoint, illumination