Tour Into the Picture: Using a Spidery Mesh to Make Animation from a Single Image Youichi Horry,...

Tour Into the Picture:Using a Spidery Mesh to Make Animation from a Single

Image

Youichi Horry, Ken-ichi Anjyo, and Kiyoshi Arai

SIGGRAPH 1997

presented by Dave Edwards10/31/2000

A Unique Problem

Can we create animation from a single image? Paintings Single photographs

Extremely difficult for a computer No information for depth or 3D structure

Not as difficult for a human brain

A Unique Solution

A system that allows humans and computers to do what they do best Human supplies 3D scene information Computer supplies efficient interface

for:Specifying vanishing pointsDifferentiating foreground & backgroundBuilding the scene with simple polygons

Tour Into the Picture (TIP)

System that allows efficient specification of 3D information

Basically, construct a diorama-like model of the scene Simple, generic representation

5 (or fewer) polygons for backgroundSmall number of polygons for each

foreground object

Overview

Scene modeling Separate foreground & background Specify a vanishing point Model the background Model foreground objects

Scene animation & renderingConclusionsDemo

Scene Modeling

Assumptions Scene is basically a box Sides of the box are orthogonal to each

other Virtual camera may not be tilted

View up vector is perpendicular to “floor”

Separate Fore- & Background

Start with a single image:

Separate Fore- & Background

Create two images with standard image editor

Foreground mask: One gray region for each object Grayscale map of alpha values

Background image: Original image w/o foreground Use neighboring pixels to fill gaps

Separate Fore- & Background

Example foreground mask:

Separate Fore- & Background

Example background image:

Specify a Vanishing Point

Use a “spidery mesh” Inner rectangle

Specifies “rear wall” of scene

Radial perspective linesSpecify the location of the vanishing pointShould be parallel to guide lines in the

picture

Interface allows rapid changes in the mesh

Specify a Vanishing Point

Example of Interface:

Model the Background

Partition the background into 5 pieces: Rear wall (inner rectangle) Floor (y = 0) Ceiling Left Wall Right Wall

Calculate 3D coords for the vertices of the box

Model the Background

Example of background model:

Model the Foreground

Foreground objects defined by maskConstruct polygon model of each

object Must be orthogonal to one side of the box Calculate 3D coords for corners of

polygon

Model the Foreground

More complex objects can be made of multiple polygons Each additional polygon must be

connected to the previous one This allows the system to determine its

3D coordinates

Model the Foreground

Example of foreground model:

Scene Animation & Rendering

Camera transformations: Rotation (change view-plane normal) Translation Zoom (move viewpoint forward) Change view angle Look around (orbit about central object)

Scene Animation & Rendering

We know 3D coordinates for points in the scene

Given a pixel to render, find its 3D image in the original scene

Apply smoothing for the backgroundApply alpha blending for the

foreground objects

Conclusions

TIP is very flexible In general, assumptions do not create

severe constraints Generic foreground/background model

works with many types of imagesInterface is (fairly) simple & intuitive,

even in the demo version It could use a little internationalization...

Future Possibilities

Handle two-point perspective drawings

Allow multiresolution images

Improve the model for foreground objects

Demonstration