Correcting Projector Distortions on Planar Screens

via Homography

Method and Setup Proposal for Correcting Distorted Projected Images

on Planar Screens Under Difficult Conditions

Daniel Hirt

Introduction to Computational and Biological Vision Project

Computer-Science Department

Ben-Gurion University

2013

Abstract

In the following paper we will discuss and present an approach and implementation for an

automated setup for correcting a projection of an image from a projector device.

A single, user-friendly method will be established so that any user can operate such a setup

even at home, provided a projector device, from harsh (sometimes unimaginable) angles, and

get proper satisfactory results, i.e. a corrected image on the projection screen.

1. Introduction In our day and age we rely heavily on projector systems. Whether it is for educational (such

as presentation displaying), or even entertainment purposes (television, movies etc.), it is

clear that projector systems had become a popular, and sometimes essential, tool to fulfill our

needs in our daily lives. Moreover, costs of projectors had decreased dramatically over the

years, so that today an average person could afford one.

However, it seems that though costs for the projector unit alone had become affordable, in a

lot of cases the designated environment for it is far from being appropriate for its operation.

In a lot of cases the space in which the projector will be operating doesn‟t allow for an ideal

setup for proper viewing on the screen.

An ideal setup for the common commercial projector consists of an overhead mounting of the

projector, in a perpendicular angle to the projection screen. Any considerable deviations from

the recommended setup easily result in a distorted unattractive image displayed on the

projection screen.

The common projector device has built-in tweaking capabilities, allowing the user to correct

some distotion, but at a limited range.

Although an ideal recommended setup does have its virtues, aesthetically-wise, the main

claim is that what the common consumer wants is flexibility, especially when there is little

choice on where to place the projector device.

The following sections of this paper we will propose an approach and implementation for a

system capable of handling harsh scenarios for projector device placement with little user

intervention, avoiding preliminary requirements for such a solution from other methods.

2. Perspective Projection Layout

Consider the following layout depicting the course our original image goes through, starting

from our computer/projector to our flat projection screen.

Denote (x1, y1) be a point in our image on the computer, then (x2,y2) is the projection of that

point on the screen.

An ideal positioning of a projector i.e. perpendicular to the flat screen will result in an image

identical to the image on the computer in its proportions.

However, if the projector is located in a notable angle from the screen, the projection will

result in an uneven image on it.

Figure 1 - Projection of an Object on a Flat Screen from a Difficult Angle (not a real-life depiction)

(x1,y1)

(x2,y2)

Many common projectors offer correction for limited types of distortions, the most popular

being a “keystone-effect correction”.

Figure 2 - "Keystone effect" - when the projection axis is not perpendicular (below here) to the screen

In common projector devices such built-in correction mechanisms are not only limited, but

also require user intervention to manually determine the suitable amount of correction values.

Our approach will help both problems: provide corrections solutions for a wide range of

distortions and with little user-intervention.

However, there is another projection in our system – the projection to viewer‟s eye. We will

consider this projection as the projection from the screen to a camera device.

Eventually, we wind-up with a system as follows:

Figure 3 - an appropriate depiction of the common projector system

Projector device

Projector device

Viewers/camera

Each of the factors obey perspective geometry principles. Under regular representations of

the images we need to know information about each system. These parameters are both

intrinsic (focal length, principal point) and extrinsic (rotation, translation) parameters. With

that amount of unknowns we are bound to take another approach in order limit user-

intervention and preliminary actions.

3. Homography in a Nutshell

Recall that in our system we deal with planar projections – each of the images is projected on

a flat screen, thus z=0 for every point on the plane.

The camera model in homogenous form:

The properties of homogenous representation, where z=0 give us the following relationship:

Setting w=1, we end up with an 8-DOF equation system.

In order to solve an 8-DOF equation system of the algebraic distance ⃗ we require a

set of four points (x,y). A common method of solving this is by applying Least Squares on

the algebraic distance of the homogeneous system ⃗ , where ⃗ is a set of points.

In our approach we will apply the robust RANSAC algorithm to guarantee outlier exclusion,

while providing a sufficient amount of sample points (more than 10).

Every projection of the three in our setup can be represented by the above equation.

Moreover, the projection from a point in an image, which is itself a flat projection screen, to

another planar screen is reversible.

4. Approach and Method

In this section of the article we will draw the outlines of one-time procedure for a specific

placement of a projector. For any new placement of the projector, this procedure should be

reapplied. In any case, the procedure itself is easy and requires little as possible user

intervention. The implementation developed (in C++) provides a demonstration, but can be

easily improved with more user-friendly features to allow an easy user-experience.

This suggested method provides high flexibility even at an oriented projected device (though

up to a certain limit of rotation) without falling into ambiguities.

In the solution we require the following:

A Computer

A standard resolution webcam

A projector device

Some approaches suggested mild deviations in the projection axis from the projector to the

projection screen. Such approaches relied solely on the four corners of the projected

boundaries. If difficult angles such as some slight rotations were applied, it would be hard to

determine how the projector was rotated. Moreover, the mixture of an unknown webcam and

unknown projector will add impurities to the whole set of equations. Thus, using more than

four points in a sampling consensus to solve the equations seems logical, to get proper results

in difficult scenarios.

Note that in this method we rely that the viewer sits in an ideal position in front of the

projection screen. However, we do not limit the projection screen to be nothing more than a

flat surface (even a wall, which has no boundaries).

First, we project a specific pattern through the projector, to allow grabbing more than 4

correspondence points. Second, we compare an ideal projection of the same pattern the

viewer would‟ve expected to see if the projector was properly placed. Next, we calculate the

Homography matrix that transfer the points from the wall-projected image to the reference

(correct) image, denoted H, via a random sampling consensus (RANSAC). Finally, we apply

perspective warp with H on a chosen sample image. At this stage the image should be

properly warped to be displayed correctly under the projection conditions.

Figure 4 - initially we project the pattern

Pattern

Figure 5 - after applying the homography on the input pattern, the output is projected correctly

5. Implementation

The method suggested in this paper was implemented in a C++ program for demonstration

purposes and verifying results under tough conditions.

The code relies heavily on the OpenCV v2.4.31 open-source library in a linux operating

system. Source code 2for this implementation is provided.

The algorithm:

1. Verify camera device is connected, otherwise give error and exit

2. Provide the user a reference 6x8 chessboard pattern to project via the projector

3. In fixed frame intervals, look for chessboard pattern in the camera‟s view, and in each

time a pattern is found, update the display with numbering of all inner-corners of the

pattern. If numbering haven't been updated, the pattern was not yet found.

Also, wait for a key input „c‟ to capture a proper detected chessboard frame. If „c‟

was pressed, proceed to step 4.

Repeat step 3.

4. Create corresponding points of the expected 6x8 chessboard pattern3, denoted pts1, to

those detected at the end of step 3, denoted pts2. Expected corresponding pairs of

points (inner corners): 35.

5. Improve4 set of points pts1 by resizing with a factor denoted by the ratio d2/d1, where

di is the diagonal length between ptsi[0] to ptsi[11].

1 Currently, OpenCV 2.4.4 is the most up-to-date version

2 To build the program, consult the official OpenCV site: www.opencv.org

3 A high number of corners was used in order to establish more sampling at the corners of the projected screen,

where high distortions could appear in difficult positioning of the projector

Pattern

6. Calculate via RANSAC the Homography matrix from points set pts2 to pts1, denoted

H.

7. Apply (pre-) warp perspective: H∙sample_image, where “sample_image” is a sample

image provided by the user on execution of the program.

8. Display both original and warped images and prompt user to display (drag) it to the

projector screen.

9. End demonstration.

4 This step was added to maintain proportions of output image, as the captured pattern on the projection wall always

turns out smaller than the original pattern image, leading to an oversized result from the calculated homography.

6. Test Results

Provided are the test results produces by a setup of a low-cost projector device, a standard laptop,

and a standard 640x480 resolution webcam.

In each test we examine a different placement of the projector against the projection screen

(white wall in this case).

Pico Projector (low-cost) 1024x768

Laptop (Linux OS)

Standard 640x480 webcam

Original Image Fixed Image

7. Conclusions

Homography indeed offers a somewhat hassle-free approach to solve the adressed problem

with common projectors. We have managed to establish a process that approximates proper

corrections for a wide range of images and positions of the projector. However, as mentioned

there are some unwanted results, due to the fact that we can not model our system as a pure

homography system. Nevertheless, we can get closer to a universal solution if we adopt other

method than homography.

8. Future Development

Integrate the process to graphic display, so that the whole desktop is projected correctly,

and not only images.

Better patterns for better correspondence matching, to allow an almost full-rotation of the

projected plane. Right now we may get some ambiguities once we get close to 90 degrees

rotations.

Extend to non-planar screens - can‟t be solved with homography.

Handle cropping of corrected images: an optimal resized image in the distorted projection

screen.

9. References

9.1. Computer Vision, Algorithms and Application, Richard Szeliski

9.2. Smarter Presentations: Exploiting Homography in Camera-Projector Systems, Rahul S.,

Robert G. S. and Matthew D. M.

9.3. Introduction to Computer and Biological Vision Lecture Notes, Prof. Ohad Ben-Shahar,

Computer-Science Dept. ,Ben-Gurion University.