SHAPE INFERENCE FOR SHEET OF PAPER WITH TEXT VIA

CHARACTERISTIC STRIPS

Project byArie Kozak

Introduction and goals

Given photograph with sheet of paper with text only, infer shape of the surface and plot it in 3d.

Single (infinite) light source from above, using reflectance map (paper is nearly Lambertian surface):

The surface is assumed to be constant in one direction.

Step 1Locate the sheet in the image

Mark it using personal biological visual system.

Step 1Locate the sheet in the image

Divide the image into two connected sub-images divided by red border.

Step 2Locate text

Use thresholding twice: after high pass and original image. Text found in the intersection.

Step 3Remove text

Constant albido assumption for ink, doesn’t work, use (cubic) interpolation.

Smooth image with Gaussian kernel before to reduce “sharpening effect” (lateral inhibition), and also after.

Step 4Starting points for characteristic strips

Maximum intensity point in image => p = q = 0. Use parabolic approximation according to B.K.P. Horn's chapter 11:

Step 4Starting points for characteristic strips

Solution

Only solutions with a<0,c<0 are relevant.

Step 4Starting points for characteristic strips

Identify “clusters” – areas of local maxima/minima. All points within certain % of highest intensity values.

Step 5Apply characteristic strips

Start with H = 0, perform for each cluster separately.

Step 6Merge clusters

Find closest clusters A and B; B with known height.

For points in A close to B, calculate expected height according to B.

Find closest points using Voronoi diagram.

Step 6Merge clusters

Find relative height between A and B. If is current and expected height of point i accordingly, find relative height x, such that error will be minimal:

Step 7Rebuild the surface of the sheet Find direction v, in which H is constant

=> derivative is 0.

Find least square line, its directions is perpendicular to v.

Step 8Rebuild the surface of the sheet If v is new x-axis, calculate projection of

all points to YZ plane.

Step 8Rebuild the surface of the sheet Use polyline approximation. Given

number of desired points = number of clusters + 2, the desired error can be approximated using binary search.

Example – 5 points:

Step 8Rebuild the surface of the sheet Finally, use spline, on polyline edge

points.

Results

Not perfect, usually works sufficiently.

Results

Future work

Detect sheet of paper automatically. Relax assumptions (light direction, H is

constant in one direction). Improve clusters search. Replace/improve polyline approximation. Use this for text recognition.