Sharpness Estimation for Document and Scene Images

Jayant Kumar, PhD Student, UMD

Francine Chen, FXPAL

David Doermann, UMD

Vision

OCR engine

Poor accuracy

Poor readability

Tilt, Scale, Lighting, Blur

Sharpness Estimation

• Measure of best in-focus image

• Measure of how sharp details (like edge) are preserved

> >

Automatic creation of photo-books

Select the least blurry of several similar images

Images can be scaled so that blurrier images are smaller (hence look nicer!)

Real-time Feedback for better Capture

Sharpness score

User can adjust camera for better sharpness

Causes of Blur

(a) (b)

(c) (d)

Focal blur Focal blur

Motion blur Blur due to limited depth-of-field

Limitations of Previous Work

• Assumption on good Edge-detection

• Offset in Gradient Computation [Batten 2000]

• Poor time-performance (DCT Coefficients in blocks [Brandao et

al. 2008, JNB, CPBD]

• Blur-width quantized as number of pixels – Just-Noticeable-Blur (JNB) [Ferzli et al. 2009], Cumulative probability of blur detection (CPBD)

[Narvekar and Karam 2011]

• Dependence on image content – Qmetric [Zhu et al. 2010]

Sharpness as Edge-width?

Many previous approaches rely on edge-width for sharpness estimation1,2

1. R. Ferzli and L. Karam. A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Transactions on Image Processing, 2009.

2. N. Narvekar and L. Karam. A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Transactions on Image Processing, 2011.

These measures are somewhat coarse for document images composed of sharp edges with high contrast and frequent transition between back-ground/foreground

Ideal Sharpness Measure

• Correlates well with human perceived quality of sharpness

• Works for images from different domains: scenes, objects, and documents

• Images of same scene/document as well as across images of different scenes/documents

• Indicative of OCR accuracy for document images

• Fast enough to compute as a pre-processing step in many applications

Proposed Sharpness Measure

∑|∆DOM|

∑|∆I| > T sharp

blur ∑|∆DOM|

∑|∆I| < T

Rx = NsharpX

NedgeX

Ry =

NsharpY

NedgeY

Rx + Ry 2 2 √ ( ) SI

=

Median-filtered image

∆DOM model

∆DOMx(i, j) Im(i+2, j) Im(i,j) [ ] - Im(i, j) Im(i, j-2) [ ] - - =

Im = Median filtered image Median filtering smooth variations due to noise while preserving edges

Underlying edge is non-linear and can be modeled by a non-parametric model, such as the difference of differences that can model changes in the direction of a line.

From this model we derive a measure that captures whether the slope changes quickly, a characteristic of sharp edges.

edge-width

Why ∆DOM works ?

Datasets

• Document Image Sharpness Dataset (DocSharp)

– Pages taken from magazines, technical articles etc.

– 27 sets of 5 images each (135 images)

– Mechanical Turk for pair-wise comparisons

– 20-25 judgments for each pair

• Scene Images

– LIVE dataset1: Gaussian-blurred subset of 174 images

– CSIQ dataset2: Gaussian-blurred subset of 150 images

1: H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik. Live image quality assessment database release 2. In http://live.ece.utexas.edu/research/quality, 2006.

2: E. C. Larson and D. M. Chandler. Most apparent distortion: full reference image quality assessment and the role of strategy. Journal of Electronic Imaging, 19(1), 2010.

Evaluation

• Spearman Rank Correlation between ∆DOM sharpness measure and perceived sharpness (MOS/DMOS* scores and ranked pair-wise judgments)

• Accuracy on pair-wise judgments

• Time performance

• Comparisons with JNB, CPBD and Q

*MOS: Mean opinion score

*DMOS: Difference in mean opinion score

Spearman Correlation

Paired t-test for DocSharp:

∆DOM is the only measure that performs well on both document and scene images

Computation Time

JNB CPBD Q ∆DOM

DocSharp 33.63 55.46 12.36 3.91

LIVE 2.25 1.05 0.88 0.27

CSIQ 1.71 0.68 0.66 0.26

*Tested on Android for real-time sharpness on preview frames

Comparison with CPBD and JNB

JNB CPBD ΔDOM

(a) 0.84 0.71 0.78

(b) 1.03 0.59 0.174

(c) 1.35 0.76 0.17

(a)

(b)

(c)

Comparison with Q measure

Comparison with other methods

JNB CPBD Q Proposed

Human Perception of Scene Images

Human Perception of Document Images

Images of same object/document

Images of different objects/documents

Computation time

Conclusion

• Proposed a method for sharpness estimation in document and scene images

• Much faster than previous methods

• Works for same as well as different images