The Development of Image Completion and Tutorial Plug-ins for the GIMP

Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

The Development of The Development of Image Completion and Image Completion and

Tutorial Tutorial Plug-ins for the GIMPPlug-ins for the GIMP

By: Cathy Irwin

Supervisors: Shaun Bangay and Adele Lobb


Project AimsProject Aims

Automatic image completion plug-in for GIMP (GNU Image Manipulation Package)

Tutorial for general plug-in development


Image CompletionImage Completion Remove scratches, defects, writing & objects,

reconstruct damaged images - Manual techniques

are painstaking

Automate process – user only selects region to be

removed

Plug-in: realistically fill in background regions by

analysing known regions


Related WorkRelated Work

Texture synthesis

Image inpainting Efros & Freeman (2001)

Bertalmio, Sapiro, Caselles & Ballester (2000)


Image Completion Plug-inImage Completion Plug-in

‘Fragment-Based Image Completion’– Drori, Cohen-Or & Yeshurun (2003)


My ApproachMy Approach

Re-implementing proven technique BUT Incorporating it into the GIMP Inevitable differences – due to language, environment,

coding style

Verify authors’ results Scenarios

Textured regions Smooth regions Geometric shapes


Fragment-Based Image Fragment-Based Image Completion AlgorithmCompletion Algorithm

Fast Approximation

Confidence Map

Candidate Position Map

Search

Composite Fragments


Implementation in the GIMPImplementation in the GIMP

GIMP 2.0Use ‘empty’ plug-in templates as basis for

implementation The implementation language: C Gimp and GTK libraries

manipulate the GIMP-specific elements: images, drawables, channels and layers


Fast ApproximationFast Approximation

X


Fast ApproximationFast Approximation

Each step is added as a new layer to the original image

Manipulate and view independently of one another


Confidence MapConfidence Map Certainty with which the colour value of each pixel is

known Gaussian falloff value used to decreases the confidence

level towards the center of the unknown region


Candidate Position MapCandidate Position Map Takes confidence map as input Pixel with the maximum value in the candidate map is

concluded to be the next most appropriate target pixel to be used in the search and composite phases


Search PhaseSearch Phase

Iterates through each pixel in a 20 x 20 pixel block around the target to find an appropriate source pixel

Average value of the pixels around each potential source it is taken to determine the best possible match with the neighbourhood surrounding the target pixel

Size of this neighbourhood region determined by the user at run time but is usually about 8 pixels wide

The search formula finds the pixels which have a higher confidence in the source than in the target regions and correspond well in terms of colour


Compositing FragmentsCompositing Fragments Superimpose that source fragment over the target

fragment so that the detail from the source merges seamlessly with the region around the target.

Transparent edges - blend in convincingly Use existing GIMP functions:

– Selections– Copy– Paste



ResultsResultsSuited to: Stochastic (random)

texture that occurs often in the surrounding region

Effective at making the reconstructed area blend into the known regions


Results cont…Results cont…

3D structure not taken into account

Complete image according to the direction the line was taking at the edge of the gap and not according to mathematical principles


Results cont…Results cont… Parameter values depend on the size of the gap

and the type of texture in the picture 160 x 120 pixel image: between 12 minutes and an

hour to process Smaller gaps need a smaller blur radius and

smaller neighbourhood search region. Larger, smooth or stochastic regions can be accurately completed using a larger neighbourhood region that also decreases the processing time


Results cont…Results cont…


On-line Plug-in Tutorial On-line Plug-in Tutorial General tutorial for writing plug-ins for the GIMP in

C Tutorial Page Provide a starting point for beginners, not a definitive

guide on every aspect of plug-in development Areas covered:

– areas that I found problematic during the development of the plug-in

– features that were discovered to be particularly useful– references to where more specific information can be

found


ConclusionConclusion Plug-in development aided by availability of templates and

documentation regarding GIMP specific functions Some common image manipulation operations complex to

implement Not possible to re-implement the original paper exactly

within the GIMP environment due to the tool and function constraints and the extra processing required for certain operations

A plausibly similar implementation has been achieved - takes advantage of GIMP functions to approximate the intentions of the original paper


Documents

The Development of Image Completion and Tutorial Plug-ins for the GIMP