Photoshop Quickselect & Interactive Digital Photomontage

Interactive Digital Photomontage1

Photoshop Quickselect & Interactive Digital PhotomontageBy Joseph Tighe

Interactive Digital Photomontage2

Photoshop Quickselect

Based on the graph cut technology discussed (Boykov-Kolmogorov)What might happen when we use a color model?

Interactive Digital Photomontage3

Photoshop Quickselect

Interactive Digital Photomontage4

Demo

Interactive Digital Photomontage5

Photoshop Quickselect

So we abandon the color model and only use pixel similarities.The “Refine Edge” matting is applied as a post process step to give good edges boundaries.

Interactive Digital Photomontage6

Interactive Digital Photomontage

Aseem Agarwala1 Mira Dontcheva1

Maneesh Agrawala2 Steven Drucker2

Alex Colburn2 Brian Curless1

David Salesin1,2 Michael Cohen2

1University of Washington 2Microsoft Research

Interactive Digital Photomontage7

Photomontage: “combining parts of a set of photographs into a single composite picture”1 while keeping noticeable seam to a minimum.

1 Agarwala et al.

Interactive Digital Photomontage8

Examples: Group Shot

Interactive Digital Photomontage9

Examples: Depth of Field

Interactive Digital Photomontage10

Examples: Depth of Field (Result)

Interactive Digital Photomontage11

Video

Interactive Digital Photomontage12

Overview of Approach

Use a graph cut optimization to determine what regions from which images are used (Labeling)Use gradient domain fusion based on the Poisson equation to blend any remaining artifacts away.

(note: alignment is needed as a pre processing step and is not covered in this paper.)

Interactive Digital Photomontage13

Graph-cut Optimization

∑∑ +=qp

iP

d qLpLqpCpLpCLC,

))(),(,,())(,()(

Cost function to minimize:

Data penalty(Image Objective)

Interaction penalty(Seam Objective)

Interactive Digital Photomontage14

Image Objectives

Designated ImageDesignated ColorMin/Max LuminanceMin/Max ContrastMin/Max LikelihoodEraserMin/Max Difference

Interactive Digital Photomontage15

Designated ImageFor each stroke the user specifies which source image the pixels are to come from

Interactive Digital Photomontage16

Designated Color

Specify a target color and find source images that have similar or different colors.Cost function given by: Euclidean distance in RGB space.

Interactive Digital Photomontage17

Min/Max Luminance

Min (max) of luminance channel. Good for adding shadows/highlightsCost function given by: the distance in the luminance channel between the current source pixel and the min/max for that pixel’s span (all source pixels at that location)

Interactive Digital Photomontage18

Min/Max ContrastMin: Remove small sharp obstructions (wires)Max: Good for increased depth of field (ant)Cost function given by: Min/Max of a difference of Gaussians for each pixel’s span

Interactive Digital Photomontage19

Min/Max LikelihoodGood for removing people in front of buildings.Cost function given by: the probability of that pixel given by the distribution across that pixel’s span in RGB.

Interactive Digital Photomontage20

Eraser

The color most different from the current color (works like designated color, except for the color is designated by the current composite)Cost function given by: Euclidean distance in RGB space

Interactive Digital Photomontage21

Min/Max DifferenceThe min or max difference between some specified source image at each pixelCost function given by: the Euclidean distance in RGB space between the pixel in the specified source and the pixel’s span

Interactive Digital Photomontage22

Seam Objectives

Colors (Faces)Colors & Gradients (Automatic/Global approaches)Colors & Edges (Multiple versions of the same scene)

Interactive Digital Photomontage23

Seam Objectives

z image of q and p pixels gneighborin obetween tw potential edgescalor theis ),(p pixelat z image of RGBin gradient color component -6 a is )(S

),(),(

)()()()(

)()()()(

edges" & colors" matching if/

gradients" & colors" matching ifgradients"" matching ifcolors"" matching if

L(q) L(p) if0

))(),(,,(

z

)()(

)()()()(

)()()()(

qpEp

qpEqpEZ

qSqSpSpSY

qSqSpSpSX

ZXYX

YX

qLpLqpc

z

qLpL

qLpLqLpL

qLpLqLpL

i

∇

+=

∇−∇+∇−∇=

−+−=

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧

+

=

=

Interactive Digital Photomontage24

Graph-cut Optimization

∑∑ +=qp

iP

d qLpLqpCpLpCLC,

))(),(,,())(,()(

Cost function to minimize:

Data penalty(Image Objective)

Interaction penalty(Seam Objective)

Interactive Digital Photomontage25

Graph-cut Optimization (Boykov-Kolmogorov)

f

:f f : E(f)) fE(

ffE(f')f

:f

Return 5.2 goto 1 success If 4.

1 success and set , If 3.2.

ofexpansion - one within ' among min arg Find 3.1.

Lin labeleach For 3.0 successSet 2.

labelingarbitrary an Start with 1.

===<

=

=

∧∧

∧

α

α

From: BOYKOV, Y., VEKSLER, O., AND ZABIH, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 11, 1222–1239.

Interactive Digital Photomontage26

Graph-cut Optimization (Boykov-Kolmogorov)

From: BOYKOV, Y., VEKSLER, O., AND ZABIH, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 11, 1222–1239.

Interactive Digital Photomontage27

How is this applied?

GloballyLocally (painting strokes)

Interactive Digital Photomontage28

GloballyNo user interactionCost function is minimized for all pixels and all images

Interactive Digital Photomontage29

Locally (Single Image Brush)

Interactive Digital Photomontage30

Locally (Multiple Image Brush)

Applies an Image Objective locally but draws from multiple sources to minimize that objectiveSeam Objective is applied globally

Interactive Digital Photomontage31

Gradient-domain Fusion (Blending)

Use the labels to determine where to copy the color gradients from.Create a composite image using the techniques described in Perez et al. 2003 (Poisson image editing)

Interactive Digital Photomontage32

Demo

Interactive Digital Photomontage33

Observations

Image Objectives rarely produce results that can’t be obtained just by using the designated image objective

Exceptions are the fully automatic methods