PowerPoint

Palette-based Photo Recoloring Huiwen Chang Ohad Fried Yiming LiuStephen DiVerdi Adam Finkelstein

SIGGRAPH 20151OutlineIntroduction ApproachEvaluationResultsConclusion2IntroductionDemo: http://recolor.cs.princeton.edu/demo/index.html

3IntroductionGoal4

originalIntroductionThis paper introduces a tool that is easy for novices to learn while offering a broad expressive range.

Our approach specifies both the colors to be manipulated and the modifications to these colors via a color palette.

We show that our algorithm can easily be accelerated that allows it to run at interactive frame rates.5IntroductionWe perform a study showing that with our tool untrained users can produce similar results to those of expert Photoshop users.

Finally, we show that our palette-based color transfer framework also supports other interfaces.stroke-based interfacelocalized editing via a selection maskfully-automatic palette improvementediting a collection of images simultaneously6Approach7User interfaceAutomatic Palette SelectionColor Transfer GoalsMonotonic Luminance TransferColor TransferAccelerationSimpleExpressiveIntuitiveResponsive(k + 1)-meansInterpolationIn GamutPixel ContinuityPalette ContinuityOne-to-OneMonotonicity in LDynamic RangeApproach(1)-User interfaceThese criteria are important for a color manipulation user interface:SimpleExpressiveIntuitiveResponsive

While there are many existing tools that allow users to recolor images, to our knowledge none of them simultaneously achieve these goals.8

Approach(1)-User interfaceWe describe a palette-based GUI

The user needs only to click on a palette color (C) and change it (to C) via a HSL color picker.9

Approach(2)- Automatic Palette Selection10

Approach(2)- Automatic Palette Selection11

KANUNGO, T., MOUNT, D., NETANYAHU, N., PIATKO, C., SILVERMAN, R., AND WU, A. 2002. An efficient k-means clustering algorithm: analysis and implementation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 24, 7 (Jul), 881892.Approach(2)- Automatic Palette Selection12

falloffApproach(2)- Automatic Palette SelectionWe discourage very dark palette entries as follows.

Rather than computing k-means, we actually compute (k + 1)-means, where one of them is initialized and perpetually locked to black.

The darkest colors in the image will be assigned to this mean, and will therefore not pull the other nearby means towards black.13Approach(2)- Automatic Palette SelectionFor a comparison of our approach with the previously described methods.14

(D) ODonovan et al. [2011]

(L) Lin et al. [2013]

(G) Gaussian mixture models (GMMs) used by Shapira et al. [2009]

(K) K-means

(O) Our methodODONOVAN, P., AGARWALA, A., AND HERTZMANN, A. 2011. Color Compatibility From Large Datasets. ACM Transactions on Graphics 30, 4.LIN, S., AND HANRAHAN, P. 2013. Modeling how people extract color themes from images. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 13.SHAPIRA, L., SHAMIR, A., AND COHEN-OR, D. 2009. Image appearance exploration by model-based navigation. In Computer Graphics Forum, vol. 28, 629638.10s2s60msApproach(3)- Color Transfer Goals15Approach(3)- Color Transfer GoalsInterpolation.

In Gamut.

Pixel Continuity.

Palette Continuity.

One-to-One.

Monotonicity in L.

Dynamic Range 16

A gradient in the input maps to a gradient in the output (not a single value).Approach(3)- Color Transfer GoalsTo our knowledge no existing approaches can be easily adapted to meet the requirements listed above.

To make matters worse, the requirements themselves are inconsistent.17Approach(4)-Monotonic Luminance Transfer18

in the input palettein the edited palette

Approach(4)-Monotonic Luminance Transfer19Approach(5)- Color Transfer (ab)20

Approach(5)- Color Transfer21

These intersections are found by binary search.

Approach(5)- Color Transfer22

Approach(5)- Color TransferUnfortunately solving the system of equations set up by the RBFs can lead to negative weights, with two potential hazards.It will add some component of the opposite behavior of some palette changes.It can throw the result out of gamut.

We clamp any negative weights to zero and renormalize the non-zero weights.23

a palette size of k = 3Approach(6)- Acceleration24Approach(6)- AccelerationWithout acceleration, our runtime is .With acceleration, runtime is .Throughout this paper and in our demo we use g = 12, because it performs well across a range of computers and images, so .25

Results of varying grid size g

Update time vs. grid size

Evaluation(1)-Other Recoloring Methods26

Chen et al. [2014]Shapira et al. [2009]Wang et al. [2010]Evaluation(2)-User StudyIn order to judge expressiveness, the task we give our users is a matching task.

We collected 32 original-target pairs.-Half (16) were selected from eight papers.-The other half were created by two expert Photoshop users.

For the study we used Amazons Mechanical Turk framework.

Each worker was assigned randomly to one of three different conditions: our method, GMM, and hue-blend.27Evaluation(2)-User Study28

Evaluation(2)-User StudyAggregates all distance-from-target results.29betterEvaluation(2)-User StudyShows completion times over all subjects, for each of the three methods. 30

roughly the sameslowerslowestResultsShow examples of using our method with palettes of size 3 and 5. 31

ResultsOur method supports masks, constraining the edits to specific image regions.32

Results(1)-Video RecoloringOur method is fast enough to be applied to a video, in real time. The interaction uses the same interface as for photos.

33

Results(2)-DuotoneDuotone reproduction is a traditional printing technique that typically involves two colors of ink applied via halftone patterns over white paper. Our method can easily produce a similar effect.34

Results(3)-Automatic Color ManipulationWe can automatically pick a top rated palette and apply it to an image.35

Results(4)-Editing an Image CollectionWe show that our system can trivially operate on a collection of images simultaneously.36

original collectionresultResults(5)-Stroke-Based InterfaceOur palette based editing method can easily be augmented with a stroke-based interface that relies on the same algorithms described.

We set the palette size:

Stroke-based editing.37

localglobalinputinputoutputoutput

the approach of Chen et al. [2012] our methodinputinputoutputoutputConclusionThis paper introduces a simple, intuitive and interactive tool that allows non-experts to recolor an image by editing a color palette.

We show results for recoloring, image collection editing and video editing, and validate via a user study.

38Referenceshistogram adjustment methods in Photoshop https://goo.gl/AbJPHTHSL https://goo.gl/UqnlZb

39