Color strategies for object identification

Qasim ZaidiMarques Bostic

State University of New YorkCollege of Optometry

Color-Based Object Identification

Can you identify which parts are sunny or shady? How?Why do you infer illuminant rather than material changes?Can you identify similar foliages from sunny to shady parts?

Strategies for color-based object identification

Color Constancy: Constancy of subjective appearance by adapting to the illuminant ( Ives, Helson, Smithson & Zaidi etc.).

Constancy of Color Contrast/Differences

Inverse Optics Methods: Estimate spectrum of illuminant and infer reflectance spectra which are physical invariants (Maloney & Wandell, D’Zmura & Iverson, Brainard & Freeman etc.).

Neural algorithms: Based on invariants in cone-catches.

Physical invariants for color identification

For natural and man-made materials, changes in cone-absorptions of reflected lights for large sets are simply multiplicative, hence rank-orders and cone-contrasts are preserved (Dannemiller; Nascimento & Foster; Zaidi, Spehar & DeBonet).

Neural invariants for color identification

In post-receptoral cordinates, the color conversion is translational for L/(L+M), and multiplicative for S/(L+M) (Zaidi, Spehar & DeBonet).

HEURISTIC ALGORITHM FOR IDENTIFICATION

0

0.01

0.02

0.03

0.04

0.05

0.06

0.5 0.6 0.7 0.8

L/(L+M)

Equal Energy Light

0.5 0.6 0.7 0.8

L/(L+M)

Skylight

0.5 0.6 0.7 0.8

L/(L+M)

EE --> Skylight

Recovery of Lambertian Objects

L/(L+M)

Identify a subset of objects under equal-energy light with the superset under skylight: Assume that the affine transformation holds, and derive its parameters by matching patterns of chromaticities (Zaidi, 1998). This process should almost always lead to correct identification, which is easy to test.

Using parallel color difference vectors will lead to correct identifications for many but not all reflectances (Khang & Zaidi). Test this alternative by comparing distractors on lines parallel and orthogonal to the line joining the illuminants.

COLOR-DIFFERENCE/CONTRAST STRATEGY

Three of the objects are identical, but the fourth is made from a different paper. The two backgrounds are of the same material, but the

two illuminants have different spectra.

Which object is made from a different paper than the other three?

Exp 1: COLOR BASED OBJECT IDENTIFICATION

Given that neither of the discriminable objects on one side looked exactly like the objects on the other side, what strategy did you follow?

Number 1 is made from a different paper than the other three objects.

Stimuli

LegendEach data point is the result of 10 observations

O = side correct < 80%, i.e. observer cannot discriminate between standard and test under the same illuminant.

O = object correct ≥ 80%, i.e. observer identifies correct objects across illuminants.

O = object correct ≤ 20%, i.e. observer misidentifies the standard as the odd object.

O = 20% < object correct < 80%, i.e. no reliable identification across illuminants.

O = test object under bluish light

O = test object under yellowish light

Observer: RH – uninformed, experienced

Two other observers gave very similar results

RESULTS: COLOR BASED OBJECT IDENTIFICATION

RESULTS: COLOR BASED OBJECT IDENTIFICATION

RESULTS: COLOR BASED OBJECT IDENTIFICATION

RESULTS: COLOR BASED OBJECT IDENTIFICATION

Object Identification Results Summary•Object identification across illuminants is not perfect, even when tests can be discriminated from the standards.

•Because the standard object is a possible choice on every trial, the systematic misidentifications rule out color-constancy, inverse-optics and the affine-heuristic algorithm.

•Near the color of the standard, misidentified and correctly identified objects tend to fall on opposite sides, regardless of the color-directions of the distractors. This rules out the color-difference/contrast strategy.

•Can a simple perceptual strategy account for the performance results, similar to the strategy for lightness based object identification (Robillotto & Zaidi)?

Exp 2: RELATIVE COLOR CATEGORIZATION

Instructions: Compared to the standard object, is the test predominantly yellower, bluer, redder, or greener?

Observer: QZ – informed, experienced. Three other observers gave very similar results

Standard and test under the same light

Legend

The C O L O R of the circle corresponds to observer QZ’s judgment of the test object being predominately yellower, bluer, redder or greener

than the standard

O = no judgment could be made because the standard and test could not be discriminated

O = test object under bluish light

O = test object under yellowish light

RESULTS: RELATIVE COLOR CATEGORIZATION

RESULTS: RELATIVE COLOR CATEGORIZATION

RESULTS: RELATIVE COLOR CATEGORIZATION

RESULTS: RELATIVE COLOR CATEGORIZATION

Summary of Color Categorization Results

Standard and test under the same lights

* Tests separated into two relative color groups with respect to the standard. The division was similar under the two lights, and corresponded to clusters of correct and incorrect object identification.

Similarity based algorithm for object identification across illuminants

1.Identify the illuminant on the test (It) by finding the side on which the two objects have different colors.

2. Compare the colors of the backgrounds to judge the change in color from Is to It.

3. Pick the object most dissimilar to the others along the perceived It - Is color dimension as the odd object.

Predictions for tests under the blue/green light

Tests perceived as bluer/greener than the standard, will be picked correctly as the odd object. Tests perceived as yellower/redder than the standard, will not be picked as the odd object, unless they are yellower/redder than the standards in the yellow light. Opposite for tests under the yellow/red light

Similarity-based Predictions vs Results

I G M TOTAL

I 43 7 4 54

M 4 5 15 24

TOTAL 47 12 19 78

Correct predictions: 92% of identifications & 79% of misidentifications.

Observed

Predicted

From QZ’s relative colors to RH’s object identification.

I = correct identification, M = misidentification, G = neither

“Opportunistic” Bayesian Strategy Selection?

•Why do observers use color similarity for object identification despite the demonstrable absence of color constancy?•Why are more accurate alternatives not used despite sufficient information in natural scenes?•Is the visual system an “Opportunistic” Bayesian which includes the costs of additional computations in strategy selection?