Exact and Distributed Algorithms for

Collaborative Camera Control

Dezhen Song*

A. Frank van der Stappen†

Ken Goldberg*

* UC Berkeley, USA

† Utrecht University, Netherlands

Dezhen Song

Webcams

n users 1 pan, tilt, zoom robotic camera

“ShareCam”

Example input: 7 requested frames:

One Optimal Frame

ShareCam Problem: Given n requests, find optimal frame

Taxonomy (Tanie, Matsuhira, Chong 00)

Multiple Operator, Single Robot (MOSR):

Single Operator, Single Robot (SOSR):

Single Operator, Multiple Robot (MOSR):

Related Work• MOSR systems

– Cinematrix (91)– Cannon, McDonald, et al. (97) – Goldberg, Chen, et al. (00, 01) – Goldberg, song, et al. (02)

• Internet robots– Tanie, K., Chong, N. Et al(01)– Jia, S. And K. Takase (01)– Hu, H., Yu, L., Tsui, P., Zhou, Q (01)– Safaric, R. Et al. (01)– Goldberg and Siegwart (02)

Related Work• Facilities Location Problems

– Megiddo and Supowit [84]– Eppstein [97]– Halperin et al. [02]

• Rectangle Fitting – Grossi and Italiano [99,00]– Agarwal and Erickson [99]– Mount et al [96]– Kapelio et al [95]

Related Work• Similarity Measures

– Kavraki [98]– Broder et al [98, 00]– Veltkamp and Hagedoorn [00]

• Distributed robot algorithms – Sagawa et al [01], Safaric[01]– Parker[02], Bulter et al. [01]– Mumolo et al [00], Hayes et al [01]– Agassounon et al [01], Chen [99]

Problem Definition

Requested frames: i=[xi, yi, zi], i=1,…,n

Problem Definition• Assumptions

– Camera has fixed aspect ratio: 4 x 3– Candidate frame = [x, y, z] t

– (x, y) R2 (continuous set)– z Z (discrete set)

(x, y)3z

4z

Problem Definition• “Satisfaction” for user i: 0 Si 1

Si = 0 Si = 1

= i = i

•Symmetric Difference

•Intersection-Over-Union

SDArea

AreaIOU

i

i

1)(

)(

)(

)()(

i

ii

Area

AreaAreaSD

Satisfaction Metrics

Nonlinear functions of (x,y)

• Intersection over Maximum:

),(

)(

),max(

)1,)/min(()/(),(

i

i

i

i

biiii

Max

Area

aa

p

zzaps

Requested frame i , Area= ai

Candidate frame

Area = a

Satisfaction Metrics

pi

Intersection over Maximum: si( ,i)

si = 0.20 0.21 0.53

Requested frame i

Candidate frame

),(),( yxpyxs iii

),( yxpi

Requested frame i Candidate

frame (x,y)

)1,)/min(()/(),( biiii zzaps

(for fixed z)

Satisfaction Function

– si(x,y) is a plateau •One top plane•Four side planes•Quadratic surfaces at corners•Critical boundaries: 4 horizontal, 4

vertical

Objective Function• Global Satisfaction:

n

iii

n

i

biii

yxpyxS

zzapS

1

1

),(),(

)1,)/min(()/()(

for fixed z

ShareCam problem: Find * = arg max S()

S(x,y) is non-differentiable, non-convex, but

piecewise linear along axis-parallel lines.

Properties of Global Satisfaction

ShareCam Algorithms

Bruteforce Algorithm– Compute S at each pixel (x,y)– O(whmn):

•w, h: width and height of panoramic image

•m: number of zoom levels•n: # users

Approximation Algorithm

spacing zoom :

spacing lattice :

zd

dx

y

d

Compute S(x,y) at lattice of sample points:

Approximation Algorithm

– Run Time: – O(w h m n / d2)

* : Optimal frame

: Optimal at lattice ~

: Smallest frame at lattice that encloses *

)ˆ()~

()( * sss

)(

)ˆ(

)(

)~

(1

**

s

s

s

s

ddz

z

z

min

min...

Exact Algorithm

• Define “Virtual corners” – Consider a pair of requested frames

and – Their critical boundariesy

x

Exact Algorithm• Virtual corner: Intersection between

boundaries– Self intersection:– Frame intersection:

y

x

Exact Algorithm

• Claim: An optimal point occurs at a virtual corner. Proof:– Along vertical boundary, S(y) is a 1D

piecewise linear function: extrema must occur at x boundaries

Exact Algorithm

Exact Algorithm:Check all virtual corners

(mn2) virtual corners(n) time to evaluate S for each

(mn3) total runtime

Improved Exact Algorithm• Sweep horizontally: solve at each

vertical – Sort critical points along y axis: O(n

log n)– 1D problem at each vertical boundary

O(nm) – O(n) 1D problems– O(mn2) total runtime

O(n) 1D problems

Distributed Algorithm

More users More computers available

Distributed Algorithm• At the Server

– Sort horiz. boundaries– O(n log n)

• At the Client– Solve 1D problem

for own vertical boundaries.

– O(nm)

• O(n(m+ log n)) Total Four 1D problems

Examples

Examples

www.tele-actor.net / sharecam

• New Approx Algorithms:– With Har-Peled, Koltun– Stair-like

approximation– Dynamic segment tree– O(n log n)

• Weighted Requests

Current Work

Future Work

• Continuous zoom (m=)• Multiple outputs:

– p cameras – p views from one camera

• “Temporal” version: fairness– Integrate si over time: minimize accumulated

dissatisfaction for any user

• Network / Client Variability: load balancing

• Obstacle Avoidance

The Tele-Actor

Operators

Server

Summary• Satisfaction Metric: • Intersection over Maximum• ShareCam Problem : find * = arg max

S()• Critical Points at Virtual Corners• Exact Algorithms: • Distributed Algorithm:• tele-actor.net/sharecam

O(mn2)

O(mn)

Summary

• A collaborative camera control system

• Satisfaction metric• Virtual corner based

algorithms• Distributed algorithm• www.tele-actor.net/

sharecam

Introduction

• Regular web-cam • Collaborative camera control

Queue

Internet Interface:

Results & Discussion

• Speed of naive (B) and fast (V): – AMD K7 950Mhz– 1.2 GB memory– JDK 1.3.1– For a fixed z