September 30, Probabilistic Modeling

Multi-Robot Systems

CSCI 7000-006Monday, September 30, 2009

Nikolaus Correll

So far

• Reactive and deliberative distributed algorithms

• Formal models describing sub-sets of the systems

• Deterministic models for deliberative algorithms

• Convex cost functions and feedback control for reactive systems

Problems so far

• How to model– Sensor uncertainty (localization, vision, range)– Communication uncertainty– Actuation uncertainty (e.g. wheel-slip)

• Deterministic algorithms break• Reactive algorithm become unpredictable

Problem Statement

• Predict the performance of a system given– Problem– Algorithms– Capabilities/Uncertainty

• Find most suitable coordination scheme / set of resources

4

Robot 1 Robot 2

Task A

Task B

Problem

Algorithms

Random Deliberative

CentralizedDecentralized

Collaborative Greedy

N. Correll. Coordination schemes for distributed boundary coverage with a swarm of miniature robots: synthesis, analysis and experimental validation. EPFL PhD thesis #3919, 2007.

Capabilities / Probabilistic Behavior

Navigation Localization Communication

5

Master equations and Markov Chains

• The system state ({robot states} X {environment state}) is finite• Non-deterministic elements of the system follow a known statistical distribution

: Conditional probability to be in state w when in state w’ a time-step before

Transition probability from w’ to w in a Markov Chain

: Probability for the system to be in state w at time k

w’ w

6

From Master to Rate Equations: probability to be in state x at time k x can be a robot’s or a system’s state

: Total number of robots

Average number of robots in state x:

Example 1: Collision Avoidance

Two states, search and avoidN0 robots

State duration of avoid– Probabilistic– Deterministic

Possible implementations:Obstacle

“Proximal”

Obstacle

“180o turn”

What are the parameters of this system and what are their distributions?

How to get them?

Parameters

Encountering probability pR

– Probability to encounter another robot per time step

Interaction time Ta

– Average time a collision lasts– Geometric distribution or Dirac pulse

Interaction time

Average time Ta constant regardless whether probabilistic or deterministicDistribution Ta is different depending on– Controller– Model abstraction level

Model is only an approximation!

Interaction times

• Probabilistic• Deterministic• Non-Parametric

Distribution

Systematic experiments with 1 or 2 robots.

Deterministic Time-Out“180o turn avoidance”

Agent-based simulation

Simulationegocentric

Simulationallocentric

A

S

S

Example 1a: Collision AvoidanceProbabilistic Delay

Search Avoidance

pR

1/Ta

Example 1b: Collision AvoidanceDeterministic Delay

Search

pR

1

AvoidanceTa

Example 2: Collaboration

Two states: search and waitN0 robots M0 collaboration sites

State duration of wait – probabilistic: robots wait a random time– deterministic: robots wait a fixed time

robot

site

Parameters

Encountering probability ps

– Probability to encounter one site

Interaction time Tw

– (Average) time a robot waits for collaboration before moving on

Robot-Robot collisions are ignored in this example

Example 2a: CollaborationProbabilistic Delay

Search Wait

psNs(k)

1/Tw

ps(M0-Nw(k))

Example 3a: CollaborationDeterministic Delay

Search Wait

psNs(k)

ps(M0-Nw(k))

ps(M0-Nw(k-Tw))G(k;k-Tw)

Summary: Memory-less systems

Systems with no or little memory (Time-outs), essentially reactiveMaster equation for a single robot allows estimation of population dynamics

How to deal with deliberative systems

that use memory?

Example 3: Task Allocation

Scenario: 2 robots, 2 tasks A and BRobots prefer task A over task BGlobal metric requires solution of both tasksTask evaluation subject to noise, robots choose the wrong task with probability p

A B

1-p p

A B

1-p 1-p

“Greedy” “Coordinated”

Example 3a: Task AllocationNon-Collaborative, Greedy

Both robots will go for task A, then BExpected time: 2 time-stepsNoise! Effective outcomes might be AA, AB, BB, BAThere is a possibility to complete in one time-step (due to noise): AB or BA

What is the state transition diagram of this system?

Master-Equations for Deliberative Systems: Greedy algorithm

AA AB

BBBA

Why does this system asymptotically converge to AB or BA?

Example 3b: Task AllocationCollaborative

Robots will allocate the tasks among themRobot 1 will go for task A, robot 2 go for task BIf only one task is left, both try to accomplish itEffective outcomes AA, AB, BB, BAExpected time to completion 1 time-step for: AB and BA

Expected Time to Completion

greedy

collaborative

Summary

Master equation: change of probability to be in state xEnumerate all possible states of a systemCalculate all possible state transition probabilitiesSolve difference equations (numerically, analytically, Lyapunov, …)Useful for analyzing dominant collaboration dynamics of a system

Upcoming

Formal approaches to obtain model parametersHow to model systems with large state space?