CS-INFO 372:Explorations in Artificial Intelligence

Prof. Carla P. Gomesgomes@cs.cornell.edu

Module 2Examples of Different Modeling Formalisms

http://www.cs.cornell.edu/courses/cs372/2008sp

Example of a reasoning formalism:

Constraint Satisfaction Problems

Escher:Waterfall, 1961

Escher:Belvedere, May 1958

Escher:Ascending and Descending, 1960

How do we Interpret the Scenes in Escher’s Worlds?

Analysis of Polyhedral Scenesorigins of Constraint Reasoning

researchers in computer vision in the 60s-70s were

interested in developing a procedure to assign 3-

dimensional interpretations to scenes;

They identified

Three types of edgesFour types of junctions

Edge Types

Hidden – if one of its planes cannot be seen

represented with arrows:

Convex – from the viewer’s perspective

represented with

+

Concave – from the viewer’s perspective

represented with

-

Huffman-ClowesLabeling

Types of Junctions

Type of junction: L Fork T Arrow

Scene InterpretationConstraint Reasoning Problem:

Variables Edges;

Domains {+,-,,}

Constraints:

1- The different type junctions define constraints:

L, Fork, T, Arrow;

L = {(, ) , ( , ), (+, ), (,+), (-, ), (,-)}

Fork = { (+,+,+), (-,-,-), (,,-), (,-,),(-,,)}

L(A,B) the pair of values assigned to variables A,B

has to belong in the set L;

Fork(A,B,C) the trio of values assigned to variables A,B,C

has to belong in the set Fork;

Constraint Satisfaction Problem (CSP)

• T = {(, , ) , ( ,,), (,,+), (,-)}

• Arrow = { (,,+), (+,+,-), (-,-,+)}

T(A,B,C) the trio of values assigned to variables A,B,C

has to belong in the set T;

Arrow(A,B,C) the trio of values assigned to variables A,B,C

has to belong in the set Arrow;

2- For each edge XY its reverse YX has a compatible value

Edge = { +,+), (-,-), (,),(,)}

Edge(A,B) the pair of values assigned to variables A,B

has to belong in the set Edge;

CSP Model - CubeCSP Model - Cube

A B

C D

E F

G

How to label the cube?

Constraint Satisfaction Problem (CSP Model)

Variables: Edges: AB, BA,AC,CA,AE,EA,CD,

DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA;

Domains {+,-,,}

Constraints:

L(AC,CD); L(AE,EF); L(DG,GF);

Arrow(AC,AE,AB); Arrow(EF,FG,BF); Arrow(CD,DG,DB);

Fork(AB,BF,BD);

Edge(AB,BA); Edge(AC,CA); Edge(AE,EA);

Edge(EF,FE); Edge(BF,FB); Edge(FG,GF);

Edge(CD,DC); Edge(BD,DB); Edge(DG,GD);

A B

C D

E F

G

CSP Model

Variables: Edges: AB, BA,AC,CA,AE,EA,CD,

DC,BD,DB,DG,GD,GF,FG,EF,FE,AE,EA;

Domains {+,-,,}

Constraints:

L(AC,CD); L(AE,EF); L(DG,GF);

Arrow(AC,AE,AB); Arrow(EF,FG,BF); Arrow(CD,DG,DB);

Fork(AB,BF,BD);

Edge(AB,BA); Edge(AC,CA); Edge(AE,EA);

Edge(EF,FE); Edge(BF,FB); Edge(FG,GF);

Edge(CD,DC); Edge(BD,DB); Edge(DG,GD);

A B

C D

E F

G+

+

+

One (out of four) possible labelings(upper right corner)

The Impossible Objects is Escher’s Worlds

Penrose & Penrose Stairs

Penrose Triangle

Impossible Objects:No labeling!

                                                       

Other examples using a Constraint Satisfaction formalism

SudokuSudoku

9 55 ~ 3x 10 52 possible completionsConstraint Satisfaction Problem (CSP) (but also Satisfiability and Integer Programming)

Given an N X N matrix, and given N colors, a Latin Square of order N is a a colored matrix, such that:

-all cells are colored.

- each color occurs exactly once in each row.

- each color occurs exactly once in each column.

Quasigroup or Latin Square(Order 4)

Constraint Satisfaction Problem (CSP) and Satisfiability and Integer Programming

Latin Squares

Given an N X N matrix, and given N colors, a Latin Square of order N is a a colored matrix, such that:

-all cells are colored

-a color is not repeated in a row

-a color is not repeated in a column

Quasigroup or Latin Square

(Order 4)Constraint Satisfaction Problem (CSP) and Satisfiability and Integer Programming

Latin Squares

Latin Square Completion ProblemLatin Square Completion Problem

Given a partial assignment of colors (10 colors in this case), can the partial latin square be completed so we obtain a full Latin square?

Example:

32% preassignment 10 68 possible completions

Fiber Optic Networks

Nodes are capable of photonic switching --dynamic wavelength routing --

which involves the setting of the wavelengths.

Nodesconnect point to point

fiber optic links

Each fiber optic link supports alarge number of wavelengths

Wavelength

Division

Multiplexing (WDM)

the most promising

technology for the

next generation of

wide-area

backbone networks.

Routing in Fiber Optic Networks

Routing Node

How can we achieve conflict-free routing in each node of the network?

Dynamic wavelength routing is an NP-hard problem.

Input Ports Output Ports1

2

3

4

1

2

3

4

preassigned channels

LSCP Application Example: Routers in Fiber Optic Networks

LSCP Application Example: Routers in Fiber Optic Networks

Dynamic wavelength routing in Fiber Optic Networks can be directly mapped into the Latin Square Completion Problem.

•each channel cannot be repeated in the same input port (row constraints);

• each channel cannot be repeated in the same output port (column constraints);

CONFLICT FREELATIN ROUTER

Inp

ut

po

rts

Output ports

3

1

2

4

Input Port Output Port

1

2

43

Design of Statistical ExperimentsDesign of Statistical Experiments

We have 5 treatments for growing beans. We want to know what treatments are effective in increasing yield, and by how much.

The objective is to eliminate bias and distribute the treatments somewhat evenly over the test plot.

Latin Square Analysis of Variance

A D E BB C

C B A E D

D C BB A E

E A C D B

B E D C A

(*) Already in use (*) Already in use in this sub-plotin this sub-plot

Spatially Balanced Latin Squares

Really hard to build balanced LS’s

Timetabling: Constraint Satisfaction Problem (CSP) and

Integer Programming

An 8 Team Round Robin Timetable

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7Period 1 0 vs 1 0 vs 2 4 vs 7 3 vs 6 3 vs 7 1 vs 5 2 vs 4

Period 2 2 vs 3 1 vs 7 0 vs 3 5 vs 7 1 vs 4 0 vs 6 5 vs 6

Period 3 4 vs 5 3 vs 5 1 vs 6 0 vs 4 2 vs 6 2 vs 7 0 vs 7

Period 4 6 vs 7 4 vs 6 2 vs 5 1 vs 2 0 vs 5 3 vs 4 1 vs 3

The problem of generating schedules with complex constraints (in this case for sports teams).

28 28 ~ 3.3 x 1040 possibilities

Sports Scheduling

Big Business!US National TV pays $500 million / year for baseballCollege basketball conferences get up to $30 millionManchester United has (had) a market cap of £400 million

No rights holder wants to pay those sums and then get a “bad” schedule Difficult to automate: Huge variety of problem typesSmall instances are difficult

Strong break between easy/hard (for all algorithms)Significant theoretical backgroundCP and IP differ in modeling

CP has clean models with [1..n] variables IP uses 0-1 variables reasonably naturally

Practical interest in instances at the easy/hard interfaceSource:Mike Trick

Graph Coloring

Coloring the nodes of the graph:What’s the minimum number of colors such that any two nodes

connected by an edge have different colors?

nn ~ possible colorings for n nodes

Graph Coloring

Constraint Satisfaction Problem (CSP) and Satisfiability and Integer Programming

Graph coloring formulations can be used to solve different problems.

Can we color agraph such that no two nodesconnected by an edge have the same color?

CSP:Variables Nodes Domains ColorsConstraints Edges

Scheduling of Final Exams

How can the final exams at Cornell be scheduled so that no student has

two exams at the same time? (Note not obvious this has anything to do

with graphs or graph coloring.)

Graph:A vertex correspond to a course.An edge between two vertices denotes that there is at least one common student in the courses they represent.Each time slot for a final exam is represented by a different color.

A coloring of the graph corresponds to a valid schedule of the exams.

1

7 2

36

5 4

Scheduling of Final Exams

1

7 2

36

5 4

What are the constraints between courses?Find a valid coloring

1

7 2

36

5 4

TimePeriod

IIIIIIIV

Courses

1,62

3,54,7

AI PLANNING

In AI, planning involves the generation of an actionplan (i.e. a sequence of actions) for an agent, such as a robot ora software system or a living artefact, that can alter its surroundings.

Planning implies the notion of synthesis: synthesis of actions, to go from an initial state to a goal state.Examples:

•plan to perform astronomical observations for the Hubble space telescope;

•plan for a robot to assemble pieces in a factory

Planning Example: Blocks world

• objects: blocks and a table• actions: move blocks ‘on’ one object to ‘on’

another object• goals: configurations of blocks• plan: sequence of actions to achieve goals

TA B C

D

Initial State

A

B

C

D

Goal State

Blocks world: propositional and first order logic representation

Knowledge Base:On(A,T)^On(B,T)^On(C,T)^On(D,C)^Block(A)^Block(B)^Block(C)^Block(D) )^Table(T)^Clear(A)^Clear(B)^Clear(D)

TA B C

D

KB:

On(A,D)^On(B,T)^On(C,T)^On(D,C)

^Block(A)^Block(B)^Block(C)^Block(D)^Table(T)

^Clear(A) ^Clear(B)T

A

B C

D

Move(A,T,D)

Another example of a reasoning formalism

A restricted form of Constraint Satisfaction:

Satisfiability

Propositional Satisfiability problem

Satifiability (SAT): Given a formula in propositional calculus, is there

an assignment to its variables making it true?

We consider clausal form, e.g.:

( a OR NOT b OR NOT c ) AND ( b OR NOT c) AND ( a OR c)n2

possible assignments

SAT: prototypical hard combinatorial search and reasoning problem. Problem is NP-Complete. (Cook 1971)

Surprising “power” of SAT for encoding computational problems.

Significant progress in Satisfiability Methods

Software and hardware verification – complete methods are critical - e.g. for verifying the correctness of chip design, using SAT encodings

Current methods can verify automatically the correctness of > 1/7 of a Pentium IV.

Going from 50 variable, 200 constraints to 1,000,000 variables and 5,000,000 constraints in the last 10 years

Applications: Hardware and

Software Verification Planning,

Protocol Design, etc.

A “real world” example

i.e. ((not x1) or x7) and ((not x1) or x6)

and … etc.

Bounded Model Checking instance:

(x177 or x169 or x161 or x153 … or x17 or x9 or x1 or (not x185))

clauses / constraints are getting more interesting…

10 pages later:

…

4000 pages later:

…

!!!!!!a 59-cnf clause…

Finally, 15,000 pages later:

The Chaff SAT solver solves this instance in less than one minute.

Note that: … !!!

Another example of a reasoning formalism

Integer Programming

Knapsack Problem (one resource)

A hiker trying to fill her knapsack to maximum total value. Each item she considers taking with

her has a certain value and a certain weight. Goal – maximize the value of the contents of the

knapsack considering the overall weight constraint.

• This problem is an abstraction with many practical applications:

Project selection and capital budgeting allocation problems

Storing a warehouse to maximum value given the indivisibility of goods and space limitations

Sub-problem of other problems e.g., generation of columns for a given model in the course of optimization – cutting stock problem (beyond the scope of this course)

Investment 1 2 3 4 5 6

Cash Required (1000s)

$5

$7

$4

$3

$4

$6

NPV added (1000s)

$16

$22

$12

$8

$11

$19

Capital Budgeting Example

Investment budget = $14,000

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6

subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14

xj binary for j = 1 to 6

Binary Optimization: Applications in Regional

Planning

I

Mxw

xc

I

iii

I

iii

,...1i and }1,0{x

Subject to

Maximize

i

1

1

Stream FootagePhosphorousPathogenParcel SizeParcel ValueBudget Constraint

Riparian Buffer in the Skaneateles Lake Watershed

Town of Skaneateles:-1834 parcels-12341 acres52 land use class.

Preservation in NY State

2,345 barns registered in year 2000464 barns in Finger Lakes Region only.

Contribution to a scenic landscape or agricultural

setting Historic significance

Budget: $2 million; Max of $25,000 grant per barn

Office of Parks, Recreation and Historic Preservation Unique Natural Areas in

Tompkins County

Important Natural CommunityGeological ImportanceAesthetic/Cultural QualitiesBudget Constraint

ModelsKnapsack and VariantsZevi Azzaino

Jon ConradCarla Gomes

Objective: Identify the best collection of parcels to include

in a riparian buffer subject to a budget constraint

Southwestern Airways Crew Scheduling

• Southwestern Airways needs to assign crews to cover all its upcoming flights.

• Simple example assigning 3 crews based in San Francisco (SFO) to 11 flights.

Question: How should the 3 crews be assigned 3 sequences of flights so that every one of the 11 flights is covered?

Southwestern Airways FlightsSeat tl e (SEA)

San Francisco (SFO)

Los Angel es (LAX)

Denver (DEN)

Chicago ORD)

Data for the Southwestern Airways Problem

Feasible Sequence of Flights (pairings)

Flights 1 2 3 4 5 6 7 8 9 10 11 12

1. SFO–LAX 1 1 1 1

2. SFO–DEN 1 1 1 1

3. SFO–SEA 1 1 1 1

4. LAX–ORD 2 2 3 2 3

5. LAX–SFO 2 3 5 5

6. ORD–DEN 3 3 4

7. ORD–SEA 3 3 3 3 4

8. DEN–SFO 2 4 4 5

9. DEN–ORD 2 2 2

10. SEA–SFO 2 4 4 5

11. SEA–LAX 2 2 4 4 2

Cost, $1,000s 2 3 4 6 7 5 7 8 9 9 8 9

Algebraic Formulation

Let xj = 1 if flight sequence (paring) j is assigned to a crew; 0 otherwise. (j = 1, 2, … , 12).

Minimize Cost = 2x1 + 3x2 + 4x3 + 6x4 + 7x5 + 5x6 + 7x7 + 8x8 + 9x9 + 9x10 + 8x11 + 9x12

(in $thousands)

subject to

Flight 1 covered: x1 + x4 + x7 + x10 ≥ 1

Flight 2 covered: x2 + x5 + x8 + x11 ≥ 1

: :

Flight 11 covered: x6 + x9 + x10 + x11 + x12 ≥ 1

Three Crews: x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10 + x11 + x12 ≤ 3

and

xj are binary (j = 1, 2, … , 12).

pairings

Combinatorial Problems

• Many computational tasks, such as planning or scheduling, can in principle be reduced to an exploration of a large set of all possible scenarios.

• Try all possible schedules, try all possible plans, pick the best.

Combinatorial Problems

Problem: combinatorial explosion!

Planning ComplexityPlanning (single-agent): find the right sequence of actions

HARD: 10 actions, 10! = 3 x 106 possible plans

REALLY HARD: 10 x 92 x 84 x 78 x … x 2256 = 10224 possible contingency plans!

Contingency planning (multi-agent): actions may or may not produce the desired effect!

…1 outof 10

2 outof 9

4 outof 8

100 ! = 9.33262154 × 10157

• “Nice” combinatorial problem (Shortest Path) – exception to combinatorial explosition polynomial scaling !

• General formulation for special problems:– shortest paths

– transportation problem

– assignment problem

– plus more

• Important subproblem of many optimization problems, including multicommodity flows

Nice Problems !

EXPONENTIAL FUNCTION

POLYNOMIAL FUNCTIONHard Computational

ProblemsScale Exponentially

EXPONENTIAL-TIMEALGORITHMS

EXPLOSIVECOMBINATORICS

ExperimentDesignGoal

Start

Software & HardwareVerification

Satisfiability

(A or B) (D or E or not A)

Data Analysis& Data Mining

Fiber optics routing

Capital BudgetingAnd Financial Appl. Information

Retrieval

Protein Folding

And Medical ApplicationsCombinatorial

Auctions

Planning and SchedulingAnd Supply Chain Management

Many more applications!!!

Require powerful computational and

mathematical tools!

NP-Complete andNP-Hard Problems

But most interesting real-world problems are:

Goals of INFO 372

Introduce the students to a range of computational modeling approaches and solution strategies using examples from AI and Information Science.

Formalisms:Logical representations;Constraint-based languages, Mathematical programming – Linear and Integer programming;Multi-agent formalisms (including adversarial games);

Solution strategies: Logical inference;General complete backtrack search; (e.g., Iterative Deepening)Local search;Dynamic Programming;Game tree search (e.g., alpha-beta pruning)

Goals of INFO 372

Special models: Satisfiability (SAT); Maximum SAT; Horn

Constraint Satisfaction; Binary Constraint Satisfaction;

Mixed Integer Programming, Linear Programming and

Network Flow Models;Themes:

Expressiveness and efficiency tradeoffs of the various representation formalisms

Students learn about the tradeoffs in modeling choices.;Concrete examples to move from one representation modeling formalism to another formalism;