PuzzleSolver

Joe  Zeimen

Monday, April 29, 13

Goal

Monday, April 29, 13

General  Outline

• Scan  Pieces• Find  pieces• Extract  edges• Compare  edges• Assemble  Puzzle• Generate  output  image

Monday, April 29, 13

 Thresholding

Monday, April 29, 13

Find  Contours

• use  cv::findContours()  on  the  binary  image.• Filter  out  contours  smaller  than  an  expected  puzzle  piece  size.• Provides  shape  informaFon• Provides  a  mask  to  produce  the  final  output  image.

5

Monday, April 29, 13

Find  the  corners

• cv::goodFeaturesToTrack()• cv::cornerSubPix()

6

Monday, April 29, 13

7

Edges  and  classifica7on

Monday, April 29, 13

Comparing  Edges

8

First  and  LastUsed  Matches  from  Angry  

Birds  Puzzle

Best:  0.453672 Last  Used:  0.865829

Monday, April 29, 13

Assembly  Algorithm

–Goal:  Join  pieces  together  to  result  in  lowest  cost.–4nn!  combinaFons  where  n  is  the  number  of  pieces

9

n combina(ons

24 1.74641E+38

48 9.83532E+89

104 4.23677E+228

Monday, April 29, 13

Assembly  Algorithm

• Compare  every  pair  of  edges,  store  in  a  sorted  list.• While  the  puzzle  is  not  finished:–Pop  the  next  best  match  off  the  list–Merge  the  2  pieces  using  the  2  edges  from  the  list

• Merge(piece  1,  edge  1,  piece2,  edge  2)– If  piece  1  and  piece  2  are  in  same  set  return– If  merging  piece  1  and  piece  2  result  in  overlap  return–merge  piece  1  and  piece  2

10

Monday, April 29, 13

Best  Matches  List

11

Cost Piece  1   Edge  1 Piece  2 Edge  20.453672 12 2 13 1

0.539882 14 1 23 0

0.557301 4 0 9 1

0.589648 11 0 23 2

0.598454 0 0 20 2

0.61856 3 3 20 1

0.625861 5 3 9 0

0.630817 11 2 22 3

... ... ... ... ...

Monday, April 29, 13

Combine  Pieces:  

12

Monday, April 29, 13

Combine  Pieces:  

12

Monday, April 29, 13

Combine  Pieces:  

12

Monday, April 29, 13

Combine  Pieces:  

12

Monday, April 29, 13

Side  numbering  conven7on

13

0

1

2

3

Monday, April 29, 13

How  the  computer  does  it:

14

4 5

9 -­‐1

0 1

3 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

Monday, April 29, 13

How  the  computer  does  it:

14

4 5

9 -­‐1

0 1

3 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

1

Rotate  the  matrix  set  with  5  in  it

Monday, April 29, 13

How  the  computer  does  it:

15

5 -­‐1

4 9

0 1

3 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

Monday, April 29, 13

How  the  computer  does  it:

15

5 -­‐1

4 9

0 1

3 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

1

Update  the  rotaFon  matrix

Monday, April 29, 13

How  the  computer  does  it:

16

5 -­‐1

4 9

2 0

1 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

Monday, April 29, 13

How  the  computer  does  it:

16

5 -­‐1

4 9

2 0

1 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

2

Rotate  6

Monday, April 29, 13

How  the  computer  does  it:

17

5 -­‐1

4 9

2 0

1 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

Monday, April 29, 13

How  the  computer  does  it:

17

5 -­‐1

4 9

2 0

1 0

6

0

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

2

Update  6’s  rotaFon  matrix

Monday, April 29, 13

How  the  computer  does  it:

18

5 -­‐1

4 9

2 0

1 0

6

2

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

Monday, April 29, 13

How  the  computer  does  it:

19

5 -­‐1

4 9

2 0

1 0

6

2

LocaFons

RotaFons

Want  to  merge  piece  5,  edge  0  with  piece  6  edge  2

5 6

4 9

2 2

1 0

+ =

Monday, April 29, 13

Saving  the  Image

• Given  the  locaFons  an  rotaFons  of  the  soluFon,  the  pieces  can  be  put  into  place  using  affine  transforms.• Can  figure  out  all  corner  locaFons  except  lower  right.  • Use  the  3  corners  to  calculate  the  transformaFon  matrix.• Scan  horizontally  then  verFcally  using  the  corners  computed  previously  to  place  the  next  piece.  

20

Monday, April 29, 13

Other  Solved  Puzzles

21

Monday, April 29, 13

Other  Solved  Puzzles

2248  Pieces

Monday, April 29, 13

Other  Solved  Puzzles

23

Monday, April 29, 13

Other  Solved  Puzzles

24Back  of  Toy  Story  puzzle,  numbered  based  on  soluFon  by  hand.

Monday, April 29, 13

25104  piece  puzzle  (back)Monday, April 29, 13

26

Ques(ons?

Code  and  images  at:github.com/jzeimen/PuzzleSolver

Monday, April 29, 13