L1 Sudoku

Sudoku solving with CVXOPT

Sudoku solving with CVXOPT

Ben Moran

http://trans�nite.wordpress.com

09 December 2009

Sudoku solving with CVXOPT

Background

1 Background

2 Sudoku as matrix and vectors

3 Sparsity and solving

4 Conclusion

Sudoku solving with CVXOPT

Background

Sudoku solving with CVXOPT

Background

Based on. . .

a paper by Babu, Pelckmans, Stoica:

Linear Systems, Sparse Solutions, and Sudoku 1

Solve Sudoku by turning it into a linear programming problem,

inspired by new signal processing paradigm: compressedsensing 2

I thought it was interesting and reimplemented it in Pythonwith CVXOPT

1http://www.it.uu.se/katalog/praba420/Sudoku.pdf2http://nuit-blanche.blogspot.com/

Sudoku solving with CVXOPT

Background

Why?

Ben Laurie - Sudoku is a denial of service attack on human

intellect

It's a solved problem!

Sudoku solving with CVXOPT

Background

Existing solutions

Knuth's Dancing Links (DLX) 3

Norvig's constraint propagation (also Python) 4

Sinkhorn method 5

And many others

3http://en.wikipedia.org/wiki/Dancing_Links4http://norvig.com/sudoku.html5http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=4802290&arnumber=4804943&count=41&index=24

Sudoku solving with CVXOPT

Background

But. . .

This is still an interesting way into the brand new �eld of"compressed sensing"

And what have Sudoku got to do with. . .

signal processingsensor networkssingle-pixel camerassatellite imaging?

Our solution will be interesting, as the solver won't know therules of Sudoku at all!

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

What are linear systems?

Matrix - Vector multiplication(1 34 −1

)(23

)=

(115

)Ax = b

In Python:

>>> import numpy>>> numpy . mat r i x ( ' 1 3 ;4 −1 ' ) ∗ \

numpy . mat r i x ( ' 2 ; 3 ' )mat r i x ( [ [ 1 1 ] ,

[ 5 ] ] )

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

Solving a linear system

Given

A =

(1 34 −1

)and b =

(115

), what is x?

In Python:

>>> import numpy>>> numpy . l i n a l g . s o l v e (numpy . mat r i x ( ' 1 3 ; 4 −1 ' ) ,

numpy . mat r i x ( ' 11 ;5 ' ) )mat r i x ( [ [ 2 . ] ,

[ 3 . ] ] )

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

Sudoku board as an indicator vector

We can turn a 9x9 Sudoku board into a single vector with9x9x9 = 729 elements

Each 9 entries corresponds to one cell, most are zero

Put a 1 in the �rst place for 1, in the second place for 2. . .

>>> p = Problem ( " 1 2 . . " ,N=2)>>> numpy . mat r i x ( p . t o_ ind i c a t o r_ve c t o r ( ) , ' i ' )mat r i x ( [ [ 1 ] ,

[ 0 ] ,[ 0 ] ,[ 1 ] ,[ 0 ] ,[ 0 ] ,[ 0 ] ,[ 0 ] ] )

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

Sudoku rules as a matrix system

Now we can set up a special matrix to enforce the rules ofSudokuIt will have one row for each constraint, and 9x9x9 = 729columnsWhen you multiply this with the indicator vector, you

get all 1's if the board is valid

Here are some of the rules for 2x2 Sudoku:

>>> p = Problem ( " 1 2 . . " ,N=2)>>> numpy . mat r i x ( p . mat r i x ( ) )mat r i x ( [ [ 1 , 1 , 0 , 0 , 0 , 0 , 0 , 0 . ] ,

[ 0 , 0 , 1 , 1 , 0 , 0 , 0 , 0 . ] ,[ 0 , 0 , 0 , 0 , 1 , 1 , 0 , 0 . ] ,[ 0 , 0 , 0 , 0 , 0 , 0 , 1 , 1 . ] ,[ 1 , 0 , 1 , 0 , 0 , 0 , 0 , 0 . ] ,

. . .[ 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 . ] ] )

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

Rules for 2x2 Sudoku

(1 2_ _

)

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

_ _ _ 1 5 _ _ 7 _1 _ 6 _ _ _ 8 2 _3 _ _ 8 6 _ _ 4 _9 _ _ 4 _ _ 5 6 7_ _ 4 7 _ 8 3 _ _7 3 2 _ _ 6 _ _ 4_ 4 _ _ 8 1 _ _ 9_ 1 7 _ _ _ 2 _ 8_ 5 _ _ 3 7 _ _ _

Figure: Full size 9x9 board

Sudoku solving with CVXOPT

Sudoku as matrix and vectors

Figure: Full size 9x9 rules matrix

Sudoku solving with CVXOPT

Sparsity and solving

What do we gain by writing Sudoku like that?

We can check solutions with a matrix-vector multiply

Turn the proposed solution into an indicator vector

Multiply it with the the rules matrix

If the solution is valid, the answer will be all ones

Solve Ax = [1 1 1 1 1] to �nd the answer?

. . . No - there is an in�nite number of answers, most won'tgive valid boards

Need something else to �nd our true correct answer

Sudoku solving with CVXOPT

Sparsity and solving

Sparsity

The true solution if it exists will be the sparsest

(Sparse vector == mostly zeros)

Sparse

0100010

vs non-sparse

310.14−21.5

That still doesn't help. . . no e�cient method to solve thatdirectly

Sudoku solving with CVXOPT

Sparsity and solving

Di�erent kinds of distance: L-2 norm, Pythagoras

You measure the length of a vector with a norm

This is an L-p norm (for a 3d vector):

(|x1|p + |x2|p + |x3|p)1/p

If you plug in p=2, you get normal Euclidean distance(Pythagoras' theorem):√

(|x1|2 + |x2|2 + |x3|2)

We can choose the solution with the smallest L2-norm. . .

Sudoku solving with CVXOPT

Sparsity and solving

Di�erent kinds of distance: L-2 norm, Pythagoras

But we don't get a valid answer

# Leas t s qua r e s s o l u t i o n>>> M = numpy . mat r i x ( sample ( ) . mat r i x ( ) )>>> ones =numpy . ones ( (M. shape [ 0 ] , 1 ) )>>> v = numpy . l i n a l g . p i n v (M)∗ ones>>> (M∗v ) [ : 4 ]mat r i x ( [ [ 1 . ] ,

[ 1 . ] ,[ 1 . ] ,[ 1 . ] ] )

>>> v [ : 4 ]mat r i x ( [ [ −0 .0735114 ] ,

[ 0 . 24178992 ] ,[ 0 . 00310795 ] ,[ 0 . 2 8 139045 ] ] )

Sudoku solving with CVXOPT

Sparsity and solving

Di�erent kinds of distance: L-0 norm, sparsity

Plug in p=0 and you get L0, sparsity (number of non-zeros):

(|x1|0 + |x2|0 + |x3|0

)The right solution will have the lowest L0 norm

But we've no way of �nding it, besides solving the Sudoku!

Sudoku solving with CVXOPT

Sparsity and solving

Di�erent kinds of distance: L-1 norm

With p=1 you get the absolute sum, taxicab distance

(|x1|1 + |x2|1 + |x3|1

)

Sudoku solving with CVXOPT

Sparsity and solving

Why is L-1 important?

There are e�cient methods of solving it (linear programming)

And for many matrices, the minimum L1 norm solution turnsout to also minimize L0

(Smallest L1 then �nds the sparsest solution)

This insight is central to the compressed sensing revolution

Sudoku solving with CVXOPT

Sparsity and solving

Python - CVXOPT

Now we can install python-cvxopt 6 and solve the problem!

CVXOPT is a GPL library for optimization

linear programmingOther kinds of convex optimization

We can use it to �nd the solution x of Ax=b, with the smallestL1-norm

The result comes out as a binary vector of 0s and 1s (withoutbeing constrained to do so!)

6http://abel.ee.ucla.edu/cvxopt/

Sudoku solving with CVXOPT

Sparsity and solving

Python - CVXOPT

After all that work to set up the question, the answer is oneline!

>>> he l p cvxopt . s o l v e r s . l pHelp on f u n c t i o n l p i n module cvxopt . coneprog :

l p ( c , G, h , A=None , b=None , s o l v e r=None , . . . )S o l v e s a p a i r o f p r ima l and dua l LPs

min imize c ' ∗xs u b j e c t to G∗x + s = h

A∗x = bs >= 0

. . .

# So l v e w i th CVXOPT>>> cvxopt . s o l v e r s . l p ( c0 , G, hh )

Sudoku solving with CVXOPT

Conclusion

Caveats

This won't solve every Sudoku - it tends to solve the easierones

It fails when the L1 minimum doesn't �nd the L0 minimum

There are other methods, from the compressed sensingliterature, like iterative reweighting, that solve more

For Sudoku, which puzzles are hard & why is still an openproblem at the frontiers of research

Sudoku solving with CVXOPT

Conclusion

Compressed sensing

Very many important, real world signals are sparse: images,sounds, gene expressions

Instead of capturing the raw signal and compressing. . .

. . . compressed sensing means you can capture thempre-compressed, with fewer measurements

(below the Nyquist rate: 44.1kHz CDs to play back 22kHzfrequencies)

Less hardware needed will mean better images or cheaper kit

Single-pixel camera 7

7http://dsp.rice.edu/cscamera

Sudoku solving with CVXOPT

Conclusion

Code, links and slides

Code is at http://github.com/benmoran/L1-Sudoku/

Sporadic blog: http://trans�nite.wordpress.com

Find CVXOPT athttp://abel.ee.ucla.edu/cvxopt/examples/index.html

And the nice CVXMOD wrapper at http://cvxmod.net