L1 Sudoku

Sudoku solving with CVXOPT


Ben Moran

http://trans�nite.wordpress.com

09 December 2009


Background

1 Background

2 Sudoku as matrix and vectors

3 Sparsity and solving

4 Conclusion


Background


Background

Based on. . .

a paper by Babu, Pelckmans, Stoica:

Linear Systems, Sparse Solutions, and Sudoku 1

Solve Sudoku by turning it into a linear programming problem,

inspired by new signal processing paradigm: compressedsensing 2

I thought it was interesting and reimplemented it in Pythonwith CVXOPT

1http://www.it.uu.se/katalog/praba420/Sudoku.pdf2http://nuit-blanche.blogspot.com/

http://www.it.uu.se/katalog/praba420/Sudoku.pdf

http://nuit-blanche.blogspot.com/


Background

Why?

Ben Laurie - Sudoku is a denial of service attack on human

intellect

It's a solved problem!


Background

Existing solutions

Knuth's Dancing Links (DLX) 3

Norvig's constraint propagation (also Python) 4

Sinkhorn method 5

And many others

3http://en.wikipedia.org/wiki/Dancing_Links4http://norvig.com/sudoku.html5http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=4802290&arnumber=4804943&count=41&index=24

http://en.wikipedia.org/wiki/Dancing_Links

http://norvig.com/sudoku.html

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?isnumber=4802290&arnumber=4804943&count=41&index=24


Background

But. . .

This is still an interesting way into the brand new �eld of"compressed sensing"

And what have Sudoku got to do with. . .

signal processingsensor networkssingle-pixel camerassatellite imaging?

Our solution will be interesting, as the solver won't know therules of Sudoku at all!


Sudoku as matrix and vectors

What are linear systems?

Matrix - Vector multiplication(1 34 −1

)(23

)=

(115

)Ax = b

In Python:

>>> import numpy>>> numpy . mat r i x ( ' 1 3 ;4 −1 ' ) ∗ \

numpy . mat r i x ( ' 2 ; 3 ' )mat r i x ( [ [ 1 1 ] ,

[ 5 ] ] )



Solving a linear system

Given

A =

(1 34 −1

)and b =

(115

), what is x?

In Python:

>>> import numpy>>> numpy . l i n a l g . s o l v e (numpy . mat r i x ( ' 1 3 ; 4 −1 ' ) ,

numpy . mat r i x ( ' 11 ;5 ' ) )mat r i x ( [ [ 2 . ] ,

[ 3 . ] ] )



Sudoku board as an indicator vector

We can turn a 9x9 Sudoku board into a single vector with9x9x9 = 729 elements

Each 9 entries corresponds to one cell, most are zero

Put a 1 in the �rst place for 1, in the second place for 2. . .

>>> p = Problem ( " 1 2 . . " ,N=2)>>> numpy . mat r i x ( p . t o_ ind i c a t o r_ve c t o r ( ) , ' i ' )mat r i x ( [ [ 1 ] ,

[ 0 ] ,[ 0 ] ,[ 1 ] ,[ 0 ] ,[ 0 ] ,[ 0 ] ,[ 0 ] ] )



Sudoku rules as a matrix system

Now we can set up a special matrix to enforce the rules ofSudokuIt will have one row for each constraint, and 9x9x9 = 729columnsWhen you multiply this with the indicator vector, you

get all 1's if the board is valid

Here are some of the rules for 2x2 Sudoku:

>>> p = Problem ( " 1 2 . . " ,N=2)>>> numpy . mat r i x ( p . mat r i x ( ) )mat r i x ( [ [ 1 , 1 , 0 , 0 , 0 , 0 , 0 , 0 . ] ,

[ 0 , 0 , 1 , 1 , 0 , 0 , 0 , 0 . ] ,[ 0 , 0 , 0 , 0 , 1 , 1 , 0 , 0 . ] ,[ 0 , 0 , 0 , 0 , 0 , 0 , 1 , 1 . ] ,[ 1 , 0 , 1 , 0 , 0 , 0 , 0 , 0 . ] ,

. . .[ 0 , 0 , 0 , 1 , 0 , 0 , 0 , 0 . ] ] )



Rules for 2x2 Sudoku

(1 2_ _

)



_ _ _ 1 5 _ _ 7 _1 _ 6 _ _ _ 8 2 _3 _ _ 8 6 _ _ 4 _9 _ _ 4 _ _ 5 6 7_ _ 4 7 _ 8 3 _ _7 3 2 _ _ 6 _ _ 4_ 4 _ _ 8 1 _ _ 9_ 1 7 _ _ _ 2 _ 8_ 5 _ _ 3 7 _ _ _

Figure: Full size 9x9 board



Figure: Full size 9x9 rules matrix


Sparsity and solving

What do we gain by writing Sudoku like that?

We can check solutions with a matrix-vector multiply

Turn the proposed solution into an indicator vector

Multiply it with the the rules matrix

If the solution is valid, the answer will be all ones

Solve Ax = [1 1 1 1 1] to �nd the answer?

. . . No - there is an in�nite number of answers, most won'tgive valid boards

Need something else to �nd our true correct answer



Sparsity

The true solution if it exists will be the sparsest

(Sparse vector == mostly zeros)

Sparse

0100010

vs non-sparse

310.14−21.5

That still doesn't help. . . no e�cient method to solve thatdirectly



Di�erent kinds of distance: L-2 norm, Pythagoras

You measure the length of a vector with a norm

This is an L-p norm (for a 3d vector):

(|x1|p + |x2|p + |x3|p)1/p

If you plug in p=2, you get normal Euclidean distance(Pythagoras' theorem):√

(|x1|2 + |x2|2 + |x3|2)

We can choose the solution with the smallest L2-norm. . .



Di�erent kinds of distance: L-2 norm, Pythagoras

But we don't get a valid answer

# Leas t s qua r e s s o l u t i o n>>> M = numpy . mat r i x ( sample ( ) . mat r i x ( ) )>>> ones =numpy . ones ( (M. shape [ 0 ] , 1 ) )>>> v = numpy . l i n a l g . p i n v (M)∗ ones>>> (M∗v ) [ : 4 ]mat r i x ( [ [ 1 . ] ,

[ 1 . ] ,[ 1 . ] ,[ 1 . ] ] )

>>> v [ : 4 ]mat r i x ( [ [ −0 .0735114 ] ,

[ 0 . 24178992 ] ,[ 0 . 00310795 ] ,[ 0 . 2 8 139045 ] ] )



Di�erent kinds of distance: L-0 norm, sparsity

Plug in p=0 and you get L0, sparsity (number of non-zeros):

(|x1|0 + |x2|0 + |x3|0

)The right solution will have the lowest L0 norm

But we've no way of �nding it, besides solving the Sudoku!



Di�erent kinds of distance: L-1 norm

With p=1 you get the absolute sum, taxicab distance

(|x1|1 + |x2|1 + |x3|1

)



Why is L-1 important?

There are e�cient methods of solving it (linear programming)

And for many matrices, the minimum L1 norm solution turnsout to also minimize L0

(Smallest L1 then �nds the sparsest solution)

This insight is central to the compressed sensing revolution



Python - CVXOPT

Now we can install python-cvxopt 6 and solve the problem!

CVXOPT is a GPL library for optimization

linear programmingOther kinds of convex optimization

We can use it to �nd the solution x of Ax=b, with the smallestL1-norm

The result comes out as a binary vector of 0s and 1s (withoutbeing constrained to do so!)

6http://abel.ee.ucla.edu/cvxopt/

http://abel.ee.ucla.edu/cvxopt/



Python - CVXOPT

After all that work to set up the question, the answer is oneline!

>>> he l p cvxopt . s o l v e r s . l pHelp on f u n c t i o n l p i n module cvxopt . coneprog :

l p ( c , G, h , A=None , b=None , s o l v e r=None , . . . )S o l v e s a p a i r o f p r ima l and dua l LPs

min imize c ' ∗xs u b j e c t to G∗x + s = h

A∗x = bs >= 0

. . .

# So l v e w i th CVXOPT>>> cvxopt . s o l v e r s . l p ( c0 , G, hh )


Conclusion

Caveats

This won't solve every Sudoku - it tends to solve the easierones

It fails when the L1 minimum doesn't �nd the L0 minimum

There are other methods, from the compressed sensingliterature, like iterative reweighting, that solve more

For Sudoku, which puzzles are hard & why is still an openproblem at the frontiers of research


Conclusion

Compressed sensing

Very many important, real world signals are sparse: images,sounds, gene expressions

Instead of capturing the raw signal and compressing. . .

. . . compressed sensing means you can capture thempre-compressed, with fewer measurements

(below the Nyquist rate: 44.1kHz CDs to play back 22kHzfrequencies)

Less hardware needed will mean better images or cheaper kit

Single-pixel camera 7

7http://dsp.rice.edu/cscamera

http://dsp.rice.edu/cscamera


Conclusion

Code, links and slides

Code is at http://github.com/benmoran/L1-Sudoku/

Sporadic blog: http://trans�nite.wordpress.com

Find CVXOPT athttp://abel.ee.ucla.edu/cvxopt/examples/index.html

And the nice CVXMOD wrapper at http://cvxmod.net

http://github.com/benmoran/L1-Sudoku/

http://transfinite.wordpress.com

http://abel.ee.ucla.edu/cvxopt/examples/index.html

http://cvxmod.net

Technology

L1 Sudoku