A bit on the Linear Complementarity Problem and a bit about me (since this is EPPS) Yoni Nazarathy...

A bit on the Linear Complementarity Problemand a bit about me (since this is EPPS)

Yoni Nazarathy

EPPSEURANDOM

November 4, 2010

* Supported by NWO-VIDI Grant 639.072.072 of Erjen Lefeber

Overview• Yoni Nazarathy (EPPS #2):

– Brief past, brief look at future…

• The Linear Complementarity Problem (LCP)– Definition– Basic Properties– Linear and Quadratic Programming– Min-Linear Equations– My Application: Queueing Networks

Just to be clear: Almost nothing in this presentation (except for pictures of my kids), is original work, it is rather a “reading seminar”

Some Things From the Past

High School in USA

Primary School in Israel (Haifa)

Israeli Army

Israeli Army Reserve

Married

Divorced

Married AgainEmily Born

Kayley Born

Undergraduate Statistics/Economics

Masters in Applied Probability

Software Engineer in High-Tech

Industry

Ph.D with Gideon Weiss

Cycle Racing

Born 1974

Netherlands (Feb 2009 – Nov 2010)

Collaborations: Matthieu, Yoav, Erjen, Johan, Ivo, Gideon, Stijn, Dieter, Michel, Bert, Ahmad, Koos, Harm, Oded, Ward, Rob,

Gerard, Florin…

Yarden Born!!!

Nederlands: Ik dank dat het is heel gezelich om te pratten…

Raising young kids in Eindhoven:

HIGHLY RECOMMENEDED!!!

EURANDOM / Mechanical Engineering / CWI Amsterdam

Pedaling to see the Low Lands

``

Future in Oz…

Melbourne

Melbourne…

Maybe live here

Work here: Swinburne University

Also collaborate here: Melbourne University

Maybe also collaborate here: Monash University

Swinburne University of Technology

Looking for Ph.D Students…

What is driving my travels??Maybe fears of some things that can kill…

In the Middle East…

In the Netherlands

A slow death…

Australia must be a safe place….

Or is it?

In Summary…I hope to stay lucky, also in Oz…

Finally…The Linear Complementarity Problem

(LCP)

Definition,

( , ) :Find , such that,

,

0, 0,

' 0.

n n n

n

q M

LCP q M z w

w Mz q

w z

w z

The last (complemenatrity) condition reads:

0 0 and 0 0.i i i iw z z w

It’s all about Choosing a Subset…For {1,..., } denote by ( ) a matrix with

collumns taken from (identity matrix)

and collumns {1,..., } \ taken from .

n B

I

n M

is about finding and 0

such that

( )

In this case:

LCP x

B x q

0, .

0i

i ii

ix iw z

x ii

Illustration: n=2

1 0 11 20 1 2

1 12 11 20 22 2

011 11 2121 2

11 12 11 2

21 22 2

{1,2}:

{1}:

{2}:

:

qw w

q

m qw z

m q

m qz w

m q

m m qz z

m m q

1 11 12 1 1

2 21 22 2 2

1 0

0 1

w m m z q

w m m z q

{1,2}C

Complementary cones:

1

0

0

1

12

22

m

m

11

21

m

m

1

2

q

q

{1}C

{2}C

{ : ( ) , 0}C y y B u u

C

Immediate naïve algorithm with complexity 3 32 2n nn or n

Existence and UniquenessDefinition: A matrix, is a P-matrix if the

determinants of all (2 1) principal submatrices are positive.

n n

n

M

Theorem (1958): ( , ) has a unique solution

for all if and only if is a P-matrix.n

LCP q M

q M

11 22 11 22 12 21e.g.for 2 : 0, 0, 0n m m m m m m

P-matrix means that the complementary cones "parition" n

P-Matrixes

Symmetric Matrixes PD Matrixes

Relation of P-matrixes to positive definite (PD) matrixes:

Reminder(PD) :

' 0 0x Mx x

Reminder(PSD) :

' 0x Mx x

Computation (Algorithms)• Naive algorithm, runs on all subsets alpha• Generally, LCP is NP complete• Lemeke’s Algorithm, a bit like simplex• If M is PSD: polynomial time algs exists• PD LCP equivalent to QP• Special cases of M, linear number of iterations• For non-PD sub-class we (Stijn & Eren) have an

algorithm. Where does it fit in LCP theory?We still don’t know…

• Note: Checking for P-Matrix is NP complete, checking for PD is quick

2n

LCP References And Resources• Linear Complementarity, Linear and Nonlinear

Programming, Katta G. Murty, 1988. Internet edition.• The Linear Complementarity Problem, Second Edition, Richard

W. Cottle, Jong-Shi Pang, Richard E. Stone. 1991, 2009.• Richard W. Cottle, George B. Dantzig, Complementary Pivot

Theory of Mathematical Programming, Linear Algebra and its Applications 1, 103-125, 1968.

• Related (to queueing networks): Unpublished paper (~1989), Avi Mandelbaum, The Dynamic Complementarity Problem.

• Open problems in LCP…. I am now not an expert (but a user) .... So I don’t know…

• Gideon Weiss, working on relations to SCLP

Some Applications(and Sources) of LCP

Linear Programming (LP)

min '

. .

0

c x

s t Ax b

x

max '

. . '

0

b y

s t A y c

y

Primal-LP: Dual-LP:

Theorem: Complementary slackness conditions

min '

. .

, 0

c x

s t Ax b v

x v

max '

. . '

, 0

b y

s t u c A y

y u

Assume , , , are feasible for primaland dual:

0, 0 Theyareoptimalsolutionsi i i i

x v y u

x u y v

0 ',

0

c ALCP

b A

0 '

0

u A x c

v A y b

, , , 0u v x y

' 0u x ' 0v y

The LCP of LPFind:

Such that:

And (complementary slackness):

Lekker!

Quadratic Programming1

min ( ) ' '2

. .

0

Q x c x x D x

s t Ax b

x

Lemma: An optimizer, , of the QP also optimizesmin ( ) '

. .

0

c Dx x

s t Ax b

x

Proof:( )x x x x

( ) ( ) 0Q x Q x ( ' ) '( ) ( ) ' ( )

2c Dx x x x x D x x

x

QP-LP:

QP-LP gives a necessary condition for optimality of QP in terms of an checking optimality of an LP

QP:

0 1, Let be feasible.x

( ' ) '( ) 0c Dx x x

( ' ) ' ( ' ) 'c Dx x c Dx x

The Resulting LCP of QP

',

0

c D ALCP

b A

Allows to find “suspect” points that satisfy the necessary conditions: QP-LP

Theorem: Solutions of this LCP are KKT (Karush-Kuhn-Tucker) points for the QP

Corollary: If D is PSD then x solving the LCP optimizes QP.

Proof: Write down KKT conditions and check.

Note: When D is PSD then M is PSD. In this case it can be shown that the LCP is equivalent to a QP (solved in polynomial time). Similarly, every PSD LCP can be formulated as a PSD QP.

Our Application: Min-Linear Equations( )B

0

0

( ) '( ) 0

B

,w z ( ) ( )

0, 0

' 0

u I B v I B

z w

w z

( ( ) , )LCP I B I B

Find :

Open Jackson NetworksJackson 1957, Goodman & Massey 1984, Chen & Mandelbaum 1991

, ,M M M MP

1

'

( ') , ( ')

M

i i j j j ij

p

P

LCP I P I P

i

i

Traffic Equations:

i jp

1

M

1

1M

i jij

p p

Problem Data:

Assume: open, no “dead” nodes

Modification: Finite Buffers and Overflows Wolff, 1988, Chapter 7 & references there in & after

ii

Exact Traffic Equations for Stochastic System:

i jp

M

1

1M

i jij

p p

Problem Data:

, , , ,M M M M M M MP K Q

Explicit Stochastic Stationary Solutions:

Generally NoiK

MK1

1M

i jij

q q

i jq

11K

Generally No

Assume: open, no “dead” nodes, no “jam” (open overflows)

Traffic Equations for Fluid System

Yes

Traffic Equations

1 1

M M

i i j j ji j j jij j

p q

out rate

overflow rate ( ) ( )

1 1( ') ( ( ') ) , ( ') ( ')LCP I Q I P I Q I P

Wrapping Up

• LCP: Appears in several places (we didn’t show game-theory)

• Would like to fully understand the relation of our limiting traffic equations and LCP

• In progress paper with Stijn Fleuren and Erjen Lefeber, “Single Class Fluid Networks with Overflows” makes use of LCP theory (existence and uniqueness)

• I will miss EURANDOM and the Netherlands very much!• Visit me in Melbourne!!!

The End