Solving Systems of Polynomial Equations: Algebraic ... · Solving Systems of Polynomial Equations: Algebraic Geometry, Linear Algebra, and Tensors? Philippe Dreesen Bart De Moor Katholieke

Univariate Polynomials Multivariate Polynomials Applications Tensors Conclusions and Future Work

Solving Systems of Polynomial Equations:

Algebraic Geometry, Linear Algebra, and Tensors?

Philippe Dreesen Bart De Moor

Katholieke Universiteit Leuven – ESAT/SCD

Workshop on Tensor Decompositions and Applications (TDA2010)Monopoli, Italy, September 2010

1 / 27


Outline

1 Univariate Polynomials

2 Multivariate Polynomials

3 Applications

4 Tensors

5 Conclusions and Future Work

2 / 27


Outline



3 Applications

4 Tensors


3 / 27


Polynomials, Matrices and Eigenvalue Problems

Characteristic Polynomial

The eigenvalues of A are the roots of

p(λ) = det(A− λI) = 0

Companion Matrix

Solvingq(x) = 7x3

− 2x2− 5x + 1 = 0

leads to

0 1 00 0 1

−1/7 5/7 2/7

1xx2

= x

1xx2

4 / 27


Sylvester Matrix

Sylvester Resultant

Consider two polynomials f(x) and g(x):

f(x) = x2− 3x + 2

g(x) = x3− 4x2

− 11x + 30

Common roots iff S(f, g) = 0

S(f, g) = det

2 −3 1 0 00 2 −3 1 00 0 2 −3 1

30 −11 −4 1 00 30 −11 −4 1

James Joseph Sylvester

5 / 27


Sylvester Matrix

Sylvester’s construction can be understood from

1 x x2 x3 x4

f(x) = 0 2 −3 1x · f(x) = 0 2 −3 1x2 · f(x) = 0 2 −3 1g(x) = 0 30 −11 −4 1x · g(x) = 0 30 −11 −4 1

1xx2

x3

x4

= 0

evaluate the vector containing the powers of x at x⋆ = 2

6 / 27


Sylvester Matrix

Find a vector in the nullspace of the Sylvester matrix,

2 −3 12 −3 1

2 −3 130 −11 −4 1

30 −11 −4 1

−0.0542−0.1083−0.2166−0.4332−0.8664

= 0

normalize such that the first entry equals 1:

2 −3 12 −3 1

2 −3 130 −11 −4 1

30 −11 −4 1

124816

= 0

7 / 27


Conclusion: Main Ingredients

Linear Algebra turns out to be suitable framework

Main Ingredients:

Linearize problem by separating coefficients and monomialsSolutions live in the nullspace of coefficient matrixExploit structure in monomial basisEigenvalue problems

Multivariate case?

8 / 27


Outline



3 Applications

4 Tensors


9 / 27


Outline of Algorithm

Macaulay: multivariate Sylvester construction

Linearize by separating coefficients and monomials

Algorithm:

1 Build coefficient matrix M

2 Find basis for nullspace of M

3 Find solutions from eigenvalue problem

Etienne Bezout James Joseph Sylvester Francis Sowerby Macaulay

10 / 27


Build Matrix M

Consider

p(x, y) = x2 + 3y2− 15 = 0

q(x, y) = y − 3x3− 2x2 + 13x − 2 = 0

Construct M

Write the system in matrix-vector notation:

2

6

6

4

1 x y x2 xy y2 x3 x2y xy2 y3

p(x, y) −15 1 3q(x, y) −2 13 1 −2 −3x · p(x, y) −15 1 3y · p(x, y) −15 1 3

3

7

7

5

11 / 27


Build Matrix M

Continue to enlarge M:

1 x y x2

xy y2

x3

x2

y xy2

y3

x4

x3

yx2

y2

xy3

y4

x5

x4

yx3

y2

x2

y3

xy4

y5

. . .

p − 15 1 3

q − 2 13 1 − 2 − 3

xp − 15 1 3

yp − 15 1 3

x2

p − 15 1 3

xyp − 15 1 3

y2

q − 2 − 15 1 3

xq − 2 13 1 − 2 − 3

yq − 2 13 1 − 2 − 3

x3

p − 15 1 3

x2

yp − 15 1 3

xy2

p − 15 1 3

y3

p − 15 1 3

x2

q − 2 13 1 − 2 − 3

xyq − 2 13 1 − 2 − 3

y2

q − 2 13 1 − 2 − 3

.

.

.

...

...

...

...

...

...

...

...

...

...

...

...

...

......

...

12 / 27


Solutions in Nullspace of M

Coefficient matrix M:

M =

"

× × × × 0 0 00 × × × × 0 00 0 × × × × 00 0 0 × × × ×

#

Solutions generate vectors in nullspace of

M:

Mv = 0

Number of solutions s follows from corank

Canonical nullspace V built

from s solutions (xi, yi):

2

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

6

4

1 1 . . . 1

x1 x2 . . . xs

y1 y2 . . . ys

x21 x2

2 . . . x2s

x1y1 x2y2 . . . xsys

y21 y2

2 . . . y2s

x31 x3

2 . . . x3s

x21y1 x2

2y2 . . . x2sys

x1y21 x2y2

2 . . . xsy2s

y31 y3

2 . . . y3s

x41 x4

2 . . . x44

x31y1 x3

2y2 . . . x3sys

x21y2

1 x22y2

2 . . . x2sy2

s

x1y31 x2y3

2 . . . xsy3s

y41 y4

2 . . . y4s

......

......

3

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

7

5

13 / 27


Solutions in Nullspace of M

Nullspace of M

Find a basis for the nullspace of M using an SVD:

M =

× × × × 0 0 00 × × × × 0 00 0 × × × × 00 0 0 × × × ×

= [ X Y ][

Σ1 00 0

] [

WT

ZT

]

Hence,MZ = 0

14 / 27


Find Solutions

Shift property in monomial basis

[

1 0 0 0 0 00 1 0 0 0 00 0 1 0 0 0

]

1x

y

x2

xy

y2

x =

[

0 1 0 0 0 00 0 0 1 0 00 0 0 0 1 0

]

1x

y

x2

xy

y2

[

1 0 0 0 0 00 1 0 0 0 00 0 1 0 0 0

]

1x

y

x2

xy

y2

y =

[

0 0 1 0 0 00 0 0 0 1 00 0 0 0 0 1

]

1x

y

x2

xy

y2

Finding the x-roots: let D = diag(x1, x2, . . . , xs), then

S1VD = S2V,

where S1 and S2 select rows from V wrt. shift property

Reminiscent of Realization Theory

15 / 27


Find Solutions

We haveS1VD = S2V

However, V is not known, instead a basis Z is computed as

ZT = V

Which leads toS1ZTD = S2ZT

orS1Z

(

TDT−1

)

= S2Z

Hence,TDT

−1 = (S1Z)†S2Z

16 / 27


Algorithm Summary

Algorithm

1 Construct coefficient matrix M

2 Compute basis for nullspace of M, Z

3 Choose shift function, e.g., x

4 Write down shift relation in monomial basis v for the chosen shiftfunction using row selection matrices S1 and S2

5 The construction of above gives rise to a generalized eigenvalueproblem

S1Z(

TDT−1

)

= S2Z

of which the eigenvalues correspond to the, e.g., x-solutions of thesystem of polynomial equations.

17 / 27


Algorithm Summary

Approach has been generalized to Multivariate Polynomials

Elegant link with Linear Algebra, especially eigenvalue problems

Finds all solutions (or alternatively, global solutions)

(Numerical) LA: embed into well-known matrix computations

Limiting computational complexity (but inherent to the problem)

18 / 27


Outline



3 Applications

4 Tensors


19 / 27


Relevant applications are found in

Polynomial Optimization Problems

Structured Total Least Squares

Model order reduction

Analyzing identifiability nonlinear model structures

Robotics: kinematic problems

Computational Biology: conformation of molecules

Algebraic Statistics

Signal Processing

. . .

20 / 27


6 × 3 Hankel STLS

minv

τ2 = vTA

TD

−1v Av

s. t. vTv = 1.

0 0.5 1 1.5 2 2.5 30

0.5

1

1.5

2

2.5

3

theta

phi

STLS Hankel cost function

TLS/SVD soln

STSL/RiSVD/invit steps

STLS/RiSVD/invit soln

STLS/RiSVD/EIG global min

STLS/RiSVD/EIG extrema

method TLS/SVD STLS inv. it. STLS eigv1 .8003 .4922 .8372

v2 -.5479 -.7757 .3053

v3 .2434 .3948 .4535

τ2 4.8438 3.0518 2.3822

global solution? no no yes

(eigenvalue decomposition on 437 × 437 matrix)

21 / 27


Outline



3 Applications

4 Tensors


22 / 27


Links with Tensors

A homogeneous polynomial of degree d is a d-mode tensor,e.g.,

a0 + aT1 x + xT

A2x + A3(x, x, x) + . . . + Ad(x, . . . , x) + . . .

where

a0 ∈ R

a1 ∈ Rn

A2 ∈ Rn×n

A3 ∈ Rn×n×n

...

23 / 27


Is it useful to ‘decouple’ along tensor directions?

24 / 27


Outline



3 Applications

4 Tensors


25 / 27


Conclusions

Link polynomial system solving and linear algebra

Polynomial system solving reduces to eigenvalue problems!

Problem tackled using matrix computations

Computational complexity

Topic touches on the fundamentals of applied mathematics

Will it be interesting to look at tensors?

26 / 27


Thank You!

27 / 27

Documents

Solving Systems of Polynomial Equations: Algebraic ... · Solving Systems of Polynomial Equations: Algebraic Geometry, Linear Algebra, and Tensors? Philippe Dreesen Bart De Moor Katholieke