Bart Vanluyten

Maandag 5 mei 2008

KATHOLIEKE

UNIVERSITEIT

LEUVEN

Promotoren: Prof.dr.ir. Bart De Moor, promotor

Prof.dr.ir. Jan Willems, copromotor

Juryleden: Prof.dr.ir. H. Van Brussel, voorzitter Prof.dr. A. Bultheel Prof.dr. V. Blondel (UCL, Louvain-la-Neuve) Prof.dr. P. Spreij (UVA, Amsterdam) Prof.dr.ir. L. Finesso (ISIB-CNR, Padova) Prof.dr.ir. K. Meerbergen

Ph.D. defence

Realization, identification and filtering for hidden Markov models

using matrix factorization techniques

Bart Vanluyten

1. INTRODUCTION 2. MATRIX FACTORIZATIONS 3. REALIZATION 4. IDENTIFICATION 5. ESTIMATION 6. CONCLUSIONS

SLIDE

2 / 43

04/’06 06/’06 08/’06 10/’06 12/’06 02/’07 04/’07 06/’07 08/’07 10/’07 12/’07 02/’08 04/’08

Mathematical modeling

Bel-20

Process with finite valued output: { , , = }

1. INTRODUCTION

Modeling—HMMs—Finite valued process—Open problems—Relation to LSM


SLIDE

3 / 43

Hidden Markov model

Example: Bel-20• Output process {up, down, unchanged}• State process {bull market, bear market, stable market}

State process has Markov property and is hidden

Andrey Markov (1856 - 1922)

Bull Market

StableMarket

Bear Market

30% BEL20 30% BEL20 40% BEL20 =

50%

20%

60% 30%

20%

10%

40%

20%

50%

70% BEL20 10% BEL20 20% BEL20 =

10% BEL20 60% BEL20 30% BEL20 =

1. INTRODUCTION



SLIDE

4 / 43

BEL20

4-06 8-06 12-06 4-07 8-07 12-07 4-08

4.800

4.600

4.400

4.200

4.000

3.800

3.600

Finite-valued processes

FINITE-VALUED

PROCESSES

{ , , = }

{ A, C, G, T } { 1, 2, ..., 6 }

{ head, tail }

{ i:, e, æ, a:, ai, ..., z }

BEL20

Bio-informatics

Speech recognitionEconomics

Coin flipping - dice-tossing (with memory)

TGGAGCCAACGTGGAATGTCACTAGCTAGCTTAGATGGCTAAACGTAGGAATACACGTGGAATATCGAATCGTTAGCTTAGCGCCTCGACCTAGATCGAGCCGATCGGACTAGCTAGCTCGCTAGAAGCACCTAGAAGCTTAGACGTGGAAATTGCTTAATC

1. INTRODUCTION



SLIDE

5 / 43

Open problems for HMMs

Obtain model from data

Use model for estimation

Estimation problem

Given: output sequence

Find: state distribution at time

Identification problem


Find: HMM that models the sequence

Realization problem

Given: string prob’s

Find: HMM generating string prob’s

1. INTRODUCTION



SLIDE

6 / 43

Relation to linear stochastic model (LSM)

Mathematical model for stochastic processes

• Output process continuous range of values• State process continuous range of values

+

NOISE

NOISE

STATE OUTPUT

+

1. INTRODUCTION



SLIDE

7 / 43

Relation to linear stochastic model

Realization Identification

Estimation

Hidden Markov model


Estimation

Linear stochastic model

1. INTRODUCTION



SLIDE

8 / 43



Estimation

Hidden Markov model


Estimation


Singular value decomposition

1. INTRODUCTION



SLIDE

9 / 43



Estimation

Hidden Markov model


Estimation


Singular value decomposition

Nonnegative matrix

factorization

1. INTRODUCTION



SLIDE

10 / 43

Outline

Estimation problem






Realization problem



Matrix factorizations

Given: matrix

Find: low rank approximation of2nd objective

1st objective


SLIDE

11 / 43

Outline

Estimation problem






Realization problem




Given: matrix

Find: low rank approximation of


SLIDE

12 / 43

Matrix – Decomposition – Rank : example

Matrix

Matrix decomposition

Matrix rank

minimal inner dimension of exact decomposition

2. MATRIX FACTORIZATIONS

Introduction—Existing factorizations—Structured NMF—NMF without nonneg. factors


SLIDE

13 / 43

Low rank matrix approximation

Rank approximation of

James Sylvester

(1814 - 1897)

orthogonal

SVD yields (global) optimal low rank approximation in Frobenius distance

Singular value decomposition (SVD)




SLIDE

14 / 43

Nonnegative matrix factorization

In some applications is nonnegative and and need to be nonnegative too

Nonnegative matrix factorization (NMF) of

Algorithm (Kullback-Leibler divergence) [Lee, Seung]

This thesis: 2 modifications to NMF

NONNEGATIVE NONNEGATIVE

NONNEGATIVE




SLIDE

15 / 43

Structured NMF

Structured nonnegative matrix factorization of

Algorithm (Kullback-Leibler divergence)

Convergence to stationary point of divergence

NONNEGATIVE NONNEGATIVE

NONNEGATIVENONNEGATIVE




SLIDE

16 / 43

Structured NMF: application

Applications apart from HMMs: clustering data points

Setosa Versicolor Virginica

– petal width– petal length– sepal width– sepal length

Asked: Divide 150 flowers into clusters

Given:

of 150 iris flowersPETAL

SEPAL




SLIDE

17 / 43

Structured NMF: application

Clustering obtained by: • Computing distance matrix between points

• Applying structured nonnegative matrix factorization on distance

matrix

cluster 1

cluster 2

cluster 3

PE

TA

L W

IDT

HS

EP

AL

WID

TH

SEPAL LENGTH

PETAL LENGTH

SE

PA

L L

EN

GT

H

SEPAL WIDTH

PE

TA

L L

EN

GT

H

PETAL WIDTH




SLIDE

18 / 43

NMF without nonnegativity of the factors

NMF without nonnegativity constraints on the factors of

We provide algorithm (Kullback-Leibler divergence)

Problem allows to deal with upper bounds in an easy way

NONNEGATIVE NO NONNEGATIVITY CONSTRAINTS NONNEGATIVE

Example 3

3




SLIDE

19 / 43

NMF without nonnegativity of the factors

ORIGINALNMF without

nonneg. factors

Upperbounded NMF

without nonneg. fact.NMF

Applications apart from HMMs: database compressionGiven: Database containing 1000 facial images of size 19 x 19 = 361 pixels

Asked: Compression of database using matrix factorization techniques

361

1000 20

. . .

Kullback-Leibler divergence: 339 383564

> 1




SLIDE

20 / 43

Outline

Estimation problem






Realization problem




Given: matrix



SLIDE

21 / 43

Hidden Markov models: Moore - Mealy

Moore HMM

Mealy HMM

=

NONNEGATIVE

NONNEGATIVE

ORDER

3. REALIZATION

Introduction—Realization—Quasi realization—Approx. realization—Modeling DNA


SLIDE

22 / 43

Realization problem

String from String probabilities

String probabilities generated by Mealy HMM

POSITIVE REALIZATION

NONNEGATIVE

such that

3. REALIZATION



SLIDE

23 / 43

Realization problem: importance

Theoretical importance: transform ‘external’ model into ‘internal’ model Realization can be used to identify model from data

POSITIVE REALIZATION

NONNEGATIVE

3. REALIZATION



SLIDE

24 / 43

Realizability problem

Generalized Hankel matrix

Necessary condition for realizability: Hankel matrix has finite rank

No necessary and sufficient conditions for realizability are known

No procedure to compute minimal HMM from string probabilities

This thesis: two relaxations to positive realization problem• Quasi realization problem• Approximate positive realization problem

Hermann Hankel

(1839 - 1873)

3. REALIZATION



SLIDE

25 / 43

Quasi realization problem

QUASI REALIZATION

such that

NO NONNEGATIVITYCONSTRAINTS !

Finiteness of rank of Hankel matrix = N & S condition for quasi realizability Rank of hankel matrix = minimal order of exact quasi realization Quasi realization is more easy to compute than positive realization Quasi realization typically has lower order than positive realization

Negative probabilities • No disadvantage in several estimation applications

3. REALIZATION



SLIDE

26 / 43

Partial quasi realization problem

Given: String probabilities of strings up to length t

Asked: Quasi HMM that generates the string probabilities

This thesis:

• Partial quasi realization problem has always a solution

• Minimal partial quasi realization obtained with quasi realization algorithm if a rank condition on the Hankel matrix holds

• Minimal partial quasi realization problem has unique solution (up to similarity transform) if this rank condition holds

3. REALIZATION



SLIDE

27 / 43

Approximate quasi realization problem Given: String probabilities of strings up to length t Asked: Quasi HMM that approximately generates the string probabilities This thesis: algorithm

• Compute low rank approximation of largest Hankel block subject to consistency and stationarity constraints

• Reconstruct Hankel matrix from largest block

We prove that rank does not increase in this step• Apply partial quasi realization algorithm

Upperbounded NMF without nonnegativity of the factors with additional

constraints

3. REALIZATION



SLIDE

28 / 43

Approximate positive realization problem

APPROXIMATE POSITIVE REALIZATION

NONNEGATIVE

such that

Given: String probabilities of strings up to length t Asked: Positive HMM that approximately generates the string probabilities

3. REALIZATION



SLIDE

29 / 43

Approximate positive realization problem

Moore, t = 2

• Define

• If string probabilities are generated by Moore HMM

Structured nonnegative matrix factorization

Mealy, general t Generalize approach for Moore, t = 2

where

3. REALIZATION



SLIDE

30 / 43

Modeling DNA sequences

DNA

40 sequences of length 200

String probabilities of strings up to length 4 stacked in Hankel matrix

ORDER 1 2 3 4 5 6 7

Quasi realization 0.1109 0.0653 0.0449 0.0263 0.0220 0.0211 0.0210

Positive realization 0.3065 0.1575 0.0690 0.0411 0.0374 0.0373 0.0371

Kullback-Leibler divergence

SIN

GU

LA

R V

AL

UE

TGGAGCCAACGTGGAATGTCACTAGCTAGCTTAGATGGCTAAACGTAGGAATACCCTACGTGGAATATCGAATCGTTAGCTTAGCGCCTCGACCTAGATCGAGCCGATCGGTCTACTAGCTAGCTCGCTAGAAGCACCTAGAAGCTTAGACGTGGAAATTGCTTAATCTAG

3. REALIZATION



SLIDE

31 / 43

Outline

Estimation problem






Realization problem




Given: matrix



SLIDE

32 / 43


Given: Output sequence of length T

Asked: (Quasi) HMM that models the sequence

NONNEGATIVE NO NONNEGATIVITYCONSTRAINTS!

Approach

Baum-Welch identification

Linear Stochastic

Models

HiddenMarkovModels

Subspace basedidentification

Subspace inspiredidentification

Prediction error identification SVD

NMF

4. IDENTIFICATION

Introduction—Subspace inspired identification—HIV modeling


SLIDE

33 / 43


output sequence

system matrices

system matrices

state sequence

state sequence

Baum-Welch Subspace inspired

4. IDENTIFICATION



SLIDE

34 / 43

Subspace inspired identification

Estimate the (quasi) state distribution

• quasi state predictor can be built from data using upperbounded NMF without nonnegativity of the factors

• state predictor can be built from data using NMF

Compute the system matrices: least squares problem

Quasi HMM:

Positive HMM:

. . .

. . .

. . .

. . .

. . .

. . .

. . .

4. IDENTIFICATION


We have shown that:


SLIDE

35 / 43

Modeling sequences from HIV genome

Mutation

25 mutated sequences of length 222 from the part of the HIV1 genome that codes for the envelope protein [NCBI database]

• Training set

• Test set

HMM model using Baum-Welch – Subspace inspired identification

A

HIV virus

ENVELOPE

MATRIXCORE

4. IDENTIFICATION



SLIDE

36 / 43

Modeling sequences from HIV genome

Kullback-Leibler divergence (string probabilities of length-4 strings)

Mean likelihood of the given sequences

Likelihood of using third order subspace inspired model

Model can be used to predict new viral strains and to distinguish between different HIV subtypes

ORDER 1 2 3 4 5

Baum-Welch 3.15 4.65 8.27 21.02 22.93

Subspace 3.15 2.14 1.13 1.08 1.10

ORDER 1 2 3 4 5

Baum-Welch 8.13 10-5 9.03 10-5 1.40 10-4 1.45 10-4 1.50 10-4

Subspace 8.14 10-5 8.84 10-5 9.84 10-5 9.60 10-5 9.83 10-5

TEST-SEQUENCE

Likelihood 9.18 10-5 9.15 10-5 9.26 10-5 8.82 10-5 9.15 10-5

4. IDENTIFICATION



SLIDE

37 / 43

Outline

Estimation problem






Realization problem




Given: matrix



SLIDE

38 / 43

Estimation for HMMs

State estimation – output estimation

We derive recursive formulas to solve state and output filtering, prediction and smoothing problems

HMM HMM

5. ESTIMATION

Estimation for HMMs—Application

Filtering – smoothing – prediction

TIME

TIME

TIME

t

t

t

FILTERING:

SMOOTHING:

PREDICTION:

= span of available measurements


SLIDE

39 / 43

Estimation for HMMs

Example:• Recursive algorithm to compute

Recursive output estimation algorithms effective with quasi HMM

Finiteness of rank of Hankel matrix = N & S condition for quasi realizability Rank of hankel matrix = minimal order of exact quasi realization Quasi realization is easier to compute than positive realization Quasi realization typically has lower order than positive realization Negative probabilities

• No disadvantage in output estimation problems

5. ESTIMATION



SLIDE

40 / 43

Finding motifs in DNA sequences

Find motifs in muscle specific regulatory regions [Zhou, Wong]• Make motif model

• Make quasi background model (see Section realization)

• Build joint HMM

• Perform output estimation

Results (compared to results from Motifscanner [Aerts et al.])

POSITION POSITION

MO

TIF

PR

OB

AB

ILIT

Y

MO

TIF

PR

OB

AB

ILIT

Y

Mef-2MyfSp-1SRFTEF

5. ESTIMATION



SLIDE

41 / 43

Conclusions

Two modification to the nonnegative matrix factorization• Structured nonnegative matrix factorization• Nonnegative matrix factorization without nonnegativity of the factors

Two relaxations to the positive realization problem for HMMs• Quasi realization problem• Approximate positive realization problem Both methods were applied to modeling DNA sequences

We derive equivalence conditions for HMMs

We propose a new identification method for HMMs Method was applied to modeling DNA sequences of HIV virus

Quasi realizations suffice for several estimation problems Quasi estimation methods were applied to finding motifs in DNA sequences

6. CONCLUSIONS

Conclusions—Further research—List of publications


SLIDE

42 / 43

Further research

Matrix factorizations Develop nonnegative matrix factorization with nesting property (cfr. SVD)

Hidden Markov models Investigate Markov models (special case of hidden Markov case) Develop realization and identification methods that allow to

incorporate prior-knowledge in the Markov chain

Method to estimate minimal order of positive HMM from string probabilities Canonical forms of hidden Markov models Model reduction for hidden Markov models System theory for hidden Markov models with external inputs

. . .

6. CONCLUSIONS



SLIDE

43 / 43

List of publications

Journal papers• B. Vanluyten, J.C. Willems and B. De Moor. Recursive Filtering using Quasi-Realizations. Lecture Notes in Control and

Information Sciences, 341, 367–374, 2006.

• B. Vanluyten, J.C. Willems and B. De Moor. Equivalence of State Representations for Hidden Markov Models. Systems and Control Letters, 57(5), 410–419, 2008.

• B. Vanluyten, J.C. Willems and B. De Moor. Structured Nonnegative Matrix Factorization with Applications to Hidden Markov Realization and Filtering. Accepted for publication in Linear Algebra and its Applications, 2008.

• B. Vanluyten, J.C. Willems and B. De Moor. Nonnegative Matrix Factorization without Nonnegativity Constraints on the Factors. Submitted for publication.

• B. Vanluyten, J.C. Willems and B. De Moor. Approximate Realization and Estimation for Quasi hidden Markov models. Submitted for publication.

International conference papers• I. Goethals, B. Vanluyten, B. De Moor. Reliable spurious mode rejection using self learning algorithms. In Proc. of the

International Conference on Modal Analysis Noise and Vibration Engineering (ISMA 2004), Leuven, Belgium, pages 991–1003, 2004.

• B. Vanluyten, J. C.Willems and B. De Moor. Model Reduction of Systems with Symmetries. In Proc. of the 44th IEEE Conference on Decision and Control (CDC 2005), Seville, Spain, pages 826–831, 2005.

• B. Vanluyten, J. C. Willems and B. De Moor. Matrix Factorization and Stochastic State Representations. In Proc. of the 45th IEEE Conference on Decision and Control (CDC 2006), San Diego, California, pages 4188-4193, 2006.

• I. Markovsky, J. Boets, B. Vanluyten, K. De Cock, B. De Moor. When is a pole spurious? In Proc. of the International Conference on Noise and Vibration Engineering (ISMA 2007), Leuven, Belgium, pp. 1615–1626, 2007.

• B. Vanluyten, J. C. Willems and B. De Moor. Equivalence of State Representations for Hidden Markov Models. In Proc. of the European Control Conference 2007 (ECC 2007), Kos, Greece, 2007.

• B. Vanluyten, J. C. Willems and B. De Moor. A new Approach for the Identification of Hidden Markov Models. In Proc. of the 46th IEEE Conference on Decision and Control (CDC 2006), New Orleans, Louisiana, 2007.

6. CONCLUSIONS



SLIDE

44 / 43

Documents

Bart Vanluyten