Viterbi Algorithm

Saravanan Vijayakumaransarva@ee.iitb.ac.in

Department of Electrical EngineeringIndian Institute of Technology Bombay

October 30, 2015

1 / 33

Encoder State Diagram

G(D) =[1 + D 1 + D2 1 + D + D2

]

+

+

+

u

v(0)

v(1)

v(2)

S0

S1

S3

S2

1/111

0/000

1/010

0/101

0/110

1/001

1/100

0/011

2 / 33

Encoder Trellis Diagram

S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1 S1S1

S3 S3 S3 S3 S3 · · ·

· · ·

· · ·

· · ·

0 1 2 3 4 5 6t =

000 000 000 000 000 000

001 001 001 00111

1

111

111

111

111

111

011

011

011

011

010

010

010

010

010

101

101

101

101

101

100

100

100

100

110

110

110

110

• The initial state of the encoder is the all-zeros state

• Every path in the trellis starting from the initial state gives a codeword

3 / 33

Terminated Trellis Diagram

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

000 000 000 000 000 000 000

001 001 00111

1

111

111

111

111

011

011

011

011

011

010

010

010

010

101

101

101

101

101

100

100

100

110

110

110

110

• The inputs are chosen to terminate the trellis in the all-zeros state

• Every path from the initial state to the final state is a codeword

4 / 33

Terminating the Trellis of Feedforward Encoders+

+

+

u

v(0)

v(1)

v(2)

• Two consecutive zero input bits will drive the aboveencoder to the all-zeros state

• In a feedforward encoder with memory order m, mconsecutive zero input bits in each of the k inputs willterminate the trellis

5 / 33

Terminating the Trellis of Feedback Encoders

wi−1 wi−2 wi−m+

+ + +

++

+

+

f0 f1 f2 fm−1 fm

q1 q2 qm−1 qm

· · ·

· · ·

· · ·

ui wi

vi

• To reach the all-zeros state, the input to the shift registerhas to be zero for m time units

wi = 0 =⇒ ui =m∑

j=1

qjwi−j

6 / 33

Maximum Likelihood Decoder for BSC

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

000 000 000 000 000 000 000

001 001 00111

1

111

111

111

111

011011

011011

011

010

010

010

010

101101

101101

101

100

100

100

110

110

110

110

• Let r =[110 110 110 111 010 101 101

]be the BSC output

• The ML decoder will output a codeword v such that dH(r, v) is minimum

7 / 33

Maximum Likelihood Decoder for BSC

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2 2 2 3 1 2 2

3 2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• Branch metric is the Hamming distance between the codeword bits of astate transition and the corresponding received bits

• Path metric is the sum of all the branch metrics along a path

8 / 33

Viterbi Algorithm (1)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

2 2 2 3 1 2 2

3 2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• At t = 2, there is only path entering each state

• The shortest path has one of these four paths as a prefix

9 / 33

Viterbi Algorithm (2)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

2 2 2 3 1 2 2

3 2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• At t = 3, there are two paths entering each state

• The shortest path has one of these eight paths as a prefix

10 / 33

Viterbi Algorithm (3)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

2 2 2 3 1 2 2

3 2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• At t = 3, the two paths entering state S3 have path metrics 5 and 4

• We eliminate the path with metric 5

11 / 33

Viterbi Algorithm (4)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

2 2 2 3 1 2 2

2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• The state S3 at time t = 3 is now labelled with its partial path metric

12 / 33

Viterbi Algorithm (5)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

2 2 2 3 1 2 2

2 21 1 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• At t = 3, the two paths entering state S1 have path metrics 4 and 5

• We eliminate the path with metric 5

13 / 33

Viterbi Algorithm (6)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

4

2 2 2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• The state S1 at time t = 3 is now labelled with its partial path metric

14 / 33

Viterbi Algorithm (7)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

4

2 2 2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 2 1 3 0

1 2 20 1 1 2

• At t = 3, the two paths entering state S2 have path metrics 2 and 5

• We eliminate the path with metric 5

15 / 33

Viterbi Algorithm (8)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

4

2

2 2 2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• The state S2 at time t = 3 is now labelled with its partial path metric

16 / 33

Viterbi Algorithm (9)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

4

2

2 2 2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• At t = 3, the two paths entering state S0 have path metrics 5 and 6

• We eliminate the path with metric 6

17 / 33

Viterbi Algorithm (10)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

2

3

3

4

4

4

2

5

2 2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• The state S0 at time t = 3 is now labelled with its partial path metric

18 / 33

Viterbi Algorithm (11)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

4

2

5

2 3 1 2 2

2 21 1 0 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• At t = 3, there is now only one surviving path entering each state

• The shortest path has one of these four paths as a prefix

19 / 33

Viterbi Algorithm (12)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

4

2

5

6

2 3 1 2 2

21 1 0 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• We eliminate one of the paths with metric 6 entering S3 at t = 4

20 / 33

Viterbi Algorithm (13)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

4

2

5

6

4

2 3 1 2 2

21 1 2

2 1 1 2 2

1 1 2 0

2 1 3 0

1 2 20 1 1 2

• We eliminate the path with metric 5 entering S1 at t = 4

21 / 33

Viterbi Algorithm (14)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

4

2

5

6

4

5

2 3 1 2 2

21 1 2

2 1 1 2 2

1 1 2 0

2 3 0

1 2 20 1 1 2

• We eliminate one of the paths with metric 5 entering S2 at t = 4

22 / 33

Viterbi Algorithm (15)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

4

2

5

6

4

5

3

2 1 2 2

21 1 2

2 1 1 2 2

1 1 2 0

2 3 0

1 2 20 1 1 2

• We eliminate the path with metric 8 entering S0 at t = 4

23 / 33

Viterbi Algorithm (16)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

6

4

5

3

2 1 2 2

21 1 2

1 1 2 2

1 1 2 0

2 3 0

1 2 20 1 1 2

• At t = 4, there is now only one surviving path entering each state

• The shortest path has one of these four paths as a prefix

24 / 33

Viterbi Algorithm (17)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

6

4

5

3

4

5

7

4

2 1 2 2

1 1 2

1 2 2

1 1 2 0

2 0

1 2

0 1 1 2

• Using the same procedure, we eliminate one path per state at t = 5

25 / 33

Viterbi Algorithm (18)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

5

7

4

1 2 2

1 2

1 2 2

1 2 0

2 0

1 2

0 1 2

• At t = 5, there is now only one surviving path entering each state

• The shortest path has one of these four paths as a prefix

26 / 33

Viterbi Algorithm (19)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

4

5

7

4

5

6

1 2 2

1 2

1 2

1 2 0

2 0

1 2

0 1

• Using the same procedure, we eliminate one path per state at t = 6

27 / 33

Viterbi Algorithm (20)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

5

6

1 2 2

1 2

1 2

1

0

0• At t = 6, there is now only one surviving path entering each state

• The shortest path has one of these two paths as a prefix

28 / 33

Viterbi Algorithm (21)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

5

6

1 2

1 2

1 2

1

0

0• We eliminate the path with metric 8 entering S0 at t = 7

29 / 33

Viterbi Algorithm (22)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

7

1 2

1 2

1

0

0• A single path from state S0 at t = 0 to state S0 at t = 7 has survived

30 / 33

Viterbi Algorithm (23)

S0 S0 S0 S0 S0 S0 S0 S0

S2 S2 S2 S2 S2

S1 S1 S1 S1S1

S3 S3 S3 S3

0 1 2 3 4 5 6 7t =

111

111

011011

010

101

110• v̂ =

[111 010 110 011 111 101 011

]is the output of the

Viterbi algorithm

31 / 33

Viterbi Algorithm in General

• Consider a convolutional code with k inputs, n outputs,memory order m and constraint length ν

• The trellis has at most 2ν states at each time instant• At t = m, there is one path entering each state• At t = m + 1, there are 2k paths entering each state, out of

which 2k − 1 have to be eliminated• At each time instant t , at most 2ν surviving paths exist• For a discrete memoryless channel with channel transition

probability function p(·|·), we maximize

log p(r|v) =h+m−1∑

i=0

log p(ri |vi)

where h is the number of k -bit input blocks, ri and vi are nsymbols long

32 / 33

Questions? Takeaways?

33 / 33