Chapter 5 SOURCE CODING FOR THE SYMBOLS OF …shodhganga.inflibnet.ac.in/bitstream/10603/4577/14/14...m = 0 for the last 2 bits of SRPS. Example : A source of five symbols is considered

57

Chapter 5

SOURCE CODING FOR THE SYMBOLS OF TYPE-II AND TYPE I

DISCRETE INFORMATION SOURCES

5.1 INTRODUCTION

This chapter deals with a new algorithm proposed for constructing

the source codes of the symbols of Type II as well as Type I discrete

information sources. The work presented relates to Module 2 of the

research work. The proposed algorithm uses code lengths of Golden

sstring terms which are Fibonacci numbers like 2, 3, 5, 8.......etc.

The scheme of coding gives prefix, variable length codes which are

exactly similar to Huffman codes. The proposed scheme does not involve

listing the actual probabilities of symbols at each stage of operation, which

makes the scheme simpler. The probabilities of symbols pi = P[si],

i = 1, 2 , 3…..m are allotted unique Fibonacci numbers or code lengths of

golden sequence terms as position identification numbers abbreviated as

PINs, like 2,3,5,8,13…etc. The resultant sum of the least two probabilities

is allotted a nearest Non Fibonacci number as PIN 4,6,9,10…etc. Starting

with the least two probabilities, all possible pairs of probability additions

i.e., a set of pairs of probabilities in terms of their Position Identification

Numbers is formed. This set of pairs formed with Fibonacci and Non

Fibonacci numbers is used finally for constructing the codes of symbols.

The work carried out in this module is presented in the journal paper “

Construction of Variable Length Codes for symbols of Discrete information

58

Sources using code lengths of Golden sequence Terms ” accepted for

publication in IETECH Journal of Communication Techniques.

5.2 ORGANIZATION OF THE CHAPTER

The notation adopted for the algorithm is explained in section 5.3

followed by description of formation of SRPS in section 5.4. The algorithm

developed for the generation of source codes in the form of step by step

procedure is explained in section 5.5. The justification for the proposed

algorithm is presented in 5.6. In section 5.7, few typical examples

including the specific type I source of alphabet size six and the specific

type II source of an English textual message considered in chapter 1 are

taken up and the source codes are derived. In section 5.8, the flowchart

used for the software simulation is presented. The results of simulation of

the proposed algorithms in C language are presented in section 5.9. In

section 5.10, results and discussion are presented. The conclusions are

presented in section 5.11.

5.3 NOTATION

si : si indicates ith symbol in the list of symbols arranged in descending

order of their probabilities.

m : Number of symbols of the discrete information source.

pi : pi = P [si ] : si = The probability of occurrence of ith symbol.

m

Pi : Pi = ∑ pj j=i+1

SRPS : This is the abbreviation for Symbol Relative Probability Sequence.

This is a sequence of l’s and 0’s. The length of the sequence is the length

59

of source alphabet size. If the source has m discrete symbols, then the

SRPS is a m-bit sequence.

This is of general form b1, b2, b3…..bm , bi = 0 or 1.

Fi : The ith Fibonacci number in the list of 2, 3, 5, 8, 13…...etc. The

Fibonacci numbers 0, 1, 1 which are present in the standard list are not

considered here. F1 = 2 , F2 = 3 , F3 = 5 , F4 = 8…....etc.

Fi : The Fibonacci number allotted as the PIN of the ith symbol after the

symbols are arranged in the descending order of the probabilities.

In general, Fi = Fm-i+1

If m=6 , F1 = F6-0 = F6 = 21 F2 = F6-1 = F5 = 13 F3 = F6-2 = F4 = 8

F4 = F6-3 = F3 = 5 F5 = F6-4 = F2 = 3 F6 = F6-5 = F1 = 2

PIN : This is the abbreviation for Position Identification Number of any

probability. This number indicates the relative position of a probability in

the original set of probabilities of symbols arranged in the descending

order or the relative position of the result of addition of two probabilities in

the new set of probabilities after the addition of the least two probabilities

in every step.

Fibonacci PINs : F numbers : These are 2, 3, 5, 8,…which are allotted for

probabilities of the given m number of source symbols. These are referred

to as F numbers. pm = 2, pm-1 =3 , pm-2 =5, pm-3 = 8 …..etc. In general the

Fibonacci PIN of ith symbol is Fi. F1 being the highest Golden Sequence

code length allotted to p1 and Fm is the lowest Fibonacci number i.e., 2

allocated to the smallest probability pm.

60

Non – Fibonacci PINs : G numbers : These are used for representing the

sum of the least two probabilities in every stage. These are referred to as G

numbers. For any source with equi-probable symbols the Non-Fibonacci

PINs or the G-numbers are all above F1.

The Non Fibonacci Numbers between Fi and Fi+1 are denoted by Gi+1.

F-Row : This is the row formed by all Fi’s. i = 1 to m . For a five symbol

source the F-Row is 2, 3, 5, 8, 13.

G-Row : This is the row formed by all Non-Fibonacci position identification

numbers which are allotted for the sums resulting out of adding the least

two probabilities in every stage.

PAP : This is the abbreviation for Probability Addition Pair. This is a pair of

two Fibonacci numbers or two non Fibonacci numbers or one Fibonacci

and one non Fibonacci number. The members of the pair represent

probabilities of symbols or the result of addition of two probabilities during

the previous stages. For example, the first addition pair is always (2, 3)

representing the PINs of the least two probabilities pm and pm-1.

CTPAP : Code Terminating Probability Addition Pair. This is defined for a

particular symbol. This is the PAP which contains the probability of the

particular symbol. Since the final binary code element of the symbol

terminates at this pair, this is called the Code Terminating PAP of that

symbol.

61

ICTPAP : Index of Code Terminating Probability Addition Pair. This is

defined for a particular symbol. This is the index allotted for the CTPAP of

the symbol.

SOS of ICTPAP : This is the abbreviation for Sequence of Successors for

any index of Code Terminating Probability Addition Pair. This contains all

the Non Fibonacci or G numbers that result successively from the code

terminating pair.

5.4 FORMATION OF SRPS

The probabilities of the given symbols are arranged in descending order.

m

For each probability pi, the Value Pi is calculated as Pi = ∑ pj j=i+1

If pi > Pi bi = 1 in SRPS

≤ Pi bi = 0 in SRPS .…………(5.1)

Always bm-1 = 1 and bm = 0 for the last 2 bits of SRPS.

Example : A source of five symbols is considered to illustrate the

procedure for the formation of SRPS. The symbols of the source occur with

the probabilities 0.61, 0.17, 0.12, 0.06 and 0.04. The values of Pi and the

bits of SRPS are posted below.

pi Pi SRPS

0.61 – 0.39 p1 > P1 1

0.17 – 0.22 p2 < P2 0

0.12 – 0.10 p3 > P3 1 0.06 – 0.04 1

0.04 – 0.00 0

Hence the SRPS for the source is 1 0 1 1 0.

62

5.5 STEP BY STEP PROCEDURE FOR CODE GENERATION ALGORITHM PROPOSED

The following is the step by step procedure :

1. From the given values of pi, the values of Pi are calculated for i = 1,

2,. ……. (m-2). This requires (2m-2) number of entries. SRPS is

formulated from these values. This needs (m-2) number of entries.

2. The F row is constructed. This has m Fibonacci numbers

representing the probabilities of m number of discrete symbols.

3. From the bits of SRPS the PINs for Pi are allotted in the form of G-

numbers.

4. The first addition pair is considered. Pm-2 = pm + pm-1 . This is the

sum of the least two probabilities. This sum is indicated by the PAP

(2,3)1 . The PIN of (2,3)1 is 4 if bm-2 is 1, and this is the first entry in

the G row. If bm-2 is 0 and pm-k < Pm-2 < pm-(k+1) then the PIN of (2,3)1

is Gm-k which is the first entry in the G row. The second addition

pair will be (4,5) or (5,6) or (5,8) depending on the PIN of (2,3). The

result of this will be second entry in the G row. Continuing this

procedure, the least two probabilities are added in every stage and

the result is given an appropriate G number until the last pair is

covered whose sum becomes 1. The G row is thus formed. This has

(m-2) Non-Fibonacci numbers representing PINs of resulting sums of

probability addition.

If any PAP is of the form (Gk+1 , Fk ) where Gk+1 = Pk , the result of the

addition is Pk-1 . If bk-1 = 1 the PIN allotted to the result is Gk .

63

If any PAP is of the form (Fk , Gk) where Gk = Pk , the result of the

addition is Pk-1 . If bk-1 = 1 then the PIN allotted to the result is the

smallest Gk available.

Index for PAP : For easier tracking an index number is allotted for each

addition pair written out side the parenthesis. For example the index of

the PAP (2, 3) is always 1. This is indicated by (2,3)1 . Similarly the index of

the second PAP is indicated by either (4,5)2 or (5,6)2 or (5,8)2. In general if

any PAP involves probabilities for ith stage addition the index for that PAP

is i.

Index of result of PAP : This is another index for the PIN of the Non-

Fibonacci or G number in any pair which is same as the index of that PAP

whose result is this G number. For example if the first PAP (2,3)1 results in

the G number 6, the subsequent PAP in which 6 appears is written as

(5,61)2 . In general the index number of the resulting G number from the ith

addition pair is i and it is indicated by Gi in the subsequent pair.

5. Index numbers are given for each PAP and for the G number

resulting out of each PAP. Starting from (Fm , Fm-1), all PAPs are

listed from left to right.

6. The CTPAP and its corresponding index ICTPAP are identified for

each symbol. Starting with Fi , the SOS for the Index of symbol Fi is

listed. The Fibonacci PIN and the G numbers of its associated SOS

are represented by either a 0 or a 1 according as the PIN is

64

the lower one (i.e., on the left side) or the higher one (i.e., on the right

side) in the ordered pair.

7. This binary sequence written in the reverse order gives the code of

the ith symbol.

SPECIAL CASES :

Special case 1 : Sources with even number of equi – probable symbols :

Out of (m-1) total number of probability addition pairs m/2 number

of pairs will be such that both elements of the pair are Fibonacci numbers

and the remaining (m/2)-1 number of pairs are such that both elements of

the pair are non-Fibonacci or G numbers. For equi-probable symbols the

sum of each pair of additions is greater than p1 and the value of the sum

progressively increases after (m/2) number of additions. A row of m

number of F numbers followed by (m-2) number of G numbers above F1

are prepared. From this row (m-1) number of pairs are drawn up.

The general form of the list of PAPs is, (Fm, Fm-1)1 (Fm-2, Fm-3 )2 …….… (F2,

F1)m/2 (G1 G2)((m+2)/2) (G3 G4)((m+4)/2) ……...(Gm-3 , Gm-2)m-1 .

From this list of PAPs followed by SOS of the index for CTPAP of Fi , the

code of si can be read easily.

Special case 2 : Sources with odd number of equi-probable symbols :

Out of (m-1) total number of probability addition pairs (m-1)/2 number

of pairs will be such that both the elements are Fibonacci numbers, one

pair will be with one F number and one G number.

65

The remaining ((m–3)/2) number of pairs are such that both the elements

are G numbers. For equi-probable symbols the sum of each pair of (m/2)

number of additions is greater than p1 and the value of the sum

progressively increases there after. A row of m number of F numbers

followed by (m-2) number of G numbers above F1 are prepared. From this

row (m-1) number of pairs are drawn up.

The general form of the list of PAPs is, (Fm, Fm-1)1 (Fm-2, Fm-3 )2 ………..(F3,

F2)(m-1)/2 (F1, G1) ((m+1)/2) (G2 , G3) ((m+3)/2)…..….(Gm-3, Gm-2) m-1

From this list of PAPs followed by SOS of the index for CTPAP of Fi , the

code of si can be read easily.

Special case 3 : Type I sources

For Type I sources, SRPS in the reverse order can be used to list the PAPs

very easily as explained below.

First stage PAP : (Fm , Fm-1) = (2 , 3)

Second stage PAP : (Gm-1 , Fm-2) if bm-2 = 1

(Fm-2 , Gm-2) if bm-2 = 0

Third stage PAP : (Gm-2 , Fm-3) if bm-3 = 1

(Fm-3 , Gm-3) if bm-3 = 0 …….etc.

In general kth stage : (Gm-k+1 , Fm-k) if bm-k = 1

(Fm-k , Gm-k) if bm-k = 0

From the list of PAPs and their indices, codes can be read out easily.

66

5.6 JUSTIFICATION

The conventional Huffman scheme suffers from two drawbacks. The

first drawback is that there is no provision for distinction between

probabilities of symbols of source and the probabilities resulting out of

addition of two probabilities. The second drawback is that tracking of odd

decimal values of probabilities is very inconvenient while reading the codes

in binary form at the end. The first drawback can be removed by

representing the values of pi with Fibonacci numbers and the addition

values by non-Fibonacci numbers as is done in the proposed algorithm.

The second drawback is overcome if the proposed algorithm offers a

feature by which tracking of odd decimal values can be avoided while

reading the codes at the end.

The following Table 5.1 shows the theoretical possibilities for the

number of probability figures that may arise between any two successive

values of pi during the coding by conventional method and the way they

can be realized as non-Fibonacci position identification numbers between

two Fibonacci PINs.

67

Table 5.1 : NUMBER OF NEW PROBABILITIES ARISE DURING HUFFMAN CODING

AND THE NUMBER OF NON FIBONACCI NUMBERS AVAILABLE BETWEEN TWO FIBONACCI POSITION IDENTFICATION NUMBERS

pm-k , pm-k-1 Fm-k , Fm-k-1 New

probabilities

That arise

Available

PINs

Actual

PINs

pm-1 , pm-2 Fm-1 , Fm-2

3 , 5

1 1 4

pm-2 , pm-3 Fm-2 , Fm-3

5 , 8

2 2 6, 7

pm-3 , pm-4 Fm-3 , Fm-4

8 , 13

3 4 9, 10, 11, 12

pm-4 , pm-5 Fm-4 , Fm-5

13 , 21

4 7 14, 15, 16, 17,

18, 19, 20

pm-5 , pm-6 Fm-5 , Fm-6

21 , 34

5 12 22, 23, 24, 25,

26, 27, 28, 29,

30, 31, 32, 33

…….

……. …… ….. ……

pm-k , pm-k-1 Fm-k , Fm-k-1 K F(m-k+1) – 1 G(m-k)i ;

i = 1, 2, ….etc.

5.7 EXAMPLES

Example 5.1 : A six symbol source is considered for which source codes

are to be constructed. The probabilities of occurrence of symbols are 0.4,

0.20, 0.14, 0.11, 0.10 and 0.05.

The values of pi , Pi ,the bits of SRPS and the allotted Fibonacci PINs

are listed below.

pi Pi bi Fi

0.4 0.6 0 21

0.20 0.4 0 13

0.14 0.26 0 8 0.11 0.15 0 5

0.10 0.05 1 3

0.05 0 0 2

This is a Type II source since P3 > p2 . SRPS : 0 0 0 0 1 0.

68

The list of PAPs that occur in the sequential order and the result of these

additions are listed below along with the allotted G-numbers and their

indices.

1. (0.05 , 0.10) 0.15 ; (2,3)1 91

2. (0.11 , 0.14) 0.25 ; (5,8)2 142

3. (0.15 , 0.20) 0.35 ; (91,13)3 153

4. (0.25 , 0.35) 0.60 ; (142,153)4 224

5. (0.40 , 0.60) 1.00 ; (21,224)5 23

The F and G rows are shown in the Table 5.2 below.

Table 5.2 : F-ROW AND G-ROW FOR THE SIX SYMBOL SOURCE

F Row 2 3 5 8 13 21

G Row 9 14 15 22

Probability Addition Pairs: (2, 3)1 (5, 8)2 (91, 13)3 (142, 153)4 (21, 224)5

The following are the codes :

s1 = 0, s2 = 111, s3 = 101, s4 = 100, s5 = 1101, s6 = 1100.

Considering that this source transmits three symbols s2 s3 s4

sequentially the binary data stream pertaining to these three message

symbols is : s2 s3 s4

111 101 100

This binary data is arranged into blocks of three bits each. This is the data

used for checking the scheme proposed in chapter 6 to find easily a

suitable generator matrix for single bit error correction linear block codes

69

and subsequently for testing the error correction scheme proposed in

chapter 7.

Example 5.2 : In this example the case of transmitting the following

English text message of example 1.2 consisting of several letters and

special characters like space and dot is considered for which source codes

are to be constructed. The following is the message.

Source coding helps to describe the information sources more

efficiently by removing the redundancy between the messages. The

requirements of memory or bandwidth resources needed to store or to

transmit the messages are reduced by source coding.

Number of symbols in the message : 24

Total number of characters in the string : 245

From the example 1.2 different parameters of this source which are shown

in Table 1.1 are reproduced in Table 5.3 below.

70

Table 5.3 : DISCRETE SYMBOLS AND THEIR PROBABILITIES OF

AN ENGLISH TEXTUAL MESSAGE

Sl. No Symbol Frequency Probabilities, pi Pi

1 E 36 0.14693 0.85307

2 (space) 36 0.14693 0.70614

3 R 19 0.07755 0.62859

4 O 18 0.07346 0.55513

5 S 17 0.06940 0.48573

6 T 16 0.06530 0.42043

7 N 13 0.05306 0.36737

8 D 11 0.04490 0.32247

9 I 11 0.04490 0.27757

10 C 10 0.04081 0.23676

11 M 9 0.03673 0.20003

12 A 7 0.02858 0.17145

13 H 7 0.02858 0.14287

14 U 7 0.02857 0.11430

15 B 5 0.02040 0.09390

16 G 5 0.02040 0.07350

17 Y 5 0.02040 0.05310

18 F 4 0.01632 0.03678

19 L 2 0.00816 0.02862

20 W 2 0.00816 0.02046

21 . (dot) 2 0.00816 0.01230

22 P 1 0.00410 0.00820

23 Q 1 0.00410 0.00410

24 V 1 0.00410 0.00000


additions are listed below along with the allotted G-numbers.

(2, 3)=0.0082 22

(5, 8)=0.01226 23

(13, 21)=0.01632 35

(22, 23)=0.02046 145

(34, 35)=0.03264 611

71

(55, 89)=0.04080 988

(144,145)=0.04086 1598

(233,377)=0.05715 6766

(610,611)=0.06122 6767

(987,988)=0.07753 28658

(1597,1598)=0.08167 46369

(2584,4181)=0.0898 46370

(6765,6766)=0.11.21 46371

(6767,10946)= 0.12652 46372

(17711,28657)= 0.14286 46373

(28658,46368)= 0.15508 121394

(46369,46370)= 0.17147 121395

(46371,46372)= 0.23673 121396

(46373,75025)= 0.28979 121397

(121393,121394)= 0.30201 121398

(121395,121396)=0.4082 121399

(121397,121398)=0.5918 121400

(121399,121400)=1.00 121401

72


Table 5.4 : F-ROW AND G-ROW FOR A 24 SYMBOL SOURCE OF

THE ENGLISH TEXTUAL MESSAGE

F Row 2 3 5 8 13 21 34 55

G Row 22 23 35

F Row 89 144 233 377 610 987 1597 2584

G Row 145 611 988 1598

F Row 4181 6765 10946 17711 28657 46368 75025 121393

G Row 6766,

6767

28658 46369,

46370,

46371,

46372,

46373

121394,

121395,

121396,

121397,

121398,

121399,

121400

Probability Addition Pairs with indices for all PAPs and their resulting sum

values :

(2,3)1 (5,6)2 (13,21)3 (221,232)4

(34,353)5 (55,89)6 (144,1454)7 (233,377)8

(610,6115)9 (987,9886)10 (1597,15987)11

(2584,4181)12 (6765,67668)13

(67679,10946)14 (17711,28657)15

(2865810,46368)16

(4636911,4637012)17

(4637113,4637214)18

(4637315,75025)19

(121393,12139416)20

(12139517,12139618)21

(12139719,12139820)22

(12139921,12140022)23

73

The codes for the symbols are shown in the following Table 5.5 .

Table 5.5 : SOURCE CODES FOR THE 24 SYMBOL

ENGLISH TEXTUAL MESSAGE

Symbol Code

E 101

(space) 110

R 1111

O 1001

S 1000

T 0111

N 0100

D 0010

I 0011

C 0000

M 11100

A 01011

H 01100

U 01010

B 111010

G 111011

Y 00010

F 011011

L 0001111

W 0110100

. (dot) 0110101

P 0001100

Q 0001101

V 0001110



0.32, 0.12, 0.09, 0.05 and 0.02.


are listed below.

74

pi Pi bi Fi

0.4 0.6 0 21

0.32 0.28 1 13

0.12 0.16 0 8

0.09 0.07 1 5

0.05 0.02 1 3

0.02 0 0 2

This is a Type I source.

SRPS : 0 1 0 1 1 0. The SRPS written in the reverse order : 0 1 1 0 1 0.


Table 5.6 : F-ROW AND G-ROW TABLE FOR THE SIX SYMBOL SOURCE

F Row 2 3 5 8 13 21

G Row 4 9 10 22


additions are listed below along with the allotted G-numbers and their

indices.

(2,3)1 (41,5)2 (8,92)3 (103,13)4 (21,224)5

The following are the codes :

s1 = 0 , s2 = 11 , s3 = 100, s4 = 1011, s5 = 10101, s6 = 10100.

75

Example 5.4 : In this example a source with even number of equi-probable

symbols is considered with m = 10 for which source codes are to be found.

The equal probabilities are : 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1.

F row : 2 , 3, 5, 8, 13, 21, 34, 55, 89, 144 m F-numbers

G row : 145, 146, 147, 148, 149, 150, 151, 152 (m-2) G-numbers

The Probability addition pairs formed with indices : (2,3)1 (5,8)2 (13,21)3

(34,55)4 (89,144)5 (1451,1462)6 (1473,1484)7 (1495,1506)8 (1517,1528)9

The codes are: 1100 , 1101, 1110, 1111, 000, 001, 010, 011, 100, 101.

Example 5.5 : In this example a source with odd number of equi-probable

symbols is considered with m = 15 for which source codes are to be

constructed.

The equal probabilities are: 0.0667, 0.0667, 0.0667, 0.0667, 0.0667,

0.0667, 0.0667, 0.0667, 0.0667, 0.0667, 0.0667, 0.0667, 0.0667, 0.0667,

0.0667.

F row : 2 , 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597.

G row : 1598, 1599, 1600, 1601, 1602, 1603, 1604, 1605, 1606, 1607,

1608, 1609, 1610.

The Probability addition pairs formed with indices :

(2,3)1 (5,8)2 (13,21)3 (34,55)4 (89,144)5 (233, 377)6 (610, 987)7 (1597,

15981)8 (15992, 16003)9 (16014, 16025)10 (16036, 16047)11 (16058, 16069)12

(160710, 160811)13 (160912, 161013)14.

76

The codes are : 0010, 0011, 0100, 0101, 0110, 0111, 1000, 1001, 1010,

1011, 1100, 1101, 1110, 1111, 000.

Example 5.6 : In this example a source with even number of equi-probable

symbols is considered with m = 6 for which source codes are to be derived.

F – numbers : 2 , 3, 5, 8, 13, 21.

G – numbers above 21: 22, 23, 24, 25

List of PAPs with indices : (2, 3)1 (5, 8)2 (13, 21)3 (221, 232)4 (243,254)5

The codes are : 10 , 00 , 111, 110, 101, 100.

Example 5.7 : In this example a source with odd number of equi-probable

symbols is considered with m = 9 for which source codes are to be

generated.

F – numbers : 2 , 3, 5, 8, 13, 21, 34, 55, 89.

G – numbers above 89 : 90, 91, 92, 93, 94, 95, 96.

List of PAPs with indices :

(2, 3)1 (5, 8)2 (13, 21)3 (34, 55)4 (89, 901)5(912, 923)6 (934, 945)7 (956, 967)8

The codes are : 110 , 101 , 100 , 011 , 010 , 001 , 000 , 1111 , 1110.



0.21, 0.13, 0.05, 0.04 and 0.03.


are listed below.

77

pi Pi bi Fi

0.54 0.46 1 21

0.21 0.25 0 13

0.13 0.12 1 8

0.05 0.07 0 5

0.04 0.03 1 3

0.03 0 0 2

This is a Type I source . Hence the special Case – 3 can be applied for the

source.

SRPS: 1 0 1 0 1 0. SRPS written in the reverse order : 0 1 0 1 0 1.


Table 5.7 : THE F-ROW AND G-ROW TABLE FOR A SIX SYMBOL SOURCE

F Row 2 3 5 8 13 21

G Row 6 7 14 15

Probability Addition Pairs: (2,3)1 (5,61)2 (72,8)3 (13,143)4 (154,21)5

Codes: s1 = 1 , s2 = 00 , s3 = 011 s4 = 0100 , s5 = 01011, s6 = 01010.

78

5.8 FLOW CHART :

Yes

res[i][0] = 0 ; i++

i < n

fib = fib1 + fib2

fib1 = fib2

a

fib2 = fib

i = n-1

No

i ≥ 0

N = n , i = 0

i++ (i=i+1)

O[i].p = s[i].p = x

Yes

Read probabilities into ‘x’

i < n

No

start

Read n

Initialise i to 0

79

Continuation of the flowchart ……

Yes

No i < n

Print s[i].p, s[i].fbn

i++

i = 0

s[i].resultfib = -1

s[i].fibn = fib

o [i].fibn = fib

a

s[i].flag = 1

Yes

l = N – 1

b

C

No

i = n-1, j = 0

x=s[i].p+s[i-1].p

q[j].x=s[i]fibn

q[j].y=s[i-1]fibn

D

i > 0

80


No

Yes N++

t=0

k>N

s[k+1].p=s[k].p

s[k+1].fibn=s[k].fibn

s[k+1].flag=s[k].flag

s[k+1].resultflag=s[k].resfib

No

B t< N

Print s[t].p, s[t].fibn ,

s[t].flag , s[t].resfib, t

t++

Yes

l = - 1

x == 1

k = N - 1

Yes

No

l = l - 1

x ≠ 1

s[l].p ≠1 ; l ≥ 0

b

81


d

C

No

Yes

Print q[l].x , q[l].y

l ≤ j

B

i=i+1

l=0

l=l+1

t = 0 ; r = 2

No

Yes

Print o[i].p, o[i].fibn

i< n

D

i=0

i++

Yes

I

H

Key = o[t].fibn

h=0

t< n

82


d

key=q[h].y

break

h=h+1

Break

res[t][1]=1

res[t][o]=res[t][o]+1

No

Yes

Yes

No E

key=s[m].fibn

m<N

yes

No

Yes

res[t][1]=0


No h<j

key=q[h].x

Yes

key=q[j-1].y H G

m=0

key=s[m].resfib

n

Break

m = m+1

No

83


Yes

res[t][r]=0


r=r+1

break

Yes

No

key=q[h].x

h=0

h<j

m≠N

G

E

Yes

No

No

key=q[h].y

h=h+1

Res[t][r]=1

Res[t][o]++

r=r+1

break

Yes

84


No

Yes

t< n

t=0

I

Print o[t].fibn ,

o[t].p

r=res[t][o]

No

stop

Yes

r = r - 1

t=t+1

Print res[t][r]

r > 1

85

5.9 SIMULATION RESULTS :

Source codes for the Type II source of Example 5.1

86

Source codes for the Type II English text message source of Example 5.2

87

Source codes for the Type I source of Example 5.3

Special case of equi probable symbols of Example 5.6

88

5.10 RESULTS AND DISCUSSION

RESULTS :

The proposed algorithm is applied for few Type II and Type I discrete

information sources for finding the codes of their symbols. The generated

codes are found to be exactly similar to those obtained from Huffman

scheme. Of particular interest are, the sources of example 5.1 , 5.2 and

5.3 since the codes of these sources are to be used in the succeeding

modules for sake of convergence and connectivity.

The Type I source of example 4.11 is considered again in the

example 5.3 and its codes are constructed using the proposed scheme of

this chapter. The codes generated are tallying with the codes obtained in

the example 4.11.

The codes for these three sources are listed below:

For the source of example 5.1,

s1 = 0, s2 = 111, s3 = 101, s4 = 100, s5 = 1101, s6 = 1100.


s1 = 110 , s2 = 101 , s3 = 1111 , s4 = 1001 , s5 = 1000, s6 = 0111 ,

s7 = 0100 , s8 = 0011 , s9 = 0010 , s10 = 0000 , s11 = 11100 , s12 = 01100 ,

s13 = 01011 , s14 = 01010 , s15 = 00010 , s16 = 111011 , s17 = 111010 ,

s18 = 011010 , s19 = 0110111 , s20 = 0110110 , s21 = 0001111 ,

s22 = 0001110 , s23 = 0001101 , s24 = 0001100.


s1 = 0 , s2 = 11 , s3 = 100, s4 = 1011, s5 = 10101, s6 = 10100.

89

DISCUSSION :

The algorithm offers a way of deriving the source codes by posting

smaller total number of entries of probabilities or their PINs when

compared with the entries of odd decimal values of probabilities and the

total number of entries to be made in Huffman scheme.

a) Comparison for the number of entries of odd decimal values of

probabilities to be made :

The algorithm offers a way of deriving the source codes by making

far less number of entries of odd decimal values of probabilities when

compared with the entries to be made in Huffman scheme.

Number of entries of probabilities in the Huffman scheme :

m+(m-1)+(m-2)…..+2+1 = m(m+1)/2 …………….(5.2)

Number of entries of probabilities in the proposed scheme :

m entries of pi + (m-2) entries of Pi = 2m–2 ………….(5.3)

The comparative study made between the two schemes with respect

to this parameter of number entries of odd decimal probability values is

shown in the Table 5.8.

90

Table 5.8 : COMPARISON WITH THE CONVENTIONAL METHOD

FOR THE NUMBER OF ENTRIES OF DECIMAL PROBABILITY VALUES

m

Huffman method : Number

of entries of decimal

probabilities = m(m+1)/2

Proposed method :

Number of entries of decimal

probabilities = 2m – 2

3 4

5

6 10

20

25 40

50

6 10

15

21 55

210

325 820

1275

4 6

8

10 18

38

48 78

98

b) Comparison for the total number of entries to be made :

Conventional method :

Number of entries of the odd probability values : m(m+1)/2 …..(I)

Number of entries to be made to allot 1s and 0s for every pair added : 2

Total number of addition pairs : (m-1)

Hence number of entries to be made for

binary digit allotment : 2(m-1)…….….(II)

Total number of entries to be made :

(I) + (II) = (m(m+1)/2 )+ 2(m-1) = (m2 +5m-4)/2 ………….(5.4)

Proposed method :

Entering the probabilities pi of symbols : m …………….(A)

Entering the probabilities, Pi : m-2…………..(B)

Entering the bits of SRPS, bi : m-2……….….(C)

Entering F numbers : m………….…(D)

Entering G numbers : m-2…………..(E)

91

Entering the indices allotted for PAPs : m-1……………(F)

Entering the indices allotted for SOS : m-2……………(G)

Entering the binary code elements 1 and 0 : 2(m-1) = 2m-2……(H)

Total number of entries :

(A)+(B)+(C)+(D)+(E)+(F)+(G)+(H) = (9m–11)………………….(5.5)

For the case of equi-probable symbols , Number of entries in the

proposed scheme :

(D)+(E)+(F)+(G)+(H) = 6m–7 ………………….(5.6)

The comparative study made between the two schemes with respect

to this parameter of total number of entries is shown in the Table 5.9. For

the case of equi-probable sources the comparative analysis is shown in

Table 5.10.

Table 5.9 : COMPARISON WITH THE CONVENTIONAL METHOD

FOR THE TOTAL NUMBER OF ENTRIES TO BE MADE

Number of

Symbols, m

Conventional

method

(m2 + 5m – 4)/2

Proposed

method

9m-11

3 5

10

20 25

30

40 50

80

100

10 23

73

248 376

523

898 1373

3398

5248

16 34

79

169 214

259

349 439

709

889

92

Table 5.10 : COMPARISON WITH THE CONVENTIONAL METHOD FOR

THE TOTAL NUMBER OF ENTRIES TO BE MADE FOR SOURCES OF EQUI-PROBABLE SYMBOLS

Number of

Symbols, m

Conventional method

(m2 + 5m – 4)/2

Proposed method

6m-7

3 5

10

20

25 30

40

50 80

100

10 23

73

248

376 523

898

1373 3398

5248

11 23

53

113

143 173

233

293 473

593

Another distinct positive feature of the proposed scheme is that the

probabilities need not be arranged in the descending order every time and

hence it takes relatively less time to develop the codes.

93

5.11 CONCLUSIONS

A connection is established between source coding and code

lengths of Golden sequence terms as an application of Fibonacci numbers.

A simpler algorithm for generating source codes of symbols of discrete

information sources is presented in this Chapter.

The following are the merits of the scheme proposed in this module.

1. The algorithm is simpler as the actual probabilities of all symbols

which may be decimal values need not be repeatedly listed in the

descending order as in the case of Huffman scheme. Hence it takes

less time for code construction.

2. In Huffman scheme the total number of entries of probability values

for an m-symbol source is N=m(m+1)/2. Where as in the proposed

scheme this number is N=(2m-2). This number is much smaller for

m>5, when compared to Huffman scheme and hence results in less

time for code construction.

3. The tracking of probabilities of symbols while reading the codes is

avoided, because the odd numerical values need not be tracked but

only the PINs in the form of Fibonacci and Non- Fibonacci numbers

are to be tracked, which are integers.

Documents

Chapter 5 SOURCE CODING FOR THE SYMBOLS OF …shodhganga.inflibnet.ac.in/bitstream/10603/4577/14/14...m = 0 for the last 2 bits of SRPS. Example : A source of five symbols is considered