Information Theory, Coding and Cryptography Unit-5 by Arun Pratap Singh

7/13/2019 Information Theory, Coding and Cryptography Unit-5 by Arun Pratap Singh

1/79

UNIT VINFORMATION THEORY, CODING & CRYPTOGRAPHY (MCSE 202)

PREPARED BY ARUN PRATAP SINGH 5/26/14 MTECH2nd SEMESTER


2/79

PREPARED BY ARUN PRATAP SINGH 1

1

ADVANCE CODING TECHNIQUES : REED SOLOMON CODES

UNIT : V


3/79


2


4/79


3


5/79


4


6/79


5


7/79


6

SPACE TIME CODES :

A spacetime code (STC) is a method employed to improve the reliability of

datatransmission in wireless communication systems using multiple transmitantennas.STCsrely on transmitting multiple,redundant copies of a data stream to thereceiver in the hope that at

least some of them may survive thephysical path between transmission and reception in a good

enough state to allow reliable decoding.

Space time codes may be split into two main types:
http://en.wikipedia.org/wiki/Transmission_(telecommunications)http://en.wikipedia.org/wiki/Wirelesshttp://en.wikipedia.org/wiki/Antenna_(radio)http://en.wikipedia.org/wiki/Redundancy_(information_theory)http://en.wikipedia.org/wiki/Receiver_(radio)http://en.wikipedia.org/wiki/Transmission_mediumhttp://en.wikipedia.org/wiki/Transmission_mediumhttp://en.wikipedia.org/wiki/Receiver_(radio)http://en.wikipedia.org/wiki/Redundancy_(information_theory)http://en.wikipedia.org/wiki/Antenna_(radio)http://en.wikipedia.org/wiki/Wirelesshttp://en.wikipedia.org/wiki/Transmission_(telecommunications)


8/79


7

Spacetime trellis codes (STTCs) distribute atrellis code over multiple antennas and multiple

time-slots and provide both coding gain anddiversity gain.

Spacetime block codes (STBCs) act on a block of data at once (similarly toblock codes)and

also providediversity gain but doesn't provide coding gain.

STC may be further subdivided according to whether the receiver knows thechannel impairments.

Incoherent STC,the receiver knows the channel impairments through training or some other form

of estimation. These codes have been studied more widely, anddivision algebras over number

fields have now become the standard tool for constructing such codes.

Innoncoherent STC the receiver does not know the channel impairments but knows the statistics

of the channel. In differential spacetime codes neither the channel nor the statistics of the

channel are available.
http://en.wikipedia.org/wiki/Space%E2%80%93time_trellis_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_trellis_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_trellis_codehttp://en.wikipedia.org/wiki/Convolutional_codehttp://en.wikipedia.org/wiki/Diversity_receptionhttp://en.wikipedia.org/wiki/Space%E2%80%93time_block_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_block_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_block_codehttp://en.wikipedia.org/wiki/Block_codehttp://en.wikipedia.org/wiki/Diversity_receptionhttp://en.wikipedia.org/wiki/Channel_(communications)http://en.wikipedia.org/w/index.php?title=Coherent_space-time_code&action=edit&redlink=1http://en.wikipedia.org/wiki/Division_algebrashttp://en.wikipedia.org/wiki/Noncoherent_STChttp://en.wikipedia.org/wiki/Differential_space%E2%80%93time_codehttp://en.wikipedia.org/wiki/Differential_space%E2%80%93time_codehttp://en.wikipedia.org/wiki/Differential_space%E2%80%93time_codehttp://en.wikipedia.org/wiki/Differential_space%E2%80%93time_codehttp://en.wikipedia.org/wiki/Noncoherent_STChttp://en.wikipedia.org/wiki/Division_algebrashttp://en.wikipedia.org/w/index.php?title=Coherent_space-time_code&action=edit&redlink=1http://en.wikipedia.org/wiki/Channel_(communications)http://en.wikipedia.org/wiki/Diversity_receptionhttp://en.wikipedia.org/wiki/Block_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_block_codehttp://en.wikipedia.org/wiki/Diversity_receptionhttp://en.wikipedia.org/wiki/Convolutional_codehttp://en.wikipedia.org/wiki/Space%E2%80%93time_trellis_code


9/79


8

CONCATENATED CODES :

Concatenated codesare error-correcting codes that are constructed from two or more simplercodes in order to achieve good performance with reasonablecomplexity.Originally introduced by

Forney in 1965 to address a theoretical issue, they became widely used in space communications

in the 1970s.Turbo codes and other modern capacity- approaching codes may be regarded as

elaborations of this approach.
http://www.scholarpedia.org/article/Complexityhttp://www.scholarpedia.org/article/Turbo_codehttp://www.scholarpedia.org/article/Turbo_codehttp://www.scholarpedia.org/article/Complexity


10/79


9

The inner code is a short block code like that envisioned by Shannon, with rate rclose to C,blocklength n, and therefore 2nrcodewords. The inner decoder decodes optimally, so its complexity

increases exponentially with n; for large enough nit achieves a moderately low decoding error

probability.

The outer code is an algebraic Reed-Solomon (RS) code (Reed and Solomon, 1960) of

length 2nr over the finite field with 2nr elements, each element corresponding to an inner

codeword. (The overall block length of the concatenated code is therefore N=n2nr , which is

exponential in n, so the complexity of the inner decoder is only linear in N.) The outer decoder

uses an algebraic error-correction algorithm whose complexity is only polynomial in the RS code

length 2nr; it can drive the ultimate probability of decoding error as low as desired.
http://www.scholarpedia.org/article/Concatenated_codes#RS60http://www.scholarpedia.org/article/Concatenated_codes#RS60


11/79


10


12/79


11


13/79


12


14/79


13

Figure 3: Simple repeat-accumulate code with iterative decoding.

All capacity-approaching codes are now regarded as ``codes on graphs," in which a (possibly

large) number of simple codes are interconnected according to some graph topology. Any such

code may be regarded as a (possibly elaborate) concatenated code.
http://www.scholarpedia.org/article/File:Concatenated_codes_figure_3.gif


15/79


14

TURBO CODING AND LDPC CODES :

In information theory, turbo codes (originally in French Turbocodes) are a class of high-

performance forward error correction (FEC) codes developed in 1993, which were the first

practical codes to closely approach thechannel capacity,a theoretical maximum for thecode

rate at which reliable communication is still possible given a specific noise level. Turbo codes are

finding use in3G mobile communications and (deep space)satellitecommunications as well as

other applications where designers seek to achieve reliable information transfer over bandwidth-

or latency-constrained communication links in the presence of data-corrupting noise. Turbo codes

are nowadays competing withLDPC codes,which provide similar performance.

The name "turbo code" arose from the feedback loop used during normal turbo code decoding,

which was analogized to the exhaust feedback used for engine turbocharging.Hagenauer has

argued the term turbo code is a misnomer since there is no feedback involved in the encoding

process.

An example encoder

There are many different instances of turbo codes, using different component encoders, input/output

ratios, interleavers, and puncturing patterns. This example encoder implementation describes a

classic turbo encoder, and demonstrates the general design of parallel turbo codes.

This encoder implementation sends three sub-blocks of bits. The first sub-block is the m-bit block of

payload data. The second sub-block is n/2parity bits for the payload data, computed using a recursive

systematic convolutional code (RSC code). The third sub-block is n/2 parity bits for a

knownpermutation of the payload data, again computed using an RSC convolutional code. Thus, two

redundant but different sub-blocks of parity bits are sent with the payload. The complete block

has m+ nbits of data with a code rate ofm/(m+ n). Thepermutation of the payload data is carried out

by a device called aninterleaver.

Hardware-wise, this turbo-code encoder consists of two identical RSC coders, 1and C2, as depicted

in the figure, which are connected to each other using a concatenation scheme, called parallel

concatenation:
http://en.wikipedia.org/wiki/Information_theoryhttp://en.wikipedia.org/wiki/Forward_error_correctionhttp://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theoremhttp://en.wikipedia.org/wiki/Code_ratehttp://en.wikipedia.org/wiki/Code_ratehttp://en.wikipedia.org/wiki/3Ghttp://en.wikipedia.org/wiki/Deep_Space_Networkhttp://en.wikipedia.org/wiki/Satellitehttp://en.wikipedia.org/wiki/Telecommunicationhttp://en.wikipedia.org/wiki/LDPC_codehttp://en.wikipedia.org/wiki/Joachim_Hagenauerhttp://en.wikipedia.org/wiki/Convolutional_codehttp://en.wikipedia.org/wiki/Permutationhttp://en.wikipedia.org/wiki/Permutationhttp://en.wikipedia.org/wiki/Interleaverhttp://en.wikipedia.org/wiki/Interleaverhttp://en.wikipedia.org/wiki/Permutationhttp://en.wikipedia.org/wiki/Permutationhttp://en.wikipedia.org/wiki/Convolutional_codehttp://en.wikipedia.org/wiki/Joachim_Hagenauerhttp://en.wikipedia.org/wiki/LDPC_codehttp://en.wikipedia.org/wiki/Telecommunicationhttp://en.wikipedia.org/wiki/Satellitehttp://en.wikipedia.org/wiki/Deep_Space_Networkhttp://en.wikipedia.org/wiki/3Ghttp://en.wikipedia.org/wiki/Code_ratehttp://en.wikipedia.org/wiki/Code_ratehttp://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theoremhttp://en.wikipedia.org/wiki/Forward_error_correctionhttp://en.wikipedia.org/wiki/Information_theory


16/79


15

In the figure, Mis a memory register. The delay line and interleaver force input bits dkto appear in

different sequences. At first iteration, the input sequence dk appears at both outputs of the

encoder,xkand y1kor y2kdue to the encoder's systematic nature. If the encoders C1and C2are used

respectively in n1and n2iterations, their rates are respectively equal to

The decoder

The decoder is built in a similar way to the above encoder. Two elementary decoders are

interconnected to each other, but in serial way, not in parallel. The decoder operates on lower

speed (i.e., ), thus, it is intended for the encoder, and is

for correspondingly. yields asoft decision which causes delay. The same delay is caused

by the delay line in the encoder. The 's operation causes delay.
http://en.wikipedia.org/wiki/Turbo_code#Soft_decision_approachhttp://en.wikipedia.org/wiki/File:Turbo_encoder.svghttp://en.wikipedia.org/wiki/Turbo_code#Soft_decision_approach


17/79


16

An interleaver installed between the two decoders is used here to scatter error bursts coming

from output. DIblock is a demultiplexing and insertion module. It works as a switch, redirecting

input bits to at one moment and to at another. In OFF state, it feeds

both and inputs with padding bits (zeros).

Consider a memorylessAWGN channel, and assume that at k-th iteration, the decoder receives a pair

of random variables:

where and are independent noise components having the same variance . is a k-th bit

from encoder output.

Redundant information is demultiplexed and sent through DI to (when ) and

to (when ).

yields a soft decision; i.e.:

and delivers it to . is called the logarithm of the likelihood

ratio(LLR). is the a posteriori probability(APP) of the data bit which

shows the probability of interpreting a received bit as . Taking the LLR intoaccount, yields a hard decision; i.e., a decoded bit.

It is known that the Viterbi algorithm is unable to calculate APP, thus it cannot be used

in . Instead of that, a modifiedBCJR algorithm is used. For , theViterbi algorithms

an appropriate one.
http://en.wikipedia.org/wiki/Additive_white_Gaussian_noisehttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/BCJR_algorithmhttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/File:Turbo_decoder.svghttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/BCJR_algorithmhttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/Additive_white_Gaussian_noise


18/79


17

However, the depicted structure is not an optimal one, because uses only a proper

fraction of the available redundant information. In order to improve the structure, a feedback

loop is used (see the dotted line on the figure).

Soft decision approach-

The decoder front-end produces an integer for each bit in the data stream. This integer is a measure

of how likely it is that the bit is a 0 or 1 and is also called soft bit. The integer could be drawn from the

range [127, 127], where:

127 means "certainly 0"

100 means "very likely 0"

0 means "it could be either 0 or 1"

100 means "very likely 1"

127 means "certainly 1"

etc.

This introduces a probabilistic aspect to the data-stream from the front end, but it conveys more

information about each bit than just 0 or 1.

For example, for each bit, the front end of a traditional wireless-receiver has to decide if an internal

analog voltage is above or below a given threshold voltage level. For a turbo-code decoder, the front

end would provide an integer measure of how far the internal voltage is from the given threshold.

To decode the m+ n-bit block of data, the decoder front-end creates a block of likelihood measures,

with one likelihood measure for each bit in the data stream. There are two parallel decoders, one for

each of the n2-bit parity sub-blocks. Both decoders use the sub-block of mlikelihoods for the payload

data. The decoder working on the second parity sub-block knows the permutation that the coder used

for this sub-block.

LDPC CODES :

Ininformation theory,a low-density parity-check(LDPC) codeis alinearerror correcting code,

a method of transmitting a message over anoisy transmission channel,[1][2]and is constructed

using a sparsebipartite graph.[3]LDPC codes arecapacity-approaching codes,which means that

practical constructions exist that allow the noise threshold to be set very close (or

even arbitrarilyclose on theBEC)to the theoretical maximum (theShannon limit)for a symmetric

memoryless channel. The noise threshold defines an upper bound for the channel noise, up to

which the probability of lost information can be made as small as desired. Using iterativebelief

propagation techniques, LDPC codes can be decoded in time linear to their block length.
http://en.wikipedia.org/wiki/Information_theoryhttp://en.wikipedia.org/wiki/Linear_block_codehttp://en.wikipedia.org/wiki/Error_correcting_codehttp://en.wikipedia.org/wiki/Signal_noisehttp://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-1http://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-1http://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-1http://en.wikipedia.org/wiki/Bipartite_graphhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-3http://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-3http://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-3http://en.wikipedia.org/wiki/Category:Capacity-approaching_codeshttp://en.wikipedia.org/wiki/Binary_erasure_channelhttp://en.wikipedia.org/wiki/Shannon-Hartley_theoremhttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/Shannon-Hartley_theoremhttp://en.wikipedia.org/wiki/Binary_erasure_channelhttp://en.wikipedia.org/wiki/Category:Capacity-approaching_codeshttp://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-3http://en.wikipedia.org/wiki/Bipartite_graphhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-1http://en.wikipedia.org/wiki/Low-density_parity-check_code#cite_note-1http://en.wikipedia.org/wiki/Signal_noisehttp://en.wikipedia.org/wiki/Error_correcting_codehttp://en.wikipedia.org/wiki/Linear_block_codehttp://en.wikipedia.org/wiki/Information_theory


19/79


18

LDPC codes are finding increasing use in applications requiring reliable and highly efficient

information transfer over bandwidth or return channel-constrained links in the presence of

corrupting noise. Implementation of LDPC codes has lagged behind that of other codes,

notablyturbo codes.The fundamental patent for Turbo Codes expired on August 29, 2013.

Function-

LDPC codes are defined by a sparseparity-check matrix.Thissparse matrix is often randomly

generated, subject to thesparsity constraintsLDPC code construction is discussedlater.These

codes were first designed by Robert Gallager in 1962.

Below is a graph fragment of an example LDPC code usingForney's factor graph notation.In this

graph, nvariable nodes in the top of the graph are connected to (nk) constraint nodes in the

bottom of the graph. This is a popular way of graphically representing an (n, k) LDPC code. The

bits of a valid message, when placed on the T'sat the top of the graph, satisfy the graphicalconstraints. Specifically, all lines connecting to a variable node (box with an '=' sign) have the

same value, and all values connecting to a factor node (box with a '+' sign) must sum,modulo two,

to zero (in other words, they must sum to an even number).

Ignoring any lines going out of the picture, there are eight possible six-bit strings corresponding

to valid codewords: (i.e., 000000, 011001, 110010, 101011, 111100, 100101, 001110, 010111).

This LDPC code fragment represents a three-bit message encoded as six bits. Redundancy is

used, here, to increase the chance of recovering from channel errors. This is a (6, 3) linear code,

with n= 6 and k= 3.

Once again ignoring lines going out of the picture, the parity-check matrix representing this graphfragment is
http://en.wikipedia.org/wiki/Turbo_codehttp://en.wikipedia.org/wiki/Parity-check_matrixhttp://en.wikipedia.org/wiki/Sparse_matrixhttp://en.wikipedia.org/wiki/Sparsityhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#Code_constructionhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#Code_constructionhttp://en.wikipedia.org/wiki/Factor_graphhttp://en.wikipedia.org/wiki/Modular_arithmetichttp://en.wikipedia.org/wiki/Linear_codehttp://en.wikipedia.org/wiki/File:Ldpc_code_fragment_factor_graph.svghttp://en.wikipedia.org/wiki/Linear_codehttp://en.wikipedia.org/wiki/Modular_arithmetichttp://en.wikipedia.org/wiki/Factor_graphhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#Code_constructionhttp://en.wikipedia.org/wiki/Low-density_parity-check_code#Code_constructionhttp://en.wikipedia.org/wiki/Sparsityhttp://en.wikipedia.org/wiki/Sparse_matrixhttp://en.wikipedia.org/wiki/Parity-check_matrixhttp://en.wikipedia.org/wiki/Turbo_code


20/79


19

In this matrix, each row represents one of the three parity-check constraints, while each column

represents one of the six bits in the received codeword.

In this example, the eight codewords can be obtained by putting theparity-check matrix H into

this form through basicrow operations:

From this, thegenerator matrix Gcan be obtained as (noting that in the special case of

this being a binary code ), or specifically:

Finally, by multiplying all eight possible 3-bit strings by G, all eight valid codewords are obtained.

For example, the codeword for the bit-string '101' is obtained by:

Example Encoder Figure 1 illustrates the functional components of most LDPC encoders.
http://en.wikipedia.org/wiki/Parity-check_matrixhttp://en.wikipedia.org/wiki/Row_operationshttp://en.wikipedia.org/wiki/Generator_matrixhttp://en.wikipedia.org/wiki/Generator_matrixhttp://en.wikipedia.org/wiki/Row_operationshttp://en.wikipedia.org/wiki/Parity-check_matrix


21/79


20

LDPC Encoder

During the encoding of a frame, the input data bits (D) are repeated and distributed to a set of

constituent encoders. The constituent encoders are typically accumulators and each accumulator is

used to generate a parity symbol. A single copy of the original data (S0,K-1) is transmitted with the parity

bits (P) to make up the code symbols. The S bits from each constituent encoder are discarded.

In some cases a parity bit is encoded by a second constituent code (serial concatenation), but more

typically the constituent encoding for the LDPC is done in parallel.

In an example using the DVB-S2 rate 2/3 code the encoded block size is 64800 symbols (N=64800)

with 42300 data bits (K=43200) and 21600 parity bits (M=21600). Each constituent code (check node)

encodes 16 data bits except for the first parity bit which encodes 8 data bits. The first 4680 data bits

are repeated 13 times (used in 13 parity codes), while the remaining data bits are used in 3 parity

codes (irregular LDPC code).


22/79


21

For comparison, classic turbo codes typically use two constituent codes configured in parallel, each

of which encodes the entire input block (K) of data bits. These constituent encoders are recursive

convolutional codes (RSC) of moderate depth (8 or 16 states) that are separated by a code interleaver

which interleaves one copy of the frame.

The LDPC code, in contrast, uses many low depth constituent codes (accumulators) in parallel, each

of which encode only a small portion of the input frame. The many constituent codes can be viewed

as many low depth (2 state) 'convolutional codes' that are connected via the repeat and distribute

operations. The repeat and distribute operations perform the function of the interleaver in the turbo

code.

The ability to more precisely manage the connections of the various constituent codes and the level

of redundancy for each input bit give more flexibility in the design of LDPC codes, which can lead to

better performance than turbo codes in some instances. Turbo codes still seem to perform better than

LDPCs at low code rates, or at least the design of well performing low rate codes is easier for Turbo

Codes.

As a practical matter, the hardware that forms the accumulators is reused during the encoding process.

That is, once a first set of parity bits are generated and the parity bits stored, the same accumulator

hardware is used to generate a next set of parity bits.

Decoding

As with other codes, the maximum likelihood decoding of an LDPC code on the binary symmetric

channel is an NP-complete problem. Performing optimal decoding for a NP-complete code of any

useful size is not practical.

However, sub-optimal techniques based on iterativebelief propagation decoding give excellent results

and can be practically implemented. The sub-optimal decoding techniques view each parity check that

makes up the LDPC as an independent single parity check (SPC) code. Each SPC code is decoded

separately using soft-in-soft-out (SISO) techniques such as SOVA, BCJR, MAP, and other derivates

thereof. The soft decision information from each SISO decoding is cross-checked and updated with

other redundant SPC decodings of the same information bit. Each SPC code is then decoded again

using the updated soft decision information. This process is iterated until a valid code word is achieved

or decoding is exhausted. This type of decoding is often referred to as sum-product decoding.

The decoding of the SPC codes is often referred to as the "check node" processing, and the cross-

checking of the variables is often referred to as the "variable-node" processing.

In a practical LDPC decoder implementation, sets of SPC codes are decoded in parallel to increase

throughput.
http://en.wikipedia.org/wiki/Maximum_likelihood_decodinghttp://en.wikipedia.org/wiki/Binary_symmetric_channelhttp://en.wikipedia.org/wiki/Binary_symmetric_channelhttp://en.wikipedia.org/wiki/NP-completehttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/Belief_propagationhttp://en.wikipedia.org/wiki/NP-completehttp://en.wikipedia.org/wiki/Binary_symmetric_channelhttp://en.wikipedia.org/wiki/Binary_symmetric_channelhttp://en.wikipedia.org/wiki/Maximum_likelihood_decoding


23/79


22

In contrast, belief propagation on thebinary erasure channel is particularly simple where it consists of

iterative constraint satisfaction.

For example, consider that the valid codeword, 101011, from the example above, is transmitted across

a binary erasure channel and received with the first and fourth bit erased to yield ? 01? 11. Since

the transmitted message must have satisfied the code constraints, the message can be represented

by writing the received message on the top of the factor graph.

In this example, the first bit cannot yet be recovered, because all of the constraints connected to it

have more than one unknown bit. In order to proceed with decoding the message, constraints

connecting to only one of the erased bits must be identified. In this example, only the second constraint

suffices. Examining the second constraint, the fourth bit must have been zero, since only a zero in that

position would satisfy the constraint.

This procedure is then iterated. The new value for the fourth bit can now be used in conjunction with

the first constraint to recover the first bit as seen below. This means that the first bit must be a one to

satisfy the leftmost constraint.

Thus, the message can be decoded iteratively. For other channel models, the messages passed

between the variable nodes and check nodes are real numbers, which express probabilities and

likelihoods of belief.

This result can be validated by multiplying the corrected codeword rby the parity-check matrix H:

Because the outcome z (the syndrome) of this operation is the three one zero vector, the

resulting codeword ris successfully validated.
http://en.wikipedia.org/wiki/Binary_erasure_channelhttp://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Decoding_methods#Syndrome_decodinghttp://en.wikipedia.org/wiki/File:Ldpc_code_fragment_factor_graph_w_erasures_decode_step_2.svghttp://en.wikipedia.org/wiki/Decoding_methods#Syndrome_decodinghttp://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Binary_erasure_channel


24/79


23

While illustrative, this erasure example does not show the use of soft-decision decoding or soft-

decision message passing, which is used in virtually all commercial LDPC decoders.

NESTED CODES :


25/79


24

BLOCK CODES :


26/79


25


27/79


26


28/79


27


29/79


28


30/79


29

CONVOLUTIONAL CHANNEL CODING :


31/79


30


32/79


31


33/79


32


34/79


33


35/79


34


36/79


35


37/79


36


38/79


37


39/79


38


40/79


39


41/79


40

OR


42/79


41


43/79


42


44/79


43


45/79


44


46/79


45


47/79


46


48/79


47


49/79


48


50/79


49

DISTANCE PROPERTIES AND TRANSFER FUNCTION REPRESENTATION :


51/79


50


52/79


51


53/79


52


54/79


53


55/79


54


56/79


55


57/79


56


58/79


57


59/79


58


60/79


59

DECODING CONVOLUTIONAL CODES :

Several algorithms exist for decoding convolutional codes. For relatively small values of k,

the Viterbi algorithm is universally used as it provides maximum likelihood performance and is

highly parallelizable. Viterbi decoders are thus easy to implement in VLSI hardware and in

software on CPUs withSIMD instruction sets.

Longer constraint length codes are more practically decoded with any of several sequential

decoding algorithms, of which the Fano algorithm is the best known. Unlike Viterbi decoding,

sequential decoding is not maximum likelihood but its complexity increases only slightly with

constraint length, allowing the use of strong, long-constraint-length codes. Such codes were used

in thePioneer program of the early 1970s to Jupiter and Saturn, but gave way to shorter, Viterbi-

decoded codes, usually concatenated with large Reed-Solomon error correction codes that

steepen the overall bit-error-rate curve and produce extremely low residual undetected error

rates.

Both Viterbi and sequential decoding algorithms return hard decisions: the bits that form the most

likely codeword. An approximate confidence measure can be added to each bit by use of theSoft

output Viterbi algorithm.Maximum a posteriori (MAP) soft decisions for each bit can be obtained

by use of theBCJR algorithm.
http://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/Maximum_likelihoodhttp://en.wikipedia.org/wiki/VLSIhttp://en.wikipedia.org/wiki/SIMDhttp://en.wikipedia.org/wiki/Sequential_decodinghttp://en.wikipedia.org/wiki/Sequential_decodinghttp://en.wikipedia.org/wiki/Robert_Fanohttp://en.wikipedia.org/wiki/Pioneer_programhttp://en.wikipedia.org/wiki/Reed-Solomon_error_correctionhttp://en.wikipedia.org/wiki/Soft_output_Viterbi_algorithmhttp://en.wikipedia.org/wiki/Soft_output_Viterbi_algorithmhttp://en.wikipedia.org/wiki/Maximum_a_posteriorihttp://en.wikipedia.org/wiki/BCJR_algorithmhttp://en.wikipedia.org/wiki/BCJR_algorithmhttp://en.wikipedia.org/wiki/Maximum_a_posteriorihttp://en.wikipedia.org/wiki/Soft_output_Viterbi_algorithmhttp://en.wikipedia.org/wiki/Soft_output_Viterbi_algorithmhttp://en.wikipedia.org/wiki/Reed-Solomon_error_correctionhttp://en.wikipedia.org/wiki/Pioneer_programhttp://en.wikipedia.org/wiki/Robert_Fanohttp://en.wikipedia.org/wiki/Sequential_decodinghttp://en.wikipedia.org/wiki/Sequential_decodinghttp://en.wikipedia.org/wiki/SIMDhttp://en.wikipedia.org/wiki/VLSIhttp://en.wikipedia.org/wiki/Maximum_likelihoodhttp://en.wikipedia.org/wiki/Viterbi_algorithmhttp://en.wikipedia.org/wiki/Algorithm


61/79


60


62/79


61


63/79


62


64/79


63


65/79


64


66/79


65

VITERBI ALGORITHM FOR MLSE :


67/79


66


68/79


67


69/79


68


70/79


69


71/79


70


72/79


71


73/79


72


74/79


73


75/79


74


76/79


75


77/79


76


78/79


77


79/79

78

Documents

Information Theory, Coding and Cryptography Unit-5 by Arun Pratap Singh