Of 14 01/03/2015ISCA-2015: Reliable Meaningful Communication1 Reliable Meaningful Communication...

of 1401/03/2015 ISCA-2015: Reliable Meaningful Communication 1

Reliable Meaningful Communication

Madhu SudanMicrosoft, Cambridge,

USA

of 14

Reliable Communication?

Problem from the 1940s: Advent of digital age.

Communication media are always noisy But digital information less tolerant to noise!

01/03/2015 ISCA-2015: Reliable Meaningful Communication 2

Alice Bob

We are not

ready

We are now

ready

of 14

Coding by Repetition

Can repeat (every letter of) message to improve reliability:

WWW EEE AAA RRR EEE NNN OOO WWW …

WXW EEA ARA SSR EEE NMN OOP WWW … Calculations:

repetitions Prob. Single symbol corrupted To transmit symbols, choose Rate of transmission as Belief (pre-1940s): Rate of any scheme as

01/03/2015 ISCA-2015: Reliable Meaningful Communication 3

of 14

Shannon’s Theory [1948]

Sender “Encodes” before transmitting Receiver “Decodes” after receiving

Encoder/Decoder arbitrary functions.

Rate = Requirement: w. high prob.

What are the best (with highest Rate)?

01/03/2015 ISCA-2015: Reliable Meaningful Communication 4

Alice BobEncoder Decoder

of 14

Shannon’s Theorem

If every bit is flipped with probability Rate can be achieved.

This is best possible. Examples:

Monotone decreasing for Positive rate for ; even if

01/03/2015 ISCA-2015: Reliable Meaningful Communication 5

of 14

Challenges post-Shannon

Encoding/Decoding functions not “constructive”. Shannon picked at random, brute force. Consequence:

takes time to compute (on a computer). takes time to find!

Algorithmic challenge: Find more explicitly. Both should take time to compute

01/03/2015 ISCA-2015: Reliable Meaningful Communication 6

of 14

Explicit Codes: Reed-Solomon Code

Messages = Coefficients of Polynomials. Example:

Message = Think of polynomial Encoding: First four values suffice, rest is redundancy!

(Easy) Facts: Any values suffice where = length of message. Can handle erasures or errors.

Explicit encoding Efficient decoding? [Peterson 1960]

01/03/2015 ISCA-2015: Reliable Meaningful Communication 7

of 14

More Errors? List Decoding

Why was the limit for #errors? is clearly a limit – right?

First half = evaluations of Second half = evaluations of What is the right message: or ?

(even ) is the limit for “unique” answer. List-decoding: Generalized notion of decoding.

Report (small) list of possible messages. Decoding “successful” if list contains the

message polynomial.

01/03/2015 ISCA-2015: Reliable Meaningful Communication 8

of 14

Reed-Solomon List-Decoding Problem

Given: Parameters: Points: in the plane

(finite field actually) Find:

All degree poly’s that pass thru of points i.e., all s.t.

: Answer unique; [Peterson 60] finds it. [S. 96, Guruswami+S. ‘98]: ; small list

01/03/2015 ISCA-2015: Reliable Meaningful Communication 9

of 14

Decoding by example + picture [S’96]

Algorithm idea:

Find algebraic explanation of all points.

Stare at the solution (factor the polynomial)

Ssss

01/03/2015 ISCA-2015: Reliable Meaningful Communication 10

𝑛=14 ;𝑘=1;𝑡=5

(𝑥+𝑦 )(𝑥− 𝑦 )(𝑥2+𝑦2−1)

𝑥4−𝑦 4−𝑥2+ 𝑦2=0

of 14

Decoding by example + picture [S’96]

Algorithm idea:

Find algebraic explanation of all points.

Stare at the solution (factor the polynomial)

Ssss

01/03/2015 ISCA-2015: Reliable Meaningful Communication 11

𝑛=14 ;𝑘=1;𝑡=5

(𝑥+𝑦 ) (𝑥−𝑦 )(𝑥2+𝑦2−1)

𝑥4−𝑦 4−𝑥2+ 𝑦2=0

ISCA-2015: Reliable Meaningful Communication 12 of 1401/03/2015

Decoding Algorithm

Fact: There is always a degree polynomial thru points Can be found in polynomial time (solving linear

system).

[80s]: Polynomials can be factored in polynomial time [Grigoriev, Kaltofen, Lenstra]

Leads to (simple, efficient) list-decoding correcting fraction errors for

of 14

Summary and conclusions

(Many) errors can be dealt with: Pre-Shannon: vanishing fraction of errors Pre-list-decoding: small constant fraction Post-list-decoding: overwhelming fraction

Future challenges? Communication can overcome errors introduced by

channels. Can communication overcome errors in

misunderstanding between sender and receiver? [Goldreich,Juba,S. ‘2011]; [Juba,Kalai,Khanna,S.’2011]

….

01/03/2015 ISCA-2015: Reliable Meaningful Communication 13

of 14

Thank You!

01/03/2015 ISCA-2015: Reliable Meaningful Communication 14