Embed Size (px)
Citation preview
Michael A. Nielsen
University of Queensland
Quantum entropy
Goals: 1. To define entropy, both classical and quantum. 2. To explain data compression, and its connection
with entropy.3. To explain some of the basic properties of entropy,
both classical and quantum.
What is an information source?
011000101110011100101011100011101001011101000
We need a simple model of an information source.
The model might not be realistic, but it should give rise to a theory of information that can be applied torealistic situations.
Discrete iid sources
Definition: Each output from a discrete information source comes from a finite set.
We will mostly be concerned with the case where thealphabet consists of 0 and 1.
01100010111001110010101110001…
More generally, there is no loss of generality insupposing that the alphabet is 0,…,n-1.
Discrete iid sources
We will model sources using a probability distribution for the output of the source.
01100010111001110010101110001…
Definition: Each output from an iid (independent and identically distributed) source is independent of the other outputs, and each output has the same distribution.Example: A sequence of coin tosses of a biased coinwith probability p of heads, and 1-p of tails.
More generally, the distribution on alphabet symbolsis denoted p0,p1,…,pn.
What other sources are discrete iid?
Most interesting sources are not.
However, lots of sources can be approximated as iid –even with English text this is not a bad approximation.
“What a piece of work is a man! how noble in reason! how infinite infaculties! in form and moving how express and admirable! in action how like an angel! in apprehension how like a god! the beauty of the world, theparagon of animals! And yet to me what is this quintessence of dust?”
Many sources can be described as stationary, ergodicsequences of random variables, and similar results apply.
Research problem: Find a good quantum analogueof “stationary, ergodic sources” for, and extendquantum information theory to those sources.(Quantum Shannon-Macmillan-Breiman theorem?)
How can we quantify the rate at which information is being produced by a
source?Two broad approaches
Axiomatic approach: Write down desirable axiomswhich a measure of information “should” obey, and find such a measure.
Operational approach: Based on the “fundamentalprogram” of information science.
How many bits are needed to store the output of the source, so the output can be reliably recovered?
Historical origin of data compression
“He can compress the mostwords into the smallest ideasof any man I ever met.”
Data compression
abcde… {
n uses
nR bits
compress decompress
abcde…
What is the minimal value of R that allowsreliable decompression?
We will define the minimal value to be theinformation content of the source.
Shannon's noiseless channel coding theorem: Shannon
The minimalachievable value of is given by the ofthe source distribution, log ,where
entrop
logarithms are taken to base two.
y
x x xx
R
H X H p p p
Data compression
Suppose we flip coins, getting heads with probability p, and tails with probability 1-p.
For large values of n, it is very likely that we will getroughly np heads, and n(1-p) tails.
Typical sequenc 1 #Headss: 1e np np 1 1 #Tails 1 1n p n p
Data compression
Typical sequenc 1 #Headss: 1e np np
Sequence is typical with probability 1.
Atypical sequences
1 1 #Tails 1 1n p n p
Pr x 1 11 1n pnpp p
1 11 1
n pnpp p
log 1 log 1Pr 2np p n p px ,12 nH p p ,1#Typical sequences 2nH p p
Data compression: the algorithm
Let be the source outputy
,1#Typical sequences 2nH p p
Sequence is typical with probability 1
I f is atypical then send the bit 0 and then the bit string
yy
The two critical facts
,1I n principle it is possible to construct a containing an of all 2 typical sequenc
lookup tableindexed list es.nH p p
else send 1 and the index of in the lookup tabley
n+1 bitsnH(p,1-p)+1 bits
On average, only H(p,1-p) bits were required tostore the compressed string, per use of the source.
1. x1
2. x2
3. x3
4. x4
…
Variants on the data compression algorithm
Our algorithm is for large n, gives variable-length output that achieves the Shannon entropy on average. The algorithm never makes an error in recovery.
Algorithms for small n can be designed that do almostas well.
Fixed-length compressionLet be the source outputy
I f is atypical then send ,1 1 0's
ynH p p
else send 1 and the index of in the lookup tabley
Errors must always occur in a fixed-length scheme,but it does work with probability approaching one.
Why it’s impossible to compressbelow the Shannon rate
Typical sequences
Atypical sequences
Suppose ,1R H p p
At most 2 sequences can be correctly compressed andthen decompressed by a fi xed-length scheme of rate .
nR
R
Pr 0
,1Pr 2 2 nH p pnR
,1Pr 2 0n R H p p
Basic properties of the entropy
logx x xxH X H p p p 0log0 0
,1 is known as binary entropy. the H p H p p
The entropy is non-negative and ranges between 0and log .d
Why’s this notion called entropy, anyway?
“When the American scientist Claude Shannon foundthat the mathematical formula of Boltzmann defineda useful quantity in information theory, he hesitatedto name this newly discovered quantity entropy becauseof its philosophical baggage. The mathematician JohnVon [sic] Neumann encouraged Shannon to go aheadwith the name entropy, however, since`no one knowswhat entropy is, so in a debate you will always havethe advantage.’ ”
From the American Heritage Book of EnglishUsage (1996):
What else can be done with the Shannon entropy?
Quantum processes
teleportation
communication
cryptography
theory of entanglement
Shor’s algorithm
quantum error-correction
Complexity
quantum phase transitions
1. Identify a physical resource – energy, time, bits, space, entanglement.2. Identify an information processing task – data compression, information transmission, teleportation.
3. Identify a criterion for success.
How much of 1 do I need to achieve 2, while satisfying 3?
What else can be done with the Shannon entropy?
Classical processes
data compression
networks
cryptography
thermodynamics
reliable communication in the presence of noise
Complexity
gambling
quantum information
What is a quantum information source?
Example: “Semiclassical coin toss”
10 with probability
21
1 with probability 2
Example: “Quantum coin toss”1
0 with probability 2
0 1 1 with probability
22
A quantum inf ormation sourceproduces states wGeneral
ith probdefi
abilnition
iti .:
es j jp
Quantum data compression
0
decompression
0
1j
compression2j
3j
4j
1,..., nJ j j
5j
1...
nj jJp p p
1...
nj jJ
,
(Recall that .)
J J JJ
J J J
F p F
F
1F
J
What’s the best possible rate for quantum data compression?
“Semiclassical coin toss” 1 10 w. p. , 1 w. p.
2 2
“Quantum coin toss”0 11 1
0 w. p. , w. p. 2 22
1 1.
2Answer: H
1 1/ 2Answer: 0.6.
2H
1 1?
2Answer: H
betteI n general, we can do than Shannon's rater .jH p
Quantum entropy
j j jj
p
von Neumann entropyDefi ne the , log .k k kk
S H
Suppose has diagonal representation
( tr log ).k k kk
e e
TheShu mimacher nimal
a's noiseless
chievable va channel coding theorem:
lue of the rate .
is R S
Basic properties of the von Neumann entropy
0 logS d
, where are the eigenvalues of .k kS H
A B
Show thaExercise: t
.A AB BS S S
AB
Subadditivity:
.AAB BS S S
The typical subspace
1 2Typical sequence ,...s: , nSx x
Atypical sequences
0 0 1Example 1: 1 , .p p S H p
1 2Typical subspace: spanned by ,..., , .nS j j
j
x x P x x
Outline of Schumacher’s data compression
Measure to determine whetherwe're in the typical subspace or not. ,
j
P Q I P
Unitarily transf orm0 ... 0jx j
Send .j
Append 0 's: 0 ... 0 .j
I nverse transf orm 0 ... 0 .jj x
Send 0 .nS
Claim 1.F
P Q
Recall classical to quantum circuits
x( )f x
classical f
x
20
f x
xquantum fU
Verif y that the eff ect on the fi rst register is
with probability
Ex
with probabilit
ercise:
y
PP
PQ
How to measure P, Q
x 0 if typical( )
1 if atypical
xT x
x
classical circuit
x
0 Measure.T x
x
TU
Outline of Schumacher’s data compression
Measure to determine whetherwe're in the typical subspace or not. ,
j
P Q I P
Unitarily transf orm0 ... 0jx j
Send .j
Append 0 's: 0 ... 0 .j
I nverse transf orm 0 ... 0 .jj x
Send 0 .nS
Claim 1.F
P Q
jxj
classicallook-uptable
0 ... 0jx jHow to unitarily transform
jxj
inverselook-uptable
jx
j
0j j
n
x x
30
jx
j
quantumlook-up
table
inversequantumlook-up
table
0 ... 0jx jHow to unitarily transform
jx j
00
U
Outline of Schumacher’s data compression
Measure to determine whetherwe're in the typical subspace or not. ,
j
P Q I P
Unitarily transf orm0 ... 0jx j
Send .j
Append 0 's: 0 ... 0 .j
I nverse transf orm 0 ... 0 .jj x
Send 0 .nS
Claim 1.F
P Q
Schumacher compression
Uj
0TU
with probability JJ J
J J
PP
P
measure: 0
†U0
0
,J J J J J JF F
JFidelity for
Ensemble f or :J with probability JJ J
J J
PP
P
junk with probability J JQ
junk junkJ JJ J J J J
J J
P PP Q
P
junk junkJ J J JP P Q
J J J J JF P P
J JP
,J J J J J JJ J
F p F p P 1F Reliability:
trJ J JJ
p P times
But ... .n
J J JJ
p
tr nF P
0 1 0 10 0 1 1; ; 1 .p p p p p p
nyy
p y y typicalxP x x
typical, tryx y
p x x y y typical, y xyx y
p typical xxp 1
S Proof that it's impossible to compress to a rate below
The idea of the proof is similar to Shannon’s proof.
Two known proofs:
One is a complicated kludge, done from firstprinciples.
The other proof is an elegant “easy” proof thatrelies on other deep theorems.
Find an easy fi rst-principles proof
that is the Research
best ach prob
ievablem
le:
.
rateS
Prove that the von Neumann entropysatisfi es the inequality
.( You may fi nd it usef ul to use the Schumacher andShannon no
Worked exer
iseless cha
cise:
Hinnel coding theorem
nt:s.)
j j j jjS p H p
Prove that Exercise: .j j j j jj jS p H p p S
Research problem(?):exactly
one
Find a low-rate quantum data compression scheme that, with probability
, produces decompressed states with fi delity tothe source output approaching one.
Suppose a source outputs with probabilityResearch problem - mixed-state quantum data compressi n
o :
. What isthe best achievable rate of compression f or such a source?
j jp
