Embed Size (px)
Citation preview
arX
iv:2
010.
0989
6v4
[cs
.IT
] 1
9 M
ay 2
021
On Effective Convergence in Fekete’s Lemma and
Related Combinatorial Problems in Information Theory
Holger Boche
Theoretical Information Technology
Technical University of Munich
Munich Center for Quantum Science and Technology
D-80333 Munich, Germany
Email: [email protected]
Yannik Böck
Theoretical Information Technology
Technical University of Munich
D-80333 Munich, Germany
Email: [email protected]
Christian Deppe
Institute for Communications Engineering
Technical University of Munich
D-80333 Munich, Germany
Email: [email protected]
Abstract
Fekete’s lemma is a well known result from combinatorial mathematics thatshows the existence of a limit value related to super- and subadditive sequencesof real numbers. In this paper, we analyze Fekete’s lemma in view of the arith-metical hierarchy of real numbers by Zheng and Weihrauch and fit the resultsinto an information theoretic-context. We introduce special sets associated tosuper- and subadditive sequences and prove their effective equivalence to Σ1 andΠ1. Using methods from the theory established by Zheng and Weihrauch, wethen show that the limit value emerging from Fekete’s lemma is, in general, nota computable number. Furthermore, we characterize under which conditionsthe limit value can be computed and investigate the corresponding modulus ofconvergence. We close the paper by a discussion on how our findings affectcommon problems from information theory.
Keywords: Fekete’s lemma, arithmetical hierarchy, computability, effective
convergence.
Preprint submitted to Journal of LATEX Templates Thursday 20th May, 2021
1. Introduction
A sequence (an)n∈N of real numbers is called superadditive if it satisfies
an+m ≥ an + am for all n,m ∈ N. It is called subadditive if it satisfies an+m ≤
an + am for all n,m ∈ N. Fekete’s lemma proves the existence of a limit value
related to sequences that are either super- or subadditive:
Lemma 1 (Fekete [2]).
1. Let (~an)n∈N be a sequence of real numbers that satisfies superadditivity.
Then, there exists x∗ ∈ R ∪ {+∞} such that
supn∈N
~ann
= limn→∞
~ann
= x∗ (1)
holds true.
2. Let ( ~an)n∈N be a sequence of real numbers that satisfies subadditivity.
Then, there exists x∗ ∈ R ∪ {−∞} such that
infn∈N
~ann
= limn→∞
~ann
= x∗ (2)
holds true.
Since the works of Bruijn and Erdös [3, 4], several authors have developed
generalizations of Fekete’s lemma (see [5]). For an up-to-date overview and
interesting discussions, we refer to the work of Füredi and Ruzsa [6]. In the
present paper, we aim to characterize the computability-properties of the limit
value by applying a theory established by Zheng and Weihrauch [1], known as
the arithmetical hierarchy of real numbers.
In addition to common problem statements from information theory, which
we will treat in detail throughout the paper, super- and subadditivity of se-
quences of real numbers occurs throughout different other areas of mathematics.
In economics, it is an essential property of some cost functions. Similar relations
also appear in physics and in combinatorial optimization (see [7]). Further appli-
cations of Fekete’s lemma exist in the research of cellular automatas [8] and for
analogues on cancellative amenable semigroups [9]. Another important example
2
can be found in Ramsey theory. The proof of Roth’s fundamental theorem [10]
also makes use of Fekete’s lemma. Thus, our results are of high relevance for a
great variety of research areas.
The majority of problems from information theory involve the calculation
of channel capacities. In most cases, a certain channel model gives rise to a
monotonically non-decreasing sequence (Mn)n∈N of natural numbers, where n
corresponds to the number of channel uses and M corresponds to the number of
messages transmittable through n uses of the channel. Subsequently, the quan-
tities lim infn→∞1/n · log2 Mn and lim supn→∞
1/n · log2 Mn, which are referred
to as pessimistic and optimistic capacity, respectively, are of interest. The map-
ping n 7→ Mn is fully characterized by a sequence of optimal codes (with respect
to a fixed error constraint) for the channel under consideration, since each op-
timal code determines the maximal number of messages transmittable given a
certain number of channel uses. Assuming finite input and output alphabets,
the set of possible codes is finite for each number of channel uses n ∈ N. Thus,
the construction of suitable codes is a purely combinatorial task, and a great
deal of research has been conducted in this area. For a comprehensive intro-
duction to combinatorial methods in information theory, see [11]. Surprisingly,
the mapping n 7→ Mn does not have to be a recursive function. This observa-
tion, which we will discuss further in Remark 25, yields an interesting relation
between information theory, combinatorics and the theory of computability.
In general, optimistic and pessimistic capacity are not equal. However,
some channel models exhibit a mathematical structure known as achiev-
ability and converse. Essentially, this structure is characterized by a pair(
(~wn)n∈N, ( ~wn)n∈N
)
of monotonic sequences that satisfy
infn∈N
~wn ≤ lim infn→∞
1
nlog2 Mn ≤ lim sup
n→∞
1
nlog2 Mn ≤ sup
n∈N
~wn (3)
as well as
supn∈N
~wn ≤ infn∈N
~wn, (4)
3
in which case the quantity
lim infn→∞
1
nlog2 Mn = lim sup
n→∞
1
nlog2 Mn = lim
n→∞
1
nlog2 Mn (5)
is referred to as the capacity of the channel (the sequence ( ~wn)n∈N will be
referred to as "the converse" in this context). This case is particulary desirable
with respect to computability, as we will discuss in Section 3. For a concise
discussion on the approach of achievability and converse in information theory,
we refer to [12].
A second possibility of proving the existence of a well-defined capacity is
by the application of Fekete’s lemma, in which case the equality of pessimistic
and optimistic capacity follows without achievability and converse. However,
the applicability of Fekete’s Lemma highly depends on the individual channel
model. In particular, it is necessary to prove that the sequence (1/n·log2 Mn)n∈N
satisfies superadditivity. The arguably most prominent example is Shannon’s
zero-error capacity, where the number Mn equals the independence number of
the nth tensor-power of a specific simple graph:
Theorem 2 (Shannon [13]). Denote by C0(W ) the zero-error capacity of the
discrete, memoryless channel W , Θ(G) the Shannon capacity of the confusabil-
ity graph G corresponding to W , and α(G⊠n) the independence number of the
n-fold strong graph product of G with itself. Then, we have
2C0(W ) = Θ(G) := lim infn→∞
α(G⊠n)1
n = supn∈N
α(G⊠n)1
n = limn→∞
α(G⊠n)1
n . (6)
For a number of other channel models, cf. e.g. [14, 15, 16], the application of
Fekete’s lemma leads to an infinite-letter description of the respective capacity.
To the authors’ knowledge, it remains an open question whether single-letter
descriptions for the capacities in [14, 15, 16] exist. Given a fixed, computable
channel, the descriptions in [14, 15, 16] provide a computable sequence (1/n ·
log2 Mn)n∈N of computable numbers that meet the requirements of Fekete’s
lemma. Up to now, however, it remains unknown if the respective capacities
always attain computable values. In other words, it is not yet known whether
4
these capacites can always be algorithmically computed, e.g. as a floating point
number, up to any specified precision. As a corollary of our results, we obtain
that in contrast to the achievability and converse approach, Fekete’s lemma is
structurally insufficient to provide a computable capacity.
Since Turing published his work on the theory of computation [17, 18], it
has been known that almost all real numbers are uncomputable. In an attempt
to characterize different degrees of (un)computability, Zheng and Weihrauch [1]
introduced the arithmetical hierarchy of real numbers, which is strongly related
to the Kleene–Mostowski hierarchy of subsets of natural numbers [19, 20]. In
order for a real number a ∈ R to be computable, it has to be on the lowest
Zheng-Weihrauch hierarchical level. The hierarchical level of a real number
solely depends on the logical structure that is used to define the number. With
respect to channel capacities, this logical structure is given by the corresponding
channel model. In information theory, or, in the broadest sense, any mathemat-
ical theory that investigates asymptotic quantities, a comprehensive character-
ization of the hierarchical levels induced by the logical structures involved may
be desirable for several reasons. Above all, it is necessary in order to be able
to decide which of the asymptotic quantities can be numerically approximated
in a feasible manner for practical purposes. Our work aims to contribute to
this characterization in view of Fekete’s lemma. In particular, we show that a
whole class of uncomputable problems can be represented in terms of sub- and
superadditive sequences of real numbers.
The outline of the remainder of our work is as follows. Section 2 yields an
overview on the fundamentals of computability theory, which will be applied
subsequently. In Section 3, we will give a brief introduction on the sets Σ1 ⊂ R
and Π1 ⊂ R, which form the lowest Zheng-Weihrauch hierarchical level, and
present the first part of our results: Theorems 13 and 15 address the problem
of determining the modulus of convergence for monotonic sequences, as they
occur in the information-theoretic achievability and converse approach. In Sec-
tion 4, we place Fekete’s lemma in the context of the arithmetical hierarchy of
real numbers. In particular, we prove in Theorem 21 an effective equivalence
5
between Σ1(Π1) and the set of real numbers that can be charcaterized by a
super(sub)additive sequence through Fekete’s lemma. As an immediate conse-
quence (Corollary 22), we obtain that convergence related to Fekete’s lemma is
generally ineffective. Several implications of this issue are discussed in Remark
25 and Remark 26. In Section 5, we characterize under which circumstances the
convergence related to Fekete’s lemma is effective. These results are presented
in Theorem 28. Theorem 31, furthermore, addresses the problem of determin-
ing a suitable corresponding modulus of convergence. The Section concludes
with Theorem 32, an analogue to Theorem 13 in the context of Fekete’s lemma.
Section 6 discusses a subtle difference in the representation of computable se-
quences. This difference turns out to have significant implications, as is high-
lighted by Theorem 33. The paper closes in Section 7 with a discussion on the
consequences of our findings for modern-day approaches in combinatorial coding
and information theory.
2. Preliminaries from the Theory of Computation
In order to investigate Fekete’s lemma in view of computability, we apply
the theory of Turing machines [17, 18] and recursive functions [21]. For brevity,
we restrict ourselves to an informal description. A comprehensive formal intro-
duction on the topic may be found in [22, 23, 24, 25].
Turing machines are a mathematical model of what we intuitively understand
as computation machines. In this sense, they yield an abstract idealization of
today’s real-world computers. Any algortithm that can be executed by a real-
world computer can, in theory, be simulated by a Turing machine, and vice
versa. In contrast to real-world computers, however, Turing machines are not
subject to any restrictions regarding energy consumption, computation time or
memory size. Furthermore, the computation of a Turing machine is assumed to
be executed free of any errors.
Recursive functions, more specifically referred to as µ-recursive functions,
form a special subset of the set⋃∞
n=0
{
f : Nn →֒ N}
, where we use the symbol
6
"→֒" to denote a partial mapping. The set of recursive functions characterizes
the notion of computability through a different approach. Turing machines and
recursive functions are equivalent in the following sense: a function f : Nn →֒ N
is computable by a Turing machine if and only if it is a recursive function [26].
In the following, we will introduce some definitions from computable analysis
[23, 24, 25], which we will apply subsequently.
Definition 3. A sequence of rational numbers (rn)n∈N is said to be computable
if there exist recursive functions fsi, fnu, fde : N → N such that
rn = (−1)fsi(n)fnu(n)
fde(n)(7)
holds true for all n ∈ N. Likewise, a double sequence of rational num-
bers (rn,m)n,m∈N is said to be computable if there exist recursive functions
fsi, fnu, fde : N× N → N such that
rn,m = (−1)fsi(n,m) fnu(n,m)
fde(n,m)(8)
holds true for all n,m ∈ N.
Definition 4. A sequence (xn)n∈N of real numbers is said to converge effectively
towards a number x∗ ∈ R if there exists a recursive function κ : N → N such
that |x∗ − xn| < 1/2N holds true for all n,N ∈ N that satisfy n ≥ κ(N).
The function κ is referred to as (recursive) modulus of convergence for the
sequence (xn)n∈N.
Definition 5. A real number x is said to be computable if there exists a com-
putable sequence of rational numbers that converges effectively towards x.
We denote the set of computable real numbers by Rc. Given a computable
number x, a pair(
(rn)n∈N, κ)
consisting of a computable sequence (rn)n∈N of
rational numbers that satisfies limn→∞ rn = x and a corresponding recursive
modulus of convergence κ is called a standard description of the number x. We
denote by Rc the set of standard descriptions of computable numbers.
7
Remark 6. The set Rc is a subfield of the set R. The field properties of Rc are
effective in the following sense: there exist Turing machines TM+ : Rc ×Rc →
Rc and TM× : Rc × Rc → Rc such that if rx ∈ Rc and ry ∈ Rc are standard
descriptions of x ∈ Rc and y ∈ Rc, then TM+(rx, ry) and TM×(rx, ry) are
standard descriptions of x + y and x · y, respectively. Furthermore, the set Rc
is countable and closed with respect to effective convergence.
Definition 7. A sequence (xn)n∈N of computable numbers is called computable
if there exists a computable double sequence (rn,m)n,m∈N of rational numbers as
well as a recursive function κ : N× N → N such that
|xn − rn,m| <1
2M(9)
holds true for all n,m,M ∈ N that satisfy m ≥ κ(n,M).
The pair(
(rn,m)n,m∈N, κ)
is referred to as a standard description of the
sequence (xn)n∈N.
An essential component in proving uncomputability of some kind is the no-
tion of recursive and recursively enumerable sets, which we will introduce in the
following.
Definition 8. A set A ⊆ N is said to be recursively enumerable if there exists
a recursive function f : N →֒ N with domain equal to A.
Definition 9. A set A ⊆ N is said to be recursive if the corresponding indicator
function 1A : N → {0, 1} is a recursive function.
Remark 10. A set A ⊆ N is recursive if and only if both A and Ac := N\A are
recursively enumerable sets. The halting problem for Turing machines ensures
the existence of recursively enumerable sets that are nonrecursive.
3. Characterizing Computable Numbers by Monotonic Sequences
Throughout this section, we address the description of computable numbers
in terms of monotonic sequences, which structurally corresponds to a description
in the sense of achievability and converse.
8
A real number x∗ is called upper semi-computable if there exist a computable
sequence ( ~rn)n∈N of rational numbers that converges towards x∗ from above, i.e.,
( ~rn)n∈N satisfies ~rn ≥ ~rn+1 for all n ∈ N as well as limn→∞ ~rn = x∗.
Likewise, a real number x∗ is called lower semi-computable if there exists
a computable sequence (~rn)n∈N of rational numbers that converges towards x∗
from below, i.e, (~rn)n∈N satisfies ~rn ≤ ~rn+1 for all n ∈ N as well as limn→∞ ~rn =
x∗.
In the above context, the arrow symbols indicate the "direction" (on the real
line) towards which the sequences (~rn)n∈N and ( ~rn)n∈N converge.
Definition 11. We denote by Π1 the set of upper semi-computable real numbers.
Likewise, we denote by Σ1 the set of lower semi-computable real numbers.
We have Rc = Σ1 ∩ Π1 as well as Rc \ Σ1 6= ∅ and Rc \Π1 6= ∅ [1]. In other
words, a real number x∗ is computable if and only if it is both upper and lower
semi-computable, and there exist real numbers which satisfy only one of the two
conditions.
The representation of computable numbers through monotonic sequences
of rational numbers extends to computable sequences of computable numbers.
That is, a real number x∗ is computable if and only if there exists a computable
sequence (~wn)n∈N of computable numbers that converges monotonically non-
decreasingly towards x∗ and a computable sequence ( ~wn)n∈N of computable
numbers that converges monotonically non-increasingly towards x∗. Subse-
quently, we will work towards a strengthening of this result. In particular,
we want to prove that both (~wn)n∈N and ( ~wn)n∈N converge effectively towards
x∗ (convergence towards a computable number is not a sufficient condition for
effective convergence, cf. [27]) and present an effective construction of the re-
spective moduli of convergence.
9
Lemma 12. Denote by WQ the set of pairs(
(~rn)n∈N, ( ~rn)n∈N
)
of computable
sequences of rational numbers that satisfy the following:
1. ~rn ≤ ~rn+1 for all n ∈ N;
2. ~rn ≥ ~rn+1 for all n ∈ N;
3. there exists x∗ ∈ R with limn→∞ ~rn = limn→∞ ~rn = x∗.
There exists a Turing machine TM : WQ × N → N such that
|x∗ − ~rn| <1
2M∧ |x∗ − ~rn| <
1
2M(10)
holds true for all n ∈ N that satisfy n ≥ TM(
(~rn)n∈N, ( ~rn)n∈N,M)
.
Proof. For M ∈ N arbitrary, let n0 ∈ N be the smallest number that satisfies
|~rn − ~rn| < 1/2M . That is, we define
n0 := min
{
n ∈ N : |~rn − ~rn| <1
2M
}
. (11)
The sequence ( ~rn − ~rn)n∈N is a computable sequence of rational numbers
that converges monotonically non-increasingly to zero. Thus, the mapping(
(~rn)n∈N, ( ~rn)n∈N,M)
7→ n0 is recursive and there exists a Turing machine
TM : WQ × N → N that computes the number n0 in dependence of (~rn)n∈N,
( ~rn)n∈N and M . For all n ∈ N that satisfy n ≥ n0, we have
|~rn − ~rn| = ~rn − ~rn ≤ |~rn0− ~rn0
| <1
2M(12)
as well as ~rn ≤ x∗ ≤ ~rn. Consequently,
|x∗ − ~rn| <1
2M∧ |x∗ − ~rn| <
1
2M(13)
holds true for all n ∈ N that satisfy n ≥ n0. Thus, the Turing machine TM
satisfies the required properties. �
In the following, we establish the description of computable numbers in terms
of computable, monotonic sequences of computable numbers.
A computable sequence (~wn)n∈N of computable numbers is referred to as
lower monotonic representation of the real number x∗ if it satisfies ~wn ≤ ~wn+1
for all n ∈ N as well as limn→∞ ~wn = x∗.
10
Likewise, a computable sequence ( ~wn)n∈N of computable numbers is referred
to as an upper monotonic representation of the real number x∗ if it satisfies
~wn ≥ ~wn+1 for all n ∈ N as well as limn→∞ ~wn = x∗.
A pair(
(~wn)n∈N, ( ~wn)n∈N
)
is referred to as a (complete) monotonic repre-
sentation of the real number x∗ if (~wn)n∈N is a lower monotonic representation
of x∗ and ( ~wn)n∈N is an upper monotonic representation of x∗. Furthermore,
we denote by W the set of standard descriptions
w =(
(~rn,m)n,m∈N, ~κ, ( ~rn,m)n,m∈N, ~κ)
(14)
of (complete) monotonic representations of real numbers.
Theorem 13. There exists a Turing machine TM : W × N → N that satisfies
the following:
• If(
(~wn)n∈N, ( ~wn)n∈N
)
is a monotonic representation of x∗ ∈ R and w is
a standard description of(
(~wn)n∈N, ( ~wn)n∈N
)
, then
|x∗ − ~wn| <1
2M∧ |x∗ − ~wn| <
1
2M(15)
holds true for all n ∈ N that satisfy n ≥ TM(w,M).
Proof. Consider the standard description w =(
(~rn,m)n,m∈N, ~κ, ( ~rn,m)n,m∈N, ~κ)
of(
(~wn)n∈N, ( ~wn)n∈N
)
and observe that for all n, k, l ∈ N that satisfy k, l ≥ n,
we have
~rn,~κ(n,n) −1
2n< ~wn ≤ ~wk ≤ x∗ ≤ ~wl ≤ ~wn < ~rn, ~κ(n,n) +
1
2n. (16)
Define the sequences ( ′~rn)n∈N and ( ′~rn)n∈N through setting
′~rn : = max
{
~rm,~κ(m,m) −1
2m: 1 ≤ m ≤ n
}
, (17)
′~rn : = min
{
~rm, ~κ(m,m) +1
2m: 1 ≤ m ≤ n
}
(18)
for all n ∈ N. Then, for all n, k, l ∈ N that satisfy k, l ≥ n, we have
∣
∣x∗ − ~wk
∣
∣ <∣
∣x∗ −′~rn∣
∣ ∧∣
∣x∗ − ~wl
∣
∣ <∣
∣x∗ −′~rn∣
∣. (19)
11
Both ( ′~rn)n∈N and ( ′~rn)n∈N are computable sequences of rational numbers and
the mapping w 7→(
( ′~rn)n∈N, (′~rn)n∈N
)
is recursive. Furthermore, the pair(
( ′~rn)n∈N, (′~rn)n∈N
)
satisfies the requirements of Lemma 12 with limn→∞′~rn =
limn→∞′~rn = x∗. Based on (19), the theorem then follows by the application of
Lemma 12. �
Remark 14. Recall the standard description r =(
(rn)n∈N, κ)
∈ Rc of a com-
putable number x∗. If we can find two standard descriptions ~r =(
(~rn)n∈N, ~κ)
∈
Rc and ~r =(
( ~rn)n∈N, ~κ)
∈ Rc of x∗ that satisfy
1. ~rn ≤ ~rn+1 for all n ∈ N,
2. ~rn ≥ ~rn+1 for all n ∈ N,
we know by Lemma 12 that the pair(
(~rn)n∈N, ( ~rn)n∈N
)
is sufficient for a Turing
machine to determine a suitable modulus of convergence, i.e., the corresponding
pair (~κ, ~κ) is obsolete in respect thereof. In this sense Theorem 13 is not a di-
rect generalization of Lemma 12, since the respective Turing machine receives
a quadruple w =(
( ′~rn,m)n,m∈N,′~κ , ( ′~rn,m)n,m∈N,
′~κ)
∈ W as part of its input. In
particular, the pair ( ′~κ , ′~κ ) is essential in the proof of Theorem 13. This sig-
nificant difference emerges from the mathematical strength of the representation
of x∗ in terms of computable sequences of rational numbers. If the assumptions
made in Lemma 12 are weakened even slightly, the corresponding proof becomes
invalid. This phenomenon will be addressed further in Section 6.
Consider a family of pairs (( m~rn )n∈N, (m~rn )n∈N)m∈N that satisfy the require-
ments of Lemma 12 and each of which has their own limit value xm∗ . In infor-
mation theory, this may correspond to a specific channel model that features a
model parameter m ∈ N. From a practical point of view, the statement of the
existence of a modulus of convergence for each pair (( m~rn )n∈N, (m~rn )n∈N)m∈N is
only usable because this modulus of convergence exhibits a construction algo-
rithm that is uniformly recursive in (( m~rn )n∈N, (m~rn )n∈N)m∈N (in a certain sense,
we want the mapping m 7→ xm∗ to be Turing computable). This highlights the
necessity of the converse in the sense of the information-theoretic achievability
12
and converse approach. Following our previous considerations, it may not be
surprising that without the converse, it is generally not possible to determine
the modulus of convergence of a monotonically non-decreasing sequence in a
recursive way.
Theorem 15. There exists a family ( m~rn )n,m∈N of computable sequences of ra-
tional numbers that simultaneously satisfies the following:
• The family ( m~rn )n,m∈N is recursive, i.e, there exists a computable double
sequence ( ′~rm,n)n,m∈N of rational numbers that satisfies ′~rm,n = m~rn for all
m,n ∈ N.
• For all m ∈ N, the sequence ( m~rn )n∈N is non-negative as well as monotoni-
cally non-decreasing in n and there exists xm∗ ∈ Rc such that limn→∞
m~rn =
xm∗ holds true.
• There does not exist a Turing machine TM : N × N → N such that
|xm∗ − m~rn | < 1/2M holds true for all n,m,M ∈ N that satisfy n ≥
TM(m,M).
Proof. We prove the statement by contradiction. Let A ⊂ N be a recursively
enumerable, non-recursive set and suppose TM does exist. Since A is recursively
enumerable, there exists a recursive bijection fA : N → A. For all n ∈ N, define
An : ={
fA(j) : 1 ≤ j ≤ n}
(20)
m~rn : =
1 m ∈ An
0 otherwise. (21)
Then for all n,m ∈ N, we have An ⊂ An+1 and thus 0 ≤ m~rn ≤ m~rn+1. Fur-
thermore, ( m~rn )n,m∈N is a recursive family of computable sequences of rational
numbers that satisfies
limn→∞
m~rn = 1A(m) (22)
for all m ∈ N. Set 1A(m) =: xm∗ for all m ∈ N. By construction, we have
∣
∣xm∗ − m~rn
∣
∣ =∣
∣1A(m)− m~rn∣
∣ ∈ {0, 1} (23)
13
for all n,m ∈ N. On the other hand, by definition of TM , we have
∣
∣
∣xm∗ − m~rTM(m,1)
∣
∣
∣=
∣
∣
∣1A(m)− m~rTM(m,1)
∣
∣
∣<
1
2(24)
for all n,m ∈ N. Thus, for all m ∈ N, we obtain
1A(m) =
1 if m~rTM(m,1) = 1
0 otherwise. (25)
Consequently, 1A(m) is a recursive function, which contradicts the assumption
of A being non-recursive. �
In Section 5 we will discuss whether Fekete’s lemma, in analogy to Theorem
13, allows for the recursive construction of a suitable modulus of convergence
for the sequence under consideration. Theorem 15 will be essential in providing
a negative answer to this question.
4. Fekete’s Lemma in the Context of the Arithmetical Hierarchy of
Real Numbers
As indicated in Section 1, Fekete’s lemma may prevent the general necessity
of the information-theoretic converse in a certain sense: given a superadditive
sequence (~an)n∈N of real numbers, it provides the existence of the limit value
limn→∞ ~an ·n−1. On the other hand, as we have seen in Section 3, the existence
of a suitable converse does not only supply the existence of the limit value, but
also yields a recursive construction for the modulus of convergence. As we will
see in the following, this does not hold true for Fekete’s lemma. In fact, the
corresponding limit value may not even be a computable number. On a meta-
mathematical level, this result implies that there cannot exist a constructive
proof for Fekete’s lemma, since, by definition, that would allow for an effective
construction of the associated limit value.
Throughout this section, we place Fekete’s lemma in the context of the
arithmetical hierarchy of real numbers. That is, we define the sets Lsub and
Lsup, which consist of real numbers that can be characterized by sub- and
14
superadditive sequences and show their equivalence to the sets Π1 and Σ1.
The general ineffectiveness of convergence related to Fekete’s lemma then follows
as a corollary.
The following lemma is a basic result by Zheng and Weihrauch, characteriz-
ing the sets Σ1 and Π1 through the suprema and infima of computable sequences
of rational numbers.
Lemma 16 (Zheng and Weihrauch [1]).
1. A real number x∗ satisfies x∗ ∈ Σ1 if and only if there exists a computable
sequence (~rn)n∈N of rational numbers such that supn∈N ~rn = x∗ holds true.
2. A real number x∗ satisfies x∗ ∈ Π1 if and only if there exists a computable
sequence ( ~rn)n∈N of rational numbers such that infn∈N ~rn = x∗ holds true.
Note that we do not require the sequences (~rn)n∈N and ( ~rn)n∈N to be mono-
tonic in this context.
Subsequently, we will make use of a generalized version of Lemma 16: the
established characterization of Σ1 and Π1 extends to the suprema and infima
of computable sequences of computable numbers, which we will prove in the
following.
Lemma 17.
1. A real number x∗ satisfies x∗ ∈ Σ1 if and only if there exists a computable
sequence (~zn)n∈N of computable numbers such that supn∈N ~zn = x∗ holds
true.
2. A real number x∗ satisfies x∗ ∈ Π1 if and only if there exists a computable
sequence ( ~zn)n∈N of computable numbers such that infn∈N ~zn = x∗ holds
true.
15
Proof.
1. As every computable sequence of rational numbers is also a computable se-
quence of computable numbers, the implication immediately follows from
Lemma 16.
For the converse, consider a computable sequence (~zn)n∈N of com-
putable numbers that satisfies supn∈N ~zn = x∗ and a standard description(
(~rn,m)n,m∈N, ~κ)
thereof. Furthermore, consider the inverse of the Cantor
pairing function n 7→ (π1(n), π2(n)) ∈ N×N. The Cantor pairing function
π : N×N → N is a total and bijective recursive function, as is its inverse.
Define the computable sequence ( ′~rn)n∈N of rational numbers by setting
′~rn := ~rπ1(n),~κ(π1(n),π2(n)) −1
2π2(n)(26)
for all n ∈ N. Then, we have
′~rn ≤ ~zπ1(n) ≤ supm∈N
~zm = x∗ (27)
for all n ∈ N, i.e., supn∈N′~rn ≤ x∗. It remains to show that supn∈N
′~rn = x∗
holds true. Let 0 < ǫ < 1 be arbitrary. Since we have supn∈N ~zn = x∗ as
well as
limm→∞
(
~rn,~κ(n,m) −1
2m
)
= νn (28)
for all n ∈ N, there exist l, k ∈ N such that
x∗ − ǫ ≤ ~rl,~κ(l,k) −1
2k≤ x∗ (29)
holds true. Furthermore, there exists n ∈ N such that (π1(n), π2(n)) =
(l, k) is satisfied. Therefore, there exists n ∈ N such that
x∗ − ǫ ≤ ′~rn ≤ x∗ (30)
holds true. Since ǫ was chosen arbitrarily, we have supn∈N′~rn = x∗. Hence,
x∗ ∈ Σ1 follows by Lemma 16.
16
2. The proof of the second claim follows along the same line of reasoning as
the proof of the first claim. Thus, we restrict ourselves to a brief summary
of the steps.
Again, as every computable sequence of rational numbers is also a com-
putable sequence of computable numbers, the implication immediately
follows from Lemma 16.
For the converse, consider a standard description(
( ~rn,m)n,m∈N, ~κ)
of a
suitable computable sequence of computable numbers and define
′~rn := ~rπ1(n), ~κ(π1(n),π2(n)) +1
2π2(n)(31)
for all n ∈ N. The computable sequence ( ′~rn)n∈N of rational numbers then
satisfies infn∈N′~rn = x∗. Hence, x∗ ∈ Π1 follows by Lemma 16.
�
As indicated in Section 1, the arithmetical hierarchy by Zheng and Weihrauch
closely relates to the Kleene–Mostowski hierachy of sets of natural numbers.
In view of the denomination "Kleene–Mostowski hierachy," we will indentify
the characterization of real numbers by means of the suprema and infima
of computable sequences of computable numbers by the term "(n-th order)
Zheng-Weihrauch representation."
We refer to a bounded, computable sequence (~zn)n∈N of computable numbers
as a lower first-order Zheng-Weihrauch (ZW) representation of the real number
x∗ = supn∈N ~zn. The set of standard descriptions ~z =(
(~rn,m)n,m∈N, ~κ)
of lower
first-order ZW representations of real numbers is denoted by ~Z.
Likewise, we refer to a bounded, computable sequence ( ~zn)n∈N of com-
putable numbers as an upper first-order Zheng-Weihrauch (ZW) representa-
tion of the real number x∗ = infn∈N ~zn. The set of standard descriptions
~z =(
( ~rn,m)n,m∈N, ~κ)
of lower first-order ZW representations of real numbers
is denoted by ~Z.
Ultimately, our goal is to introduce a characterization of (semi-)computable
numbers through super- and subadditive sequences, which subsequently allows
17
us to relate Fekete’s lemma to the arithmetical hierarchy of real numbers. Thus,
in analogy to Π1 and Σ1, define the sets Lsup ⊆ R and Lsub ⊆ R as follows:
Lsub :={
x∗ ∈ R : There exists a subadditive, computable
sequence ( ~an)n∈N of computable numbers
that satisfies limn→∞
~an · n−1 = x∗.}
, (32)
Lsup :={
x∗ ∈ R : There exists a superadditive, computable
sequence (~an)n∈N of computable numbers
that satisfies limn→∞
~an · n−1 = x∗.}
. (33)
Similar to the notion of first-order ZW representations, we refer to a su-
peradditive, computable sequence (~an)n∈N of computable numbers that satisfies
limn→∞ ~an ·n−1 = x∗ for some real number x∗ as a superadditive representation
of x∗. The set of standard descriptions ~a =(
(~rn,m)n,m∈N, ~κ)
of superadditive
representations of real numbers is denoted by ~A.
Likewise, we refer to a subadditive, computable sequence ( ~an)n∈N of com-
putable numbers that satisfies limn→∞ ~an · n−1 = x∗ for some real number
x∗ as a subadditive representation of x∗. The set of standard descriptions
~a =(
( ~rn,m)n,m∈N, ~κ)
of subadditive representations of real numbers is denoted
by ~A.
Given the definition of the sets Lsup and Lsub as well as Lemma 17, we can
immediately prove the following relation to the sets Π1 and Σ1:
Lemma 18. For all real numbers x∗, the following holds true:
1. If there exists a superadditive representation for x∗, then there exists a
lower first-order ZW representation for x∗. Consequently, we have Lsup ⊆
Σ1.
2. If there exists a subadditive representation for x∗, then there exists an
upper first-order ZW representation for x∗. Consequently, we have Lsub ⊆
Π1.
18
Proof.
1. Consider a superadditive representation (~an)n∈N for x∗ and set ~zn :=
~an · n−1 for all n ∈ N. Following Lemma 1, we conclude that
x∗ = limn→∞
~an · n−1 = supn∈N
~zn (34)
holds true. Thus, (~zn)n∈N is a first-order ZW representation of x∗, and,
by Lemma 17, we have x∗ ∈ Σ1.
2. Consider a subadditive representation ( ~an)n∈N for x∗ and set ~zn := ~an ·n−1
for all n ∈ N. Following Lemma 1, we conclude that
x∗ = limn→∞
~an · n−1 = infn∈N
~zn (35)
holds true. Thus, ( ~zn)n∈N is a first-order ZW representation of x∗, and,
by Lemma 17, we have x∗ ∈ Π1.
�
Observe that the proof of Lemma 18 is constructive. Hence, we can transform
a super(sub)additive representation of a number x∗ ∈ R effectively into a lower
(upper) first-order ZW representation of x∗. As we will prove in the following,
the converse is true as well.
Lemma 19. There exists a Turing machine TM : ~Z → ~A that satisfies the
following:
• If the input of TM is the standard description of a lower first-order ZW
representation of a number x∗, then the output of TM is the standard
description of a superadditive representation of x∗.
Consequently, we have Σ1 ⊆ Lsup.
Proof. Denote again by n 7→ (π1(n), π2(n)) ∈ N × N the inverse of the Cantor
pairing function. Let(
(~rn,m)n,m∈N, ~κ)
be the standard description of a lower
first-order ZW representation (~zn)n∈N of x∗. We define
′~rn := max
{
~rπ1(m),~κ(π1(m),π2(m)) −1
2π2(m): 1 ≤ m ≤ n
}
(36)
19
for all n ∈ N. Following the line of reasoning presented in the proof of Lemma
17, we conclude that ( ′~rn)n∈N is a monotonically non-decreasing, computable se-
quence of rational numbers that satisfies limn→∞′~rn = supn∈N
′~rn = supn∈N ~zn =
x∗ and that the mapping(
(~rn,m)n,m∈N, ~κ)
7→ ( ′~rn)n∈N is recursive. Observe that
the sequence (n · ′~rn)n∈N satisfies superadditivity, since
(n+m) · ′~rn+m = n · ′~rn+m +m · ′~rn+m ≥ n · ′~rn +m · ′~rm (37)
holds true for all n,m ∈ N, following the monotonicity of ( ′~rn)n∈N.
Now consider the computable double sequence ( ′′~rn,m)n,m∈N of rational num-
bers defined via
′′~rn,m := n · ′~rn (38)
for all n,m ∈ N, as well as the recursive function ′~κ : N → N, M 7→ 1. Clearly,
( ′′~rn,m)n,m∈N satisfies limm→∞′′~rn,m = n · ′~rn for all n ∈ N, as well as
∣
∣n · ′~rn − ′′~rn,m∣
∣ = 0 <1
2=
1
2′~κ (M)
(39)
for all n,m,M ∈ N that satisfy m ≥ ′~κ(M). Thus, the pair(
( ′′~rn,m)n,m∈N,′~κ)
is the standard description of a superadditive representation of the number x∗
and the mapping(
(~rn,m)n,m∈N, ~κ)
7→(
( ′′~rn,m)n,m∈N,′~κ)
is recursive. �
Lemma 20. There exists a Turing machine TM : ~Z → ~A that satisfies the
following:
• If the input of TM is the standard description of an upper first-order ZW
representation of a number x∗, then the output of TM is the standard
description of a subadditive representation of x∗.
Consequently, we have Π1 ⊆ Lsub.
Proof. The proof of the claim follows along the same line of reasoning as the
proof of Lemma 19. Thus, we will restrict ourselves to a brief summary of the
steps.
20
Again, consider a standard description(
( ~rn,m)n,m∈N, ~κ)
of an upper first-
order ZW representation of the number x∗ ∈ R. For all n,m ∈ N, we define
′′~rn,m := n ·min
{
~rπ1(l), ~κ(π1(l),π2(l)) +1
2π2(l): 1 ≤ l ≤ n
}
. (40)
Furthermore, set ′~κ : N → N, M 7→ 1. Then the mapping(
( ~rn,m)n,m∈N, ~κ)
7→(
( ′′~rn,m)n,m∈N,′~κ)
is recursive and the pair(
( ′′~rn,m)n,m∈N,′~κ)
is a standard de-
scription of a subadditive representation of x∗. �
We conclude that we are likewise able to transform any lower (upper) first-
order ZW representation effectively into a super(sub)additive representation of
the same number, leading up to the main result of this section:
Theorem 21. For the sets Σ1, Lsup, Π1, Lsub and Rc, the following equalities
hold true:
1. Σ1 = Lsup, 2. Π1 = Lsub, 3. Rc = Lsup ∩ Lsub.
Proof. Equality 1 is the logical conjunction of Lemma 18 and Lemma 19, while
equality 2 is the logical conjunction of Lemma 18 and Lemma 20. As indicated
in the beginning of Section 3, we have Rc = Π1 ∩ Σ1 [1], which subsequently
implies equality 3. �
Corollary 22. There exists a superadditive, computable sequence (~an)n∈N
of computable numbers as well as a real number x∗ ∈ R \ Rc such that
limn→∞ ~an · n−1 = c∗ holds true.
Proof. As indicated in the beginning of Section 3, we have Σ1 \Rc 6= ∅ [1]. The
assertion then follows from Theorem 21 and the definition of the set Lsup. �
Remark 23. According to Specker [28], we can find a non-negative, mono-
tonically non-decreasing, computable sequence (~rn)n∈N of rational numbers such
that the corresponding limit value x∗ = limn→∞ ~rn is not a computable number.
The sequence ( ′~rn)n∈N, defined via ′~rn := n · ~rn for all n ∈ N then yields an
alternative, direct proof of Corollary 22.
21
Remark 24. As indicated by Remark 6, the field properties of the set Rc are
effective with respect to the standard description computable numbers. Likewise,
the sets Σ1 and Π1 satisfy effective field properties with respect to first-order ZW
representations [1]. As pointed out previously, Lemmata 19 and 20 as well as
the proof of Lemma 18 imply that a lower (upper) first-order ZW representation
can be transformed effectively into a super(sub)additive representation of the
same number, and vice versa. Thus, we conclude that the field properties of
Σ1 and Π1, Lsup and Lsub, respectively, are effective with respect to super- and
subadditive representations as well.
Remark 25. Corollary 22 highlights the boundaries of the state-of-the-art meth-
ods in information theory. For the number x∗, there cannot exist a monoton-
ically non-increasing, computable sequence of computable numbers with limit
value x∗. This is due to the fact that x∗ is the limit value of a monotonically
non-decreasing computable sequence of rational numbers on the one hand, but
an uncomputable number on the other. Thus, the approach of characterizing x∗
by deriving a suitable converse (in the sense of information theory) is unfeasible
in this case. Surprisingly, this phenomenon is not exclusive to the information
theoretic converse, as was shown in [29]:
Denote by CHc({0, 1}, {0, 1}) the set of computable stochastic 2×2 matrices.
There exists a compound channel V∗ := {Vs}s∈N with Vs ∈ CHc({0, 1}, {0, 1})
for all s ∈ N, such that the mapping K 7→ VK := {Vs}s∈{1,2,...,K} is recur-
sive and the capacity C(V∗) exists, but is not a computable number. On the
other hand, the capacity C(VK) exists and is computable for all K ∈ N, since
C(VK) only contains a finite number of components. It was furthermore shown
that limK→∞ C(VK) = C(V∗) holds true, with (C(VK))K∈N being a monotoni-
cally non-increasing sequence. Thus, there cannot exist a computable sequence
of computable numbers that converges towards C(V∗) from below. Again, de-
note by M(n) the maximum number of messages transmittable by n uses of the
channel. As indicated in the introduction, the existence of an optimal code for
each blocklength n is straightforward, since the number of feasible codes is fi-
22
nite. Furthermore, for the channel V∗, we have 1/n · log2 M(n) ≤ C(V∗) for all
n ∈ N (with respect to an error constraint of 0 < ǫ < 1/2) and, by the existence
of C(V∗), limn→∞1/n · log2 M(n) = C(V∗). Thus, the mapping n 7→ M(n) is
nonrecursive in this context. Consequently, the common information theoretic
approach of finding optimal codes for an increasing number of channel uses n is
unfeasible for the channel V∗.
Remark 26. As indicated in the Section 1, Fekete’s lemma provides an infinite-
letter description for the capacities associated with a number of channel models
[14, 15, 16]. In his Shannon lecture in 2008, Rudolf Ahlswede pointed out that
a lot of research concerning single-letter descriptions of channel capacities has
been conducted during the past 60 years. For many capacities of practical inter-
est, however, it remains unclear whether a single-letter description does exist.
In view of this problem, Ahlswede asked whether it would instead be possible to
prove the effective convergence of multi-letter formulas towards the correspond-
ing capacity. According to Definition 4, it would then be possible to compute
the numerical value of the capacity up to any desired accuracy. For many prac-
tical applications, this would be sufficient for the near future. As implicated
by Corollary 22, Ahlswede’s question cannot be answered positively on the ba-
sis of Fekete’s lemma, since the corresponding limit value does not necessarily
have to be a computable number. Furthermore, we can observe that the non-
computability of the limit value occurs for the de Bruijn-Erdös condition as well
([3, 4]).
5. An Effective Version of Fekete’s Lemma
In this section, we prove an effective version of Fekete’s lemma for superad-
ditive sequences that satisfy non-negativity.
Given a computable sequence (rn)n∈N of rational numbers that converges
towards a real number x∗ with recursive modulus of convergence κ : N → N,
we immediately have x∗ ∈ Rc, due to the definition of computable numbers.
As indicated before, the converse is not true in general. Given a computable
23
sequence (rn)n∈N of rational numbers that converges towards a computable
number x∗, there may not exist a recursive modulus of convergence. A general
description of this phenomenon may be found in [27]. In the following, we will
show that the equivalence holds true for non-negative sequences in the context of
Fekete’s lemma. The majority of sequences considered in discrete mathematics
and information theory do, in fact, satisfy non-negativity. For didactic reasons,
we first establish the equivalence for computable sequences of rational numbers
and present the equivalence for computable sequences of computable numbers
subsequently.
Lemma 27. Let (~rn)n∈N be a superadditive, computable sequence of rational
numbers that satisfies 0 ≤ ~rn for all n ∈ N. Furthermore, let x∗ be a real
number such that limn→∞ ~rn · n−1 = x∗ holds true. The sequence (~rn · n−1)n∈N
converges effectively towards x∗ if and only if x∗ is a computable number.
Proof. If the sequence (~rn ·n−1)n∈N converges effectively towards x∗, then x∗ is
computable, as is clear from the definition of computable numbers.
For the converse, consider a monotonically non-increasing sequence ( ~rn)n∈N
of rational numbers that satisfies limn→∞ ~rn = x∗. The existence of such a
sequence is ensured by x∗ being a computable number. Furthermore, by Lemma
1, we have ~rn · n−1 ≤ x∗ for all n ∈ N. For M ∈ N, we define
n0 := min
{
n ∈ N :~rnn
<1
2M+1∧ ~rn −
1
2M+1<
~rnn
}
. (41)
Since both (~rn · n−1)n∈N and ( ~rn)n∈N converge to the same number, we know
that n0 must exist. Furthermore, both conditions in (41) can be decided al-
gorithmically, since they exclusively involve comparisions on rational numbers.
Thus, given the sequences (~rn)n∈N and ( ~rn)n∈N, the mapping M 7→ n0 is recur-
sive. For all n ∈ N that satisfy n ≥ n0, there exist two unique numbers q, s ∈ N
24
with 0 ≤ s ≤ n0 − 1, such that n = qn0 + s holds true. We have
~rqn0+s ≥ ~rqn0+ ~rs
≥ ~rn0+ ~r(q−1)n0
+ ~rs
≥ ~rn0+ ~rn0
+ ~r(q−2)n0+ ~rs
≥...
= q~rn0+ ~rs. (42)
Thus, q~rn0+ ~rs ≤ ~rqn0+s = ~rn is satisfied, and consequently,
~rnn
≥q~rn0
n+
~rsn
=qn0
n
~rn0
n0+
~rsn
>qn0
n~rn0
−qn0
n
1
2M+1+
~rsn
>qn0
n~rn0
−1
2M+1(43)
holds true. Now consider n ∈ N satisfying n ≥ n20. Since any n ∈ N that
satisfies n ≥ n20 certainly satisfies n ≥ n0 as well, there again exist unique
numbers q, s ∈ N with 0 ≤ s ≤ n0 − 1, such that n = qn0 + s holds true. Thus,
we have
0 ≤ ~rn −~rnn
≤ ~rn0−
~rnn
≤ ~rn0−
qn0
n~rn0
+1
2M+1
≤
(
1−qn0
n
)
~rn0+
1
2M+1
≤~rn0
n0+
1
2M+1(44)
<1
2M+1+
1
2M+1
=1
2M, (45)
where (44) holds true because
1−qn0
n=
n− qn0
n≤
n0
n≤
n0
n20
=1
n0(46)
25
is satisfied. Define the mapping κ : N → N,M 7→ n20, which, given the sequences
(~rn)n∈N and ( ~rn)n∈N, is a recursive function. Then, we have∣
∣
∣
∣
x∗ −~rnn
∣
∣
∣
∣
= x∗ −~rnn
≤ ~rn −~rnn
≤1
2M(47)
for all n,M ∈ N that satisfy n ≥ κ(M). Thus, κ yields the required modulus of
convergence for the sequence (~rn · n−1)n∈N. �
Theorem 28. Let (~an)n∈N be a superadditive representation of the number
x∗ ∈ R that satisfies 0 ≤ ~an for all n ∈ N. The sequence (~an ·n−1)n∈N converges
effectively towards x∗ if and only if x∗ is a computable number.
Proof. In the following, let(
(~rn,m)n,m∈N, ~κ)
be a standard description of the
sequence (~an)n∈N.
If (~an)n∈N converges effectively towards x∗ with recursive modulus of con-
vergence κ : N → N, it is straightforward to construct a computable sequence of
rational numbers from(
(~rn,m)n,m∈N, ~κ)
and κ that converges effectively towards
x∗. Then, x∗ is a computable number.
For the converse, we will show the existence of a non-negative, superadditive,
computable sequence ( ′′~rn)n∈N of rational numbers that satisfies ′′~rn ≤ ~an for all
n ∈ N as well as limn→∞′′~rn ·n−1 = x∗. The theorem then follows by application
of Lemma 27. Define the sequence ( ′~rn)n∈N via
′~rn := max
{
0, ~rn,~κ(n,n) −1
2n
}
(48)
for all n ∈ N. Then, ( ′~rn)n∈N is a computable sequence of rational numbers that
satisfies ′~rn ≤ ~an for all n ∈ N as well as limn→∞′~rn · n−1 = x∗. We now define
the sequence ( ′′~rn )n∈N in an inductive manner. Set ′′~r1 := ′~r1, and for all n ∈ N
that satisfy n ≥ 2,
′′~rn := max{
′~rn, max{
′′~rl + ′′~rk : l, k ∈ N, l + k = n}
}
. (49)
Then, ( ′′~rn )n∈N is a non-negative, superadditive, computable sequence of rational
numbers that satisfies ′~rn ≤ ′′~rn for all n ∈ N. Assume ′′~rn = ′~rn for some n ∈ N.
26
Then, ′′~rn ≤ ~an holds true. Assume now ′′~rn = ′′~rl +′′~rk for some l, k, n ∈ N that
satisfy n = l + k. Then, by induction,
′′~rn = ′′~rl +′′~rk ≤ ~al + ~ak ≤ ~al+k = ~an (50)
holds true. Thus, the sequence ( ′′~rn)n∈N satisfies ′~rn ≤ ′′~rn ≤ ~an for all n ∈ N.
Therefore, we have ′~rn · n−1 ≤ ′′~rn · n−1 ≤ ~an · n−1 ≤ x∗ for all n ∈ N, as
well as limn→∞′′~rn · n−1 = x∗. Any modulus of convergence for the sequence
( ′′~rn · n−1)n∈N is thus a modulus of convergence for the sequence (~an · n−1)n∈N
as well. The theorem then follows from Lemma 27. �
Remark 29. Observe that apart from the necessity of finding a suitable sequence
( ~rn)n∈N, both the proof of Lemma 27 and the proof of Theorem 28 are construc-
tive. Thus, given a suitable pair(
( ~rn)n∈N, (~an)n∈N
)
, we can find a modulus of
convergence κ for the sequence (~an · n−1)n∈N by means of a Turing machine.
Remark 30. In the above sense, we have a positive answer to Ahlswede’s ques-
tion (cf. Remark 26) for computable capacities that exhibit a non-negative,
superadditive representation.
We have discussed the necessity of algorithmic constructions for moduli of
convergence to a great extent in the previous part of our work. While Lemma
27 and Theorem 28 prove the existence of a recursive modulus of convergence
in the context of the effective version of Fekete’s lemma, they do so in a non-
constructive way. In the following, we will apply Theorem 15 to show that
even the effective version of Fekete’s lemma is not strong enough to allow for a
recursive construction of the associated modulus of convergence.
27
Theorem 31. There exists a family ( m~rn )n,m∈N of computable sequences of ra-
tional numbers that simultaneously satisfies the following:
• The family ( m~rn )n,m∈N is recursive, i.e, there exists a computable double
sequence ( ′~rm,n)n,m∈N of rational numbers that satisfies ′~rm,n = m~rn for all
m,n ∈ N.
• For all m ∈ N, the sequence ( m~rn )n∈N satisfies non-negativity as well as
superadditivity in n, and there exists xm∗ ∈ Rc such that limn→∞
m~rn ·n−1 =
xm∗ holds true.
• There does not exist a Turing machine TM : N × N → N such that
|xm∗ − m~rn · n−1| < 1/2M holds true for all n,m,M ∈ N that satisfy n ≥
TM(m,M).
Proof. We prove the statement by contradiction. Let ( ′′~rn,m)n,m∈N be com-
putable double sequences of rational numbers that satisfy the requirements of
Theorem 15, and suppose TM does exist. For all n,m ∈ N, define
m~rn := n · ′′~rn,m. (51)
Then, ( m~rn )n,m∈N is a recursive family of computable sequences of rational num-
bers that satisfies non-negativity as well as superadditivity in n for all m ∈ N.
Furthermore, for all m ∈ N, there exists xm∗ ∈ Rc such that
limn→∞
m~rn · n−1 = limn→∞
′′~rn,m = xm∗ (52)
holds true. By assumption, the Turing machine TM satisfies
∣
∣xm∗ − m~rn · n−1
∣
∣ =∣
∣xm∗ − ′′~rn,m
∣
∣ < 2−M (53)
for all n,m,M ∈ N that satisfy n ≥ TM(m,M), thereby contradicting Theorem
15. �
In Section 3, we have introduced the notion of monotonic representations for
computable numbers. By Theorem 13, we know that given a monotonic repre-
sentation(
(~wn)n∈N, ( ~wn)n∈N
)
for some number x∗ ∈ R, we can algorithmically
28
deduce moduli of convergence for the sequences (~wn)n∈N and ( ~wn)n∈N. On the
other hand, we know by Theorem 21 that a number x∗ ∈ R is computable if
and only if it exibits both a super- and a subadditive representation. In view of
this result, we conclude the section by proving a claim similar to Theorem 13
for non-negative super- and subadditive representations.
In the following, we refer to a pair(
(~an)n∈N, ( ~an)n∈N
)
as a P/B-additive
representation of x∗ ∈ R if (~an)n∈N is a superadditive representation of x∗ and
( ~an)n∈N is a subadditive representation of x∗. Furthermore, we denote by A the
set of standard descriptions
a =(
(~rn,m)n,m∈N, ~κ, ( ~rn,m)n,m∈N, ~κ)
(54)
of P/B-additive representations of real numbers.
Theorem 32. There exists a Turing machine TM : A × N → N that satisfies
the following:
• If(
(~an)n∈N, ( ~an)n∈N
)
is a P/B-additive representation of x∗ ∈ R that
satisfies 0 ≤ ~an for all n ∈ N and a is a standard description of(
(~an)n∈N, ( ~an)n∈N
)
, then
∣
∣x∗ − ~an · n−1∣
∣ <1
2M(55)
holds true for all n ∈ N that satisfy n ≥ TM(a,M).
Proof. As Remark 29 points out, both the proof of Lemma 27 and the proof of
Theorem 28 are constructive, apart from the necessity of finding a monotonically
non-increasing, computable sequence that converges towards x∗ from above. We
will thus restrict ourselves to a brief summary of the necessary steps.
Consider the standard description a =(
(~rn,m)n,m∈N, ~κ, ( ~rn,m)n,m∈N, ~κ)
of(
(~an)n∈N, ( ~an)n∈N
)
and define
′~rn := min
{
~rm, ~κ(m,m) +1
2m: 1 ≤ m ≤ n
}
(56)
for all n ∈ N. The sequence ( ′~rn)n∈N is a monotonically non-increasing, com-
putable sequence of rational numbers that satisfies limn→∞′~rn = x∗ and the
29
mapping(
( ~rn,m)n,m∈N, ~κ)
7→ ( ′~rn)n∈N is recursive. Furthermore, define
′′~rn : = max
{
0, ~rn,~κ(n,n) −1
2n
}
, (57)
′~rn : =
′′~r1 if n = 1,
max({
′~rl +′~rk : l + k = n
}
∨{
′′~rn})
otherwise,(58)
κ(M) : =
(
min
{
n ∈ N :′~rnn
<1
2M+1∧ ′~rn −
1
2M+1<
′~rnn
})2
, (59)
for all n,M ∈ N. Setting TM(a,M) := κ(M) yields the required Turing ma-
chine. �
6. Computable Sequences of Rational Numbers versus Computable
Sequences of Computable Numbers
In this section, we want to prove an additional result concerning the rep-
resentation of number sequences. In particular, we will consider computable
sequences (xn)n∈N of computable numbers that additionally satisfy xn ∈ Q for
all n ∈ N. At first sight, the difference to a representation of (xn)n∈N in terms
of Definition 3 may not be apparent. However (xn)n∈N being a computable se-
quence of rational numbers is a strictly stronger assertion that (xn)n∈N being a
computable sequence of computable numbers that additionally satisfies xn ∈ Q
for all n ∈ N, as we will see in the following.
Theorem 33. There exist non-negative, computable sequences (~wn)n∈N and
(~an)n∈N of computable numbers that satisfy the following:
1. The sequence (~wn)n∈N is a lower monotonic representation of a rational
number and satisfies ~wn ∈ Q for all n ∈ N, but is not a computable
sequence of rational numbers.
2. The sequence (~an)n∈N is a superadditive representation of a rational num-
ber and satisfies ~an ∈ Q for all n ∈ N, but is not a computable sequence
of rational numbers.
30
Proof.
1. We prove the statement by contradiction. Suppose all sequences (~wn)n∈N
that satisfy all other requirements of Theorem 33 (Statement 1) are com-
putable sequences of rational numbers. Let A ⊂ N be a recursively enu-
merable, non-recursive set and consider a Turing machine TM that accepts
A, i.e. the domain of the corresponding recursive function fTM : N →֒ N
equals A. In other words, the Turing machine TM halts for input n ∈ N
if and only if n ∈ A is satisfied. The recursive enumerability of A ensures
the existence of such a Turing machine. For all n,m ∈ N, define
l(n,m) :=
m0 if, for input n ∈ N, TM halts
after m0 < m steps,
m otherwise.
(60)
Furthermore, for all n,m ∈ N, define
rm,n :=
l(n,m)∑
k=0
1
2k= 2−
1
2l(n,m). (61)
The sequence (rm,n)m,n∈N is a computable double sequence of rational
numbers that is monotonically non-decreasing in m for all n ∈ N. Assume
n ∈ A holds true for some n ∈ N. Then, there exists m0 ∈ N such that
for all m ∈ N that satisfy m0 ≤ m, we have l(n,m) = m0. Thus, if n ∈ N
holds true, we have
limm→∞
rm,n = 2−1
2m0
∈ Q. (62)
On the other hand, assume n /∈ A holds true for some n ∈ N. Then, for
all m ∈ N, we have l(n,m) = m. Thus, if n /∈ N holds true, we have
limm→∞
rm,n = limm→∞
(
2−1
2m
)
= 2 ∈ Q. (63)
Define the sequence (xn)n∈N via xn := limm→∞ rm,n for all n ∈ N. Then,
(xn)n∈N is a sequence of numbers in Q. By construction, we furthermore
have |xn − rm,n| < 1/2M for all n,m,M that satisfy m ≥ M + 1. Thus,
31
setting
κ : N → N,M 7→ M + 1, (64)
we conclude that the pair(
(rm,n)m,n∈N, κ)
is a standard description of
the computable sequence (xn)n∈N of computable numbers. For all n ∈ N,
define
~wn := 1−1
n+
1
2n(n+ 1)xn. (65)
Then, (~wn)n∈N is a non-negative, computable sequence of computable
numbers attaining values in Q. We have 1 ≤ xn ≤ 2 and thus
∣
∣1− ~wn
∣
∣ =
∣
∣
∣
∣
1
n−
1
2(n(n+ 1))xn
∣
∣
∣
∣
<1
n(66)
for all n ∈ N, i.e., the sequence (~wn)n∈N converges effectively towards 1
for all n ∈ N. Furthermore, we have
~wn+1 − ~wn =1
n−
1
n+ 1+
1
2(n+ 1)(n+ 2)xn+1
−1
2n(n+ 1)xn
>1
n(n+ 1)−
1
2n(n+ 1)xn
≥1
n(n+ 1)−
1
n(n+ 1)= 0, (67)
for all n ∈ N, i.e., the sequence (~wn)n∈N is monotonically non-decreasing in
n. In summary, (~wn)n∈N is a non-negative, monotonically non-decreasing,
computable sequence on computable numbers that satisfies ~wn ∈ Q for all
n ∈ N and converges effectively towards 1. By assumption, (~wn)n∈N is also
a computable sequence of rational numbers. As (~wn)n∈N is furthermore
non-negative, there exist recursive functions fnu, fde : N → N such that
~wn = fnu(n)/fde(n) is satisfied for all n ∈ N. Since comparisons on rational
numbers are recursive operations, there exists a Turing machine TM ′ :
N → N that satisfies
TM ′(n) =
1 if fnu(n)fde(n)
< 1− 1n+1
0 otherwise(68)
32
for all n ∈ N. We have TM ′(n) = 1 if and only if xn < 2. Consequently,
TM ′ = 1A holds true, which contradicts the assumption of A being non-
recursive.
2. Consider a sequence (~wn)n∈N that satisfies the requirements of Theorem
33 (Statement 1). For all n ∈ N, define ~an := n · ~wn. Then, the sequence
(~an)n∈N satisfies the requirements of Theorem 33 (Statement 2).
�
7. Summary and Discussion
By proving the equalities Σ1 = Lsup and Π1 = Lsub, we have established a
full characterization of Fekete’s lemma in view of the arithmetical hierarchy of
real numbers and shown that a whole class of uncomputable problems can be
represented by super- and subadditive sequences. As a corollary, we have seen
that for a computable sequence (~an)n∈N of computable numbers, superadditivity
is not a sufficient condition for the sequence (~an ·n−1)n∈N to converge effectively.
If the conditions (limn→∞ ~an · n−1) ∈ Rc and 0 ≤ ~an for all n ∈ N are
satisfied additionally, then (~an ·n−1)n∈N does converge effectively. Furthermore,
if a suitable "converse" ( ~an)n∈N is available, a modulus of convergence for (~an ·
n−1)n∈N can be determined algorithmically.
On the other hand, given a non-negative, computable sequence (xn)n∈N of
computable numbers that converges towards a computable number, superaddi-
tivity of the sequence (n·xn)n∈N is a sufficient condition for effective convergence
of (xn)n∈N. As pointed out in the beginning of Section 5, convergence towards a
computable number is not a sufficient condition for effective convergence in gen-
eral. Whether the superadditivity-requirement on (n ·xn)n∈N to ensure effective
convergence of (xn)n∈N can be further weakened is a pressing open question.
On a different note, our results imply that no constructive proof for Fekete’s
lemma can exist, and furthermore, no algorithm can exist that generally allows
for the effective computation of the associated limit values. As already men-
tioned in Section 1, other methods of information theory have recently been
33
investigated regarding their computability properties, and it has been shown
that they are non-constructive. In particular, the associated problems cannot
be solved algorithmically. Despite the fact that this behavior occurs in multiple
cases, it does not seem to be sufficiently understood. It is present in Shannon’s
fundamental publication in information theory, where he proves the existence
of capacity-achieving codes, but is unable to construct them effectively [30]. It
was only recently shown in [31] that an effective construction of capacity achiev-
ing codes as a function of the channel is not possible. Even for several results
that are significantly less complex in their structure, an analysis shows that the
corresponding proofs are non-constructive. That is, the mathematical object of
interest is proven to exist, but no algorithm to explicitly generate it is devised.
The approach by Blahut and Arimoto for the numerical approximation of the
capacity of discrete memoryless channels yields a particularly simple example
[32, 33]. The computation of the capacity as a number is straightforward, due
to the properties of the mutual information. Blahut and Arimoto describe a
procedure for the characterization of a corresponding optimizer, but the proof of
their method is non-constructive [33, 34, 35, 36] (in [36], a related problem has
been considered). Thus, their approach cannot be used to devise an algorithm
which effectively generates the optimizer. For interesting weakening of the su-
peradditivity requirement, reference is made to the work of Erdös [3, 4] and the
current work [6] by Füredi and Rushka.
Acknowledgments
Holger Boche was supported by the Deutsche Forschungsgemeinschaft (DFG,
German Research Foundation) under Germany’s Excellence Strategy - EXC
2092 CASA - 390781972. Christian Deppe was supported by the Bundesminis-
terium für Bildung und Forschung (BMBF) through grant 16KIS1005.
34
References
References
[1] X. Zheng, K. Weihrauch, The Arithmetical Hierarchy of Real Numbers,
Mathematical Logic Quarterly 47 (1) (2001) 51–65.
[2] M. Fekete, Über die Verteilung der Wurzeln bei gewissen algebraischen
Gleichungen mit ganzzahligen Koeffizienten, Mathematische Zeitschrift 17
(1923) 228–249.
[3] N. de Bruijn, P. Erdös, Some linear and some quadratic recursion for-
mulas. I, Proceedings of the Section of Sciences of the Koninklijke Ned-
erlandse Akademie van Wetenschappen. Series A, Mathematical Sciences
54 (5) (1951) 374–382.
[4] N. de Bruijn, P. Erdös, Some linear and some quadratic recursion formulas.
II, Proceedings of the Koninklijke Nederlandse Akademie van Wetenschap-
pen: Series A: Mathematical Sciences 14 (1952) 152–163.
[5] B. Bollobás, O. Riordan, Percolation, Cambridge University Press, 2006.
[6] Z. Füredi, I. Z. Ruzsa, Nearly subadditive sequences, Acta Mathematica
Hungarica 161 (2020) 401–411.
[7] J. Steele, Probability Theory and Combinatorial Optimization, SIAM,
1997.
[8] S. Capobianco, Multidimensional cellular automata and generalization of
Fekete’s lemma, Discrete Mathematics & Theoretical Computer Science 10.
[9] T. Ceccherini-Silberstein, F. Krieger, M. Coornaert, An ana-
logue of Fekete’s lemma for subadditive functions on cancella-
tive amenable semigroups, Journal d’Analyse Mathématique 124,
doi:10.1007/s11854-014-0027-4.
35
[10] K. Roth, Sur quelques ensembles d’entiers, C. R. Acad. Sci., Paris 234
(1952) 388–390.
[11] I. Csiszár, J. Körner, Information Theory: Coding Theorems for
Discrete Memoryless Systems, Cambridge University Press, 2nd edn.,
doi:10.1017/CBO9780511921889, 2011.
[12] R. Ahlswede, On Concepts of Performance Parameters for Chan-
nels, vol. 4123 of Lecture Notes in Computer Science, chap. V,
Springer Berlin Heidelberg, Berlin, Heidelberg, 1 edn., ISBN
978-3-540-46245-3, 639–663, doi:10.1007/11889342_40, URL
https://doi.org/10.1007/11889342_40, 2006.
[13] C. Shannon, The zero error capacity of a noisy channel, IRE Transactions
on Information Theory 2 (3) (1956) 8–19.
[14] R. Ahlswede, I. Bjelakovic, H. Boche, J. Nötzel, Quantum Capacity Un-
der Adversarial Quantum Noise: Arbitrarily Varying Quantum Channels,
Communications in Mathematical Physics 317 (1) (2013) 103–156.
[15] I. Bjelakovic, H. Boche, J. Sommerfeld, Secrecy Results for Compound
Wiretap Channels, Problems of Information Transmission 49 (1) (2013)
73–98.
[16] M. Wiese, J. Nötzel, H. Boche, A Channel Under Simultaneous Jamming
and Eavesdropping Attack–Correlated Random Coding Capacities Under
Strong Secrecy Criteria, IEEE Transactions on Information Theory 62 (7)
(2016) 3844–3862.
[17] A. Turing, On Computable Numbers, with an Application to the Entschei-
dungsproblem, Proceedings of the London Mathematical Society 42 (1)
(1936) 230–265.
[18] A. M. Turing, On computable numbers, with an application to the Entschei-
dungsproblem. A correction, Proceedings of the London Mathematical So-
ciety s2-43 (1) (1937) 544–546.
36
[19] S. C. Kleene, Recursive Predicates and Quantifiers, Transactions of the
American Mathematical Society 53 (1) (1943) 41–73, ISSN 00029947, URL
http://www.jstor.org/stable/1990131.
[20] A. Mostowski, On definable sets of positive integers, Fundamenta Mathe-
maticae 34 (1) (1947) 81–112, URL http://eudml.org/doc/213118.
[21] S. C. Kleene, General Recursive Functions of Natural Numbers, Mathe-
matische Annalen 112 (1) (1936) 727–742, doi:10.1007/BF01565439, URL
https://doi.org/10.1007/BF01565439.
[22] C. P. Schnorr, Rekursive Funktionen und ihre Komplexität,
Vieweg+Teubner Verlag, 1st edn., doi:10.1007/978-3-322-94701-7,
1974.
[23] K. Weihrauch, Computable Analysis: An Introduction, Springer Publishing
Company, Incorporated, 1st edn., 2013.
[24] R. I. Soare, Recursively Enumerable Sets and Degrees, Springer-
Verlag, Berlin, Heidelberg, 1st edn., ISBN 0387152997,
doi:10.1007/978-3-322-94701-7, 1987.
[25] M. B. Pour-El, J. I. Richards, Computability in Analysis and
Physics, Perspectives in Logic, Cambridge University Press,
doi:10.1017/9781316717325, 2017.
[26] A. M. Turing, Computability and λ-definability, Journal of Symbolic Logic
2 (4) (1937) 153–163, doi:10.2307/2268280.
[27] V. V. V’yugin, Effective Convergence in Probability and an Ergodic The-
orem for Individual Random Sequences, Theory of Probability & Its
Applications 42 (1) (1998) 39–50, doi:10.1137/S0040585X97975915, URL
https://doi.org/10.1137/S0040585X97975915.
[28] E. Specker, Nicht Konstruktiv Beweisbare Sätze der Analysis, Journal of
Symbolic Logic 15 (1) (1950) 67–68.
37
[29] H. Boche, R. F. Schaefer, H. V. Poor, Communication Under Chan-
nel Uncertainty: An Algorithmic Perspective and Effective Construc-
tion, IEEE Transactions on Signal Processing 68 (2020) 6224–6239,
doi:10.1109/TSP.2020.3027902.
[30] C. E. Shannon, A mathematical theory of communication,
The Bell System Technical Journal 27 (3) (1948) 379–423,
doi:10.1002/j.1538-7305.1948.tb01338.x.
[31] H. Boche, R. Schaefer, V. Poor, Turing Meets Shannon: Algorithmic Con-
structability of Capacity-Achieving Codes, IEEE International Conference
on Communications (ICC ’21) .
[32] R. Blahut, Computation of channel capacity and rate-distortion func-
tions, IEEE Transactions on Information Theory 18 (4) (1972) 460–473,
doi:10.1109/TIT.1972.1054855.
[33] S. Arimoto, An algorithm for computing the capacity of arbitrary discrete
memoryless channels, IEEE Transactions on Information Theory 18 (1)
(1972) 14–20, doi:10.1109/TIT.1972.1054753.
[34] I. Csiszar, On the computation of rate-distortion functions (Corresp.),
IEEE Transactions on Information Theory 20 (1) (1974) 122–124,
doi:10.1109/TIT.1974.1055146.
[35] I. Csiszar, G. Tusnady, Information geometry and alternating minimization
procedures, Statistics and Decisions 1 (1984) 205–237.
[36] T. Cover, An algorithm for maximizing expected log investment re-
turn, IEEE Transactions on Information Theory 30 (2) (1984) 369–373,
doi:10.1109/TIT.1984.1056869.
38
