確率統計 特論 (Probability and Statistics)
May 13, 2020
来嶋 秀治 (Shuji Kijima)
システム情報科学研究院 情報学部門
Dept. Informatics, ISEE
Todays topics
• what is probability?
• probability space
確率統計 特論(Probability & Statistics)
共通基礎科目
Preview
About this class
3
Evaluation
• exercises (レポート課題) =: 𝑧 (0 ≤ 𝑧 ≤ 10); optional
Almost every class (10回程度)
Submit from moodle before the final exam.
• midterm (中間試験): (mid) June, =:𝑥 (0 ≤ 𝑥 ≤ 50)
• final (期末試験): late July or early Aug., =:𝑦 (0 ≤ 𝑦 ≤ 50)
• score =𝑥 + 𝑦 (+𝑧).
• passing mark = 60. No makeup exam (追試は無い)
supplemental option for credit
not mandatory, but recommend.
4
Lecture
• Use moodle
• begin with S4B + slides, (teams, whiteboard, etc.)
• If you have any troubles, contact with chat/e-mail.
• contact: [email protected]
• Language: speak in Japanese, write in English.
• mathematical backgrounds (undergraduate level)
e.g., calculus (微積分), linear algebra (線形代数), etc…
• Back-up page
http://tcs.inf.kyushu-u.ac.jp/~kijima/
you can find exercise, slides, keywords, and schedule.
• Office hour: Wed(水曜日) 15:00—16:30
5
References
• 藤澤洋徳, 確率と統計, 現代基礎数学 13, 朝倉書店, 2006.
• 中田寿夫,内藤貫太,確率・統計,学術図書出版社,2017.
• M. Mitzenmacher, E. Upfel,
Probability and Computing: Randomized Algorithms and Probabilistic Analysis,
Cambridge University Press, 2005.
(邦訳) 小柴健史, 河内亮周, 確率と計算 --乱択アルゴリズムと確率的解析--,
共立出版, 2009.
[確率論]
• 伏見正則, 確率と確率過程, 朝倉書店, 2002.
[統計学]
• 日本統計学会, 統計学 (統計検定1級対応), 東京図書, 2013.
• 東大教養学部統計学教室編, 統計学入門, 東京大学出版会, 1991.
Check the web page!
http://www.tcs.inf.kyushu-u.ac.jp/~kijima/
Requirement (前提知識)6
Analysis (解析学)
• Differentiation (微分)
Ex. 1. d
d𝑥log 𝑥 = ?
• Integration by parts (部分積分)
Ex. 2. 0
3𝑥 exp(−𝑥) d𝑥 =?
Linear algebra (線形代数学)
• Bases (基底)
• Eigenvalues/diagonalization (固有空間/対角化)
What is Probability?
Examples
Ex 1. Monty Hall problem --- ask Marilyn8
You are given the choice of three doors:
Behind on door is a car; behind the others goats.
You pick a door, say “A”.
The host (Monty), who knows what's behind the doors,
opens another door, say “C”, which he knows has a goat.
He then says to you, “Do you want to pick door “B”?”
Question
Is it to your advantage to switch your choice?
図: wikipedia”モンティーホール問題”より
Explain in next time!
Hint: A modified Monty Hall problem --- ask Marilyn9
You are given the choice of 26 doors:
Behind on door is a car; behind the others goats.
You pick a door, say “A”.
The host (Monty), who knows what's behind the doors,
opens all other doors, say “C” to “Z”, except for “B”,
each of which he knows has a goat.
He then says to you, "Do you want to pick door “B”?”
Question
Is it to your advantage to switch your choice?
図: wikipedia”モンティーホール問題”より
Explain in next time!
Ex 2. Bertrand paradox10
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle?
Ex 2. Bertrand paradox11
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle?
13
Without loss of generality,
assume that the radius of the circle is 1.
Then, the side length of equatorial
triangle in a circle (内接正三角形) is …
Answer 1: angle method.
Consider the “angle 𝜃” between the chord and the tangent line.
Note 0 ≤ 𝜃 ≤ 𝜋
The chord is longer than 3
iff𝜋
3≤ 𝜃 ≤
2𝜋
3
the probability is 1
3.
Ex 2. Bertrand paradox12
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle (=:x)?
Ex 2. Bertrand paradox13
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle (=:x)?
Answer 2: level method.
Consider the “height ℎ” of the chord from the horizontal line.
Note 0 ≤ ℎ ≤ 2.
The chord is longer than 3
iff1
2≤ ℎ ≤
3
2
the probability is 𝟏
𝟐.
ℎ
Ex 2. Bertrand paradox14
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle (=:x)?
Answer 3: distance from the center.
Consider the “distance 𝑑” from the center of the circle to the chord.
Note 0 ≤ 𝑑 ≤ 1.
The chord is longer than 3
Iff 0 ≤ 𝑑 ≤1
2(the chord touches the yellow circle)
the probability is 𝟏
𝟒.
(prob. that the midpoint of the chord is in the yellow circle)
Ex 2. Bertrand paradox15
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
Question
What is the probability that the chord is
longer than a side of the triangle (=:x)?
What does
“a chord of the circle is chosen at random”
mean?
Probability Space
Axiom and theorems
17
Example: Probability Space
Ex. 1. A fair coin
Tossing a coin.
Ω = ℎ, 𝑡
𝐹 = 2 ℎ,𝑡 = ∅, ℎ , 𝑡 , ℎ, 𝑡
𝑃 ∅ = 0, 𝑃 ℎ =1
2, 𝑃 𝑡 =
1
2, 𝑃 ℎ, 𝑡 = 1
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.
18
Example: Probability Space
Ex. 2. A die
Ω = {1,2,3,4,5,6},
ℱ = 2Ω = 1 , 2 , … , 1,3,5 , … , 1,2,3,4,5,6 , ∅
𝑃 𝐴 =𝐴
6for any 𝐴 ⊆ Ω.
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.
19
Definition: Probability Space
𝑃 is a probability measure (Kolmogorov axioms)
1. 𝑃 𝐴 ≥ 0 for any 𝐴 ∈ ℱ.
2. 𝑃 Ω = 1.
3. Any countable sequence of mutual exclusive events 𝐴1, 𝐴2, …
satisfies that 𝑃 𝐴1 ∪ 𝐴2 ∪⋯ = 𝑃 𝐴1 + 𝑃 𝐴2 +⋯.
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.
20
Theorem: Property of Probability (1/2)
Kolmogolov’s axiom (P1), (P2), (P3) implies the following.
Thm. 1
Let Ω,ℱ, 𝑃 be an arbitrary probability space.
(1) 𝑃 ∅ = 0.
(2) If 𝐴 ⊆ 𝐵 (𝐴, 𝐵 ∈ ℱ), then 𝑃 𝐵 ∖ 𝐴 = 𝑃 𝐵 − 𝑃 𝐴 .
(rem. 𝐵 ∖ 𝐴 = 𝐵 ∩ ഥ𝐴 )
(3) If 𝐴 ⊆ 𝐵 (𝐴, 𝐵 ∈ ℱ), then 𝑃 𝐴 ≤ 𝑃 𝐵 .
(4) 𝑃 𝐴 ≤ 1 for any 𝐴 ∈ ℱ.
(5) 𝑃 ഥ𝐴 = 1 − 𝑃(𝐴) for any 𝐴 ∈ ℱ.
21
Theorem: Property of Probability (2/2)
Thm. 1. (cont.)
(6) P 𝑃 𝐴 ∪ B = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 ∩ 𝐵 for any 𝐴 ∈ ℱ.
(7) If 𝐴1, 𝐴2, … ∈ ℱ satisfy 𝐴1 ⊆ 𝐴2 ⊆ ⋯,
then 𝑃 𝑖𝐴𝑖ڂ = lim𝑛→∞
𝑃 𝐴𝑛
(8) If 𝐴1, 𝐴2, … ∈ ℱ satisfy 𝐴1 ⊇ 𝐴2 ⊇ ⋯,
then 𝑃 𝑖𝐴𝑖ځ = lim𝑛→∞
𝑃 𝐴𝑛
(9) For any 𝐴1, 𝐴2, … ∈ ℱ,
𝑃 𝑖𝐴𝑖ڂ = 0 ⇔ ∀𝑖 ≥ 0, 𝑃 𝐴𝑖 = 0
Kolmogolov’s axiom (P1), (P2), (P3) implies the following.
Proof of (1): 𝑃 ∅ = 0. 22
(1) 𝑃 ∅ = 0.
Proof:
Proof of (1): 𝑃 ∅ = 0. 23
(1) 𝑃 ∅ = 0.
Proof: 𝑃 Ω = 1 (∗) by Axiom (2).
𝑃 Ω = 𝑃 Ω ∪ ∅ (𝑏𝑦 𝑠𝑒𝑡 𝑡ℎ𝑒𝑜𝑟𝑦)
= 𝑃 Ω + 𝑃 ∅ 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 3
= 1 + 𝑃 ∅ ∗∗ 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 2 .
Thus, 1 = 1 + 𝑃 ∅ ∗∗∗ 𝑏𝑦 ∗ 𝑎𝑛𝑑 ∗∗ .
Finally, 𝑃 ∅ = 0 𝑏𝑦 ∗∗∗ 𝑎𝑛𝑑 𝑎𝑟𝑖𝑡ℎ𝑚𝑒𝑡𝑖𝑐 𝑜𝑓 𝑟𝑒𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 .
Every “reason” must be found in
Axiom (and set theory, arithmetic of reals)
and propositions already proved.
Let’s prove (4): ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.24
Firstly, we prove (3).
(3) If 𝐴 ⊆ 𝐵 then 𝑃 𝐴 ≤ 𝑃 𝐵 .
Proof:
Every “reason” must be found in
Axiom (and set theory, arithmetic of reals)
and propositions already proved.
Let’s prove (4): ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.25
Firstly, we prove (3).
(3) If 𝐴 ⊆ 𝐵 then 𝑃 𝐴 ≤ 𝑃 𝐵 .
Proof: 𝑃 𝐵 = 𝑃 𝐵 ∖ 𝐴 ∪ 𝐴 = 𝑃 𝐵 ∖ 𝐴 + 𝑃 𝐴 (*) by Axiom (3).
𝑃 𝐵 ∖ 𝐴 ≥ 0 (**) by Axiom(1).
𝑃 𝐵 ≥ 𝑃 𝐴 by (*), (**) and arithmetic of reals.
(4) ∀𝐴 ∈ ℱ, 𝑃 𝐴 ≤ 1.
Proof. 𝐴 ⊆ Ω since 𝐴 ∈ ℱ. Thus,
𝑃 𝐴 ≤ 𝑃 Ω 𝑏𝑦 3 𝑜𝑓 𝐿𝑒𝑚𝑚𝑎
= 1 𝑏𝑦 𝐴𝑥𝑖𝑜𝑚 2 .
Every “reason” must be found in
Axiom (and set theory, arithmetic of reals)
and propositions already proved.
Appx: Integration by parts (部分積分)26
න𝑎
𝑏
𝑥 exp(−𝑥) d𝑥 = −𝑥 exp −𝑥 𝑎𝑏 −න
𝑎
𝑏
− exp −𝑥 d𝑥
= ൫−𝑏 exp −𝑏 + 𝑎 exp −𝑎
Thm.
𝑓 𝑥 𝑔 𝑥 d𝑥 = 𝑓 𝑥 𝐺 𝑥 − 𝑓′ 𝑥 𝐺 𝑥 𝑑𝑥
27
Appx: -algebra
ℱ is a -algebra
1. ℱ contains the empty set: ∅ ∈ ℱ
2. ℱ is closed under complements: 𝐴 ∈ ℱ ഥ𝐴 ∈ ℱ
3. ℱ is closed under countable unions: 𝐴𝑖 ∈ ℱ 𝑖𝐴𝑖ڂ ∈ ℱ
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.
28
Appx: Probability Space
Ex. 2. tossing 2 fair coins
Tossing 2 coins, which look alike. We can observe only
how many heads or tails are.
Ω = ℎ, ℎ , ℎ, 𝑡 , 𝑡, ℎ , 𝑡, 𝑡 (why?)
𝐹 = 2 0,1,2 (why?)
𝑃 ∅ = 0, 𝑃 0 = 1/4 , 𝑃 1 = 2/4, 𝑃 {0,1} = 3/4, …
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.
29
Appx: Probability Space
Ex. 2. tossing 3 fair coins
Tossing 3 coins, which look alike. We can observe only
how many heads or tails are.
Ω =ℎ, ℎ, ℎ , ℎ, ℎ, 𝑡 , ℎ, 𝑡, ℎ , ℎ, 𝑡, 𝑡 ,𝑡, ℎ, ℎ , 𝑡, ℎ, 𝑡 , 𝑡, 𝑡, ℎ , (𝑡, 𝑡, 𝑡)
(why?)
𝐹 = 2 0,1,2,3 (why?)
𝑃 ∅ = 0, 𝑃 0 = 1/8 , 𝑃 1 = 3/8, 𝑃 #ℎ is odd = 1/2, …
A probability space is defined by (Ω, ℱ, 𝑃)
Ω: sample space(標本空間); a set of elementally events(標本点),
an event(事象) is a subset of Ω.
ℱ: a set of events e.g., 2Ω (-algebra ⊆ 2Ω, in precise).
𝑃: probability measure(確率測度); a function ℱ → ℝ,
probability of an event.