Upload
vonga
View
213
Download
0
Embed Size (px)
Citation preview
On Biostatistical GeneticsUsing Twin Data
Jacob Hjelmborg
University of Southern Denmark
Fall 2013
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 1 / 63
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 2 / 63
The Danish Twin RegistryThe oldest twin registry
Founded in 1953 by doctors Tage Kemp, Mogens Hauge and BentHarvald.Cancer among initial focuses. Rockefeller Foundation and NIH∼100.000 twins born from 1870 till now.Population based with more than 97% completeness.Linked to National Registries.
The Danish Twin Registry
The Danish Twin Registry
Twin Symposium 2013
BRCA gene discoverer
Mary-Claire KingPioneer in research into hereditary breast cancer: New Scientist June2013
’Supreme court is right - the BRCA1 gene cannot be patented’.Portrayed in forthcoming film ’Decoding Annie Parker’.’Tales of a Minstrel Geneticist’ - Title of this year’s Hans ChristianAndersen lecture at the University of Southern Denmark, ThursdayNovember 14th.
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Males
Females
3.0
53.1
3.1
53.2
3.2
5M
ean lnB
MI
15 20 25 30 35 40 45 50 55 60 65Mean age
Longitudinal BMI
Body Mass IndexUS Twin Study 1997: Genetic influences on changes in body mass index:a longitudinal analysis of women twins. (Mary-Claire King)Finnish Twin Study 2007: Genetic influences on growth traits of BMI: alongitudinal study of adult twins.modeling change: BMI(t)i = αi + βi tSee K. Holst ’Day1-estimation’ slide: Longitudinal Biometric Analysis
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 9 / 63
What is heritability?
Darwin: Origin of the Species (1859)The inherited traits form the basis for natural Selection, Mutation andReplication.
The inherited is a blend
Mendel: Experiments in Plant Hybridization (1865)The inherited is a fifty-fifty mix of discrete units.
Mendel’s First Law (Segregation): One allele of each parent is randomlyand independently selected, with probability 1
2 , for transmission to theoffspring. The alleles unite randomly to form the offspring’s genotype.Mendel’s Second Law (Independent Assortment): The alleles underlyingtwo or more different traits are transmitted to offspring independently ofeach other. The transmission of each trait separately follows the first lawof segregation.
How can Mendel’s discrete units explain the variation in continuous humancharacteristics?
Statistical geneticsHow is variation at phenotypic level governed by variation at genetic level?
Two structures for modelling: mean and variance-covariance.R.A. Fisher (1918): The variance-covariance matrix varies by type of twinpairs.
Σ =
(variance of first twin covariance of twinscovariance of twins variance of second twin
)-a measure of twin similarity: ρ, correlation (K. Pearson 1904)
Aims
Difference in correlations between MZ and DZ twins suggests geneticinfluence on trait.What type and magnitude of genetic and environmental influences toexpect?Classical twin analysis using the polygenic model, known as theADCE-model, in which the individual outcome, Yi decomposes into
Yi = Ai + Di + Ii + Ci + Ei ,
whereI A: Additive genetic effects of allelesI D: Dominant genetic effectsI I: Epistasis genetic effectsI C: Shared environmental effectsI E : Unique environmental effects
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 13 / 63
Biometric analyses - polygenic model (Falconer 1981)
Contributing factors to the variation in outcome:
ΣY =
(σ2
A zσ2A
zσ2A σ2
A
)+
(σ2
D uσ2D
uσ2D σ2
D
)+
(σ2
I kσ2I
kσ2I σ2
I
)+
(σ2
C σ2C
σ2C σ2
C
)+
(σ2
E 00 σ2
E
)where z = u = k = 1 for MZ pairs, z = 1
2 , u = 14 and 0 ≤ k ≤ 1
4 for DZ pairs.
In particular, we obtainHeritability:
H2Y =
σ2A + σ2
D + σ2I
σ2A + σ2
D + σ2I + σ2
C + σ2E
= h2 + d2 + i2
Shared environmental effect:
C2Y =
σ2C
σ2A + σ2
D + σ2I + σ2
C + σ2E
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 14 / 63
H2 Heuristics {ρMZ = h2 + d2 + i2 + c2
ρDZ = 12 h2 + 1
4 d2 + ki2 + c2
HenceρMZ − ρDZ ≤ H2 ≤ min{ρMZ ,2(ρMZ − ρDZ )}
If no epistasis, {ρMZ = h2 + d2 + c2
ρDZ = 12 h2 + 1
4 d2 + c2
we obtain43
(ρMZ − ρDZ ) ≤ H2 ≤ min{ρMZ ,2(ρMZ − ρDZ )}
Heritability propositionsH2 = 0 always if ρMZ = ρDZ , hence existence of genetic effects can bedetected solely from correlations and is robust towards misspecificationof genetic model, nice!H2 differs by at most ρMZ − ρDZ from true value.
Correlation heuristics
Within pair similarity is measured by correlations.Correlations are further modelled by genetic and environmental variancecomponents via the polygenic ADCE model.For instance, the polygenic ACE model relates to correlations viaρmz = h2 + c2 and ρdz = 1
2 h2 + c2.
Heuristics of MZ and DZ correlationsInterpretation
Relation Genetics Environment Examplesρmz > 4ρdz Epistasis albinismρmz > 2ρdz Genetic dominance Dρmz = 2ρdz Additive effect A (mono- or polygenic) and small D Small C BMI2ρdz > ρmz > ρdz Additive genes A Shared environment C longevityρmz = ρdz > 0 No genetic effect Cρmz = ρdz = 0 No genetic effect No familial aggregation
Biometric analyses - polygenic modelMain assumptions
Equal environments assumption for MZ and DZ twins.No gene-environment interaction and correlation.No gene-gene interaction (link: epistasis).Equal mean and variance of twin 1 and twin 2, MZ and DZ.Estimation and inference by maximum likelihood principle assumingbivariate normality of paired observations (as before).
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
● ●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●●●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
2.8 3.0 3.2 3.4 3.6
2.8
3.0
3.2
3.4
3.6
ext$logbmi.1[ext$zyg == "MZ"]
ext$
logb
mi.2
[ext
$zyg
==
"MZ"
]
●
●
●
●
●
●
● ●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●●
●
●
● ●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
2.8 3.0 3.2 3.4 3.6
2.8
3.0
3.2
3.4
3.6
ext$logbmi.1[ext$zyg == "DZ"]
ext$
logb
mi.2
[ext
$zyg
==
"DZ"
]
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 18 / 63
Sex-limitation model to include OS DZ’s
Should we include opposite sexed DZ’s?Are the same genes in males and females affecting the trait of interest?(genetic pleiotropy)-howto?
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 19 / 63
Example
Height difference between men and womenMen are taller than women.Is this because of genetic differences?
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 20 / 63
Height difference between men and women
Within pair correlations for same and opposite sex DZ twins
Within pair correlationcountry DZ males DZ females OS twinsAustralia 0.42 0.49 0.46Denmark 0.47 0.55 0.50Finland (old) 0.50 0.49 0.49Finland (young) 0.53 0.53 0.49Italy 0.57 0.49 0.30The Netherlands 0.47 0.49 0.43Norway 0.49 0.49 0.44
source: Silventoinen et al., Twin Res 2000 and 2003
There is no significant difference between correlations within SSDZ andOSDZ twin pairsThere seems to be no sex-specific genetic factor affecting height.
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 21 / 63
Sex-limitation model to include OS DZ’s
Studying gene×gender interaction.Making use of all DZ pairs.
The within-pair covariance of male and female twins:
Cov(male,female) = gσAmσAf + σCmσCf
Will g = 12 as for DZ same sex pairs?
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 22 / 63
Sex-limitation model to include OS DZ’s
DZSS Males,
ΣDZSS =
(σ2
Am+ σ2
Cm+ σ2
Em
12σ
2Am
+ σ2Cm
12σ
2Am
+ σ2Cm
σ2Am
+ σ2Cm
+ σ2Em
)DZSS Females,
ΣDZSS =
(σ2
Af+ σ2
Cf+ σ2
Ef
12σ
2Af
+ σ2Cf
12σ
2Af
+ σ2Cf
σ2Af
+ σ2Cf
+ σ2Ef
)DZOS,
ΣDZOS =
(σ2
Am+ σ2
Cm+ σ2
EmgσAmσAf + σCmσCf
gσAmσAf + σCmσCf σ2Af
+ σ2Cf
+ σ2Ef
)
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 23 / 63
Sex-limitation model to include OS DZ’s
We will encounter more examples as we go along.A value of g = 1
2 for OS twins suggest no gender specific genetic effect.-a lower value will indicate such effects.R package ’mets’ estimates g when OS pairs are available.See K. Holst ’Day1-estimation’ slide: Gene-Environment Interactions.
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 24 / 63
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 25 / 63
Review of classic twin analysis aims and methodology
Inferring genetic influence without observing any gene.What is the contribution of genetic and environmental factors to thevariation?Are the same or different genes influencing the traits?{
Y = Genes + EnvironmentΣY = ΣGenes + ΣEnvironment
However, there are other fruitful use of collections of twin pairs.
SEE SLIDES: "Day02caseCotwin.pdf“.
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 27 / 63
What is dependence?
AimTwin studies may provide a frame for genetic analysis and etiology.For instance in the etiology of cancer.A measure of dependence becomes utterly important.
CorrelationF. Galton (1890)
’I can only say that there is a vast field of topics that fall underthe laws of correlation, which lies quite open to the research ofany competent person who cares to investigate it’
Pearson 1904The product moment correlation
E{(X − µ)(Y − µ)}Measures linear dependence.
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
● ● ●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●● ●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●
20 40 60 80 100
2040
6080
100
MZ
time.2
time.
1
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
● ●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
● ●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
● ●●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●
●
●●
●
●●
●
●
●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●● ●
●●●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
● ●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0 20 40 60 80 100
2040
6080
100
DZ
time.2
time.
1
R.A. Fischer 1918Heritability in terms of correlations.Genetic variation causing phenotypic variation
Variance explained by SNPs in metabonomic traits
Addendum
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 31 / 63
Agouti MZ mice
Equal but not the same
The Epigenetics RevolutionControlled reading of DNADetermines function of cellMay change during lifeTransgenerational inheritance(!) (Lamarckism?).Ovrekalix, Dutch famine, Obesity mice, . . .
Equal but not the same
Epigeneticsmakes monozygotic twins different from first replicationkey in evolutionary dynamics of cancer
Schewig et. al. , Max Planck Berlin, 2013
Equal but not the same
In parallel: Evolutionary dynamicsEvolutionary dynamics: Replication, Mutation and Selection.Statistical genetics governs mostly Mutation and Selection.Epigenetics: Replication of genetic Information.
Epigenetic treatment of acute leukemia by5-aza-cytidine
Equal but not the same
Epigenetics in Twin Studies1 Genetics of Epigenetics (genetic influences on epigenetic effects)2 Epigenetic Epidemiology (case-cotwin discordance)
AimHeritability of complex traits?-for instance taking epigenetic phenomena into account.In general: non-Mendelian inheritance?, genetic variation with time?
’MZ Nokia mobile phones’
What is the conveyed information between X and Y?
Information Theory
The Mutual Information
I(X ,Y ) =
∫∫fXY (x , y) log
fXY (x , y)fX (x)fY (y)
dxdy
E. H. Linfoot 1957
Theorem: E.H. Linfoot (1957)For (X ,Y ) ∼ F ,
ν(X ,Y ) =√
(1− exp(−2I(X ,Y )))
where I(X ,Y ) =∫∫
fXY (x , y) log fXY (x,y)fX (x)fY (y)
dxdy is the mutual information,satisfies the seven correlation properties of Renyi.
Renyi propertiesIndependence if and only if ν(X ,Y ) = 0Strict dependence if and only if ν(X ,Y ) = 1Bivariate Gaussian: Linfoot is |ρ(X ,Y )|Invariance under continuous and strictly increasing marginaltransformations
Information
The Informational CorrelationThe Linfoot transform of the Mutual Information is bridging Informationtheory with Statistics.
Generalizing heritability (joint with A. Kryger Jensen)We propose:
The informational heritability coefficient
h2∗ = 2(LMZ(X ,Y | XMZ)− LDZ(X ,Y | XDZ))
L(X ,Y ) =√
1− exp(−2I(X ,Y )) (Linfoot’s correlation)
I(X ,Y ) =
∫∫log
f (x , y)
f (x)f (y)dF (x , y) (Mutual Information)
PropertiesMarginal densities may be constrained to be exchangeable within pairsand equal for MZ and DZ types of pairsProposition: h2
∗ generalizes classic Falconer h2 for bivariate normalsh2∗ is measuring proportion of density explained by genetic variations
transmitted to phenotypic density patterns.topology may be rather complex.
Perspectives
Epigenetics: Key in evolutionary dynamics of cancer. Evidence fortransgenerational inheritance.Calls for more general heritability measure.Linfoot’s correlation is based on the mutual information and fulfills Renyiproperties.
Overview
1 Introduction
2 Heritability Heuristics and History
3 Opposite versus Same Sexed Pairs
4 The Matched Case-Cotwin Design and Analysis
5 On Epigenetics and Information Theory in Twin Studies
6 Examples
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 44 / 63
Example 1: Bivariate Gaussian simulation
Twinlm Falcon h*
0.0
0.2
0.4
0.6
0.8
1.0
500 simulations with 300 MZ and 300 DZ pairs
Agouti MZ mice
simulation
-3 -2 -1 0 1 2 3
-3-2
-10
12
3
MZ
Twin 1
Twin
2
-4 -2 0 2 4
-4-2
02
4
DZ
Twin 1
Twin
2
Twinlm Falcon Kendall Spearman h*
0.0
0.5
1.0
500 simulations with 300 MZ and 300 DZ pairs
Albumin - h2 = 0 (Bathum et al. 2004)
0.2 0.4 0.6 0.8
0.2
0.4
0.6
0.8
MZ Albumine levels
Twin
Co−
Twin
0.5
0.5
1
1.5
1.5
1.5
2
Albumin - h2 = 0 (Bathum et al. 2004)
0.0 0.2 0.4 0.6 0.8 1.0
0.2
0.4
0.6
0.8
1.0
DZ Albumine levels
Twin
Co−
Twin
0.5
0.5 1
1.5
1.5
2
Albumin - h2 = 0 (Bathum et al. 2004)
Proposed heritability for general associationsLMZ = 0.47LDZ = 0.30
h2∗ = 0.34
-estimation needs to be validated!
Time to event twin dataMain objectives:
I Time-varying genetic influenceI Taking censoring and competing risks into account,
Major component in NorTwinCan Study joint with
The Nordic Twin Cancer Study
∼ 400.000 Nordic twins linked with national cancer registries.42 common cancer sites:
1 "Lip"2 "Tongue"3 "Salivary glands"4 "Mouth"5 "Pharynx"6 "Oesophagus"7"Stomach"8 "Small intestine"9 "Colon"10 "Rectum and anus"11 "Liver"12"Gallbladder and extrahepatic bile ducts"13 "Pancreas"14 "Nose, sinuses"15"Larynx"16 "Lung (incl. trachea and bronchus)"17 "Pleura"18 "Breast"19"Cervix uteri"20 "Corpus uteri"21 "Uterus, other"22 "Ovary and uterineadnexa"23 "Other female genital organs"24 "Prostate"25 "Testis"26 "Penisand other male genital organs"27 "Kidney"28 "Bladder and other andunspecified urinary organs"29 "Melanoma of skin"30 "Skin, non-melanoma"31"Eye"32 "Brain, central nervous system"33 "Thyroid"34 "Bone"35 "Softtissues"36 "Non-Hodgkin lymphoma"37 "Hodgkin’s disease"38 "Multiplemyeloma"39 "Acute leukaemia"40 "Other leukaemia"41 "All sites butnon-melanoma skin"42 "All sites - first diagnosis"60 "Rectum"61 "Colorectal"
Multiple outcome at time t .
alive
dead
prostate cancer
α01(t)
α02(t)
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 53 / 63
Cumulative incidence
40 50 60 70 80 90 100
0.00
0.05
0.10
0.15
Age
Cum
ulat
ive
Inci
denc
e
DenmarkFinlandNorwaySweden
The cumulative incidence function:
Fcancer(t) = Prob(T ≤ t , ε = cancer) =∫ t
0λcancer(s)S(s−)ds,
NB! Same prostate cancer mortality (Bray et al. Eur Jour Cancer 2010).
Concordance
Scheike et al. 2013: Semi-parametric random effects regression model ofthe cumulative incidence function.The concordance function conditional on covariates
P(T1 ≤ t , ε1 = cancer,T2 ≤ t , ε2 = cancer|X ),
Dependence in terms of: Probandwise concordance, relative recurrencerisk, cross-odds ratio and multilocus index.Alternative frailty approach on hazard scale by F. Eriksson (2013).
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 55 / 63
Prostate cancer - Nordic - Concordance
60 70 80 90 100
0.0
0.1
0.2
0.3
0.4
0.5
Age
conc
orda
nce
60 70 80 90 100
0.0
0.1
0.2
0.3
0.4
0.5
Age
conc
orda
nce
Probandwise concordance
MZ pairsDZ pairsIndependence
Methods: Estimating heritability for cause specific mortality based on twin studies. Scheike,Holst and Hjelmborg, LIDA (2013).
Bivariate extreme value theory
-joint with J. Kaprio, Y. Goegebeur and M. Osmann.M. Osmann receives "Dansk Matematisk Forenings specialepris for 2012"
Biostatistics (Institute of Public Health) On Biostatistical Genetics Fall 2013 57 / 63
Extreme value theory for twin dataObesity
Evidence for epigenetic effects governing extreme BMI (Obesity).Twin studies can provide a frame for genetic analysis and etiology.Modelling the dependence of extremes becomes utterly important.
●●
●
●
●●
●
●
●
● ●
● ●
●
●●
●
●● ●
● ●● ●
●●
●
●
●●
●●
●●
●
●
● ●
●
●
●
● ●
● ●
●●
●
●●
●
● ●● ●
● ●
●
● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ● ●
●
●
●
● ●●
●
●
●
●
●
●●
●
●●
●● ●
● ●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
● ●● ●
●
●
●
●●●
●
●
●
●
●●●
●● ●
●●
●●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
● ● ●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●
● ●
●●
●
●
●
●
● ● ● ●●
●●
●
●
●
●
●
●
●
●
●● ●
●
● ●●
●
●
●
● ●
●
●
●●
●●
●●
●
●
●●
●
●
●
● ●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●● ●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●●
● ●
●
●●
●
●
● ●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●●
●
● ●
●
●
●●● ●
●●
● ●
●
● ●
●●
● ●
●
●●
● ●
●●
●
●
●
●
●
●
●●
● ● ● ●
●
●
●
● ● ● ●
●
●
●
● ●
●
●
●● ● ● ● ● ●
● ●●
●
● ●
●
● ●
● ●
● ●
●
●
● ●
●
●
● ●
●●
●●
●
●●
●
●
●
●
●● ●
●● ●
●
●●
●
●●
●●
● ●
●
●
● ● ● ●●
●
●
●●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
● ●●● ●
●
●●
●●
●
●
●
●
●
●
● ●
●●
●●
●
●
●●
●
●
● ●
●
● ●
●
● ● ●● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
● ●
● ●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
● ●●
●
● ●
●
● ●
●
●
● ●
●
● ● ● ●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
● ●
● ●●
●●
● ●
● ●
●●
●
●
● ●
●
●
●
●
●
●
● ●
●
●● ●
● ●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●● ● ●
● ●
●
●● ●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ● ● ●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
● ●
●
●●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●●
●
●
●●
● ●
●●
● ● ●
●
● ●●
●
●
●●
●
● ●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ● ●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
● ●
● ●
●●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●● ●
● ●
●
●
●
●
●● ●
●
●
●●
● ● ●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●●
● ● ●●●
●
●●
●●
● ●
●
● ●
●
●●
●
●● ●
●
●
●
●
●
● ●
●●
●
●●
● ●
●
●
● ●●
● ●
●
●
●
●
●●
●
●
●
●● ●
●●
● ●
●●
●●
●
●
●
●
●
●●
●
● ● ●
● ●
● ●
● ●●
● ●●
●●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●●
●
●
● ●
●
●● ● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
● ●
●
●
●●
●●
●
● ●
●
●●
●●
●
●●
● ● ● ●
● ●
●
●
●
●
●
● ●
●●
●
● ●
●
●
●
●
● ●●
●
●
●
● ●
●
●●
●
●●
●
●
●
●
● ●
● ● ●●
●●
●
●
●
●●
● ● ●
●
●●
●
● ●
●
●
●
●
●
●
●
● ●
●●
●
● ●
●
● ●
● ●
● ●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●●
●
● ●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
● ●
●●
●
●●
●
● ●●
● ●● ●
●
●
●
●
● ●
●
●
●●
●
●
● ●
●
●
● ●
●
●
●● ●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
● ●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
● ● ●
●
●
●●
●
●
●
●
● ● ●
●
●
●
●●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●●
●
●
●
●● ●
●
●
●
●
●●
●
●
●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●● ●
● ●
● ●●
●
●● ● ● ●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
● ● ● ●
●
●
●
●
●
●
● ●
●
●●
●
●
●●
●
●
●●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●●
●●
●
●
● ● ●
●
●
● ●
● ●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ● ●
●
● ● ●● ●
●●
●●
●●
●
●●
●●
●
●
●●
● ● ● ●●
●
● ●
● ●
●
●
● ●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●
● ●
●
●
●●
●
●●
●
●
● ●
● ●
●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●●
●●
● ●
●
●
●●
● ●
● ●
●
●● ●
●
● ●
● ●
●
● ● ●
●
●
● ●
● ● ●
●
●
●
●● ●
●
●
● ●
●
● ● ●●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
● ●
● ●
● ● ● ●●
●
●
● ●
●
● ●
●
●
●●
●
●
●
● ● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
● ●
● ● ● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
● ●
●
● ●
●
●
●
●●
●
●●
●
●●
●
●
●
●
● ● ● ●
● ●●
●
●
●
●
●
● ●●
● ● ●
●
●
● ● ●
● ●●
● ●●
● ●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
● ●
●
●● ●
● ●
●
●
● ●
●
●
● ●
●●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
● ●
●
●
● ●
●
● ●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●● ●
●
●
● ●
● ●
●
● ●●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●●
●
●
●●
●
●●
● ●
●●
●● ●
● ●
●
●
●
●
●
●
●
●
●
●
● ● ● ●
●
●
●●
●
●
●
● ●●
●
● ●
●
●
● ●
●●
● ● ● ● ●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●● ●
● ●
●● ●
●●
●
●
●
●
● ●
●
●●
●
●
●
● ●
●
●
●●
● ●●
●
● ●
●●
●
●
●
●
● ●
●
● ●
●
●
● ●
●●
●●
●
●
●●
●●
●
●
●
●
●
● ●●
●●
●
●
● ●
●●
●
●● ● ●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
● ●● ● ●
●
●
●
● ●
●
●
●
● ●
●●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
● ●
●●
●
●
●
●
● ● ● ● ●
● ● ●
●
●
●
●
●
●●
● ●●
●
●
● ●●
● ●
● ●
● ●
●
● ●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
● ● ●
●
●● ● ● ● ●
● ●●
●
●
●●
●
●
● ●
●
●
●●
●
● ● ●
●
● ●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
●
● ●●
●●
● ● ●
● ●
● ●●
●●
● ●● ●
● ●
●
●
●
●
●
●●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
● ●
● ●
●●
●● ●
●
●
●●
● ●
●
●●
●
●
●
●●
●
●
● ●
● ●●
●
●
●●
●
●
●
● ●
●
●
●
● ●
● ● ●
●
●
●
●
●
●
● ●
● ●
●
●
● ●
●●
●●●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●●
●●
●
●
●
●●
●
●
● ●
●
●
●● ●
●●
●
●●
● ●
●●
● ● ●●
●
● ●
●
●
●
● ●
●
● ●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●
● ●
●
● ●
●
●●
●●
●● ●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
● ●●
● ●●
●
●
●
● ● ●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●
● ●●
●
●
●
●
●
● ●
●
●
●●
● ●
●
●
●
●●
●
●●
●
● ● ● ●●
●
● ● ●
●
●
● ●●
● ●
●
● ●●● ●
●
● ●
●●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●●●
●
●●
●
●
●●
● ●● ●
●
●
●
●
●
●
●●
● ●●
●
●●
●
● ●
● ●●
●●
●
●
●
● ●●
●●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●
●●
● ●● ● ●
● ●
●
●
●● ●
●
●
● ●
● ●●
●●
● ●
●
●● ● ●
● ●
● ●
●
●●
●
●
●● ●
●
●
● ●
●
●●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
● ● ● ●●
●
●
●
●
●
●
●
●
●●
●●
● ●●
●
●
●●
●
● ●
●
●● ●
●
●
●
●
●
●
●
●●
●
● ●
●
●
● ●
● ●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●● ●
●
●●
●
●
●●
● ●
● ●●
●
● ● ●
●
● ●●
● ● ●
●
● ●
●
●
●●
●
●
●
● ●
●
●
●
●●
●
● ●
●
●
●●
●
●● ●● ● ● ●
● ● ●
●
●●
● ● ●●
●
●
●●
● ●●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
● ●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
● ● ● ●
● ●●
●●
●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●●
● ●●
●
●
● ● ●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
● ●
●●
●●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●●
●●
●
●
● ●●
●
●●
●
●
●● ●
● ● ● ●
●
●
●
●
●●
●
●
●●
● ●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
● ●
●●
●
● ●
●
●● ●
●
●
●
●
●●
●●
●
●
●
● ●●
●●
●
●
●
●
●●
●
● ● ●
●●
● ●●
●
● ●
●
●
●
● ● ●
●
●●
●
●
●
●
●
●
● ● ●
●
●● ●
●
●
●●
●●
● ●●
● ●
●
●●
●
●● ●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
● ●●
●
●● ●
● ● ●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
● ●●
● ● ●
● ● ● ●
●●
●
●
●
●● ●
●
●
●
●● ●
● ●
● ●
●
●
●
●
● ●●
●●
●
●
●
●●
●
● ●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●●
●
●
● ●●
● ●
●
●
●
●
●
● ●
● ●
●
●●
● ●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
● ●
● ●
●●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
● ● ●
●
● ●
●
●
●●
● ● ●
●●
●●●
●
●●
●●
●
●●
●● ●
●
●
●● ●
●
● ●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
● ●
●
●
●●
●
● ●●
●● ●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
● ●
●
●
● ●●
●
● ●
●
●
● ●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●● ● ●
●●
●●
●
●●
●
●
●
●
●
●
● ●
●● ●
●
● ●
● ●●
●
●
●
●●
● ●● ●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●● ● ●
● ●
●
● ● ●
●● ● ●
● ●
● ●
● ● ●●
●●
● ●
●●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
● ●
●
●
●
●
●
●
●● ● ●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
● ●
●●
●
●
●
●
●●
● ●
●●
●●
●
●●
●
●
● ● ● ●
●
●
●
●
●
●
●
●
● ●●
● ●●
●
●
● ●
●
●
●
●
● ●
●●
●
●
● ● ● ●●
●
●
●
●
●
●
● ●
●
●
●● ●
●●
●
● ●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
● ●
●
●
●●
●● ●
●
● ●●
●●
●
●
●●
●● ●
●
●
● ● ●
● ●
●●
●
● ●●
●
●
●
●
● ● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●●
●
●
● ●
●
●
● ● ●
●
●
●●
●
●
●
●●
●●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●●
● ●
●●
●● ●
●●
●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
● ●
● ●
●
●
● ●● ● ●
●
●
●
●●
● ●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●●
●
●
●●
●●
● ●
●
● ●
●
●
●
●
●●
●
●● ●
● ●
●
●
●
●●
●
● ●
● ●
●
●●
●●
●
●
●●
●
● ●
●
● ●
●
● ●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
● ●
●
●
●
● ●
●
● ● ●
●
● ●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
● ●
●
●
●
●●
●
●
●●
●●
● ●●
●● ● ●
●●
●
●
●●
●●
●●
●
● ●●
●
●●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
● ●
●
● ● ●
●
●●
●
●●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●
● ●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ● ●
●
●
● ● ● ● ●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●
● ●● ●
●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
● ●
● ● ●●
●
●
●
●
●
●
●
●●
●●
●●
●● ●
● ●●
●
●
●
●
●●
●
●
● ●● ●
●●
●
● ●
● ● ●
●●
● ●
●
●
●●
● ● ●●
● ●●
●
●●
●
● ●●
●
●●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
● ● ●●
●●
●● ●
●● ●
●
●●
● ● ●
●
●
●
●
●
● ●
●●
●
●
●
● ●
●
●
●●
●
● ●
●●
● ●●
●
●
● ●
●
●
●● ●
● ●
●
●
●
●
●
●
●
●
●●
●●
● ●
●
●
●●
● ●●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●●
●●
●
●●
● ●
●
●● ● ●
●
●
●
●
●
● ●●
●
●
●●
● ●
●
●●
●●
●
●
●
●
●●
●
●●
●
● ● ●
● ●●
● ●
●●
●
● ●
●
●
●
●●
● ●
●
●
●●
●
● ●●
●●
●●
● ●
● ● ●
●
●
●●
●
● ● ●
●
●
● ●●
●
●●
●
●
●
●
● ●
● ●●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●●
● ● ●
●
●●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●●
● ●●
●
●
●●
●
●
● ●
●
● ●
● ●
●●
●●
●●
●
●
●
●
●●
●
● ● ●
●
●
●
●
● ●
●
●
● ●
●
● ●
●
●
● ●●
●
●
●
●●
●●
● ● ●
●●
●
●
●●
● ●● ●
●
●
●
● ●
●
●
● ●
●●
●
●●
●
●
●
●
●
●
● ●
●●
●
● ●
●●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
● ●
● ●●
●
●●
●
● ●
● ●
●
●●
●
●● ●
●
● ●
●
●
●
●
● ●
●●
●● ●
●
● ●
●
● ●●
● ●
●●
● ●●
●
●
●
●●
●●
●
● ●
● ●●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
● ●
●
●●
●
● ●
●
●
● ●
●
● ●●
●
●
●●
●
● ●
● ●
●
● ● ●
●
●
●
●●
●
●
●
● ● ●
●
●
●
●●
●●
●
●
●
● ●●
● ●
●●
● ●●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ● ●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
● ● ●● ●
● ●
●
● ●
●● ●
●● ●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●●
●
● ● ●
●
●
●●
●● ●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
● ●
● ●● ● ●
●
●
● ● ●
●
● ● ●
●
●
●
●
●
●● ●
●
●●
●
●
●
● ●
●● ●
● ●●
●
●
●
●
●
● ● ●
●
● ● ● ●●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●●
● ● ●
●
●
● ●
●
●
●
●
●
●
●
● ●
● ●
● ● ●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●
●●
●
●●
●●
●
● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
● ●●
●
●
● ●
●
●
●
●
●
●
●
● ●●
● ● ●●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●●
●
●● ●
●
● ● ●●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●●
● ●●
●
●
●●
●● ● ●
●
●
●
● ●
●
●
●
●● ●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ● ● ● ●●
●●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●●
●● ●
●●
● ●
●● ●
● ●●
● ●
●
●●
●
● ● ●●
●
●● ● ● ●
● ●
●●
● ●
●
● ●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
● ● ●
●
● ●●
●●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
● ●
●
●●● ●
●●
●
●
●●
●●
● ●
●●● ●
●
●●
●●
●● ●
●
●
●
● ●
● ●
●
●
●
● ●●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
● ● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
● ●
●
●●● ● ●
●●
●
●
● ●
●
●●
● ●●
●
●●
●
●
●
●
●●
● ●
● ●●
●
●
● ●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●● ●
● ●
● ●
●●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
● ●●
● ●
●
●
● ●●
● ●
● ●
●●
●●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●●
●
● ●
●
●●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●● ● ●
● ●
● ●
●●
● ● ●
●
●●
●● ●
●
●
●●
● ●
●
●●
●●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●●●
●
●
● ●
● ●
●
●
● ● ●
●
●
●
● ●●
●
●
●●
●
●
● ●
●
●
●●
●
●
● ●●
● ●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
● ●●
●
●●
●
●
● ●
● ●
●●
●●●
●
●
● ●
●
●
●
●
● ●
●●
●●
●
●
● ●
● ●●
● ●
●
●
●
●●
●
●
●●
●
●●
●●
●
● ●
● ●● ●
●
●● ● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ● ●
●
●
●●
●
●
●
●●
●
●●
● ●
●
●●
● ●● ●
●
●●
●
●● ●
● ●
●
●●●
● ● ●
●
●●
● ●
●
●
●●
●●
● ●
●
●
●
●●
●●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●●
●
●
● ●
●●
●
●
●
●
●●
●●
●
●
●
● ●●
●
●
●
●●
●●
●
●●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●●
● ●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
● ●●
● ●●
●
●● ●
● ●
●
●● ● ●
●
● ●
●
● ● ●
●
●●
●
● ●
●
●●
●●
● ●
●
●
●
●●
●●
●●
● ● ●●
●
●
● ● ●● ● ●
●
●● ●
●●
●
●
●
● ● ● ●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
● ● ●
● ●
●
● ●
●
●
●
●●
●
●
●
●
●
●●
●● ●
●● ●
●
● ● ● ●
●
●●
●●
●●
●
●●
●
●
●●
●● ● ● ● ● ●●
●●
● ●
● ●
●
●
● ●
● ●
●
●●
●
● ●● ●
●
●
●●
●
● ●
●
●
●
●●
● ●
●
●
●●
●
● ●
● ●
●●
●
●
●
●●
● ●●
●
●●
● ●●
● ●
●●
●
●
●●
●
●●
●
●●
●
● ●
●
●●
● ●●
●
●
●
● ●
●
●
●
●● ● ●
●
● ●
●
●
● ● ●●
●
●
● ●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●● ●
● ● ●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●● ● ●
● ●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
● ● ●●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
● ● ●
●
●
●
●
●
●
●●
● ● ●
● ●●
● ●
● ●
●
●
● ●● ●
● ●
●● ● ●
●
●
●
●
●
● ●
●● ● ●●
●
●
●
●
● ●
●
●
●●
●
●●
●●
●
●●
● ●
● ●
●●
●
● ●
●
●● ● ●
●
●●
● ● ●
●
●
●●
● ●
●
●
● ●●
●
●●
●
●
●
●●
●●
●● ●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
● ●●
●
●
● ● ●
● ●
●
●
●
●
●
●
●
●
● ● ● ●●
●
●
●●
●● ●
● ●
●
●●
●
●
●
●●
●●
●
●●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
● ●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
● ●
●● ●●
●
●
● ●
●
●
● ●● ●
●
● ●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ● ●
●
●
● ●
●
● ● ●
● ● ●
● ●
● ●●
● ●
●●
●
●
● ●
●
●●
●●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●●
●
●●
● ●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
● ●
●
●
● ●
● ●
● ●
● ● ●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ● ●
●
●
●●
●● ●
●●
●●
●●
●
●
●
● ● ●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
● ●● ●
●
●●
●
●
● ● ●
●●
●●
●
●
●●
●● ●
●
● ●
●
●
●
●
●●
●●
●●●
●●
●
●● ●
●
●
● ●
●
●
● ●
●
● ●
● ●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
● ●
● ●
●
● ●●
● ●
●
●●
● ●
●
● ●
●
●●
●
●●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ● ● ● ●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●● ●
● ●
●
●
● ● ●
●
●
●●
● ● ●
●●
● ● ● ●
●
●● ●
●
●● ●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
● ● ● ●●
● ●
●●
● ● ●●
●
● ●●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
● ●
●
● ●
● ● ● ● ● ● ●
●
●●
●●
●●
● ●
● ●● ●
●●●
● ●●
●
●
●
●● ●
●
●
●
●
●
●●
● ●
●
●
●
● ●
● ●
●
●
●●
● ● ●
●
●●
●
● ●
●
●●
●●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ● ●
●●
●
●
● ●
●
● ●
● ●
●
●
●● ●
● ●
● ●
● ●●●
● ●
●
●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
● ● ● ● ● ●
●
●
●
●
●
● ●●
●
●
● ● ● ●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●
●
●●
●●
● ●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
● ●●
●●
●
●●
●
●
● ●●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●● ●
●
● ●
●
●
●
●
●
●●
● ●
●
●●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
● ●
● ● ●●
●
●●
● ●● ● ●
● ●●
●● ● ● ● ●
●
●
●
●
●
●●
● ●
●●
●
● ●
●●
● ●● ●
●
● ● ● ● ●
●
● ● ● ● ● ●
●
● ●
●
●● ●
●●
●●
●
●●
●●
●
● ●
● ●●
●
●
●
●
●
●
●
●
●●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
● ● ●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●● ● ● ●
●●
●
●
●●
●
● ●
●●
●●
●
●
●
●● ●
●
●
●
●
●
● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●●
●● ● ●
●●
●
● ●
● ●
●
●
●●
●
●●
●
● ●
●
●
●
●● ●
●
●●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●●
● ●
●●
●
● ●
●
●●
●
●
●
●●
●
●
● ● ● ●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●● ●
●●
●
●
●● ●
●●
●
●●
● ● ● ●
●
●
● ● ●
● ●
●●
●
●
●● ●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
● ●
●
● ●
●●
● ● ●● ●
●●
●
● ● ●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●● ●
●
● ●
● ●
●●
●● ●● ●
● ●
●
●
●
●
●
● ●
●
● ●
●
●●
●
●
● ●
●
● ● ●
● ●
●
●
●
●● ●
●●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
● ●
● ●●
●●
●
● ●
●
●
●
● ●
●
●
● ● ●●
●●
●●
● ●
●
●●
● ●
● ●
●
●●
●● ●
● ●
● ●
●
● ●●
●
● ● ● ●●
●
●
●
●
●
●
● ●● ● ● ●
●
●● ● ● ● ●
●
●●
●●
●
●
●
●
● ● ●
●
●
●
●
●
●●
●●
●
●
● ●
● ●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●● ● ● ●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
● ●●
● ● ●●
●
●● ●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●●
●●
●● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
● ●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●●
●● ●
●
●
●
●
●
●
● ●
●●
●● ● ● ●
●
●
● ●
●
●
●● ● ● ●
●●
●
● ●
●● ● ●
●
●
●
●
●●
● ● ●●
●
●
●
●
● ●● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
●
●
●
● ●
●●
●● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●● ●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
● ● ● ●●
●
●●
●
●
● ●
●
●
●
●●
●
●
●●
●●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
● ● ● ● ●
● ●
●
●
●
●
● ● ●
●
●●
●
●
● ●
●
●
●● ● ●
●
●
●●
●●
●●
●
●
● ●
●
●
●
●●
●●
● ●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●● ●
●
●
●● ●
● ●
● ● ●
● ●● ●
●●
●●
●
●
●
●●
● ●●
●
●
●
● ●
●
●
●
●
●●
● ●
●
●
●
●
●
● ●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●● ●
●
●
●●
● ● ●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
● ●●
●
●
●
● ● ●
●
●●
●
●
●●
●
●
●
● ● ● ●●●
●●
●● ●
●
●
●
● ●
● ● ●
●
●
●
●
●
●●
●
●
●●
●●
● ● ●●
● ● ●
●
● ● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●● ●
●
●
●
●
●
● ●
●
●● ● ●
● ●●
●
●
●●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●●
●●
●
● ●
● ●
●
●
●
●
●
● ● ●
●
● ●●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
● ●
●
●
●
●
● ● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●●
● ●
●
●●
●
●●
●
●
●
●
● ●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
● ●
● ● ●● ●
●
●
●
●
● ● ●
●
●● ● ●
● ●
●● ●
●
● ● ●●
●
●
● ●
●
●
●●
●
● ●
●
●
●●
●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●● ● ●
●●
● ●
●●
●
● ●
● ●●
●
●
●
●
●
●
●
●
● ●
●
●●●
● ●
●●
●
●
●
●
●
●
●
● ● ●
●
●●●
●
●
●
●●
●● ●● ●
●
●●● ●
● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
● ● ●
●
●●●
●● ●
● ●
●
●
● ● ●
●
●
●●
● ●●
●
●
●
●● ●
●●
●●●
●
●
●
●
●●
● ●●
●●
●● ● ● ●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
● ●●●
● ● ●●
●
●
●
● ● ●
●
●
●●
●
●●
●
●
●● ●
●●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
● ●
●
●
●
●
● ● ●
●●
●● ●
●●
● ●●● ●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●● ● ●
●
●●
●
●●
●
●
● ●●●
●
●
●
●
●● ●
●●
● ●
●
●
●● ●●
●
●●
●
●●
●
●
●●
● ● ●
●
● ●
● ●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●● ●
● ● ●●
● ●●
●
●
● ●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●●
●
●
●
●●
● ●●● ● ●
●
●
●
●
●● ●
●●
●
●●
● ●●●
●
●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●
● ●
●
● ●●●
●●
●
● ●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●● ●
● ●
●
●
●
●
●
●
●
●●
●● ●
●
●●
●●
●
● ● ●
● ●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●●●
●
●
●●
●
● ●
●
●
●
●
● ●
●
●
●
● ●
●
●
●●
● ●
●●
●
● ●
●● ●
●
●
● ●
●●
●
●
●
●
●
●
●●● ●
●●
●
●
●
●
● ● ●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●● ●
●
●
●●●
●
●
●
●
●
●
●
● ●
●●
●● ●●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●● ●●●
●
●
●
●
●● ●
● ●
●
●
●
●
●
●●●
●
●
● ●
●
●●
●
●
●
●
●●
●● ● ● ●
●
●
●
●
●●
●
●
● ●
●
● ●
●
● ●
●
●●
●
●
●
●●
● ●
●
●
●
●
● ● ● ●
● ●
●
●●
●
●
●
●
●● ●
● ●
●
●
●
● ● ● ●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
● ●
● ●
● ● ● ● ●
● ●
●
● ●
● ●●
●
●●
●
●●
● ●
●●
●
● ●●●
● ●●
●●
● ●●
● ●
● ●●
●
●
● ● ● ●
●
● ● ● ● ●
●
● ● ●
●
●
●●
● ●
●
●
●
●
●
●
● ●●
●
●
●
●
● ● ●● ●
●●
●
●
● ●
●
● ●
●
●●
● ●
●
●
● ● ● ●
●
● ●● ●
● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
● ● ●
● ●
● ●
●
●
● ●
● ●
●● ●
●●
●
● ● ● ● ● ●
●
●●
● ●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●●
●● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
● ● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ● ●
●●
●●
●
● ●
● ●
● ● ●
●
●● ●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
● ●● ● ●
● ●
●●
●
●
●
●
● ● ●
●
●
●●
●
●
●
●
●●
● ●
● ●
● ●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●●
●
● ●
●
●
●
●●
● ●●
●
● ●●
●●
●●
● ●
●● ● ● ●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
● ● ● ●
●
●
●
● ●
● ●
●●
●●
●
●●
● ●
● ●
●
●
●
●
●
●
● ●
●●
●● ●
● ● ●
●
●●
●
●
●
●
●
● ●
● ●
●●
●
●
●● ●
●
●●
● ●
●
●
●
●
●● ●
● ●
●
● ● ● ●
●
●
●
● ●●
● ● ●
●
●
●
● ●
● ●
● ●
●
●
●
●
●
● ●●
● ●●
● ●
●
●
●
●
●
● ●
● ●
●
●
●
●
●●
●
●●
●
● ●
● ●●
●●
●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ● ●● ● ● ●
●●
●
●
● ●
●
●●
●
● ●
● ●
●●
● ●●
●●
● ●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●● ●
● ●
● ●
●
● ● ●
●
●●
● ●
● ●
●
●
●
●
●
●
● ●
●
●
●
●●
●●
●
●
●● ● ● ●
●
●
●
●
●●
●●
●● ● ● ●
● ●
●
●
●
●
●
●
●
●
● ●
●●
●●
● ●
● ●●
● ● ●
● ● ●
●
●
●
●
● ●
●
●
●● ● ● ●
● ●
● ● ●
● ● ●●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●●
● ●● ●
●
●
●
●
● ●
●●
●●
● ● ● ●
● ● ●● ●
● ●
●●
●●
●●
●
●●
●
●
●● ●
● ● ●
● ●●
●
●●
●●
● ●
●
●
●
●●
●●
●
●
● ●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
● ●
● ●
●
● ● ●
●
● ●
●● ● ● ●
●
● ●
●
●
●
●
●
●
●●
● ●
●● ●
●●
● ●
●
● ● ●
●
●
●
●
● ● ● ●
● ●
● ●
●
● ● ● ●
●
● ●● ●
● ●
●
●
●
● ● ● ●
●
●
●●
●
●
●
●
●
● ● ●● ●
●
●
● ●
●
●
●
●
●
●
●
● ●
●●
●● ●
● ●
●
●
●● ●
●●●
● ●
●
●
●
●
●●
●
●●
● ●
●●
●
●
●
●●
●
●
●● ● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●● ●
●●
● ●
● ●
●
●●
●●
●
●
● ●
●
●●
●● ●
●●
●
●
●●
●
●
●
●●
●
● ● ● ●
● ●
●
● ●
●
●
●
●
● ●
●
●
● ●
●● ● ● ●
●●
●
●
●
●●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●●
● ● ●●
● ● ●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●●
●
●
●
●●
● ●
● ●
●
●●
●
●
● ● ●
● ● ● ●
●
●
●●
●
● ●
●
●
●● ●
●
●
●●
● ●
●
● ●
●
●
●
●
●
●●
●
●
●
● ● ●
● ● ● ●
●
● ●
●●
●
● ● ●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
● ●
●●
●
● ●
●
●
●
●
●
●●
●
●●
●
●●
●●
● ●
●●
●●
●● ●
●
●
● ●
●
●
●●
●
●●
●
● ●●
●
●
●
● ●●
●
●
● ●●
● ●●
●
●
●
●● ●
● ●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
● ● ●
● ●● ● ● ● ●
●
●
●
● ●
●
●
● ●
●
● ●
●
●
●
●●
● ●
● ●
●
●
●
●
● ●●
●●
●● ●
● ●
●
●
●●
●
● ●
●
●●
● ● ●●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
● ●
●
●
●
●●
●
●
●
●●
● ●● ●
●●
●
●● ●
●
●●
● ●
●
●
●
●
●
●●
●
●●
● ●●
●
● ●●
● ●
●
●
● ●
●
●●
● ●
● ● ●● ●
● ●
●
●
●
● ● ●●
●
●
●●
●
●
●
● ●
● ● ●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●●
● ●
●●
●
●
●
● ●●
●
●●
●
●
●
●
●●
● ●
●
●
● ●
●
● ●● ●
●
●
●
●
●
●●●
●●
●
●
● ●
● ●
●● ●
● ●
●●
●●
●● ●
●
●●
● ●● ●●
●● ● ●
● ●
●●
●● ● ● ●
●
●
●● ● ● ●
●●
●
● ●● ●
●●
● ● ●
●
●
●
●
●
● ● ●●
●●
●
● ●● ●
●
●
●
● ● ●
●●●
●
● ●●
● ● ●
●
●
●
●●
● ● ●
●●
●
● ●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
● ●
● ●
●
●
●
● ●
●● ●
● ●
● ●
●
●
●
●
●
● ●
●
●●
●
●
●
● ●
●
● ●●
●
●
●
●
●● ●
● ●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
● ●
● ●
●●
●
● ● ●●
●
●
● ●●
●
●
●
●
● ●
●
●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
● ●
●
●
● ● ●
● ●
●
● ●
●
● ● ●
●
● ●
●●
●
●
●
●
●
● ● ● ●
●
●
● ●
● ●
●●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
● ●
● ●
● ●●
● ● ●●
●
● ●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●●
● ●
● ●
●
●
● ● ●
● ●
●●
●● ●
●●●
●
● ● ● ●●
●
●●
● ● ●
●●
●
●● ● ● ●
●
●
● ● ●
● ●
●
●
●● ● ● ●
●
●●
● ●
● ●
●
●●
●●
●●
●
●
●● ●
●
●
● ● ●
● ●●
●
●
● ●
●
●●
●
● ●● ●
●●
●
●
●
●
●
●
●
● ● ●
● ●
●
●
● ● ● ● ●
●
●●
● ● ● ●
●
●
●
●●
● ●●●
●
●
●●
●
●
●●
●●
●
●●
●● ●
● ●●●
●● ●
● ●●● ●
●
●
●●
●●
●
● ●
● ●
●
●
●
●
●
●
●
●●
●
● ●
● ●
●
●
●●
●
●●
● ●●
● ●
● ●
● ●
●● ●
●●
●●
●
● ●
●
●
●
●
●● ●
● ●
●
●
●
●
● ●●
● ●●
● ●
● ●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●●
●
● ●
● ●
●
●
●● ●
● ●
●
●●
● ●
●
●
●
●
●
●●
●●
●
●
● ●
●●
●
●
●
● ●
● ●
●●
●
● ●
● ●
●
●
●● ●
● ●
●
●● ● ● ● ●
●
●
●
●
●
●
●
● ●●
●
●
● ●●
●
●
● ● ● ●
●
●● ●
●●
●●
● ● ●
● ●●
●● ● ●
●
●
● ●
●●
●●
●●
●● ●
● ●
● ●
●
● ●
●
●●
●●
●●
●●
●● ●
●● ● ●
●●
●
● ●
● ●
●
●
●
●●
●●
●●
●
● ●
●
●
●
●
●
●● ●
●
● ● ●
● ●
●
●
● ●
● ● ●
●●
●
●●
● ●
●
●
●
●●
● ●
● ●
●
●
●
● ●●
●
●●
●
●●
●
●
●
● ●●
●
●
●● ● ●
● ●
●●
● ●
●● ●
● ●● ●
● ● ●
●
● ●●
● ●
● ●
● ●
●
●
●
●
●
● ● ●
●● ●
●
●
●
● ● ● ●●
●●
●
● ●● ●
●
●
● ● ●
●
●
●● ●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
● ●
●●
●●
● ●
●
● ●
●
●● ● ●
●●
●
● ● ● ● ●
●●
● ● ●
●
●
●
●
●
●●
● ●
●
●
●
●● ● ●
●
●
●
●●
●●
●●
●
●●● ●
●
●
●●●
●
●
●● ●
●●
● ●●
●
●
●●●
●
●
●
●● ● ●
●
●●
●●
●
●
●●
●
●●
● ●
●
●
●
●●
●●
●●
●
●●
●
●
●
● ● ●● ● ●
●
●
●
●●
●●
●
●
●●●
● ●●●
●
●
● ●
●● ●
●●
●●●
●
●
●●
●●●
●
●●
●
●
●● ● ●
●●●
●
●
●●
●
●
●●
●●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
● ● ●●
● ●●
●
●
● ●
●● ● ●
●
●● ●●
●
●
●●
●
●●
●
●
●●
●
●●
● ●
●
●
●
●● ●
●●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
● ●
●●
●
●
● ● ●●● ●
●● ● ●
●
● ●
●●
●●
●● ●
●●
● ●
●
● ●●●
●●
●● ●
●●
● ●●●
●
●
●
●●
●
●
●●
●●
●
●●●
●
●● ●
●
●●
●●
●●●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●
● ●
●● ● ●●
●
●●
●
●
●●●
●●
●●
●
●●●
●●
● ●
●●
●
●●
●●● ● ●
●●
●
●●
●
●
●
●● ●●
●●
●
●
●●
●
●
●
●●
●●
●●
●●
●●●
●
●●
●●●
● ●
●●●
●● ● ●●●
●●● ●
●
●●●
●● ●●
●
●●
●●●
●
●●
●
●●
●
●
●●
●
●●
● ●
●
●
●
●●
●●
●
●
●●●
● ●
●
●● ●
●
●●
●
●
● ●● ●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●● ●
●●
● ●
●
●
●
●●● ●●
● ●
●
●
●
●●
●●
●●
●
●●
● ●
●
●
● ●●
●
●●●
●
●
●
●
●
●
●●
●●● ●
●● ●
●●●
●
●●
●●
●
●
●
●
●
●●
● ●
●
● ● ●●
● ●
●
●
●●
● ●
●
●● ● ●●
● ●
●● ●
●
●
●
●
●
●
●●●
●●
●●● ●
●
●●
● ●●●
●
●
●●●
● ●●
●
●●●
●
●●●
●
●●
●
●
●●
●●●
● ●
●
●
● ●●
● ●
●
● ●
●● ● ●
●
●
●
●●●
●
●
●
●
●●
●●
●● ●
●
●
●●
●
●●●
●●
●
●
●
●●
● ●
●
●
●
●●
●
●●●
● ●
●
●
●
●●
● ●●● ●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
● ● ●●●
●
●●
●
●
●
●
●
●
● ●
●●
● ●●
● ●
●
●●
●●
●
●●●
●●●
●●
●
●● ●
●●
● ●
●●
●●
●● ●
●
●
● ●●
●●●
●
●●●
● ●
●●
●●
●
●●
●●● ● ●
●
●
●
●●
●
●
●
●
●
●●
●●
●● ●
●
●
●
●
●
● ● ●●
●
●
●●● ●●
●
●
●
●
●
●●
● ●
●●●
●●
●
●●
●●
●●
●
●
●
●
●●●
● ●
●
●
● ●● ● ●
●
●
●●● ●
●
●●
●●●
● ●●
●
● ●● ●
●
●
●
●●●
● ●● ● ●● ●
●
●
●
●
●●
●●
●●●
●●
● ●
●
●
● ●●● ●
●
●
●
●● ● ●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
● ●
● ●
● ●●
●
● ●
●
● ●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
● ●●
●
●
● ●
● ●
●●
● ●
●
●● ● ●
●
●
●
● ●
● ●●
●●
●
●
●
●
●
●
● ● ●
●
●
●
●
● ● ●
●
●● ●
●●
●●
●
●
● ● ● ● ●
● ●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●● ● ● ●
● ● ●● ● ● ●
●
●
● ●● ●
●
●●
●●
●
●
●
●
●
●
●
●● ● ● ●
●
●●
● ●
●
● ●
●
● ●
● ●
●●
●● ●
●
●●
●
● ●
●
●
● ● ●
● ● ●
● ● ●● ●
●
●●
●
● ●
●
●
●
●●
● ● ●
● ● ●
● ●
●
●
●
● ●
● ●● ●
●
●●
● ● ●
●
● ●
●●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●●
● ● ●●
●●
●
● ●
●
●
●●
● ● ● ●
●
●●
●
●● ● ●
●
●●
●
●
●
●
● ●
●
●
●
●
● ●●
●● ●
● ●
●
●
●●●
●
● ● ●
●
● ●●
● ●●
●
●
●
● ● ●
●
●
●●
●● ●
●
●
●
●
●● ●
●
●
●
● ● ● ● ●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ● ●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
● ● ● ●
●●
● ●
●
●
●
●●
●
● ●
●
●
● ●
●
● ●
● ●
●
●
●
●
●
●
●●
●
● ●
● ●
●●
●
● ● ●
●
●●
● ●
●
●
●
●● ● ●
●●
●
●
●● ●
● ●
●
●
●
●
● ●
● ●
● ● ●● ●
●●
●
●
●
●
●
●
●● ● ●
●●
●● ●
●
●●
●
●
●
●
●●
●
●
●●
●● ●
●
●●
●
●
● ●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
● ●●
●
●
●
● ●
● ● ●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
● ●
●
●●
● ●
●
●
●
● ●●
●
●
●
●
● ● ●
●
●
● ●
●
●
●
● ●
●●
● ●
●● ● ● ●
●●
●● ●
●●
● ● ●
● ●
●
●● ●
●● ●
● ●
●●
●
●
●
●
●
●
●
●●
●● ●
●●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
● ●
●
● ●
●
●
● ● ●
● ●
●●
●● ●
●●
●●
●
● ●
●
●
●●
● ● ●
●
●
●
●
●
● ●
●
●
●
●
●
● ● ●
●
●
● ●
●
●
● ●● ●
●
●
●●
●
● ● ●●
●
●
●● ●
●
●
●
●● ● ●
● ●
● ● ● ●
●
●●
● ●
●
● ●●
●
●
●●
●
●
●
●
●
●
● ● ●
● ●
●
●
●
● ●
● ●
●
●
●● ●
● ●
●
●
●● ●
●
●
●
●
●
● ●
● ●
●●
● ●●
● ●●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
● ● ●
●
●
●●
● ●
●
●
● ● ● ● ● ● ●
● ●●
●
●
●
●
●●
●● ●
●
●●
●●
● ●
● ●
●
●
●
● ● ● ●
●
●
● ● ●
●
●
●
●
●●
●
●
● ● ●
●
●
●
●● ● ●
●
●
●
●●
● ●
●
●
●
● ●
●
●●
●● ●
●
●
● ●
●● ● ●
●
●
●●
●
●
●
● ●●
●●
●
●
● ● ●●
●
●
●
● ●●
● ●
● ●
●
● ● ●
●
●
● ●
●
●●
● ●
●
●
●
●●
●●
●
●
●
● ●
●
●
● ●
●● ●
●
●
● ●
●
● ● ● ●
●
●
●
●
●
●
●●●
●
● ●●
●● ●
● ●●
● ●
● ●●
●●
●
●
●●
●
●●
● ●●
●
●
● ● ●
●●
●●
●
●
●
●
●●
●● ● ●
●
●
● ● ● ● ●
●
●
●●
● ●
●
●
●● ●
●
●
●
● ● ●
● ●●
●
●
●
●●
● ●
●
●
●●
● ●
●
● ● ●
●
●
● ● ●●
●●
●
●
● ●
●
●●
●
●
● ●
●
●●
● ●
●
●
●
●●
●●
●
●
●
● ●
●●
● ●● ●
●● ●
●
●
●
●
●
●
● ●
● ●●
●● ●
● ●
● ●●
● ●
●
●
●
●
●
●●
● ● ● ● ●
●
●
●●
●
●
●
●
● ●●
● ● ●
●
●●
●
● ● ●
●
●
●
●
●
●● ●
● ● ●
●
●
● ● ●
● ●●
●
●
● ●
● ●●
●●
●●
● ●
●●
● ●
●
●●
●
●
●● ● ● ●●
● ●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●●
●
●
● ●● ●
●
●
●
●
● ●● ●
●
●
●
● ●
●
●
●
●
● ●
●●
●
● ●
● ● ● ●
●
● ●
●
●●
●
●
●
●
●
●
● ● ●
●
● ● ●●
● ●
●
●● ●
●●
●●
●
●
●● ●
● ●
●●
●
● ●●
● ●●
●
●
●
●●
●●
●●
● ●
● ●
●
●● ● ● ● ●
●●
●● ●
●
●●
●
●
● ●●
●
●
●
● ● ●
●
● ● ●●
●
●
●
● ● ● ● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
● ●
● ●
●●
●●
●●
●
●
●● ●
●
●●
● ● ● ●●
●●
●
● ● ●
●
●
●
●● ●
●
●●
●
●●
●
● ● ●
●● ●
●
●●
●
● ● ●●
●
●
●
● ● ● ●
●
●
●
●
●●
●
●● ●
●●
●
●
●
●
● ●●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●
● ●
●●
● ●
●
●
●
●
● ● ●●
●●
● ●
●
●
●
● ●
●●
●
●
●
●●
●● ● ●
●
●
●
●●
● ●
●● ●
● ●
●
●
●
●
● ●
●
●
● ●●
●●
● ●
●
●
●
●
●● ●
●●
●
● ●
●●
●
●
●
●
●
●
●
● ●●
●
● ● ●●
●●
●●
●
●
● ●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●●
●
●
● ●
●● ●
●
●
●●
●
●●
●
●●
●
● ●
● ● ●
●
●●
●
●
●●
●
● ●●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●● ● ● ●
●
●
●
●
●
● ●
● ● ●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●●
●
●
●
●
● ●
●
●
●
● ●
●
●
● ●
●
●●
● ●
●
●
●● ●
●
●●
● ●
●●
●
●● ● ●
● ● ●●
● ● ●
●
● ●
●
● ●
●
●
●
●
●
● ●●
●
●●
● ● ●
●●
●
● ●
● ●
●
● ●● ●
●●
●
●
●
● ● ● ●
●● ● ● ● ● ●
●●
●
●
●●
● ●
●
●
●● ●
●
● ●
●
●
●
●●
●
●
●
●
●
● ●
●●●
●
●●
●
● ●
●●
● ●●
● ●
●●
●
● ●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
● ●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
● ●
●● ●
● ●
●
●● ● ●
●●
●
●
● ●
●●
●● ●
●
●
●
●
●
●
●
●
● ●
●●
●●●
● ●
● ●
● ●
●●
● ● ●
● ●●
●
●
● ●
●
●● ● ●
● ●●
● ● ●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ● ●
●●
● ●
●
●
● ● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●● ●
●● ●
●● ● ●
●
●
●
●●
● ●
●
●●
●● ● ●
● ●●
● ●● ●
●
●
● ● ●
●
●●
●● ● ● ● ●
●
●
●
●
●
● ●
●●
● ●
●●
●
● ●
●
●
●
●●
●●
●●
●
●
●
●
●●
● ●●
●
●
● ● ● ●
●
●
●
●●
● ● ●
●
● ●
●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●●
●●
●●
●
● ●●
●
●
●
●●
●
●●
●
● ● ●●
●
●●
●
●
● ●
●
●
●
● ● ● ●●
●
●
●
● ●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
● ●
● ● ●
●● ●
● ● ● ●●
●● ●
● ●●
●
●
●● ●
●●
●●
●● ●
●●
●
●●
●
●
●
●
●
●
●●
●
● ●●●
●● ●
● ●
●
● ●
● ● ●
●
● ● ●● ●
● ●
●
●
●
● ●
●
●
●●●
●●
● ●
●
●
●●●
●
●
●●
●
● ●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●●
● ●
●
● ●●●
●●
●
●
●
●●
● ●
●
● ●●●
●●
●●
● ●
●
●
●
● ●
●●
● ●●
●● ● ● ● ●● ●
●
● ● ● ●
●
● ●
●●
●
●
●
● ● ● ● ●●
● ●●
●
● ●
●●
●● ●
● ●
● ● ●
● ●
●
●
●
●
●
● ●
● ●
●
●●
● ●●
●
●●
●
●
●
●
●
● ●
● ●
●
●●
● ●● ●
●●
●● ●
●
●
● ●
●
● ●
●
●
●●
●● ● ● ●
●●
●
● ●
● ● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●●
●
●
●●
● ●● ● ●
● ●●
●
●
●●
●●
●●
● ●
●
●
●
● ●
●
20 30 40 50 60
1520
2530
3540
Age
BM
I
Upper tail copulas
Twin
Co−
Twin
BM
I density
Twin
Co−
Twin
BM
I density
Dependence measures for upper tail
N η χ χ L(X ,Y ) h2∗
MZ Overall 20048 0.80 0.49 0.59 0.23 0.24Men 8212 0.78 0.45 0.56 0.22 0.13
Men (Age 18-29) 2241 0.83 0.44 0.67 0.29 0.57Men (Age 30-39) 3356 0.87 0.41 0.74 0.13 0.21Men (Age 40-60) 2615 0.70 0.39 0.39 0.22 0.14
DZ Overall 37476 0.61 0.26 0.21 0.11 -Men 16354 0.60 0.22 0.20 0.16 -
Men (Age 18-29) 4356 0.60 0.22 0.20 0.00 -Men (Age 30-39) 6657 0.55 0.16 0.10 0.03 -Men (Age 40-60) 5341 0.65 0.14 0.30 0.15 -
Practical: Estimating upper tail dependence measures
This session illustrates how to estimate the upper tail dependence measure
χ = limu→1
Prob(U1 ≥ u | U2 ≥ u)
from the classical bivariate extreme value distribution.
Bivariate Extreme Value DistributionLauch R-code on next slide to obtain an estimate of χEstimate χ for MZ and for DZ pairs.
Reference: Coles S, "An Introduction to Statistical Modeling of Extreme Values", Springer (2001).
Fitting Bivariate Peaks Over a Threshold Using Bivariate Extreme ValueDistributions
library(POT)library(mets)
data(twinbmi)
# a number, 1 or 2, is assigned to each twin in a pair.twinbmi$nr <- with(twinbmi, ave(bmi,tvparnr, FUN = seq_along))twinbmi$logbmi <- log(twinbmi$bmi)
bmiwide <- reshape(twinbmi,idvar="tvparnr",timevar="nr",v.names=c("bmi","logbmi","age"),direction="wide")
th = quantile(twinbmi$bmi,probs=0.80, na.rm=TRUE)
extlog <- fitbvgpd(cbind(bmiwide$bmi.1,bmiwide$bmi.2),c(th,th),cshape=TRUE, cscale=TRUE,model = "log")
extlog$chi
The R kiosk
’mets’ (K. Holst, T. Scheike)’etm’ (A. Allignol); ’prodlim’ (T. Gerds).