Upload
fauna
View
28
Download
0
Embed Size (px)
DESCRIPTION
Summer Inst. Of Epidemiology and Biostatistics, 2009: Gene Expression Data Analysis 8:30am-12:00pm in Room W2017. Carlo Colantuoni – [email protected]. http://www.biostat.jhsph.edu/GenomeCAFE/GeneExpressionAnalysis/GEA2009.htm. Class Outline. - PowerPoint PPT Presentation
Citation preview
Summer Inst. Of Epidemiology and Summer Inst. Of Epidemiology and Biostatistics, 2009:Biostatistics, 2009:
Gene Expression Data AnalysisGene Expression Data Analysis
8:30am-12:00pm in Room W20178:30am-12:00pm in Room W2017
Carlo Colantuoni – [email protected]
http://www.biostat.jhsph.edu/GenomeCAFE/GeneExpressionAnalysis/GEA2009.htm
Class OutlineClass Outline• Basic Biology & Gene Expression Analysis Technology
• Data Preprocessing, Normalization, & QC
• Measures of Differential Expression
• Multiple Comparison Problem
• Clustering and Classification
• The R Statistical Language and Bioconductor
• GRADES – independent project with Affymetrix data.
http://www.biostat.jhsph.edu/GenomeCAFE/GeneExpressionAnalysis/GEA2009.htm
Class Outline - DetailedClass Outline - Detailed• Basic Biology & Gene Expression Analysis Technology
– The Biology of Our Genome & Transcriptome– Genome and Transcriptome Structure & Databases– Gene Expression & Microarray Technology
• Data Preprocessing, Normalization, & QC– Intensity Comparison & Ratio vs. Intensity Plots (log transformation)– Background correction (PM-MM, RMA, GCRMA)– Global Mean Normalization– Loess Normalization– Quantile Normalization (RMA & GCRMA)– Quality Control: Batches, plates, pins, hybs, washes, and other artifacts– Quality Control: PCA and MDS for dimension reduction
• Measures of Differential Expression– Basic Statistical Concepts– T-tests and Associated Problems– Significance analysis in microarrays (SAM) [ & Empirical Bayes]– Complex ANOVA’s (limma package in R)
• Multiple Comparison Problem– Bonferroni– False Discovery Rate Analysis (FDR)
• Differential Expression of Functional Gene Groups– Functional Annotation of the Genome– Hypergeometric test?, Χ2, KS, pDens, Wilcoxon Rank Sum– Gene Set Enrichment Analysis (GSEA)– Parametric Analysis of Gene Set Enrichment (PAGE)– geneSetTest– Notes on Experimental Design
• Clustering and Classification– Hierarchical clustering– K-means– Classification
• LDA (PAM), kNN, Random Forests• Cross-Validation
• Additional Topics– The R Statistical Language– Bioconductor– Affymetrix data processing example!
DAY #3:DAY #3:
Measures of Differential Expression:Review of basic statistical conceptsT-tests and associated problemsSignificance analysis in microarrays
(SAM)(Empirical Bayes)Complex ANOVA’s (“limma” package in
R)
Multiple Comparison Problem:BonferroniFDR
Differential Expression of Functional Gene Groups
Notes on Experimental Design
Slides from Rob Scharpf
Fold-Change?Fold-Change?T-Statistics?T-Statistics?
Some genes are more variable than othersSome genes are more variable than others
Slides from Rob Scharpf
Slides from Rob Scharpf
Slides from Rob Scharpf
Slides from Rob Scharpf
Slides from Rob Scharpf
distribution of
distribution of
Slides from Rob Scharpf
Slides from Rob Scharpf
X1-X2 is normally distributed if X1 and X2 are normally distributed – is this the case in microarray data?
Problem 1Problem 1: T-statistic not t-distributed. : T-statistic not t-distributed. ImplicationImplication: p-values/inference incorrect: p-values/inference incorrect
P-values by permutationP-values by permutation
• It is common that the assumptions used to derive the statistics are not approximate enough to yield useful p-values (e.g. when T-statistics are not T distributed.)
• An alternative is to use permutations.
pp-values by permutations-values by permutations
We focus on one gene only. For the bth iteration, b = 1, , B;
• Permute the n data points for the gene (x). The first n1 are referred to as “treatments”, the second n2 as “controls”.
• For each gene, calculate the corresponding two sample t-statistic, tb.
After all the B permutations are done:
• p = # { b: |tb| ≥ |tobserved| } / B
• This does not yet address the issue of multiple tests!
The volcano plot shows, for a particular test, negative The volcano plot shows, for a particular test, negative log p-value against the effect size (M).log p-value against the effect size (M).
Another problem with t-testsAnother problem with t-tests
Remember this?Remember this?
Problem 2Problem 2: t-statistic bigger for genes: t-statistic bigger for genes with smaller standard with smaller standard
error estimates.error estimates.ImplicationImplication: Ranking might not be optimal: Ranking might not be optimal
Problem 2Problem 2
• With low N’s SD estimates are unstable
• Solutions:
– Significance Analysis in Microarrays (SAM)
– Empirical Bayes methods and Stein estimators
Significance analysis in Significance analysis in microarrays (SAM)microarrays (SAM)
• A clever adaptation of the t-ratio to borrow information across genes
• Implemented in Bioconductor in the siggenes package
Significance analysis of microarrays applied to the ionizing radiation response, Tusher et al., PNAS 2002
SAM d-statisticSAM d-statistic
• For gene i :
di y i x isi s0
y i
x i
is
0s
mean of sample 1
mean of sample 2
Standard deviation of repeated measurements for gene i
Exchangeability factor estimated using all genes
Minimize the average CV across all genes.
Scatter plots of relative difference (d) vs standard Scatter plots of relative difference (d) vs standard deviation (s) of repeated expression measurementsdeviation (s) of repeated expression measurements
Random fluctuationsin the data, measured by balanced permutations(for cell line 1 and 2)
Relative difference fora permutation of the datathat was balanced between cell lines 1 and 2.
A fix for this problem:
SAM produces a modified T-statistic (d), and has an approach to the multiple
comparison problem.
Selected genes:Selected genes:Beyond expected distributionBeyond expected distribution
• An advantage of having tens of thousands of genes is that we can try to learn about typical standard deviations by looking at all genes
• Empirical Bayes gives us a formal way of doing this
• “Shrinkage” of variance estimates toward a “prior”: moderated t-statistics – eliminates extreme stats due to small variances.
• Implemented in the limma package in R. In addition, limma provides methods for more complex experimental designs beyond simple, two-sample designs.
eBayes: Borrowing StrengtheBayes: Borrowing Strength
The Multiple The Multiple Comparison ProblemComparison Problem
(some slides courtesy of John Storey)
Hypothesis TestingHypothesis Testing
• Test for each gene:
Null Hypothesis: no differential expression.
• Two types of errors can be committed
– Type I error or false positive (say that a gene is differentially expressed when it is not, i.e., reject a true null hypothesis).
– Type II error or false negative (fail to identify a truly differentially expressed gene, i.e.,fail to reject a false null hypothesis)
Hypothesis testingHypothesis testing
• Once you have a given score for each gene, how do you decide on a cut-off?
• p-values are most common.
• How do we decide on a cut-off when we are looking at many 1000’s of “tests”?
• Are 0.05 and 0.01 appropriate? How many false positives would we get if we applied these cut-offs to long lists of genes?
Multiple Comparison ProblemMultiple Comparison Problem
• Even if we have good approximations of our p-values, we still face the multiple comparison problem.
• When performing many independent tests, p-values no longer have the same interpretation.
Bonferroni ProcedureBonferroni Procedure
= 0.05= 0.05# Tests = 1000# Tests = 1000
= 0.05 / 1000 = 0.00005 = 0.05 / 1000 = 0.00005oror
p = p * 1000p = p * 1000
Bonferroni ProcedureBonferroni Procedure
Too conservative.
How else can we interpret many 1000’s of observed statistics?
Instead of evaluating each statistic individually, can we assess a list of
statistics: FDR (Benjamini & Hochberg 1995)
FDRFDR
• Given a cut-off statistic, FDR gives us an estimate of the proportion of hits in our list of differentially expressed genes that are false.
Null = Equivalent Expression; Alternative = Differential Expression
False Discovery RateFalse Discovery Rate• The “false discovery rate” measures the proportion of false
positives among all genes called significant:
• This is usually appropriate because one wants to find as many truly differentially expressed genes as possible with relatively few false positives
• The false discovery rate gives an estimate of the rate at which further biological verification will result in dead-ends
tsignifican called#
positives false#
-0.4 -0.2 0.0 0.2 0.4
01
23
Distribution of Observed (black) and Permuted (red+blue) Correlations (r)
Correlation (r)
Den
sity
Permuted
Distribution of Statistics
Observed
Statistic
N=90
-0.45 -0.40 -0.35 -0.30
0.00
0.05
0.10
0.15
0.20
Distribution of Observed (black) and Permuted (red+blue) Correlations (r)
Correlation (r)
Den
sity
Permuted
Observed=
Distribution of Statistics
FDR =False Pos.
Total Pos.
PermutedObserved
Statistic
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.5
1.0
1.5
2.0
Distribution of p-values from Observed (Black) and Permuted Data
p-value
Den
sity
Distribution of p-values
Permuted
Observed
p-value
N=90
FDR = False Positives/Total Positive Calls
This FDR analysis requires enough samples in each condition to estimate a statistic for each
gene: observed statistic distribution.
And enough samples in each condition to permute many times and recalculate this
statistic: null statistic distribution.
What if we don’t have this?
FDR = 0.05Beyond ±0.9
FDR = 0.05Beyond ±0.9
False Positive Rate False Positive Rate versus False Discovery Rateversus False Discovery Rate
• False positive rate is the rate at which truly null genes are called significant
• False discovery rate is the rate at which significant genes are truly null
tsignifican called#
positives false#FDR
nulltruly #
positives false#FPR
False Positive Rate False Positive Rate and and P-valuesP-values
• The p-value is a measure of significance in terms of the false positive rate (aka Type I error rate)
• P-value is defined to be the minimum false positive rate at which the statistic can be called significant
• Can be described as the probability a truly null statistic is “as or more extreme” than the observed one
False Discovery Rate False Discovery Rate and and Q-valuesQ-values
• The q-value is a measure of significance in terms of the false discovery rate
• Q-value is defined to be the minimum false discovery rate at which the statistic can be called significant
• Can be described as the probability a statistic “as or more extreme” is truly null
Power and Sample Size Power and Sample Size Calculations are HardCalculations are Hard
• Need to specify:– (Type I error rate, false positives) or FDR– (stdev: will be sample- and gene-specific)– Effect size (how do we estimate?)– Power (1-, =Type II error rate)– Sample Size
• Some papers:– Mueller, Parmigiani et al. JASA (2004)– Rich Simon’s group Biostatistics (2005)– Tibshirani. A simple method for assessing sample
sizes in microarray experiments. BMC Bioinformatics. 2006 Mar 2;7:106.
Beyond Individual Genes:Functional Gene Groups
• Borrow statistical power across entire
dataset
• Beyond threshold enrichment
• Integrate preexisting biological knowledge
-0.4 -0.2 0.0 0.2 0.4
01
23
Distribution of Observed (black) and Permuted (red+blue) Correlations (r)
Correlation (r)
Den
sity
Correlation of Age with Gene Expression
Functional Annotation of Lists of Genes
KEGGPFAM
SWISS-PROTGO
DRAGONDAVID/EASEMatchMiner
BioConductor (R)
Gene Cross-Referencing and Gene Annotation Tools In BioConductor
(in the R statistical language)
annotate package
Microarray-specific “metadata” packagesDB-specific “metadata” packages
AnnBuilder package
Annotation Tools In BioConductor:annotate package
Functions for accessing data in metadata packages.
Functions for accessing NCBI databases.
Functions for assembling HTML tables.
Annotation Tools In BioConductor:Annotation for Commercial Microarrays
Array-specific metadata packages
Annotation Tools In BioConductor:Functional Annotation with other DB’s
GO metadata package
Annotation Tools In BioConductor:Functional Annotation with other DB’s
KEGG metadata package
Is their enrichment in our list of differentially expressed genes for a particular functional gene
group or pathway?
Threshold Enrichment: One Way of Assessing Differential Expression of Functional Gene Groups
Threshold Enrichment: One Way of Assessing Differential Expression of Functional Gene Groups
Threshold Enrichment: One Way of Assessing Differential Expression of Functional Gene Groups
The argument lower.tail will indicate if you are looking for over- or under- representation of differentially expressed genes within a particular functional group (using lower.tail=F for over-representation).
Can we use more of our data than Threshold Enrichment (that only uses
the top of our gene list)?
EXP#1
Swiss-Prot
PFAM
KEGG
Functional Gene Subgroups within An Experiment
Statistics for Analysis of Differential Expression of Gene Subgroups
Is THIS …
… Different from THIS?
Over-Expression of a Group of Functionally Related Genes
p<7.42e-08
T statistic
Statistical Tests:
2
Kolmogorov-SmirnovProduct of ProbabilitiesGSEAPAGEgeneSetTest (Wilcoxon rank sum)
Is THIS …
… Different from THIS?
Conceptually Distinct from Threshold Enrichment and the Hypergeometric test!
histogrambins
E
O
2
ED =
(O-E)2______
2 is the sum of D values where:
All Genes
Subset of Interest
All Genes
Subset of Interest
Kolmogorov-Smirnov
All Genes
Subset of Interest
Product of Individual Probabilities
What shape/type of distributions would each of these tests be sensitive to?
All statistics
Statistics from gene subgroup
Gene Set Enrichment Analysis (GSEA)
Subramanian et al, 2005 PNAS
Gene Set Enrichment Analysis (GSEA)
Gene Set Enrichment Analysis (GSEA)
Gene Set Enrichment Analysis (GSEA)
Gene Set Enrichment Analysis (GSEA)
Gene Set Enrichment Analysis (GSEA)
Parametric Analysis of Gene Set Enrichment (PAGE)
Kim et al, 2005 BMC Bioinformatics
Parametric Analysis of Gene Set Enrichment (PAGE)
Z =Sm-
/m0.5
The test statistic used for the gene-set-test is the mean of the statistics in the set. If ranks.only is TRUE the only the ranks of the statistics are used. In this case the p-value is obtained from a Wilcoxon test. If ranks.only is FALSE, then the p-value is obtained by simulation using nsim random selected sets of genes.
Arguement: alternative = “mixed” or “either” : fundamentally different questions.
Test whether a set of genes is enriched for differential expression.
Usage:geneSetTest(selected,statistics,alternative="mixed",type="auto",ranks.only=TRUE,nsim=10000)
geneSetTest(limma)
A simple method in Bioconductor
Wilcoxon test
Analysis of Gene Networks
Large Protein Interaction Network
Network Regulated in Sample #1
Network Regulated in Sample #1
Network Regulated in Sample #2
Large Protein Interaction Network
Network Regulated in Sample #1
Network Regulated in Sample #2
Network Regulated in Sample #3
Large Protein Interaction Network
Networkof Interest
Network Regulated in Sample #1
Network Regulated in Sample #2
Network Regulated in Sample #3
Large Protein Interaction Network
Additional Notes on Experimental Design
Old-School Experimental Old-School Experimental Design: RandomizationDesign: Randomization
Dissection of tissue
RNA Isolation
Amplification
Probelabelling
Hybridization
Biological Replicates
Technical Replicates
Replicates in a mouse model:
Common question in Common question in experimental designexperimental design
• Should I pool mRNA samples across subjects in an effort to reduce the effect of biological variability (or cost)?
Two simple designsTwo simple designs
• The following two designs have roughly the same cost:– 3 individuals, 3 arrays– Pool of three individuals, 3 technical
replicates
• To a statistician the second design seems obviously worse. But, I found it hard to convince many biologist of this.– 3 pools of 3 animals on individual arrays?
Cons of Pooling EverythingCons of Pooling Everything• You can not measure within class variation
• Therefore, no population inference possible
• Mathematical averaging is an alternative way of reducing variance.
• Pooling may have non-linear effects
• You can not take the log before you average:E[log(X+Y)] ≠ E[log(X)] + E[log(Y)]
• You can not detect outliers
*If the measurements are independent and identically distributed
Cons specific to microarraysCons specific to microarrays
• Different genes have dramatically different biological variances.
• Not measuring this variance will result in genes with larger biological variance having a better chance of being considered more important
Higher variance: larger fold changeHigher variance: larger fold change
We compute fold change for each gene (Y axis)From 12 individuals we estimate gene specific variance (X axis)
If we pool we never see this variance.
Remember this?Remember this?
Useful Books:
“Statistical analysis of gene expression microarray data”
– Speed.
“Analysis of gene expression data”– Parmigianni
“Bioinformatics and computational biology solutions using R”
- Irizarry