Upload
vanessa-sochat
View
357
Download
0
Embed Size (px)
Citation preview
Brain Maps Like Minesemantic and computational image comparison
methods for meta-analysis and reproducibility of brain statistical maps
Vanessa SochatResearch In Progress
October 20, 2015
Outline
BackgroundWhy do we want to compare images?
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
Meta-analysis to synthesize understanding of human cognition and reproducibility of brain statistical maps.
Why Compare Images?
Are we good at reproducibility?
Why Compare Images?
1. For ReproducibilityWe do not not what a replication looks like.
Why Compare Images?
1. For ReproducibilityWe do not not what a replication looks like.
2. For Meta AnalysisWhat does all the research say about “anxiety?”
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
Results
What if there is data missing?Should I tranform the images first?What am I trying to optimize?
How similar are these results?
Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison
Goal: assess influence of different degrees of image thresholding on the outcome of pairwise image comparison
VARIABLESthresholdsmetricsoptimization
Methods Results
1. Define thresholds
Z = +/- [0,13]
Methods Results
GIVEN
SINGLE VALUE IMPUTATION
COMPLETE CASE ANALYSIS
2. Define comparison method
Methods Results
3. Define our similarity metrics
Pearson’s R
where
"Correlation coefficient" by Kiatdd - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons
Spearman’s Rank Correlation Coefficient
Methods Results
4. What are we optimizing?
Methods Results
Data
465 single subjects7 tasks47 contrast images
working memoryanimalsgamblinglanguagerelationalemotionsocialmotor
“cat” vs. “dog”
Methods Results
Subsampling Procedure
For each of 500 subsamples: Subset data to unrelated groups A and B For each unthresholded map, A i in A Apply each threshold in Z = +/- 0:13, and Z = + 0:13 to all of B Calculate similarity for each of B to A i with CCA or SVI Assign correct classification if contrast A i most similar to equivalent contrast in B
Methods Results
Results
Highest classification accuracy for our dataset: Pearson Complete Case Analysis +/- 1.0
0.984
Results
...
Methods Results
https://github.com/vsoch/image-comparison-thresholding
Results
Methods Results
Conclusions
1. More data is not always better minimum degree of thresholding improves accuracy random field theory may be too much
2. Question the choice of metric, threshold, etc. complete case analysis, pearson, worked for us...
3. “What is the quantitative language that we should use to compare two images?”
Outline
BackgroundWhy do we want to compare images?
Computational Image ComparisonImpact of Image Thresholding on Similarity Metrics
Semantic Image ComparisonOntological and Graph Based Methods
cat dog
Semantic Image Comparison
Is semantic comparison of images useful to classify cognitive states?
The Approach:- “graph” based similarity- “probabilistic” based similarity- compare the two to spatial similarity
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
Goals:- completely automated- assess predictive power of semantic similarity- relate to computational (spatial) similarity
cat
dog
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
cat
dog
Semantic Similarity: OverviewIs semantic comparison of images useful to classify cognitive states?
ONTOLOGY
Tools
Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?
Goals:- completely automated
- Cognitive Atlas, NeuroVault, Pybraincompare
catdog
graph similarity( , )
IMAGE DATA METHODS
Data:- 93 brain maps tagged in NeuroVault with contrast → concept- programatically retrieve data, run methods, and output result
and visualization.
General Workflow- publish interesting results- tag with a contrast, and associated cognitive concepts- assess semantic similarity
- graph based- probabilistic
Semantic Similarity: MethodIs semantic comparison of images useful to classify cognitive states?
Semantic SimilarityGraph Based Methods
cat dog
feline canine
mammal
graph similarity( , )
Graph Similarity: Method
visual feline recognition
visual canine recognition
Wang’s Method
- aggregates semantic contributions of ancestor terms
1. We start with associated concepts.
visual canine recognition
animal recognitionis a kind of
is part ofis a kind of recognition
canine fear response
Wang’s Method
- aggregates semantic contributions of ancestor terms
2. We then take weights at intersection
visual canine recognition
animal recognitionis a kind of
is part ofis a kind of recognition
canine fear response
visual feline recognition
animal recognitionis a kind of
is part ofis a kind of recognition
feline fear response
S( , ) = sum(intersected weights) sum(all weights)
Graph Similarity: Method
Semantic
Graph Similarity: First Round
Semantic SimilarityProbabilistic Methods
Reverse InferenceP(cognitive process | activation)
database of images a new result
Reverse Inferencefor image classification and concept validation
a new result
P( | )
P( | )
P(cognitive process | a spatial map)
P(node mental process|activation) = P(activation|mental process) * P(mental process)
P(activation|mental process) * P(mental process) + P(A|~mental process) * P(~mental process)
P( | )
What does a high score say?about the cognitive concept?
P( | )contributes evidence for
Data:93 brain maps tagged in NeuroVault with contrast → conceptprogramatically retrieve data, run methods, and output result and visualization.
For each of 93 brainmaps, as query image: For each of 140 concept nodes, node, in Cognitive Atlas: calculate P(node|query image) Assign correct classification if P(node|query image) > 0.5
Probabilistic Similarity Method
Probabilistic Similarity Preliminary Results
Probabilistic Similarity Area Under the Curve Across Concepts
Summary
Image comparison is essential formeta-analysis and reproducibility
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
Semantic Image Comparisonis a promising strategy to assess reproducibility
Summary
Image comparison is essential formeta-analysis and reproducibility
A small amount of image thresholdingaids to find images of similar contrast
Semantic Image Comparisonis a promising strategy to assess reproducibility
Acknowledgements
INCF/ NidashSatra GhoshNolan NicholsJessica TurnerTom NicholsJB PolineDavid Keator
CollaboratorsTal Yarkoni
Nipy
FundingMicrosoft ResearchSGF and NSF
PoldracklabRuss PoldrackChris GorgolewskiCraig MoodieSanmi KoyejoPatrick BissettJoke DurnezIan EisenbergMac ShineJoe Wexler
BMIDaniel RubinRuss AltmanMark MusenRebecca SawyerMary JeanneNancySteven BagleyJohn DiMario
Thank [email protected]
vsoch.github.io
Coordinate-Based Approaches
column 1: raw data: big black dots showing the local maxima that are reported, dotted line is “true” simulated signal, black thick line is that signal with added noise.column 2: shows the results of ALE: the result is more of a curve because the “ALE statistic” reflects a probability value that at least one peak is within r mm of each voxel, so the highest values of course correspond to actual peaks.column 3: kernel density analysis (KDA) gives us a value at each voxel that represent the number of peaks within r mm of that voxel. If we divide by voxel resolution we can turn that into a “density”column 4: is MULTI kernel density analysis, which is the same as KDA, but the procedure is done for each study. The resulting “contrast indicator maps” are either 1 (yes, there is a peak within r mm) or 0 (nope).
ALE
KDA
MKDA
Pairwise Image
Comparison
Animals paradigm
vsoch.github.io/experiment
visual canine recognition
CONCEPTS
visual feline recognition
visual feline recognition
RELATIONSHIPS
is a kind of
animal recognition
visual canine recognition
is a kind of
GRAPH SIMILARITY
0.8
Pearson Correlation(rho)