A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Multiple independence tests for point processes:a permutation Unitary Events approach based on delayed
coincidence count
Mélisande ALBERT1, Yann BOURET2, Magalie FROMONT3,
Patricia REYNAUD-BOURET1.
1Univ. Nice Sophia Antipolis, LJAD
2Univ. Nice Sophia Antipolis, LPMC
3Univ. Européenne de Bretagne, IRMAR
September, 8th 2015
MathStatNeuro 2015 - 1/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Introduction of the biological contextProblematic and State of the art
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
time in second
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
0.0 0.5 1.0 1.5 2.0
010
020
030
0
Global problematic
Synchrony detection?
Notion of coincidence
Neurons fire nearly at thesame time
Model based methods:
Unitary Events methods [Grün (1996), Grün, et al. (1999) orTuleau-Malot, et al. (2014)].
Smoothed JPSTH methods [Ventura et al. (2005)].
Surrogate data methods:
Across time such as dithering approaches, [Louis et al. (2010)].
Across trials such as the Trial Shu!ing [Pipa et al. (2003)].
MathStatNeuro 2015 - 2/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceStatistical model
First step
Test independence on a time window.
Statistical Modeling for Neuronal Activity
Spike trains modeled by point processes on an interval (say [0, 1]).
Definition
Point process on [0, 1] = random countable set of points in [0, 1].X := the set of almost surely finite point processes on [0, 1].
Example : homogeneous Poisson process with intensity ! > 0.
"! In the following, no model assumption for the point processes.
MathStatNeuro 2015 - 3/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceThe number of delayed coincidences
Observation: Xn = (X1, . . . , Xn),
where n = number of trials, and Xi = (X 1i , X 2
i ) i.i.d. in X 2.
X12
X11
X21
X22
0 1
X13
X23
Aim:
Test (H0) : X 1 "" X 2 against (H1) : X 1 #""X 2.
Denote X!!n an sample as above satisfying (H0).
MathStatNeuro 2015 - 4/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceThe number of delayed coincidences
Observation: Xn = (X1, . . . , Xn),
where n = number of trials, and Xi = (X 1i , X 2
i ) i.i.d. in X 2.
Notion of (delayed) coincidence for point processes
#coinc! counts the number of coincidences between two point processes:
#coinc!
!
X 1, X 2"
=#
T"X1
#
S"X2
1{|T#S|$!}.
coincidence
$$
0 1
X 2
coincidence
X 1
MathStatNeuro 2015 - 4/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceThe number of delayed coincidences
Observation: Xn = (X1, . . . , Xn),
where n = number of trials, and Xi = (X 1i , X 2
i ) i.i.d. in X 2.
Notion of (delayed) coincidence for point processes
#coinc! counts the number of coincidences between two point processes:
#coinc!
!
X 1, X 2"
=#
T"X1
#
S"X2
1{|T#S|$!}.
Test statistic based on the total number of delayed coincidences
Cobs = C(Xn) =
n#
i=1
#coinc!
!
X 1i , X 2
i
"
.
MathStatNeuro 2015 - 4/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceDefinition of the test statistic
General idea
Reject independence when there are too many (resp. too few) coincidencescompared to what is expected under independence.
How to recreate the distribution under independence?
Construct a new sample Xn from the original one, i.e. Xn, such that
L!
C!
Xn
"$
$Xn
"
$ L!
C!
X!!n
""
,
whether Xn satisfies independence or not.
MathStatNeuro 2015 - 5/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
The di!erent resampling approachesTrial Shu!ing, Full Bootstrap and Permutation
NatureRandomness
Computerrandomness
Original data set
Surrogate data set
Trial-shuffling Full Bootstrap
Permutation
built as either
n=3 trials
Pick n= 3 couples (i,j) with replacement in
(1,2) (1,3)(2,1) (2,3)(3,1) (3,2)
(1,1) (1,2) (1,3)(2,1) (2,2) (2,3)(3,1) (3,2) (3,3)
Pick n= 3 couples (i,j) with replacement in
Pick only 1 permutation given by
1 2 31 3 22 1 32 3 13 1 23 2 1
in
Unconditional distribution: all possible choices of both Nature and Computer randomness
Conditional distribution: 1 fixed original data set (Nature randomness), all possible choices of Computer randomness
MathStatNeuro 2015 - 6/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Conditional distributions of the number of coincidencesHow do they perform?
L!
C!
Xn
"$
$Xn
"
$ L!
C!
X!!n
""
?
Centering issue !!!
MathStatNeuro 2015 - 7/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceDefinition of the test statistic
In view of the statistical literature, it is not possible to estimateL
!
C!
X!!n
""
directly, BUT,
L!
C!
Xn
"
% E%
C!
Xn
"
|Xn
&$
$Xn
"
$ L!
C!
X!!n
"
% E%
C!
X!!n
"&"
.
YET, E%
C!
X!!n
"&
is unknown...
Centering trick
Let C0(Xn) =1
n % 1
#
i %=j
#coinc! (X 1
i , X 2j ), s.t. E
%
C0(Xn)&
= E%
C!
X!!n
"&
,
and letU(Xn) = C(Xn) % C0(Xn).
Then
L!
U!
Xn
"
% E%
U!
Xn
"
|Xn
&$
$Xn
"
$ L!
U!
X!!n
""
.
with
E%
U!
XTSn
"$
$Xn
&
= %U(Xn)
n, and
'
E[U(X&n )|Xn] = 0,
E[U(X"n )|Xn] = 0.
MathStatNeuro 2015 - 8/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Conditional distributions of the centered number of coincidencesHow do they perform?
L!
U!
Xn
"
% E%
U!
Xn
"
|Xn
&$
$Xn
"
$ L!
U!
X!!n
""
?
MathStatNeuro 2015 - 9/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Conditional distributions of the centered number of coincidencesHow do they perform?
L!
U!
Xn
"
% E%
U!
Xn
"
|Xn
&$
$Xn
"
$ L!
U!
X!!n
""
?
Critical value:
(1 % %)-quantile of L!
U!
Xn
"
% E%
U!
Xn
"
|Xn
&$
$Xn
"
,
& with Monte Carlo approximation.
MathStatNeuro 2015 - 9/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceDefinition of the critical region
Trial Shu!ing p-values with Monte Carlo approximation:
Simulate B Trial Shu!ing samples XTS,1n , . . . ,XTS,B
n .
Compute the centered test statistics:
UTSb = U
!
XTS,bn
"
+U(Xn)
n.
Define the p-value by
%TS(Xn) =1B
B#
b=1
1{UTSb
'U(Xn)},
and reject independence if it is smaller than %.
MathStatNeuro 2015 - 10/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceDefinition of the critical region
Full Bootstrap p-values with Monte Carlo approximation:
Simulate B Full Bootstrap samples X&,1n , . . . ,X&,B
n .
Compute the centered test statistics:
U&b = U
!
X&,bn
"
.
Define the p-value by
%&(Xn) =1B
B#
b=1
1{U!
b'U(Xn)},
and reject independence if it is smaller than %.
MathStatNeuro 2015 - 10/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceDefinition of the critical region
Permutation p-values with Monte Carlo approximation:
Simulate B permuted samples X",1n , . . . ,X",B
n .
Compute the centered test statistics:
U"b = U
!
X",bn
"
, and U"B+1 = U(Xn) .
Define the p-value by [Romano and Wolf (2005)]
%"(Xn) =1
B + 1
B+1#
b=1
1{U!
b'U(Xn)},
and reject independence if it is smaller than %.
Then, thanks to [Romano and Wolf (2005)],
P(H0) (%"(Xn) ' %) ' %,
(only for the permutation approach).
MathStatNeuro 2015 - 10/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Test of independenceHow do they perform?
MathStatNeuro 2015 - 11/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Multiple TestingProblematic
Initial problematic
Detect the synchronizations.
Idea : simultaneously test independence on sliding time windows [ak , bk ],
b0 2a
(H0,k) : X 1 "" X 2 on [ak , bk ] (H0,k ) : X 1 #""X 2 on [ak , bk ].
Aim:
Control the m tests at a global level %.
! The errors accumulate !
"! Benjamini and Hochberg multiple testing procedure to control the FalseDiscovery Rate (1995).
MathStatNeuro 2015 - 12/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Multiple TestingSimulation study: parameters
0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0
20
40
60
80
Independent Dependent
Poisson
Injection Injection
Hawkes
spont=60
h =-600.1i->i [0,0.001]
h =0i->j
h =30.1i->j [0,0.005]
i->j [0,0.005]h =6.1 h =-30.1
i->j
A: Description of Experiment 1
h =0i->j
[0,0.005]
n = 50,$ varies in {0.001, 0.002, . . . , 0.04},B = 10000 steps in the Monte Carlo approximation of the quantiles,m = 191 simultaneous tests.
MathStatNeuro 2015 - 13/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Multiple TestingSimulation study: results
time
de
lta
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
0.0 0.5 1.0 1.5 2.0
0.0
00
.01
0.0
20
.03
0.0
4
time
de
lta
time
de
lta
time
de
lta
time
de
lta
time
de
lta
time
de
lta
time
de
lta
time
de
lta
time
de
lta
Permutation
MathStatNeuro 2015 - 14/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
Work still in progress and Perspectives
Conclusions and perspectives
Centering when applying bootstrap-based methods.
The permutation approach is more reliable.
Asymptotic theoretical results for full bootstrap and permutationapproaches.
Non-asymptotic results for the permutation tests?
Choice of $ for the notion of coincidences?
More than two neurons?
MathStatNeuro 2015 - 15/16
A permutationUnitary Events
method
MélisandeALBERT
Introduction ofthe Problematic
Problematic
Single testing
Statistical Model
Number ofcoincidences
Resamplingapproach
Centering issue
Centered TestStatistic
Test construction
Multiple tests
Problematic
Simulation study
Conclusion
References
M. Albert, Y. Bouret, M. Fromont, and P. Reynaud-Bouret.
Bootstrap and permutation tests of independence for point processes.Available on ArXiv: arXiv:1406.1643, (to appear in AOS), 2014.
S. Grün, M. Diesmann, F. Grammont, A. Riehle, and A. Aertsen.
Detecting unitary events without discretization of time.Journal of neuroscience methods, 94(1):67–79, 1999.
S. Louis, C. Borgelt, and S. Grün.
Generation and selection of surrogate methods for correlation analysis.In Analysis of Parallel Spike Trains, pages 359–382. Springer, 2010.
G. Pipa and S. Grün.
Non-parametric significance estimation of joint-spike events by shu!ing and resampling.Neurocomputing, 52:31–37, 2003.
J. P. Romano and M. Wolf.
Exact and approximate step-down methods for multiple hypothesis testing.Journal of the American Statistical Association, 100(469):94–108, 2005.
C. Tuleau-Malot, A. Rouis, F. Grammont, and P. Reynaud-Bouret.
Multiple Tests based on a Gaussian Approximation of the Unitary Events method with delayedcoincidence count.Neural computation, 26(7):1408–1454, 2014.
V. Ventura, C. Cai, and R. E. Kass.
Statistical assessment of time-varying dependency between two neurons.Journal of Neurophysiology, 94(4):2940–2947, 2005.
MathStatNeuro 2015 - 16/16