Using Bayes factors in biobehavioral research

Daniel S. QuintanaNORMENT, KB Jebsen Centre for Psychosis Research Oslo University Hospital & Institute of Clinical Medicine University of Oslo

Our field has a problem with p-values

The main problems with p-values (or NHSTs)

• Running more participants will get you a

significant result (eventually)

• Can’t ‘compare’ p-values between studies

• A p-value cannot provide evidence for the null,

no matter how ‘significant’ the p-value is

Bayes factors (B) indicate the relative strength of evidence for two theories - the null and alternative hypotheses

Bayes factors vary between 0 and infinity, where 1 indicates that the data do not favour any theory

Bayes factors (B) only consider the observed data, and how they relate to the alternative and null hypotheses

Bayes factors provide 3 conclusions

• Evidence for the null (B < 0.33)

• Evidence for the alternative hypothesis (B > 3)

• Evidence is not sensitive (B is between .33 & 3)

Most null results are never written up.

Bayes factors (B) can provide evidence of whether a non-significant result was due to insensitive data (i.e. underpowered) or the data favours the null

Common language rules-of-thumb

Jarosz & Wiley (2014), Journal of Problem Solving, 7

A comparison of 855 p-values and corresponding B’s

Wetzels et al. (2011) Perspectives on Psychological Science, 6

A comparison of 855 p-values and corresponding B’s

Wetzels et al. (2011) Perspectives on Psychological Science, 6

• The corresponding B of 18% of p-values only provide anecdotal evidence for the alternative hypothesis

• The corresponding B values of 14% of p-values suggest the data were simply insensitive

Bayes factors (B) not affected by stopping rules

Bayes factors (B) are ratios of probabilities so two B’s of equal value provide equivalent evidence

Example: HRV in psychosis spectrum disorders

• When comparing HRV between schizophrenia

group and clinical group, p = 0.001

• B = 133.5, providing support for the null

hypothesis. In other words, given the data, the

alternative hypothesis is 133 times more likely

than the null

Quintana et al. (2016) Acta Psychiatrica Scandinavica, 133

Example: HRV in psychosis spectrum disorders• When comparing HRV between Bipolar Disorders

and schizophrenia, p = 0.99

• This is a ‘large’ p-value, but still can’t use this to

support null hypothesis

• B = 0.21, providing support for the null hypothesis

• Although this was ‘highly significant’, the null

was only 5 times more likely under the null

In JASP you can perform common analyses using NHST and Bayes - if you can’t find your analysis, it’s possible using R scripting

Example: Personality dataset

• Run a full correlation matrix with plots

• Are not significant correlations due to data

insensitivity?

Example: t-tests

• Compare sexes on full NEO and first 3 questions

Example: t-tests

Example: t-tests

The average effect size (d) in social psych is .36, so let’s shift the cauchy prior to .36

Example: t-tests

Example: t-tests

Example: t-tests

Example: Repeated measures ANOVA

• Compare repeated measures factors

Questions?

Bayes theorem

Bae’s theorem