Statistical significance, Interactions, Statistics

Notes

The next session will be led by Hazem Toutounji who will present around a paper:

Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci link to PDF

Expect some general discussion about the challenges to do with meaningful stats in neuroscience / psychology.

The [slides from the presentation] have some more details and ideas.

Some discussion points from the session

[ds]

• intro: difference in cultures between maths/engineering/physics and psychology on how stats is taught and introduced during training.
• Psychology has a particular way of teaching this - emphasis on linear regression, ANOVA, experimental design.
• even though other discipiplines often require much stronger quantitative skills, intuition for experimental design / stats / analysis of data sets common in neuroscience and psychology is not a given
• pop-quiz: based on figure from paper… is there sufficient information in this plot + description for you to accept the conclusions?
• two levels of issue: reasoning about stats, significance, p-values, etc is hard (and even experienced pracitioners make mistakes), appropriateness of techniques/techinical details may not always be correct (people often stick with what they know, have learnt before)
• visual short-cuts from graphs that look like they may imply a significant difference can be misleading
• [see slides for some more pointers]