A mini-tutorial on the Statistical Assessment of Time Frequency Data

Recently, I encountered an article discussing the ongoing replication crisis in biology [1]. Why so often, the article stressed, results obtained by different teams using the same data (and following the same inquiries) are difficult to replicate. According to the article, scientists tend to integrate their beliefs into their hypothesis-making machinery (i.e. in the form of a toolbox) to every problem they stumble upon in the field. While the need for collective consensus is clear, potentially diverging decisions taken during a statistical assessment may bring forth confusion rather than clarity. With all its good intentions, excess trust in a method should not lead us into the realm of extreme scientific belief or scientificism/scientism (i.e. the urge to trust on the temporary answers our good ol’ metric provide rather than the underlying problem that inspired them in first place). Coincidentally, while trying to reach this consensus in my own work, I stumbled upon another noteworthy piece in the now obsolete Twitter. The post provided the much needed, so zu sagen, plumber’s perspective:

Statisticians should be less like priests and more plumbers. I don’t care what you personally believe is the right way to do things - if I have a specific problem, I want to know all possible solutions that might fix it, what their limitations are, and how much each would cost. Daniël Lackens

It all made sense then. After all, plumbing and fitting are nuanced tasks that can either yield a pipe jungle or a professionally designed system. In this mini-tutorial, I show how two related statistical approaches, when applied to the same scenario, can lead to equivalent solutions. I provide basic Python code to illustrate how these two different yet fundamentally similar pipelines can lead to slightly different but comparable results. The pipelines are based on examples provided in Fieldtrip and adapted from the book Analyzing Neural Time Series Data: Theory and Practice.

[1]

Anil Oza. Reproducibility trial, 246 biologists get different results from same data sets. Nature, 2023.

Content

Content:

• 1. On TFR statistical assessments
  • 1.1. Background
  • 1.2. Open discussion
• 2. Univariate TFR statistics, a mini-tutorial
  • 2.1. Installation
  • 2.2. Compute the null distributions
  • 2.3. Statistical thresholds overlaid onto spatio-temporal frequency maps
  • 2.4. Conclusion
• 3. Multiviariate TFR statistics, a mini-tutorial
  • 3.1. Compute RDM reliability using permutations
  • 3.2. Plot RDM reliability
• 4. References

Package:

• tfrStats

Indices

• Index

• Module Index

• Search Page