THE DOG THAT DIDN’T BARK... INTERPRETING NON SIGNIFICANCE

  • Diana Eugenie Kornbrot
  • Rachel M. Msetfi

Abstract

Hypothesis testing is a crucial component of science. This leads to guidelines (often ignored) in most discipline including psychology. Unfortunately, most focus on significant effects. Non- significant effects are sidelined, in spite of their importance to scientific progress. This study reports a survey of practicing scientist on how they would report and interpret explicit scenarios with non-significant effects. There was no consensus on interpretation in terms of predicting future results. Respondents agreed about how to report the significance of a hypothesis test. Most chose not to report any descriptive statistics, or the sample size, or anything about power, or sufficient information to enable replication or meta-. These results shed light on statistical thinking and so should enable more useable guidelines. For non- significant effects, the importance of a priori power is emphasised.

Section
Full Articles