Writing about numbers without overclaiming

Causal language where the design does not support it

The commonest overclaim in the field, and it usually happens in the discussion rather than the results. Correlational and uncontrolled designs support statements about association, prediction and covariation — not about effect, impact, influence, cause, leads to, produces or improves.

Six other things reviewers should catch and often do not

1. 1Non-significant does not mean equivalent. A null result means the study did not detect a difference; only an equivalence or non-inferiority design, powered for it, supports "no difference".
2. 2A significant difference between "significant" and "non-significant" is not a significant difference. To claim that an effect is larger in one group than another you must test the interaction, not compare two p values.
3. 3Effect sizes need confidence intervals. A d of 0.6 with a CI from 0.05 to 1.15 is not the same finding as a d of 0.6 with a CI from 0.5 to 0.7, and reporting only the point estimate conceals the difference.
4. 4Cohen’s benchmarks are conventions, not thresholds. Cohen said so. What counts as a meaningful effect depends on the outcome, the cost and the comparator.
5. 5Statistical significance is not clinical significance. Report reliable change and the proportion crossing into the functional range, not only group means.
6. 6Report the comparator in the sentence. "CBT produced a large effect" is uninterpretable; "CBT produced a large effect relative to a waitlist control" is a finding.

Sentences worth having in your fingers

Because participants were not randomly assigned, these associations cannot support a causal interpretation; alternative explanations including [specific confound] cannot be excluded.

The confidence interval includes effects ranging from clinically trivial to substantial, so the magnitude of the effect remains uncertain despite statistical significance.