The research base I: what counts as evidence
Be able to read a psychotherapy trial and say what it does and does not show.
What you should be able to do afterwards
- 1Explain the difference between an empirically supported treatment and evidence-based practice
- 2Read a randomised trial and identify the comparator, the allegiance, and the blinding problem
- 3Explain effect size, reliable change, and clinically significant change
- 4State the case for and against manualisation
The reading
The three terms are not synonyms and the differences matter. An empirically supported treatment is a specific protocol shown to work for a specific disorder in randomised trials. Evidence-based practice is a decision process: the integration of best available research with clinical expertise and patient characteristics, values and preferences — three legs, not one. Practice-based evidence is data generated from routine clinical work, which addresses the external validity that trials lack.
Reading a trial
- 1What was the comparator? Waitlist inflates effects substantially; treatment-as-usual less so; another bona fide therapy is the demanding test.
- 2Who conducted it? Researcher allegiance is one of the largest predictors of outcome in comparative trials, and is not a sign of dishonesty — it operates through a hundred small decisions.
- 3Was it blinded? In psychotherapy it cannot be blinded for the participant or the therapist, only for the assessor. Self-report outcomes are therefore particularly vulnerable.
- 4Who was excluded? Trial populations are frequently less comorbid, less suicidal and more literate than clinic populations.
- 5What was the attrition, and how was it handled? Completer analyses inflate effects.
- 6Is the effect clinically significant, or merely statistically reliable?
What belongs to this unit
- topicESTs, EBP and practice-based evidence→
- topicHow to read a psychotherapy trial→
- topicEffect sizes and clinical significance→
- formCritical appraisal of a research articleThe article discussion assignment, as a structured document→
- topicReporting guidelines, and the gaps specific to psychotherapy trials→
- topicWriting about numbers without overclaiming→
- topicThe practices that produced the replication crisis→
Questions to take away
For a seminar, a supervisor, or an argument with yourself.
- 1Why does the choice of comparator change the conclusion more than the treatment does?
- 2Is manualisation a methodological necessity or a distortion of practice? Argue both.
- 3What would a psychotherapy trial look like if it were designed to be maximally informative rather than maximally publishable?