Its been a couple of weeks (and many answering blogposts) since Dr Ben Goldacre launched his paper on research in education, specifically discussing some of the benefits of using Randomised Control Trials (RCTs) and generally suggesting that evidence is a good thing. Nothing in this post should be construed as suggesting that any part of Dr Goldacres paper is unwelcome, or that I do not agree with it completely. What I have written below is not a “but“, it is an “and”.
He gave a little bit of the history, noting that the medical profession had not always had evidence at the heart of its practice. Indeed, initially many doctors fought against this approach “as a challenge to their authority.” Evidence, he suggests, supports professional independence rather than undermining it.
For me, the key paragraph in the report was the final one:
Now we recognise that being a good doctor, or teacher, or manager, isn’t about robotically following the numerical output of randomised trials; nor is it about ignoring the evidence, and following your hunches and personal experiences instead. We do best, by using the right combination of skills to get the best job done.
This struck home in a very personal way.
A few years ago my wife was diagnosed with a pancreas problem that required surgery. Any one who knows anything at about this will know that pancreatic surgery basically requires the surgeon to dismantle the digestive system, repair the pancreas, then reassemble the digestive system (if this were The Princess Bride, at this point in the story we would cut to the Peter Falk character who would explain that everything turns out well in the end).
The surgery was planned for eight hours and took a little over ten. As you can imagine the surgery and the recovery were guided by detailed protocols as to what to do at any stage. Anyhow, a couple of days into recovery pain relief was required. At this point every thing was going well and the patient was in great spirits. I was going to bring the kids in to see mum the next day, now she was out of the ICU. So, Tramadol was prescribed for the pain. As suggested by the protocol. It was the recommended intervention at this stage.
So, hours later, the doctor returned to find the patient deep in depression. Whilst Tramadol is the recommended drug at this stage, for this particular patient it was a disaster. Known side-effects occurred. The drug was withdrawn and replaced with another. Recovery ensued.
The doctor had been following the recommended protocol. Based on the evidence base for this procedure. But his professionalism enabled him to recognise issues with the protocol when applied to this patient, and to over-ride the protocol.
This is what I want to to see for education. We need to know what interventions (or call them what you like) work best with the generality of students/teachers. I then want teachers to understand the evidence, the statistics and their students sufficiently to know when (and how) they should over-ride the protocol for the individuals they have in front of them. This is not a suggestion that because a recommended strategy doesn’t work with all students then teachers should feel free to ignore it. But it does mean that, as with medicines, we must recognise that there will rarely be an intervention that does work in the same way for everybody. And when the recommended intervention fails, the professional has to take over.
The results from RCTs are statistically measured. We need to ensure that everyone understands the meaning of the term “statistically significant”. Its not a statement of truth or falsehood. When it comes to educational research, it is often a statement that a certain majority are benefitted by the intervention. Depending on the specific statistics, it will also mean that some are not. They may actually receive more benefit from an alternative intervention ruled out for the majority by the evidence.
This is why teachers should not be concerned by the idea of evidence based education. Every day, teachers are making decisions about interventions for their students, based on detailed knowledge about who they are. This would not be asking them to do anything new. This is, as Dr Goldacre says about the medical profession and evidence based medicine:-
The opportunity to make informed decisions about what works best, using good quality evidence, represent[ing] a truer form of professional independence than any senior figure barking out their opinions.
Or, more succinctly, evidence will set you free.