Interesting conversation about measurement and metrics on Twitter at the moment which i’m not going to interrupt, a) because everyone seems to be having a great time without me, and b) so many people involved there are only about 12 characters left for real words.
Essentially, the question is ‘how do you know what you are doing is having a positive or negative impact if you aren’t measuring it’. My rather longer than 140 character answer to this follows.
I’ll start by pointing out what a terrible parent I am. For years I’ve been giving my children food and drink without really ever checking that it is having a positive effect on them. I admit to never having weighed either of them once. We stopped measuring how tall they were when we put a cupboard in front of the bit of wall we used to mark their heights. I guess I’ve been relying on a gut feel that if I give them nourishing food then they will grow. I think its working but I have no measurements to give you to prove that. Except to say that the older one is now taller than her Nan and the little one is a little bit smaller than that. Now those are metrics of a sort, but not what I would call formal or transferable.
Instinctively (looks at Statistics degree certificate on the wall) I’m on the ‘you need to be able to measure what is happening’ side of the argument. This is simply because if the intervention (or whatever you want to call it) has a positive outcome (i.e. a child learnt something better than when you intervened in a different way) then you would want to be able to know that so you could intervene in the same way in the future (assuming of course that you could correctly identify the right type of child to make the intervene with).
So I’m all for measurement and metrics. I once spoke with someone who claimed to be a black belt in Six-Sigma who contended that there wasn’t an event for which he couldn’t work out either a direct or proxy metric. I spent some time giving what I thought were obscure examples and he came up with a viable metric every time. Which makes me think that whilst it may not always be desirable to create a metric, it is probably usually possible.
But in a real world environment we need to be more nuanced. In a classroom with 30 children it is not always possible to note down the metric of choice at every moment it might be needed. This is, of course, a trade off. More metrics might be useful, but they eat into time for doing useful things. Good teachers recognise this and, by and large, get the balance right.
Where observers have it wrong is to suggest that every intervention requires a metric to decide if its impact is positive or not. Or more specifically, a metric that they (the observer) can understand and which is transferable to other teachers. I was once asked how did I know if the class I was teaching understood what I was teaching them. Well, whilst it might sound arrogant, I just knew. In reality this was due to a number of informal metrics. What was their body language like? What sort of questions were they asking me? What was their eye contact like when they were being questioned? Because I had worked with those students for months I could interpret those signs and make a judgement. Add these informal ‘metrics’ to longer term measurements that had been carried out and I could rely on my judgement. This heuristic approach is one that teachers use day in, day out.
Layered on top of this is the knowledge we have about the pedagogies that are more effective. For example, we know that formative assessment is effective. So if I use formative assessment and my longer term metrics show my students to be progressing positively, then I can rely on my informal metrics to guide me on a day to day basis. Obviously, in order to know in the first place that formative assessment is useful we would have had to resort to more formal metrics, but once that work is done we don’t need to do it every time we use it.
So the answer to the original question becomes “Of course you need to measure it. Its just that I don’t have time to explain to you how I made those measurements or to even tell you what they are. You just have to trust me. I’m a professional.”